While I don't disagree with you about the fact that it may just be that her TV career has run its course your argument about an Islamic cookbook having less appeal because less than 10% of the population is Muslim is ridiculous!
It was a lifestyle recipe book focussing on Ramadan and Eid, the importance of fasting and how eating together to break fast is an important family event for her with the recipes in there as an anchor.
Many many high profile chefs have brought out similar lifestyle / journey type cookbooks - Nigel Slater, Rick Stein, Jamie Oliver etc and their audiences are wider than people from their own backgrounds.
Do you think Yotam Ottolenghi and Sami Tamimi made a mistake in bringing out Jerusalem which focussed on Jewish and Muslim dishes from the city? By your argument it was a marketing mistake given that such a small percentage of the country is either muslim or jewish - but given it sold thousands of copies in the UK and didn't do his career any harm, that argument isn't really one that stands up to scrutiny (his latest book is currently ranked number 5 in Food Reference and Gastronomy).
Islamic recipes tend to be made up of Middle Eastern, Persian and Indian cuisine - and those sort of cookbooks are incredibly popular - as of today, 5 of the top 10 best sellers in national and international cookery on Amazon feature cook books related to Middle Eastern, Persian or Asian recipes including The Ramadan Kitchen.
Nadiya lost her deals the day she announced the book - it hadn't even been published, so there was no way of knowing whether there was an issue with the marketing or how popular it would be.
So given other lifestyle cook books based on ethnic cuisine do very well and she lost the deals before her book was even published, then there is clearly something else at play - the most obvious thing would be a combination of sexism and anti-muslim sentiment.