VAT is charged on things that are luxuries, so that wealthier people who can afford the extra luxuries of life pay more into the government's income for running the country than those who are just living with the bare essentials and making ends meet without luxuries. Therefore there is no VAT on basic food stuff, but more luxury foods like icecream, confectionary and crisps have VAT added, and food that is sold for eat-in in restaurants or sold heated (there was a whole thing about Greggs Pasties not being VATable if they are still hot straight from the oven, but would be VATable if reheated). In general, VAT is a good thing because wealthy people with a clever accountant can fiddle things to hide income and get away with paying less income tax than they should, but if you are leading a luxurious lifestyle you are going to pay the VAT on it.
Things that are considered to be generally "A Good Thing" don't get VAT added, such as sporting activities and physical education, education and training, and medical treatments - VAT is effectively a "brake" on spending on non-essentials and its seen as inappropriate to tax things like this that it's not a bad thing to spend money on. Books come under this category and so are zero-rated.
However, in real life, books really are a luxury - even without VAT a book is generally around £10 (sometimes it might be as low as £7) and you could feed a family for at least a day on that. Less wealthy families are going to use libraries and get access to books for free, so the main beneficiaries of the VAT exemption on books is wealthy families who can afford to throw away £10 on something they could access for free at the library. Some affluent households may own thousands of books, and that's a huge amount of luxury goods that could have been taxed. You won't find that kind of book collection in households struggling at close to poverty levels.
OK I admit it, I am not actually in favour of charging VAT on books, but the exact same argument - that only wealthier people can afford it - is used as justification for the suggestion that education should no longer be in the list of things that are VAT exempt. If this is the case and education should no longer be VAT free then books are in the same category.
So if you are voting that IABU to call for VAT on books then you should also be supporting non-profit-making schools - ones that are providing education as a service, not as a business, in regions where the state offering is inadequate - to be able to charge their fees at cost-price without adding VAT, even though not everyone can afford that.
So YANBU = yes books and education are only accessible to poor people in their "free" forms from public libraries and state schools, so anyone who can afford the extra money to access a better version privately should pay VAT on that luxury
YABU = books and education are ultimately good for society and not appropriate to be taxed, even though the available budget for spending on them will be unfeasible for many people.