It is important to maintain a balance between a reasonable home life and time working at school. In my opinion, schools are being unreasonable if they issue a blanket No. There can be a number of reasons why families cannot be together for holidays during school holidays, and these are not always just the extra cost involved.
I've known many people who are not allowed to take time off from work during school holidays because those are the busy times for their employers. I've even known people working in the education field who were not permitted to take time off during school holidays - because that time was busy for them completing IT projects while the premises were free.
When I've discussed it with teachers, they have always said that it has been proved that any school time missed is very damaging to a child's education.
However - it cannot purely be down to the number of weeks that a child attends school. Private schools are able to educate children very adequately with much shorter terms - in, say, 29 or 30 weeks in a school year.
In state schools, with longer school terms, a child might attend up to 42 weeks of the year!
That gives a good 10 weeks extra attendance already available for education in state schools.
It might be that in some families, the amount of time spent with their family is damaging or of negligible worth in educational terms. So, for these children, it might be better if they spent all their time at school. But it isn't fair to punish families that can offer more by restricting opportunities that they might be able to offer.
I suggest that the extra time needed in state schools is probably more down to the difficulties of teaching larger classes and lack of resources, and they COULD be solved by investing more funds in education, smaller classes, more resources such as space to do homework in evenings, etc etc. However, the government prefers to concentrate on bullying parents into giving up holidays and time together as a family.