I don't think the issue is so much that holidays less in term time (supply and demand) I think it's the pressure put on parents not to miss any school whatsoever, without taking a sensible view.
My DCs school is actually sensible about this. Friends of ours are going to Australia in term time. It had to be those dates for a certain reason, nothing to do with cost. The HT said he couldn't authorise it, but have a lovely time. On the flip side I spoke to one parent who was in tears because she was scared she was going to be fined for taking her child out of school for two days before a holiday in order to visit her mother abroad who has cancer. It dawned of me afterwards her child was still 4, and so legally didn't have to be at school. I don't think that HT handled the situation quite so well.
Obviously if parents just aren't bothering to get their DC into school, and are taking masses of "holiday" its a different matter.