The problem with some of these arguments is that it tends to be the more affluent families that take holidays in term time. That does not, of course, mean that everyone taking a holiday in term time is more affluent buy a high proportion are.
There is also evidence that many children taken out of school for a holiday fail to catch up afterwards, leaving them behind where they would have been had there been no holiday.
Then of course you get parents who do things that are just plain ridiculous like taking a child for an unauthorised holiday when they should have been sitting their GCSE exams.
If you want the right to take your child out of school for a holiday you either home educate or you go to a private school that allows such things. If you choose to use state schools you must obey the rules. That means you accept that term time holidays are at the school's discretion and you may be fined if you take an unauthorised holiday. Like it or not, that is the law and has been for years.