The problem with letting heads make the is that they could understandably think no harm will be done by family A taking a term time holiday, but it could be a very different story for family B.
Family A are genuinely only planning on a cheaper, term time, foreign holiday once throughout their children's school years, whereas family B do it at least once a year. Family A support their children's learning at home by doing reading, homework, educational activities, and they have high Ability children who would catch up on any work they missed relatively easily. Family B's children struggle academically and are behind many of their peers, and the family are unsupportive of the school with things like reading and homework. These children need all the time they can get in school, certainly more than they need a week by a swimming pool.
Why should head teachers be expected to discriminate against families because term time holidays are more detrimental to some children than others?
They shouldn't imo, it needs to be the same rule for everyone, and families should prioritise their children's education during term term.