I think Xmas would be better if the actual day itself (25 December) was not a thing, other than for sincerely religious people (rather like Whitsun for Christians or the summer soltice if you're a pagan or whatever).
Most of the drama around Xmas seems to center around the pressure to spend on particular day centering your family and not letting anyone be alone. So people who are alone at Xmas end up feeling miserable, or if they don't, other people pity them and try to force them to join their gathering. You "have to" invite unpleasant relatives you would normally avoid, out of pity or a sense of decency. Grandparents fight about who they get to spend Xmas day with, and adult children get caught in the middle (guilt trips, arguments, or spending Xmas day dragging about trying to visit both households in turn). Too many people under one roof. Few pubs and restaurants are open, and those that are open will probably serve mediocre food with a snarl and charge huge amounts for it, so someone always ends up having to host at home for far too many people.
It would be better if Xmas was more of a generalized wintery season where there were nice Xmas lights and music and special food throughout (ideally through the end of January), and where you just made plans to meet with all the various people in your life on any day. So maybe do dinner with one set of grandparents on 20 December, do lunch with the other set on 2 January. Which would give the option of going to a pub or restaurant if you want, and would involve your spending a couple of hours enjoyably with someone rather than having way too many people in the house and the whole thing dragging on all day.
Oh, and the other thing that seems to cause misery at Xmas is bloody presents (so much butthurt, hurt feelings, stress, arguments.....). I basically don't exchange any gifts at Xmas other than the kids in the family and a restaurant voucher with my parents, and my goodness it is so much nicer. I'd rather spend the money on getting together with someone I care about and having food and drinks together, frankly.