As you don't work, I presume finance is another reason.
My mum and dad haven't been on holiday for about 20 years. My dad is very set in his ways and doesn't like being away from home. I suspect he would get an ASD diagnosis if he sought one. They live in a seaside resort anyway.
When I left home at 18 I had participated in 6 family holidays, 4 of those were UK ones, all places my dad used to go with his own parents. Two of them were in the same guest house in Scotland he stayed in as a child. Dad was never keen and all the holidays were organised with a lot of persuasion from mum. He'd get very anxious a few days before leaving and keep saying he would rather stay at home.
I remember feeling quite self conscious about the lack of holidays as we were quite well off, private school etc and our friends and their parents clearly thought it was odd that we never went anywhere. Peer pressure!
After we left home they went on a couple of extended family holidays abroad, then a couple of group holidays in the UK but I think my mum got fed up with all the effort, persuasion and moaning. My dad absolutely loved lockdown! Some people are really not bothered about holidays at all and some actively dislike them!
If you don't fancy it, there's nothing wrong with not going. Your kids don't sound like they were bothered and they can make up for it later if they want to. I certainly did, I've been on lots of holidays since I left home!