The friend I know who was married to someone exactly like this, with children now in their early teens, is now divorced. She was sick of being the only one who engaged in any way with the world outside the house, even as regards taking the children to a swimming lesson or dropping them to play with nearby friends — leaving aside any kind of social life, separately or together.
I used to work with her husband in a former job, and though I didn’t know him that well at the start, I remember noticing and being puzzled by the fact that, to a casual enquiry from someone about whether he had any plans for the weekend, he would sort of puff and blow and look harassed, and say he had to take his son to an appointment, or some other routine thing that would have taken maybe half an hour in total, but which for him obviously dominated the weekend. That to him was a source of stress because it involved leaving the house. A week that contained a parents’ evening at school, or a child’s school play, was a big, stressful deal. He used to be up in his study supposedly working all weekend , or cooking an elaborate Saturday night dinner all day, largely, because I think they were alibis against having to leave the house. If you asked him if he’d had a nice weekend/holiday, he’d blow out his cheeks and say they’d spent it ‘recovering’. Everything appeared to exhaust him, although he was a healthy man in his 40s.
I think I realised how entrenched it was when someone he knew got tickets for Hamilton (which he and the kids loved) and invited them to come, he found excuses for why it was impossible to go to London for a day — he was too busy, it was too tiring — even though it was a straightforward 50-minute train trip, and his wife was going to go with them to ‘supervise’, for free tickets to something they all loved.
His wife used to socialise by herself a lot over their 16 year marriage, but I think in the end she got sick of carrying everything as regards the children, and that they were growing up with the idea that Daddy just couldn’t be bothered.
So, I don’t blame you in the least, OP.