This is a timely thread for me, OP, as I've been doing exactly the same as you - fantasy house buying. I came to the conclusion that I would have to invite extended family to live there, which is not a bad idea really in some ways. Multi-generational living will become a thing again, I think, and I'd rather do that in a big house than in a normal semi. If it was a REALLY big house then potentially I would have DH and me, 2 teens (and partners & kids maybe in a few years) and grandparents all living there. Maybe my sister and her family if they wanted to. I could keep an eye on the grandparents so less stress in that way. But everyone would need their own living & kitchen space as I'm an introvert and couldn't be doing with sharing those all the time. THere's always be someone around so you wouldn't feel as vulnerable.
If you had a swimming pool you couldn't really use it alone, could you? You could slip, crack your head and drown. Or even pass out while swimming and drown that way. General advice is never go swimming alone. But if you had people around you could arrange to have a swim together.
Imagine the fun you'd have with a proper ballroom type of room .Parties and games, with your own extended family & kids/ birthday parties would be right there. If I had such a big house i'd be rich so would get caterers in quite a lot and have a big Downton type of affair when it came to dinners in the large dining room.
And a billiards/games room - would be a laugh (in my imagination, anyway)
I'd have one gym that everyone could use.
You'd have to have a gardener otherwise there'd be arguments about whose turn it was to get on the ride-on mower to cut the massive lawn.
If you went on holiday you'd have ready made house and pet sitters.
It's the way forward!