Defo stay where you are OP.
I am in a 3 bed semi (old 1930's house,) and have a huge garden (like 90 feet long and 40 feet wide at the side of the house, so the house of the neighbour not joined on is 65-ish feet away) and I do love it (been here quite a while!) We are on top of it and have it nice, but it is still quite high maintenance. (I do love gardening though thankfully!)
However I do agree that all the garden activities that are promoted are overrated or exaggerated. BBQs are a ballache and hard work, kids pools get full of crap, and it's too cold half the year to sit out there!
I would rather have a big 4 bed Victorian home (if I had 3 kids) than a 3-bed crappy generic new build. Not ALL new build are crap, but more are crap than not. Only the 4 and 5 bed detached houses at twice the price of the semis and terraced new builds are built fairly well.
I know several people (who are in late noughties and 2010s new builds,) and they can hear their neighbours yawn, talk, fart, grunt, flush the loo, cough, switch on the light in the bathroom, and snore!!! They can also hear their kettle boiling, when the microwave is on, and the sound of the washing machine!
In addition, their homes are tiny, with small bedrooms, no storage space, and a postage stamp for a garden - front AND back. Literally - some new builds have 14 to 16 feet wide gardens.
Stay where you are @flossie86