Oh OP, I completely understand where you're coming from. Slightly different situation here in that I sold a house I loved in order to free up some money for my business. The new house I bought was a compromise on location and space, but I thought it would all be worth it in the long term. The house needed some work but not too much...mainly cosmetic I thought.
But then, a company whose services I was using for my business ripped me off of thousands of pounds. It all became very stressful. I was on the phone one evening battling with their lawyer and DS8 came down the stairs to tell me there was water coming through his bedroom ceiling. Then the boiler broke, we had no heating for months, then the storms brought all the fencing down, then a tree, and the porch leaked every time it rained, etc. etc. etc. This all happened within a few months of moving in.
I painted a few rooms, got carried away and started ripping up carpets that we haven't been able to afford to replace so we've been carpetless for 18 months now.
And I know this isn't rational, but I blame the house for my bad decision making. I'm generally a very practical and pragmatic person, but my goodness I've been an emotional wreck about/towards this house.
So I completely understand your feelings of regret. And the point you make about wanting to enjoy your lives with your young DC. I know my son is fine and happy, but I have such a strong urge in me to move from here and get settled in what will then be the house he grows up in and I grow old in. He's 10 and it feels like times is flying by...
I've now gone back to work part time on top of running the business. That money is paying to get the house ready for sale...new flooring throughout, decorator in, new fencing, etc. I'm hoping to have it on the market early next year, which will be just over two years since moving in. It will mean taking on a mortgage, after being mortgage free for 10 years, but that's how strongly I feel against the house!
That said, I also don't love the location and there are things about the upstairs that I don't like and can't change...it sounds like you do love the location and the house itself will be perfect once the work is done...?
So you might find your feelings changing over time. I hope you do. Because it sounds perfect on paper. But it's a very challenging time for you, so do go easy on yourself.
As an aside, my previous house needed full renovation. Took three times longer, cost twice as much as expected, I had to fire the builer for shoddy work and struggled to find tradespeople to do the jobs, and I was living in my parents' attic with a new born for the 10 months it took so it was a very stressful time. But I never once regretted buying that house. So sometimes, maybe, it is jusst the house.
Good luck with it all. I hope it works out for you. But if not, there's nothing wrong with selling up and moving on.