I hear you OP. When I was a young mum, I stayed at home with my children bar some unsocial/ low paid work, and my DHs wage was really enough to support us all. The downside of that, is that I built no private pension and have had no career as such.
I got divorced a few years ago, and am now stuck on a low wage, juggling 3 jobs to try to make ends meet. Paying the mortgage on the family home on my own, after buying my ex out.
The advantage to ME is knowing that my children have a roof over their heads. that they could finish their education in the school that I chose for them, rather than having to take whatever I could get.
Yes, I could have walked away from the house, I could have rented and had the rent paid for me, but I’d have had little choice as to where that was, and would have been at the mercy of a private landlord, who could sell from under me at any time (happened to me years ago, and I ended up in temporary accommodation). There isn’t enough social housing to go around, although, if you’re lucky enough to get it, it’s great. The downside is, that it’s often found in pockets surrounding below standard schools, and with a lot of anti social behaviour problems.
Your home IS an investment, and looking to the long term future, will rise in value. When your DCs decide to move out, you can cash that investment in, and downsize, giving yourself a lump sum to do whatever you choose. Travel. Buy a new car. Buy a second smaller property to rent out yourself, giving you a further income. Or whatever you choose to do with it.
I sometimes wonder whether it’s worth working myself, at the end of an 8 hr shift, I go home around £12 better off than if I hadn’t bothered, and when the trade off for that is not seeing your children, especially when they are young, it’s natural to wonder whether it’s “worth it”.
I’m not sure what the future holds for me tbh. I am debating whether to sell the house in a few years, and split the equity between the kids so they have a deposit for their own homes, and I would then rent. The upside, that they have the security that I want them to have, and I wouldn’t have an asset to be sold to pay for a care home, later in life - which, as you say, wipes out any legacy you would have left them.
In your circumstances, where you say that you need to move, I’d try to find somewhere a bit cheaper, and cut back on my hours a bit - so you can enjoy spending time with your children a bit more than you do now. I had the benefit of the time with my own, and I don’t regret it at all, but I’d encourage my own daughters to keep working and build their own career, rather than assuming that their DPs wage and pension will be enough ti support the family - because if it goes tits up - you’re screwed.
It will be worth the effort and sacrifices in the end, but don’t make your life all about work. A cheaper/ smaller house, really isn’t the end of the world.