NB all the below assumes you currently have a repayment mortgage, not interest only...
Hmmm. Re your equity "might" rise - barring a massive house price drop, which I think is unlikely, it will DEFINITELY rise as you pay down your mortgage and prices probably do continue to go up, even if not at rates seen in recent decades.
It feels on paper like the house is always going to be bigger than you need it to be. Unless you plan on having more DC, a four bed house is more than you'll ever need personally.
If you move to a cheaper property, it's worth working out what it would cost you including stamp duty and the sum you'd need to borrow, including what your future payments would be on today's high rates. You might not actually feel all that much better off month to month, so you need to run those figures. It's worth looking at what your total repayments including interest would be on that speculative new mortgage, not just monthly payments, to understand what the overall cost of that new mortgage is versus the value of the asset you'd be buying.
Another factor is that, once you have paid off this mortgage (how long is the total remaining term?), you'll have an asset worth around £500k, even without likely price rises, versus owning an asset worth £280k if you move, and with your nice current low fix you may have paid comparatively little for it because of lower interest rates than anything you could secure now. So being in a more expensive property now can be seen as a kind of "forced savings" vehicle that will leave you with more assets in the future.
Assuming your monthly outgoings would be meaningfully lower if you move (and as PP have said, you need to try to estimate running costs of a smaller property and include that in these calculations too), what you're really weighing up is more cash month to month now versus more assets to your name in future. There isn't a right answer to which of those is better - it depends on your circumstances, and personality, too.
The flip side of this is that you could move, own a less valuable asset, but use some of the money you save in monthly outgoings to put more money into other assets eg investments, so you're still building up assets in ways other than via your more expensive property. This could be useful if you want to eg save up for your DC's university fund or whatever, as they'll be more accessible than capital tied up in property.