To answer your question, no I don't think I would leave the UK, but who knows one day dependant on how things change over the coming decades. For now, I would stay, because I love my homeland and also my family, friends and community here. However it is difficult to see so many issues as are currently faced with under-resourced public services, healthcare, education and this level of pressure through taxation on those who are working very hard to contribute to the economy - in many tax brackets, combined with the state of the economy.
We are fortunate, through hard work and families working very hard over multiple generations and will end up in a position where we face a situation where we will have a greater (potential) IHT liability than this, if we do not plan for it. I don't think i'd leave the country to minimise a tax bill, if I did it would be because children were grown up and settled overseas or we had a desire to live somewhere warmer.
What I will do though, is ensure our tax planning is very thorough to ensure we can pass on as much as possible to our children. If I were you, I'd be planning for how much you realistically need to retain, to live comfortably with the retirement you have planned, taking account of any potential medical needs. I'd set up a trust each (to pass on the 325k of it which you can put in trust IHT free outside of your estate) directly to your children.
I would be looking at gifting the majority of what is left that you don't foresee yourselves needing sooner than later, so the 7 years can hopefully pass meaning it is passed on tax free. If I was pre-retirement age, I'd likely do this in stages, but if older probably just in a small number of (significant) payments. This would help financially set my children up for their futures. They will never need the money more than when they're in their 20s-40s while building careers or raising families, and so I'd want to help them when they would benefit most from it. If I pass on an inheritance by the time they're in their 50s+ they'll not have quite the same need for it, and the earlier some is passed on, the more they can grow it themselves. Perhaps you could invest in property in their name, or help them buy their own properties to live in.
I'd also consider, if either of you falls ill and leaves your full estate to the other, if they ever need care it could swallow a lot, then IHT could take a significant amount of anything that is left, so it really is a question of whether you'd want that money to be used for care, or whether you'd rather ensure at least some of it always reaches your children (they will obviously have legal rights to a third, regardless). I appreciate this is just considerations I'd have, and a lot of people will think IHT should just be paid, whereas I think some is fine but effective tax and estate planning and gifting is also worth considering.
A lot to consider, and it would be interesting to know whether you would think to pass on some now to your children, in the hope it is tax free, rather than leave the country?