I don't predict good things for the housing market so chose to do this last year. Moved to a cheaper part of the country and cleared the mortgage totally. You can lose your existing home at any time while the bank still owns it if you have a run of bad luck. Once your house is paid off it's yours forever.
Your home is worth 1m now - will it still be in 5 years time, given that house prices can and do fall at times? We are at the top of yet another bubble. If you own your family home outright the machinations of the market become so much less of a concern. The IMF and other respected bodies have stated that house prices need to come down and that an interest rate rise will impact the market.
If you look at how compound interest works and it just seems to make sense to be as debt free as you can possibly manage right now given the volatility on a global scale of the financial world.
I personally feel that our current government has done a lot of tinkering round the edges and kicking the can down the road, eventually the music will stop. Interest rates will go up, nobody can say their job is 100% secure and accident and illness can hit at any time.
I think greed gets people into horrible trouble, and that security for your family is most important. If you are debt free that gives you tremendous freedom to really ensure your kids fly.
The money you would have been spending on the mortgage could always go into a pension fund, a cheap BTL up north, a small business start up etc.
If I had your options I'd go for a family home owned outright + a couple of cheap BTL's in Wales (yields are as high as 10% in Cardiff unlike London & the SE and the market not as volatile) with no mortgages on them. Security is more my priority for old age than being "rich" iyswim. That would give you pretty much cast iron security. I might even invest some in gold, farmland or utilities just to diversify a bit, and spread my risk.