It sounds as if you have made your decision not to move to London, OP. I would advise you to keep an open mind about it though, and review the decision every few months.
My DH was seconded to a job in SE Asia. I had NO wish at all to go too and hid behind the perfectly valid excuse of two of our DCs being halfway through A levels and GCSEs. Also I had a job I valued.
I didn't think the separation would be too difficult - we'd had times apart before - but in fact it was. It seemed as if we were living diverging lives; each so remote from the minutiae of each other's day.
Anyway, after a year (once the A levels and GCSEs were done) I did follow him out here (hence my name) and I know that it's better that we, and the two youngest DCs, are all together, but it is a compromise for me and I don't find it all that easy being uprooted from my comfort zone.
You will be able to see each other most weekends, so that's good, but he will be tired from working and travelling, and you will be tired from working and child-raising.
As madwoman has just said, nothing is fixed in stone. Let him go off and test the waters.