I have a place in France, and was intending to move there permanently before finding out I was pregnant. I looked into the logistics (even though it ultimately made sense for me to have baby in the UK). Forgive me, this is just from asking, not experience, but hopefully some of it is useful...
So, I believe you need to get an S1 (or possibly S2? Not sure) form from the NHS (which can take a while to come through). Then take that, along with pretty much every useful document about you (birth certificate, passport, proof of address, French banking details etc) to a CPAM office (Caisse Primaire Assurance Maladie), and they will register you and give you an 'attestation'. Eventually your Carte Vitale will come through, but you can take your attestation to the Dr in the meantime (CV can take months...). Then you may want to look into private "top up" insurance, as the amount covered by the CV varies according to your means, I believe.
If you go to the Dr in France in before you're registered, they will charge you privately. Whether or not you can claim some/all of that back from the NHS, I'm not sure. Worth a try...?
So, I mean, it's a right faff, and the S1/S2 form essentially de-registers you from the NHS, so you wouldn't get free healthcare in the UK in the meantime. But on the plus side, the maternity care in France is one of the best rated in the world, so it's a great idea to get on to the system and have the baby there (especially if your French is good!).
Good luck with it all - you may already be aware that French systems are slow and involve A LOT of paperwork, but all will be sorted in the end.