I live in France but I had both my babies here - I had been living in the country for 10 years before I had my children so was fluent in French and knew how things worked. Your situation is rather different and I'm not surprised that you are worried - doing things with a newborn in an unfamiliar place/language is not easy on you.
However let me reassure you that the health system in France is very good and you will be offered plenty of well baby checks and the vaccination schedule is similar to the UK for the main vaccines. Watch out - you may be offered Hep B which is considered unnecessary for children in the UK. If anything, the French tend to vaccinate more than the UK and some vaccines are mandatory for crèche/school entry.
All French born children are given a 'carnet de santé' (health book like the UK red book) when they are born - it is generally given to you in the maternity unit. You might want to ask your doctor about getting one of these (he may direct you to the local PMI which is like a well baby clinic). All visits and vaxes plus height, weigh, etc are recorded in this book.
What I found lacking in France was the sort of playgroup you get in the UK where mums can get out of the house with their babies and meet other mums. I was lucky enough to have a mum friend as a neighbour so we hung out together and moaned about lack of sleep, etc with each other.
You will not be visited by a health visitor the way you are in the UK. I was visited only once about 4 weeks after getting out of hospital and the HV was useless in answering my questions about feeding.
I found breastfeeding support to not be that great in general, although there was a volunteer group near me that was open to anyone.
I have found making friends in France OK but I was working and speaking French - saying that, my closest friend here is half German and speaks great English. I have found that I have made good friends over the years but people don't always 'get me'.
Mumsnet is a godsend, put it that way!!
There are quite a lot of MNers in France - I'm nowhere near you, but you may find there is someone not far who might want to meet up.
Do you post in 'living overseas'? I don't post there a great deal but if you put a thread in there about parenting in France you will soon get replies.
Don't hesitate to ask me anything and good luck with your pregnancy and birth x