We went with dd1 at 7 weeks with Canvas hols tent, it was fine. As slightly says, it wasn't so much a holiday as a change of scene, but it was nice.
Took dd2 at 10 weeks (CH tent again) and that was a doddle, the 'pain in the arse' bit was potty training dd1 at same time in a tent. Not my choice, I hasten to add: hers.
Lots of baby baths etc onsite, I didn't find the lask of running water a problem, just let DH deal with it.
Took DS away c 3months and camping proper at 6 months. Old timers by then, we have a canvas highchair and sling so no equipment needed. We have photos of him eating a baguette in the sling on the way back from the shop in the morning.
I've BF all three of mine & lived/traveled/holidayed in France: only ever had a comment re BF when in someone's home, they had individual ishooos (DH is French so we have a lot of French relatives). Very approved of culturally, however, because they have shorter maternity leave, many of DH's cousins etc switched to bottles at c.4 months for simplicity so horses for courses, as it is here.
Go for it! If you do take a buggy etc, I personally always find it a bit of a pain in France, as the pavements are so narrow and variable outside the main cities so I prefer slings over there. Changing nappies when out and about is also difficult, I do a lot of nappy changing in the boot of the car etc.