Sorry, had to go get DD from nursery then dinner etc.
It all depends on how much your DH will interact with the child. If it's only half an hour a day and at weekends he'd rather be on the computer then it might be difficult, but if he intends to be quite hands on and is willing to spend time on a bit of one to one at weekends then it's perfectly feasible.
I was the main carer with my DD until she was 9mo when i went back to work, then she had a nanny who spoke neither Italian nor English very well. At 15mo she went to full time nursery (Italian) and I'd say I get about 2.5 hours a day with her from picking her up to bedtime, as well as the half hour getting ready in the mornings, plus about 50% of the time at weekends. All the rest of the time she hears Italian, so yes, it is her dominant language, but she understands English perfectly and can express fairly well and it's improving all the time. I do try to travel to the UK at least twice a year, plus we often have relatives to stay, which actually helps more as she has to make more effort to speak English and can't get away with speaking Italian.
Monthly visits from your DC's French grandparents will work wonders, I'd love monthly visits from mine. Can you not pop over to France yourself from time to time as well? I would imagine that as they get older (my DD is only 2.6) seeing the language in a local context might also help.
You do have to be patient, I know I was beginning to despair my DD would never say more than her stock 10 words in English but now it's all tumbling out.