Our youngest started at 3 1/2 with ESF. I deliberately chose a resort where accommodation was ski-in and -out, where the ski school, child-care and lifts were all together and a 5 minute walk or 60 second ski from the apartment. We stayed in Val Cenis. Our 3 year old really loved it, but she is bilingual French/English and also quite bossy and outgoing and quite sporty so was in her element helping the English speaking children and always wanted to be first up the magic carpet and first back to the bottom. We used the child-care a few times after her ski school, it was really a great set-up.
This year she was 4 1/2 and in her class were a few English speaking children who did not get on quite so well with the culture shock of French language and French mentality. I think you just need to consider your child's personality if they are quite young, a bad start to skiing could make it difficult to return the following year.
Make sure you choose a resort with plenty alternatives to skiing, such as a luge run, indoor pool, husky rides etc.
Ouf backpack with baby and skiing, not sure if I could really do that. I'm a level 4 ESF skier, equivalent of a child nearing competition level so not quite advanced level, I like to take risks, but I think I could only manage a magic carpet green slope with a backpack and toddler.
Hope you have a great time, I absolutely adore skiing and wish I could live closer to the Chamonix valley which is our second home.