You have a new-born baby and a partner who needs walking through the complexities of adulthood and intersubjective relationships.
That's quite a lot to deal with. In fact, it may be more than you have the energy and resources for at the moment.
Obviously, it's very sad that your partner is finding this a challenge. At some point, he has missed some of the vital building blocks that help people grow into fully developed adults, who are unafraid of responsibility, feel capable of dealing with the demands of life, and are fully committed to the demands of deep, inter-subjective relationships with others. Who knows where he missed those stages? Who knows how much work would be required to help him repair/build those stages?
You can try having a tough talk with him, explaining that he signed up for this, it's his responsibility too, he needs to do this and stop waiting for you to lead him through it, you don't - any of you - have time for him to stop being scared/acting like a non-adult. You can try gently leading him through it, assure him that he is loved, assure him that he is capable, teach him the things he needs to do.
One of those, or a mixture, might work. You might have the time and energy to do it. Or they might not work. You might not have the time and energy for this.
I think you should set yourself a time limit (and a short one at that) and also a boundary as to what needs to change - and by when (the drinking, for a start, is extremely problematic). Then you can try a couple of approaches. If something works - great. But the time limit and boundaries are incredibly important. If you don't set those, you will end up in a ridiculous situation where your relationship basically becomes one of you looking after three children - one of whom is in the difficult (for you) position of actually being an adult, with power and responsibility - who is taking the role of a child.
That is enormously draining and destructive. You weren't put on earth to pour your energy into helping an adult transition into full adulthood. There are professionals, who get paid good money, who help with that kind of thing. You, on the other hand, signed up for a different kind of relationship with this man: one in which you were going to both take responsibility for the care of each other; for the young, dependent members of your family (the genuine non-adults); and where you had sex with them. Can you really see yourself wanting to have a sexual relationship with someone who is - effectively - asking you to take on the role of his parent/therapist? And can you genuinely share equal responsibility for your (vulnerable) offspring with someone who is asking you to treat him like a non-adult/one of your children?
So, set yourself a time-limit - a short one - because otherwise this will suck the life, energy and joy out of you.