I assumed the OP was a french speaker due to using "souvenir" instead of "memory", but I could be wrong.
OP - The situation you are describing sounds like absolute hell on earth.
You are basically agreeing to be this man's maid, nanny, cook and house keeper. That's an awful situation to willingly put yourself in.
Why don't you just tell him to get out? To go to his other woman (if she will have him), or to sod off on his own?
He can still be a father to his children - there is absolutely no way that you should accept a situation like this. You are letting him have his cake and eat it too - a nice family life, sex on the side with whomever he pleases (you said yourself he would have to get it elsewhere), and absolutely no consequences to his, frankly catastrophic behaviour.
Again, your younger children will look at your situation and assume this is what a marriage is supposed to be. There is a chance your sons will grow up with the view that the wife is for the home, and other women are for romance and sex.
And I cannot even begin to comprehend you accepting that his love child be "brought into the fold" in your own household. How on EARTH are your children going to understand that? How will that child understand that?
You said something about him wanting you to be a good influence on the child - what the fuck? That is what that child's mother is there for.
It doesn't even match his original excuse - that he agreed to father a child with her out of "good will". You don't see surrogate mothers or sperm donors taking the kids out for trips afterwards do you?
Look, I'm sorry but the cold hard truth seems to be that your partner has been leading a double life - bigamy in all but a ring.
Now he is thinking he can just tell you and you will allow him to bring all his disgusting behaviour under one roof, still cook his food for him and generally act like his keeper. And heartbreakingly, you seem to think that this arrangement is your way of showing him your disgust.
It really isn't.
It's just letting him get away with absolutely everything.