"I just don't want to have any regrets".
Life necessarily involves having some regrets. We make choices that we regret or things we don't control happen than we regret.
You can also choose where you allow yourself to have regrets and then move on from it.
You had children with a rubbish partner/father but you don't regret that despite that being a far more consequential poor outcome for them and you than not having a child with this man would be.
Also, saying he doesn't think he can get a woman pregnant because he hasn't so far is a HUGE red flag.
Why was he having entirely unprotected sex with women that it doesn't sound like he was in a long term relationship where both parties had made a choice to try and have children? You say you pointed out to him that they might have been taking precautions themselves - how does he not know if they were or weren't taking contraception? If you are deliberately trying to have children that is the sort of thing you would know.
Here are the many, many reasons that you could regret having a further child. You might feel squeamish about the concept of regretting a child but many people do. It's just hard to say
For example:
Your existing children have less of your time and energy because you have a new baby and a man who doesn't very little at home, you have less money for your existing children, you can't take your existing children to activities or interests or holidays due to lack of money and complicated logistics, your existing children feel unfairly treated or left out by the partner or his wider family, he parents the children in a way they or you don't like, the new child has a disability or additional needs that compounds the situation.
Why aren't you worried about these potential regrets? They are far more consequential to your life and children's wellbeing.
I suspect because you would quite like a child and also would feel its a way of tying this man to you longer term?
We all have some regrets.