I know you don’t want to hear it, but there is someone else. You couldn’t go to the rugby because she is going with him.
We all go through phases with small babies when our partners aren’t our main focus. It evens out over time. Throw in pnd and it’s not fun for anyone.
You need to start looking at it like this; you got ill whilst looking after the baby you both wanted. Instead of caring for you and offering kindness and patience, he decided to let you know how awful you are and how much you’ve ruined his fun.
This is not the actions of a good and kind man.
He has someone else laughing at his jokes and last night he panicked because you were about to uncover all his lies.
Whether you believe there is another woman or not, the only way to get through this is to stop playing his game. Don’t let him decide this is over, decide for him. Tell him he is not the husband or father you thought he was and that you won’t wait around for him to choose to come home. Tell him he needs to arrange contact with your dc and suggest every other weekend (or whatever is appropriate). Do not let him flit in and out as it suits.
Find your anger and go into practical mode. If (and this is a big if) he’s just having a stupid moment, it might make him realise the stark reality of what he’s leaving behind. If not, you are setting your boundaries early.
I’m so sorry you’re going through this. Just because you weren’t perfect, doesn’t mean his actions are justified. You don’t walk away like that, you talk and go to counselling and try to mend things. He’s already placed his focus elsewhere and I think it’s only a matter of time before that comes to light.