OP,
Just remember nothing is accidental.
He is very deliberately working to avoid you and his child.
He is a shit husband and father.
The getting up late, working late is always the deliberate actions of a shit father.
Good to hear that you won't have another.
20 years ago my friend married someone similar and within a couple of years she left him.
He was similarly lazy and selfish.
She went to a wedding on his side some weeks before they split, when her daughter was just 2, and she was asked multiple times AGAIN, by his various family members when she would have another, including his mother, and she told them with a smile on her face that she wouldn't dream of having another child with such a lazy useless man who was a a completely uninvolved father.
Obviously it got back to him quickly, and he was furious.
She had already decided she was done and had put things in place.
She had moved out within weeks.
She is a top psychiatrist and she said at the time..19 years ago, that she should have seen the red flags, but she didn't.
She later apologised to his mother for her timing but she said she was furious with him and listening to him playing the great father, just infuriated her.
His mother ended up being very kind to her and he got a really hard time from his family.
Now 19 years later, his family are still kind and he did buck up a bit as a co parent, more because he was so shamed by my friend and his family's disgust.
She has a lovely long term relationship but never had another child.
She is very successful professionally.
Get back to work and tell family and friends the truth, he is a lazy selfish waster.
In the interim, you need to be heading out and leaving him with his child.
Start getting organised and stop doing ANYTHING like laundry or food for him.