There is probably more of 'his stuff' than you realise. Sports/hobbies/interests equipment, old clothes, shoes, cd's, records, presents and gifts given as mementos of special occasions etc. Go through the house with black bin bags and boxes and pack everything that you would consider to be his into them. You can chuck them in the garage or a corner of a room for now. Ask him to take them with him to his parents or sort out storage until you've decided what you want.
I have a feeling that he thinks that when you have the baby you are going to need his help and he will be able to just come and get his feet back under the table (accidentally staying when you fall asleep tired out - that sort of thing) and by the time you are able to cope he will then make out you're being unreasonable to force him to leave!
You need to make proper plans about when you have the baby, ask your DB or a good friend or another family member to come and stay and help out for the first few weeks when your head will be all over the place. It can be a rota of different people doesn't have to be the same person. Just so long as you are not obliged to rely upon WH.
Also be fully prepared for the guilt trip. You are depriving him of being fully involved in his new DCs life - you never get those days back type of talk - remind yourself (and him) that it was his actions that have resulted in things happening like this not yours. His fault, his responsibility, he only has himself to blame. He will most probably start the blame shifting behaviour soon as he will feel he has apologised enough and you should have got over it by now. Remind him that only you can decide when and if you feel like you are able to try again and him putting pressure on you to resume normal service is making you feel manipulated and less likely to give him another chance not more.
You are doing fabulously btw,