Rainbow, I've not read all comments, but some.
His comment was crass and stupid. You are right to be upset.
Because the baby (however big she or he may be at this moment) is in your body, and because you are already completely committed to the baby, you are viewing things differently to him.
IMHO, I would suggest you forgive his thoughtless comment, explain that you are rather sensitive at the moment, and if he had a person growing inside him, he would probably feel that way too!
Then I would spend the next few months working out if he really is partner material and if you want to stay together and I would allow him to see as much of me as he needed to work out what was best for him.
You were on the pill, it was an accident, had he been more concerned he could have used a condom too. His choice, your choice, now there is a baby on the way. If he misses out on knowing his child I am sure one day he will feel utterly gutted.
So regardless of whether you are going to be the love of his life or not, he has to work out how he feels about the baby and what his response will be.
He will need to pay maintenance to support the baby, but he doesn't have to stay with you, and you do not have to stay with him.
Give him a chance, if you can, to see what is right. for both of you.