Mammy that really is insensitive, and very personal. It's 'advice' regularly dished out to bereaved parents, and its not something the op has hinted at, so maybe you should have kept your thoughts, however well meaning, to yourself.
Op, 2 of my children have died, I have been a bereaved parent for over 2 decades, and the thing that I have learned is that we tend to want to lessen the grief of others, not say things in case it causes pain, and we put ourselves last.
You absolutely should be selfish, you have been through the worst pain a person can possibly go through.
In the head space you're in now I wouldn't advise you to end it with dp, but you really do need to have a conversation. He will also be having mixed feelings and down days and he won't want to tell you about it and cause you pain either.
There may be a compromise to be had, you can stay at your parents or he can take the children to stay at his parents for the short term, you could go to counselling together, maybe even have a break from each other for a few weeks.
Sadly its not uncommon for parents to split after their child dies, having that shared trauma is a very difficult thing to move on together from.
Please give yourself a break op. Your feelings are understandable and valid.
I'm so sorry that you have to miss your baby 