It's a long distance, that's true.
What is his stance on moving closer to you? Why does it have to be you moving to him? Is there a possibility of you two both moving and compromising over where? Your ex may be supportive of you moving an hour or so up the road, providing you give him the same rights he had before. Similar with your DP - he may not want to move 3.5 hours away, but would be OK with 1.5 down the road.
Ultimately though, if you want to make it work distance wise, there are ways and means, even if it's tough. If he's suggesting there will never be a right time, it does sound as if he has checked out and the reality of a LTR has dawned on him, as well as the sad loss of your pregnancy (I'm sorry to hear this x).
Please feel free to ignore this as it sounds as if things are very raw for you at the moment, but do you feel there is any unresolved issues from the loss of your pregnancy? If the pregnancy was unplanned, perhaps falling pregnant has unnerved your DP a little - it's common (although quite rightly, if you have sex, you should be prepared) and may not indicate that he wants to end the relationship, but rather he needs the relationship to slow down a little to fully measure up what he is capable of giving you right now. It's a sad fact of life that sometimes we over-egg our capabilities and feelings and when something sad or stressful happens, our true abilities at dealing with adversity are exposed. Your DP may very well want to move in together, and discuss having children at another time, but needs to see that it OK for him to not want those things immediately. He may also be grieving in his own way, and him communicating will help - he may feel bad for you experiencing the loss of a child, and he wouldn't be the first man to distance himself as a consequence.
Ultimately though OP - if he has a problem or concern, or needs to talk, he needs to communicate with YOU - if he can't do that, the relationship is a lost cause. Sometimes relationships that have stressful starts can falter because neither party fully knows how to communicate effectively with each other - neither person is to blame, it's just how it is.
Good luck and look after yourself 