Op
I would try not to make assumptions about what is happening. Nothing you have said tells me he is having an affair. He may be, but you certainly don't know that for sure and it doesn't help to assume that.
Clearly, getting pregnant was a trigger for some kind of crisis in his mind. It may be that the marriage was less secure than you thought before that, but the pregnancy does seem to have pushed it over the edge. It sounds to me that things since then have really melted down in terms of communication. And you are suffering so much, especially after the termination, so it is not surprising.
I really hope you are able to listen to what he is saying when he comes without telling yourself you know in advance. (Or even worse having people here tell you what he thinks and feels!) At worst, if your relationship is over and/ or he is having an affair, then you will know for sure and maybe gain some understanding. At best, listening will start to turn things around, after this huge collapse.
Finally, whatever happens I would beg you to separate your heartbreak from his relationship with your son. When he comes to see his son should not be when you try to talk to him about your relationship- please allow your son to see him without you there or at least without it becoming about your relationship breakdown. Otherwise your son will soak up the trauma of your interaction and if your DP feels he can not see his son, without putting himself and his son through that trauma, this will give him the excuse to shut off and distance himself from your child. That really isn't fair on your son.
The only reason to stop your DP having time with your son would be if you think he is not a good parent and would be inflicting something harmful on your son. Nothing you have said suggests this is the case. However hard, it is important to keep his parenting separate from the relationship with you......
I am so sorry things are so hard right now. I am wishing you good luck for the talk.