I can see how upset you are about your partner but you seem to be really insecure about the relationship. Pregnancy is stressful - I think you should consider talking to your midwife about how anxious you are feeling and the impact it is having.
Your partner sounds as if he is finding your behaviour really stressful.
I don't know whether he is having an affair but I do think your behaviour is making the situation more stressful for both of you and putting barriers between you.
I have just picked my DH up from work. He was chatting to a pretty young female member of staff in the car park when I arrived and he finished the conversation before he got in the car. I could feel myself getting twitched about it because I am so easily rattled by things at the moment. I had to stop myself making a remark. He then said as he put the seatbelt on. 'That's xxxxx, the one I was telling you about yyyyy going out with. They've been to Majorca together on holiday while we've been off so it's still going on. She was just saying he's in hospital - had to have his appendix out'.
If I had made a remark like 'I don't appreciate being kept waiting while you impress pretty young teachers' I would have caused a problem over nothing. I feel fat, I'm full of wind, my clothes have stopped fitting, I have two big spots, am scared about giving birth, irrational at times and could pick fights with him every day. But he is delighted about the baby, he loves me, he is here, he's helpful and it isn't fair to take it out on him. He's scared too. He doesn't understand pregnancy and if I don't tell him he hasn't got a clue what I am feeling.
Your DP is the same and you are pushing him away with this behaviour.