Melonballs,I hope you manage to come back on here before your h gets back.
I agree totally with SGB.This man is calling all the emotional and sexual shots here.No way is he a "poor man".You are feeling violated;whether he intends this or not,it is absolutely not ok .
You don't have to have sex with him.
Are you safe?
Is he putting pressure on you? Have you told him you don't want to have sex,and if so,what does he do.Does he sulk,or does he carry on regardless,and you end up having sex against your will? - which of course is rape.
If this is happening,or has ever happened,please speak to Women's Aid
This does not sound like the sort of relationship you want or need.By drinking and keeping your feelings to yourself,you remain stuck in the same situation since it doesn't sound as if he is sensitive,kind ,or cares enough to notice and try to address your sadness.
A decent partner would be very concerned about you,and would certainly not pop in and out,demanding sex,regardless.
You deserve a better life.Which ever way you start to tackle this,and however difficult and uncomfortable that may be initially,it has got to be better for you than this.Your dc can only benefit from a happier mum.
There is lots of practical and emotional help and support on here if you feel unable, or are not yet ready,to contact anyone in RL.
I am really concerned for you.Please keep posting.