I feel very scared of all the legal stuff and eventually moving. I know when I move it will be the best thing for me but God it feels so daunting...
This too is normal. It IS daunting. But anything worth having is worth the struggle it takes to get it. And what you will get is peace and calm. You have to try and look beyond the 'legal stuff' and keep your eyes on the prize; a peaceful and calm home in which you can relax and just 'be'.
He keeps trying to hug me and saying all nice things to me but I just wished he'd stop as it's effecting my mental health
This is where my advice to 'avoid him' comes in to play. You certainly should tell him to stop and then move away if he tries to hug you, or touch you in any way. But if you can find a place to be in the house away from him that's even better. If he's in the living room, go to the bedroom. If he's in the kitchen, go to the living room. And spend as much time away from the house as you can. Have you told anyone IRL? If not, now's the time. Find someone who will keep your confidence and tell them. Then let them help you spend time away from him.
and he is really good at minimising the affair sayings I'm making it all a big thing when it was all nothing..
It's not that he really thinks it was nothing, just that you should think it was nothing. Because if you accept it's 'nothing' then you'll just accept it, shut up, and keep providing his 'home comforts'. And he doesn't want to lose those.
I just can't wrap my head around why you'd even consider doing that to your family .I'll never understand why ?
No, you won't. Because you never would do that. So the best thing to do, and I know it's hard, is just to quit trying to understand it. It just is. Put all that mental effort into getting the divorce.