I'm so miserable right now.
Background info so as not to drip feed:
Been married to DH for 12 years (we're both early 30s.)
He's always been quite inconsistent, e.g. one minute he wants to do something, gets all excited and passionate about it, then the next minute he changes his mind.
We've mostly been happy. He suffers from depression and anxiety and goes through periods of deep, dark depression where he tends to act a bit selfishly , doesn't want to do anything or see anyone, can be quite horrible, says hurtful things like he wished he'd never got married, thinks he'd be happier being single etc 
He's cheated on me twice. Once was 10 years ago, the last occasion was 2 years ago. The last time was the worst. The affair lasted 6 weeks and he lied to me constantly during this time. I only found out by accident. My mental health was quite bad (I suffer with anxiety myself) and this almost killed me. I had to be put on strong meds and was under the care of a mental health team.
He was surprised when I said I wanted to forgive him and was somewhat taken aback I think . I made it clear that i wanted to forgive him and make it work but the road back from infidelity is a difficult one ,and I expected him to do certain things (delete her number, never contact her again) which he did.
He swore he would get help, swore he loved me to bits, swore it was a terrible terrible mistake, swore it would never happen again because he couldn't bear to do that to me again. He swore to do whatever it took, for as long as it took.
Fast forward 2 years to now..He's recently been through another selfish, depressive phase (I've tried my best to be really supportive. ) I've tried to ignore his mean comments.
The last few weeks have been hell. He wants to travel alone. Says it's always been something he wants to do, ALONE. Abroad, without me.
I'm not comfortable with it at all. I think it's because of the previous way he's behaved, but if I'm honest then I probably haven't ever been comfortable with it, even before he cheated. I feel like I would miss him too much, and my anxiety would be too much while he's gone, not knowing where he is or who he's with.
Now he has two affairs under his belt, the trust between us is understandably damaged a lot, and this is a huge factor now too.
I am starting to trust him again (or I was.)
I'm trying to leave the affair in the past and move on, but I can't pretend to be ok with something that would make me uncomfortable.
He's really angry with me for not agreeing to what he wants and is accusing me of being controlling and of keeping him "trapped. "
Last week I tried to compromise and said why didn't he book a few days away with his sister? He agreed. Then I asked why doesn't he go away travelling by himself to visit our friend who lives in Italy ? (This would make me feel more comfortable. )
He agreed and I thought we'd reached a compromise.
Today he comes in saying how unhappy he is with life, he shouldn't have got married at such a young age (22) and he's "just going to do it whether I like it or not."
Apparently I'm controlling him by dictating what he does with his life and who he sees. The compromise we reached isn't enough. He wants to go abroad ALONE and he's booking it right now.
He also made it clear he's not happy about my wishes regarding meeting up alone with female friends (This was because of his past infidelity).
I get sworn at every time I try to talk to him about the subject.
AIBU to not want my husband to travelling by himself?
Apparently "other wives let their husband do it" and I need "to let go and deal with my own anxiety issues. "