Dare I say it? Some men are super clingy. Don't know if this applies to the OP's DH or not. My DH won't go to the shops alone, or the doctor alone, or even to see one of our (adult) daughters alone. He always has to have me there. (Or sometimes someone else if I can't go!) I can go out alone all right and often do - or with someone else, but it's just that HE won't go out alone. Only to work!!!
He always used to do this 'coming to bed when I went' too, until he started working shifts about 15 years ago. (nights 10-8, and days 7-5, four days a week.) So when he was on nights I would stay up til midnight or later watching tv, then get into the big snuggly bed on my own til 9am when he came to bed bliss 
Then the next week he would be on 7-5 and often went to bed at 10 as he had to be up for work at 5.30am. So again, we had different bed times.
On the 3 days he was off we went together often (which I didn't mind because we didn't go together on 4 days of the week, so it was nice to go together sometimes at least.)
It has never bothered me that he often wanted to go to bed with me, but if it had have done, I am pretty sure that I would have told him.
It does seem odd to be so massively irked by your own husband just wanting to go to bed at the same time as you, but each to their own, a few things about my DH piss me off LOL. And I reckon some stuff about me does!
Can you not have separate rooms? Is that an option? I know it's taboo and some people think your relationship is defunct if you sleep apart, but that's a crock; some marriages have been saved by it.