I feel for you sotiredcantthink.
What your op described is not annoying, it is vile-and depending on what country you live in- probably illegal as well.
You are being emotionally abused, and held against your will (illegally imprisoned).
That you call it annoying is evidence that your self-esteem is being erroded severely. It is a systematic development of numbing your feelings down to the point where you comply...with any level of rationalization to make it justifiable in your mind. These rationalizations are like saying, "I don't mind-his way is ok for this", or "Well, I can take no for an answer-I don't need that to survive", and "Well, I am essentially irrelevant so there is no point in having feelings on this, let alone expressing them".
These rationalizations are coping mechanisms.
I think the danger to you for long term exposure to such treatment is severe depression which may lead to suicidal thoughts (and the rationalizations for that drip in very slowly).
The obvious, first choice answer is to get away from there, as previously suggested. Can not your family of origin rescue you and your dc, if any, even it is just with the clothes on your back?
But you must remain. It is not a perfect world and things are complicated (must be a seriously tangible threat of death), I understand. Well, I don't fully-I have not been in your circumstances, but you have all my empathy.
Please actively manage your self-esteem to survive. You are an intelligent and worthy human being. Consider venting your toxic frustrations by writing them down then immediately shredding and soaking the paper bits until it is no longer paper; or burning the paper. The destroying of the toxic writing can be therapeutic.
Are you allowed a hobby? A creative outlet can be invaluable. I am a quilter and endorse that wholeheartedly.
Be kind to yourself. Surround yourself with pleasurable things as much as possible. I don't necessarily mean materialistic 'things' (although that wouldn't hurt either), but things like nice music, aroma therapy, nice body lotion, books with positive messages.
'Why does the caged bird sing?'...but you really need to make an exit. Even if it is a five year or ten year plan. Make a plan because it will give you hope. You don't have to write anything down or give any clue that might initiate trouble; just in your mind, even if you know it will never happen. He can not prevent you from thinking about it, and you are under no obligation to tell him everything you are thinking!
If at all possible try to save a bit each month, hidden. Time passes quickly, it will add up.
Oh, and I doubt the insomniac story. That may be a cover for when he's doing his secret stuff.
for you.