This could also all be a way of projecting his own, unrecognised feelings of selfishness for going out drinking onto you.
If this is the case, what might be happening is he sees you're working your ass off to the degree that you are too knackered to go out, and deep down he feels he is selfish for going out drinking while you're waiting at home. But he also wants to be selfish - he enjoys going out drinking - so he suppresses those negative feelings.
But those suppressed feelings don't actually go away - they sit inside him, still feeling bad. So what he does is he projects the bad feelings onto you.
So when you're running late, he phones to ask where you are, and you say your meeting ran over (increasing his suppressed feelings that he is selfish for going out drinking when you're hauling ass). Now he secretly feels even more guilty.
So to relieve or decrease that guilty feeling, he accuses you of EXACTLY the thing he feels bad about doing himself – being out having a drink while he sits at home! Of course, you deny this (cause it's not true) so he's back stuck with those bad feelings. How can he get rid of these bad feelings? He could punish YOU for being late, by being late himself!
Two things could happen now. One: it's all too much drama, so you suck it up and accept the 'punishment' and just wait up to let him in. Maybe you even accept responsibility and say sorry for being late earlier. Now he can privately feel that he was 'right' to punish you, generously forgive you and apologise himself for being late, and feel okay about the whole thing.
Someone (should be him, but actually you) has been accused of selfishly going out drinking when they should be home. And someone (also should be him, but actually you) has also been punished for this selfish act. His bad feelings dissolve, and all is well.
OR, what could happen is you get mad and refuse to leave the key out, or tell him you're angry that he's treating you badly when he gets home. Now, he gets to yell at you, act self-righteous, tell you that you're impossible to please, and otherwise vent all his bad feelings by blaming you for causing the problem. Now the bad feelings are gone, and all is well – for him at least, not so much for you.
This is just a theory, and may be way off course, but do have a think about whether it matches up to your reality.