OP, you don't seem to be listening to what EVERY SINGLE POSTER has told you, to dump him, right away!
It doesn't matter what you were doing, you don't have to justify yourself.
A man who checks up on you like this, is not a good man, despite what he might seem like at other times. This is the start of abuse, if you stay with him, he WILL abuse you, possibly physically, but definitely emotionally, until you end up only wearing clothes that he approves of (no mini skirts or tops that show any cleavage), no make up, unless it's only for his benefit, he'll tell you your friends and family don't really like you, so you end up not seeing them, and a shit ton of other abusive behaviours.
You'll be so ground down, you won't recognise yourself. You'll be walking on eggshells around him, to stop him getting angry.
There is nothing good about men like this. Get rid, or you'll regret staying with him.