Sorry op, I don't mean to sound dramatic but the defensiveness just sets off warning signals to me.
The defensiveness is so they don't have to lie, they would rather shut you down by making you feel akward, unreasonable, jealous or any other feeling they use to shut you and the conversation down, they will also use the threat of anger, anger itself, silent treatment and stonewalling.
So why.
They deflect, they act defensive and then use lies, if the defensiveness doesn't work.
Because a small white lie, is to cover a small lie, a small lie covers up a medium lie, a medium lie is used to cover a major lie, and the major lies can cover up huge, life deceiving lies that you would not believe in a million years of them.
It can all add up to a great web of interlinked untruths, a grand illusion, and it all begins with confusion and the questioning of behaviour that seems to make no sense.
The taking offence of such minor things that to normal people take for granted, their fear of being accused for the most minor of things.
Only many years later did it make sense and even years after we separated I was finding out things that I never knew, minor things, major things, things he didn't need to conceal from me but he did, I clearly 'didn't need to know'
But it all started with this seemingly illogical behaviour.
Covert narc, I hope yours is not like this but maybe it gives you an insight as to why they sometimes can act so irrationally and be offended by such minor questions.
If you have to ask yourself "this makes no sense" then there will be an answer somewhere, there always is.
His comments "you don't need to know" which you found patronising is quite telling.