I think I might have reacted in a similar way to him, and that his action in swearing at you was much less violent than what you did to him.
He was doing nothing unusual or unreasonable, i.e. nothing that you hadn't both done during the previous hour or two. He was sitting watching TV and, because he was hungry, eating. Out of the blue he's implicitly accused of selfishness for not giving some of the food that was the solution to his hunger pangs to someone whose blood sugar he has no direct experience of, and who he would in any case have expected (in terms of established precedent) to solve such problem for herself by getting something from the fridge.
It would have been reasonable (before he started eating) for you to ask for some of his before resorting to getting your own, though once I've lined up an appropriate quantity of food, it would make me very grumpy to have some of it snatched from under my nose.
I'd take an accusation of selfishness very seriously. In terms of my emotions, it would be much worse than being sworn at, as your were, and worse than a physical assault like a hard punch on the shoulder, say.
Being asked to do something for someone is one thing. Being asked why you haven't done what it never crossed your mind to do amounts to being convicted and punished for a crime you didn't know had occurred, and had no opportunity to avoid. (The punishment being the disaproval of a loved one.)
All your background information is probably irrelevant, as there's no reason to believe he processed any of it. That's not to say he was unaware of the parts of it that existed outside your head, merely that he probably never connected any of it with what he was doing. I think his experience was: he was hungry, he ate, then he suffered an unprovoked attack from the person closest to him.