Well, going against the grain I think you are NBU.
I'm guessing you're his primary carer? Which means he's with you most of the time and you manage to feed him healthy balanced meals and milk at regularly timed intervals? Your dh has him how often? Not too often by the way you wrote your op and the fact you felt the need to prep lunch and put out the milk before you went out? Does he know how much milk he should have and in which cupboard it lives? I suspect not, otherwise you could have left him to work it out himself.
So you did all the prep for the afternoon to get 3 hours to yourself and he couldn't even be naffed to give him his lunch and milk properly when it was sat right there!
I agree with pp that baby will survive, can let you know when he's hungry, yadda yadda, of course. This is fine when you're looking after dc all day every day and occasionally you need to relax and go with the flow. I certainly have had rice cracker and babybel days with mine as I look after them 24/7 365. But what's his excuse for 3 hours ffs?
To me this just smacks of good old fashioned sexism. when a man looks after a child we're supposed to throw all basic expectations out of the window and just be pleased the baby is still alive at the end of it. Dare to complain about any aspect of frankly substandard care and you're told to be grateful and stfu. Fact is, he's saying very loudly and clearly that all the work you do is unnecessary and he can just swan off with friends having fun with baby, not worry about their routine, their nutrition or anything that takes any effort, and thereby prove how uptight and over organised you are.
I'm not saying he's a horrible person or a bad dad, just that if we treat men like children and expect so little of them in terms of parenting, it's doing them and us a disservice.