My husband used to mow the lawn for a neighbour, who was in her 70s. He's a man in his 40s. She used to make all sorts of comments about how the sight of him in shorts and ripped tee-shirt was making her come over all unnecessary (and as a caveat he has a 40-something dad-bod, he's no Adonis). She freaked him out. I just think that perhaps some women have very little awareness when a man is on his own, and don't consider that it might make him feel uncomfortable. That's all. So please, don't make this into a man vs. woman issue, because really it so very much isn't.
As soon as I read your OP, my mind went to wondering if the new neighbour was from an background that was culturally different from yours, as (and I thought this was generally well known, but I'm willing to be corrected) it is considered perfectly normal to offer food as gifts. I used to work in the home of a Sikh lady, once a week, and she was forever offering me food, either to eat on a tea break, or to take home. I discouraged it as much as I could, as I am really not capable of eating anything that's not as bland as can be, but it was well intended and I always accepted.
Similarly, I have neighbours nearby who have brought food round in the past, simply as a gesture of goodwill at Christmas, although between you, me, and the whole of Mumsnet, they had inadvertently p**sed off another neighbour (who had over-reacted in a spectacular fashion, albeit consistent with their bad temper & history of aggression) by doing something that was nothing to do with food, and therefore was very much trying to rally the troops. Safety in numbers and all that.
So yes, based on what you have said, you are being totally, totally unreasonable, and instead of fighting against how you say he made you feel, you could be using that energy to challenge why you feel the way you do. Why is it not possible that you are the one who needs to amend your ways, as opposed to him doing so to suit you?
On the other hand, if -as you have described "a new male neighbour recently knocked on my door to introduce himself and gave me a plate of food" turns out to be a white man in his 30's with a plate of eggs & bacon and a hope you will pop round sometime to enjoy the same at his place, then no, you ANBU, because frankly, that's just weird. But somehow, I'm sensing that's really not what happened.