I live in a small place, there's a local group at the village hall. They are all very friendly and have been a massive source of support through some difficult times for us recently so I try to attend as often as possible and we both volunteer in the community through them.
There is a couple that go to this group every week. The wife, like me, is in a wheelchair. For whatever reason the husband seems to think that means we will
all be best of friends cos obviously the cripples are all the fucking same and my wheelchair is an integral part of my personality and dh is obviously just like the other guy because his wife is sick too. Ugh They are nice enough people but not really our cup of tea for a bunch of reasons. They are both about 10y older than us, have several children (children are okay i guess but just not for us! Both on the spectrum and would probably find kids a bit of a struggle) and in the politest way possible I feel very uncomfortable saying this as it seems mean but i think may be relevant we are quite far apart intellectually 
The husband is well meaning but irritating, he really is just trying to be friendly (I think he's just lonely, bless him) but its starting to impact my enjoyment of living in this village and I have no idea how to make it stop!
He shows up at my door a couple times a week uninvited which really bothers me, I am a very anxious person and have some serious health issues that are massively impacted by my stress levels and people knocking at the door without warning can trigger all sorts of symptoms. I have mentioned this him and gave him a phone number saying could he please check its okay before he shows up. He's taken to texting and then not opening any reply and still just showing up 
A few times in his company he has made comments that I am not comfortable with, I think he thinks he is being funny but it just isn't. Some of the things he has said are quite shocking really, an example (not verbatim) would be along the lines of "Haha when I get calls like that I just be racist down the phone it's well funny" referring to cold callers. I can't tell if he's serious or not, the way he is constantly playing the really unfunny clown leads me to believe he is joking, but have told him in no uncertain terms that we neither find it amusing nor wish to engage with him on the topic. There are so many more of these types of scenario where he is "joking" and on some level I feel like I'm taking offence at the guys lack of intelligence because they way he talks etc lead me to believe he isn't malicious, just stupid 
I genuinely am unsure if this means I'm supposed to tolerate it on the basis he doesn't know better?
I just don't know what to do. The wife is lovely and has been very kind to me giving me some of her old unused things that I badly needed and couldn't afford. I feel like I should just put up with him because I don't want to upset anyone but both me and dh are starting to dread going to group which is awful as we both really love it here and want to be active in the village.
Advice? 
