I personally wouldn't take offence. I'm black and some people, especially the older generation aren't PC, they will still refer to brown people as coloured etc. I try to seek things I have in common and have a fairly robust threshold for offence and writing people off. It has always paid off.
I also think that just because someone voted Brexit doesn't automatically make them a racist🙄 I think that's equally as lazy as someone blaming immigrants for everything and I thought we'd got past that nonsense and acknowledged that there were a lot of factors that influenced peoples' vote. Actually calling people racist for raising issues about immigration leads to people listening to people with more extreme viewpoints.
I do think there needs to be an acknowledgement of how immigration legal or otherwise, has rapidly changed some areas, not all positive and for white indigenous people born here decades ago, the areas they grew up in might feel taken over and alien to them.
I was born and grew up in SE London the child of second wave Windrush and one day an elderly lady I used to check in on was saying she felt the local high street had changed out of all recognition. The traditional small independent grocers, stalls and specialists shops (think things like pollards) had gone, replaced with sellers of exotic foods, cosmetics, cafes etc owned by non-british people some of whom don't speak English much or only employ family members as staff and don't mix with the community or attend the local pub etc.
I was quite young at the time and thought 'how ignorant' But after a few years I thought: She's right. That's how it is from her perspective. Who am I to say otherwise? I can see myself how it has changed from Frank and Barbara running the local corner shop and post office, who you would chat and have a natter with to being replaced with a family from abroad who maybe you don't have much in common with and don't talk to English to you beyond necessity and speak in their own language around you.
Then there's schools. Is it necessarily beneficial for schools with children representing 20 different languages or more to be using/receiving extra resources to help them catch up? I'm focusing on the word beneficial, because again for some people, they may think well, everyone else is getting the help, but not the poor white school children in some deprived part of the country away from the major cities.
The NHS: There is a massive shortage of training GPs and other allied health professions. It's all good foreign doctors being fast tracked or granted visas to work within the NHS, but why has the government withdrawn or reduced bursaries over the years for university health courses to the indigenous population whilst placing emphasis on immigrants holding the NHS together? And how many white English GP's are there for a white person to go to without waiting for ages? That sounds ignorant - why do you care as long as they're qualified? - but I try to put myself in their shoes. There have been times that I as a woc have desperately wanted a black doctor because they will understand things culturally that my white doctor as competent as they are, won't quite understand in the same way, and I don't see why it's different for a White person either.
When people expressing concerns (however ignorantly) like your neighbor are not listened to and just written off with a slogan of DailyFail Reader, Brexiter, racist, and not engaged with, then they are sitting ducks for the far right who do listen and give them all the time they want.
We need to have a balanced conversation around immigration. It may be a net good, but it does have impacts that may not be wholly positive that need to be acknowledged if we are to have cohesion as a nation. We may not like those views, but you can't just dismiss them.