Well, if your experience of refugees/immigrants is one person then I expect you have never experienced any of the following -
Living with your parents, partner and 7 year old in a two bedroom bungalow for 5 years on the waiting list for a housing association property and seeing those that become available allocated to people who are definitiely not locals and can barely speak English.
Living in a nice suburb of London which suddenly gets populated by people from a country who can't/don't want to understand that you don't endear yourselves to those who have lived there for generations by having settees in the front gardens, drinking, shouting and general anti-social behaviour at all hours (because it is their culture). And don't understand the concept (even though they have been given the info in their own language) of rubbish collection and recycling and fill their gardens with crap attracting rats.
Your child being in a school where English is only the 6th language spoken.
Your 14 year old female child going to the corner shop on a hot day for ice creams and being refused service because they are not appropriately dressed (shorts, vest top and flip-flops) because this is now an islamic area.
Cutting the grass in your back garden. Seeing 2 females who have recently moved in next door, going over to say hello and introduce yourself, having the women run indoors and having the men of the house coming out and yelling at you that as a male you can't talk to their women.
Having all the properties in the cul de sac you have lived in for 50 years being occupied by new arrivals to the country. Being asked when you are moving time and again and can they buy your house because you surely don't want to be the 'only white woman in the street'.
Until you have experienced things like this OP, don't go calling those who are concerned about the rapid changes happening in their areas racists and bigots.
Of course all immigrants are not anti-social or behave like this but the rate of change in some areas is extremely concerning to a lot of people who find their way of life being changed without their consent.