Most people's problem is that the ones that are let in aren't spread out over the country but crammed into urban areas that are already overcrowded and underfunded. It's not difficult to understand the frustration with the effect this has on services. Or the feeling of displacement...
That's exactly the issue: displacement. A big part of the problem is the way migrants are clustered together into working class neighbourhoods which causes an inevitably fierce scramble for resources, particularly for housing. When social housing waiting lists jump from 4-6 years to 12-15 years local people are put in the position of having children nearly ready to leave home before they are housed. Matters aren't helped by the underhand way many councils prioritise and fast-track migrants and treat local people like fools who can't see with their own eyes how the circumstances of their lives have changed and observe the very obvious reasons for it.
Also there is the issue of schooling. How are people expected to feel when they themselves have attended the local school, their eldest children have attended the same school, then they go to enroll their youngest child and all of a sudden there are no school places available yet scores and scores of children with no ancestral connection to the nation, never mind the neighourhood, running around the schoolyard? This causes inevitable resentment, not towards the children, but towards the displacement of ones own children.
The government has no business taking in foreign people without making adequate provisions for them. Displacing indigenous people and their children and grandchildren from neighbourhoods and schools is immoral and bloody designed to cause bad feeling - and no, it's not about racism - it's about justifiable resentment.