You are right it's become so polarised.
As with most political debates there is some truth in both sides but both sides spin the facts to suit their agendas.
1.2 million migrants entered the uk in 2023.
Did we build 1 million new homes (assuming most are single households)?
No.
But but but...
Since right to buy every government has failed to replace the sold off social housing stock. There would be a shortage even with zero immigration.
So the cohort of the population who's parents generation got a council house for life if they needed/wanted it are now stuck in private lets that take over 40% of wages in many cases facing council waiting lists of 20+ years.
If someone seeking asylum is finally approved as a refugee they have to move out of their asylum accommodation which is legally classed as 'unintentional homelessness'. anyone classed as unintentionally homeless goes to the top of the council waiting list. Therefore people do see refugees 'jumping' the housing list above the long term population who are in private lets.
But what is the alternative? Where do newly granted refugee status people live? They won't have been allowed to work whilst their asylum claim was being processed so they are unlikely to be able to rent privately or buy.
I don't know what happens in other countries that don't have social housing??
Where my DM lives they built a row of 4 bed social rented townhouses. Every one was allocated to refugees. It does sting. But the fault is with governments who didn't build more (and people who voted for these governments) not with the families themselves.