I have spoken to quite a few migrants as most of my husband's workmates are. All the settled ones say the same thing. That it's not possible to bring up a family with a decent standard of living in this country on the wages currently available. They are trying to feed and clothe and house families of four or five competing against people who pitch tents in woodlands and have almost no living costs for jobs!
I don't care about the nationality or race of the people concerned, many of the people concerned are two Polish or Lithuanian parent families who've been here most of their adult lives and intend to stay and have families and lives here. (And no, outside MN most of them are sensible enough to realise they're probably going to be allowed to stay once negotiations are completed and aren't hyperventilating about forced repatriations and withdrawal of healthcare, families being ripped apart etc, etc because they're not hysterics).
This isn't the first place this has happened, off the top of my head it happened when Singapore was colonised - large numbers of transient low paid workers push down wages and push up prices to the point where only the rich can afford to bring up families. Aside from those lucky enough to get social housing before it ran out this is largely the case in the SE of England now.
And yes, I do think it is more important that people who are settled can afford a decent standard of living than transient workers who intend to work here temporarily to enrich themselves can afford to buy a fancy house in their homeland or retire early. I would always prioritise people wanting the basics to live above people wanting to actually get wealthy.
It is starting to change a little as the weak pound has meant the most mobile workers are starting to think about going elsewhere.
All this 'oh the working class is going to be hit hardest': I don't think there is a huge amount of evidence for this, particularly as most EU funding goes into jobs associated with middle class white collar labour. But even if it does happen, I find the general attitude is that it can't get much worse, so it's worth a gamble. The 'oh its going to be so much worse for the working classes argument' actually comes off as 'It will probably get no worse than being the same for the working classes. But we get all the good stuff if we stay in the EU so let's try and fool them into thinking it benefits them'.
I can't believe so many people are so credulous about the EU being some sort of unionised workers utopia either. The EU is a supranational organisation which prioritises market forces and the free movement of capital and labour to the advantage of business. By it's very nature it allows businesses to move easily to places where workers have the least rights, lowest pay and worst conditions in the EU circumventing unions and accessing the cheapest labour.
I don't think you can really expect people to be mugged off that the EU is a great sharing caring socialist utopia when it doesn't take much surface scratching to see it's very much geared towards enriching the rich at the expense of the poor.