I ve actually some sympathy for this argument.. or i did.
But think about it... who decides the min wage? who decides how much everyone in the public sector gets paid? and in turn dictates public sector wage rises... yep the Govt, so from 2010 onwards we had pay freezes or 1% rises in public sector, MW was also cut in real terms.
Sure some EU workers came here and did MW stuff but many didn't, the majority did skilled worked and did not under cut anyone, Persimmons don't suddenly pay a Polish bricklayer less than a UK one, same in the NHS or in a food factory.
..and where EU workers did undercut, whose fault is that? 40 years of demonising the unionisation of workers, Govt allow no union work places etc etc.
I'd also take issue on the so called pay rises you say people are getting, going from £10ph to £10.55 ph is not a huge pay rise (care agency my DD used to work for) and many cases, yes the advertised wage is much higher but now includes extra hours and shift payments.
Anyway, Inflation is 9 or 10%, would be very strange if workers across all sectors were not getting 5 or 6% pay rises.... nurses got 3% :( so much for Brexit getting workers more money... its a myth)
But behind all this, is the fact the UK do not have enough people of suitable skills and intelligence to do the work EU folk used to do.