I found this to be very interesting during the Brexit campaign. I found that the views changed a lot depending on who I spoke to or what they read.
But what I found remained the same was how, quite a few people who had been pushed out of the cleaning industry, i.e hotels, night clubs, and offices felt that the Uk was better off outside the EU. I don't think Poland coming to the UK in the first 5 years was the problem. What i remember happening was after the 2008 financial crush, lots of cleaners lost their jobs because the companies decided to fire all the old workers being paid a high wage and hire through an agency. Most of the new workers were Polish and this created resentment. Most of the black workers were not getting those jobs anymore.
changes in immigration with much stricter rules also played a part. knowing that the Uk would suffer if they didn't have enough workers leading to hiring more people from outside Europe.
What I found interesting, was how these voices were drowned out and no one cared to stop and think. Instead, a lot of people just assumed the UK would not leave the EU. The confidence and cockiness from remain were interesting and it showed ignorance in a lot of people who assumed that the Uk would not leave.