Focalpoint, I'm not sure this will answer your question fully though it might provide a partial answer. As far as I can tell Brexit is about ending FOM and not really about promising trade deals.
As any country's economy moves from say a mostly agricultural one to an industrial / service oriented one then more people move from the rural areas to larger towns and especially cities.
This creates a problem for the rural farmers as they have less people willing to work as farm laborers and even with improved automation there are still crops that need careful handling when being picked.
I remember reading about after the Haitian earthquake where a woman sought refuge on her cousin's farm. She was quite willing to have her cousin feed her and give her a place to sleep but she absolutely refused to do ANY farm work to help out her cousin.
After a few weeks and probably some very heated discussions she decided she would rather return to her former city and live off charity, donations, and soup kitchens.
She wasn't the only one. From what I read, the vast majority who tried this returned to the cities because no matter how poor and broke they may have been in the cities, they prided themselves on being city people and not country people.
South Tyrol in Italy use to be a fairly impoverished agricultural province. At the time there would have been no social stigma to being a farm laborer but as industrialization occurred and South Tyrol went from being a rather poor province to a pretty wealthy one, people from rural areas found better job prospects and higher social status for themselves in the cities. So South Tyrol found itself having to import foreign workers to do most of the farm work.
It's perhaps not that much of a jump to go from looking done upon farm work to looking down upon people who are willing to do that work. You may not like them but you really should try to not think too ill of them since they help get that food that you like so much onto the supermarket shelves. A lot of farmers here in the US are reluctant to admit that they are indeed farmers, especially around big city types.
This is an issue that Mike Rowe has attempted to address with his show " Dirty Jobs ", the unfortunate looking down upon people who do blue-collar and working-class jobs.
The word " foreigner " may begin to take on a different meaning though if Brexit indeed results in the great impoverishment of the UK people and then they see people in other countries getting to have things that they once had but cant have any more.