My (millionnaire) ex-boss voted leave, as did all my colleagues except the one who owns a house in France and whose dd lives in Italy. DH and dbro voted leave too.
None of them are racist. None of them are or were remotely bothered about immigration and immigrants; indeed they all agreed with me that our lifestyle, our economy, the NHS etc would collapse without immigrants, and are jolly grateful that they're here, and prepared to put up with years of austerity. All of them (the leavers I've mentioned) know very well that the UK needs immigrants.
My ex-bos and colleagues wanted to leave because they didn't like the direction the EU is going. They don't like the way Germany was tryng to make the Union more than a trading floor. They don't want to lose currency, sovereignty etc. While they do like FOM they were prepared to sacrifice it. I have explained that badly, I know, but it's not actually my own position so I've done my best, as people are still thinking that all leavers hate immigrants, when it's just not so.
I found it hard to decide, being surrounded by leavers, but in the end I voted remain.