I agree but ducking questions like antisemitism when there are still extant human beings who have recently deceased relatives who fled Europe for the safety of the U.K. Is equally divisive.
Oh FFS stop playing "look squirrel". Both the Tory party (who can forget the Nazi dressing up costumes?) and the Labour Party have issues with anti-semitism - the leadership of both, I would remind you, are in favour of brexit.
UKIP is known for attracting anti-Semites. I don't know about the record of the LDs (though I should do) but doubt they are squeaky clean.
What does this have to do with remainers? There may be anti-Semites in the remain camp, but there are also many in the leave camp. Anti-semitism is a terrible thing but it is IMO largely irrelevant this debate - though if you would like to make it relevant we can talk about UKIPs record on this issue if you would like (my bet: you won't).
In contrast to France, for eg, the UK leave movement does not appear to be fundamentally anti-Semitic, despite anti-Semites being attracted to UKIP. (similar to Trumpism IMO: Trump is likely not anti-Semitic himself, but he does seem to have a strong appeal to anti-Semites). In both cases the bulk of xenophobic feeling is being directed against other groups.