Polar opposite, no, but individual policies, results, and votes aren't always about that.
Brexit being a perfect example. I would never date anyone right-wing, because of the way right-wing policies have affected me - and do/will affect me - personally, not just how they affect other people.
I don't see how I'm supposed to believe that someone who's in favour of me having a significantly lower quality of life - for no good reason, since the same policies that cause that also damage the economy - cares about me at all.
But I've known a few lefties who were/are Leavers and who talked passionately about things like imposition of austerity, and how they felt the immigration situation prioritised white Europeans over non-white non-Europeans, and their fear of a Remain vote acting as a tacit approval of TTIP, stuff like that.
I voted Remain, but don't consider myself a Remainer at all, because there were such wildly different arguments on both sides that that would make no sense to me. My position is more that the referendum was a distraction that should never have happened in the first place.
So I guess my point is that a yes or no answer to an individual policy doesn't always tell you whether someone's political beliefs are that different to yours.