The people you have to convince are the ones who aren't keen enough on a party to join.
Like me. I've voted for three different parties (well, one was an Independent) in the last three elections. I can't say any of them are really attracting me at the moment!
(The Independent was because he seemed really impressive when he doorstepped, very clear, and he e-mailed me later to answer a question he hadn't known the answer to when I'd asked. I wanted to help him get his deposit back.)