I refuse to have my son grow up around drugs, my partner knows this and I've told him often enough
So now you have to stand by that - if you stay with you partner, you child will continue to grow up around drugs. Note I say "continue" - it is already happening.
He says he'll not do it again, but deep down I don't think I believe him
Why would you believe him? You've had serious discussions about this in the past, he knows that you don't want him to take drugs, and yet he has done it anyway.
I thought he would change and I thought he did
But now you know he didn't change.
Honestly, OP, I know that this isn't what you want to hear, but you can't trust him - all that will change is that he will put a bit more effort into hiding his drug use from you.
Are we too different people to make this work?
Yes. You think that his drug use is unacceptable and he thinks it's fine. You won't get him to change his point of view - you might get him to SAY that he won't take drugs again, that he's so sorry, that he gets it, he didn't realise how important is was but NOW he's seen the light, etc. It will be bullshit. Fundamentally, he doesn't see the problem.