OP- don't know how old you are but I had a similar situation with a boyfriend when I was 15. I thought he was the one, the only slight drawback was that he smoked weed- but never around me and it didn't seem excessive so I stayed with him. I promised myself I'd never touch the stuff.
A few months later the pair of us were both on crack cocaine, though I was far more addicted than he was. I ended up being kicked out by my parents and lived with my boyfriend for a while, then had to leave his and ended up homeless. Both of us had been expelled from school at this point. I had a pregnancy which ended with a stillborn, in which my drug abuse could have been a factor. When I was 16 I got extremely lucky and was taken into a private fostering agreement, managed to get clean, went back to school, took my GCSEs, broke up with the boyfriend and got my life back on track. I thought that was the end of it.
Then when I was 17 I had a bad experience with a different boyfriend I won't go into, I didn't know how to cope with it and long story short ended up back on the drugs, kicked out of my foster mother's, back with the first boyfriend (who was dealing the drugs) and sleeping rough. It took realising I was pregnant with DD to make me realise it had to stop. Thankfully my foster mum agreed to take me back in, I ended up having to resit my A levels after I had DD but other than that my life's back on track 3 years on and I know this time I won't ever be tempted again.
What I'm trying to say is that it doesn't seem like a huge deal to have a boyfriend using weed when he says he doesn't use often, doesn't do it around you and you know you won't be tempted. But it can get completely out of hand, very, very scarily quickly, even if you never intend it to.