No, you aren't an idiot.
You really aren't. You love him, you have done your best for him.
Now you have children, your priorities have changed. While you yourself might be able to live with a using addict, you can't let your children see that.
If it's any consolation I too am where I was a year ago. I am a much stronger person now though, and my younger children are older. They are now almost adults, and we can discuss this, and they recognise that their brother had problems beyond his addiction (as all addicts do).
We, as a family, have decided to live with it, with some provisos. And ds is growing up and getting (a bit) better all the time.
You, with much younger children, haven't got the luxury we have, to be able to discuss it as adults and make allowances.
So you need to make a list of what you can deal with and go from there. Do you go to al-anon or narc-anon meetings? Because they can be very helpful, and some have attached creches so you can attend even with small children.
I found that once I was honest to my family and friends things got a lot better. You should start by being totally honest with one person in real life who you trust (a relation or a friend, it doesn't matter). You will be able to tell by their reaction to your honesty how far down the line of self-delusion your family has gone. And go from there. I bet if you put your foot down you will get a lot of support.
And you never know - a complete "that's it, I've had enough" might be sufficient to frighten him. But he has to know you mean it - you can't compromise with an addict.
I wish you all the very best. I'm sure you and your kids will be fine 