I have had a bad family upbringing and unstable relationships. But my children are safe and happy and I've never had any previous involvement with social services
So, you are doing really well OP. You have done brilliantly to make sure that your kids did not have the childhood you did.
And your optimism and belief in change is also for the good.
BUT you must not risk yourself and your kids in order to support this man. It isn’t worth it,
And it isn’t a case of whether SS are involved but your own emotional safety. (But yes, they will be involved and if I was your kids’ father I would be seeking Residence if you took this on).
It may be that your difficult upbringing has left you with skewed judgment about boundaries and risk.
There will be people who can help and support him, but it shouldn’t be you.
WAit til He has been out 3 years, then see if he is sober and drug free, emotionally stable and making good choices.
You’ve worked hard to give your kids stability. Don’t take risks. Because apart from anything else the fact that you are even thinking of doing this shows that you are not strong Or boundaried enough to look after yourself. So if it all went wrong it could be very badly wrong. For example, no, don’t discuss this with him, you have nothing to justify. Just tell him you are not and will not be in a relationship with him.