To me it sounds like you're unhappy with several things e.g. your finances, your lack of a support network. And you're projecting this onto being single and convincing yourself that is the reason you're unhappy and that you'd be happy if you were in a relationship.
But it doesn't work like that. You'd take all your baggage, the low self-esteem that makes you feel sad about or fear being single, your financial problems, the lack of good friends, into any relationship you were in anyway, with your ex or anybody else. With the added disaster for your DC that if it was with your ex they'd be even more confused when it inevitably goes wrong because you don't even love him and he's lazy and a shit parent because he is their father and you'd have been playing happy families, when with someone new it need not involve your DC at all.
I think you not wanting to be single stems from your other issues and you think a relationship would somehow fix them. It won't.
Why not take the time while your children are small to enjoy that time with them, focus on yourself with therapy, career building to fix the financial issues and using some of your free time as they get bigger to build a support network and make friends? Why not invite some of your DCs friends over, and get to know some parents that way? Easier 1:1. Or take kids to park run or something. Focus on you. Then when you're in a better place you can think again in a few years whether a relationship is what you want, and be in a position to choose a good one if it is.
It doesn't have to involve your children at all, many single fathers will also want to keep things separate from family life. But the fact you'd even contemplate re-establishing a relationship with your ex given what you've said about him and how he treated you and DCs seems to indicate that you're not in the right frame of mind to even think about a relationship with him or anybody else until you've worked through a lot of stuff on your own.