The priority should be these children and waiting for their Mother to prioritise them isn't helping them. If you and their father find it difficult navigating their hurt with them, then seek professional help because if you're finding it difficult, they're finding it a million times worse.
Given the trauma that these kids have been through and their continuing difficulties, it really is hard to understand why you thought TTC was a good idea? You can never fill the hole that their Mother has made and continues to reinforce, that isn't your job. The role of you and your H is to provide a stable and loving environment and to support these children in their navigation of their emotions.
Continuing to feed them junk because their Mother did is a cop out and shouldn't be an excuse to not prioritise their health. He probably wasn't as active a parent when he was with their Mother and he's had to step up when being a single father but being the RP comes with major responsibilities.
DD was abandoned by her father and even though I was always the primary caregiver and we split when she was two, it still has a profound effect on her. Society has made it acceptable for Father's to be at best, the secondary parent and at the very worst, the absent one. Which probably explains why that some who have the RP role, have an expectation that any new partner become the primary caregiver.
Being the resident biological parent to a child effectively abandoned by their other biological parent is difficult, being the non-biological RP is going to be harder still. So don't beat yourself up about not being able to fill their Mother's (even though she probably wasn't the best) shoes, nobody can. Yours is a supporting role and not the primary, he is their biological parent and he can't outsource the parenting of his kids onto you, that's not fair on you or these kids.