No it can be done on your side.
Look at it any way you want, you cannot possibly hand on heart say your daughter has been your number one priority in all of this.
You have.
And you are enabling your boyfriend to be a shit father. That makes you as bad as him.
He isn't putting his children first and you are standing by and letting him?
Honestly there is nothing more disgustingly unattractive than a shit, selfish father.
There is no denying that's what he is. 'The kids will always be there'. No they won't. They really won't. They are real people with their own choices and opinions. Family isn't guaranteed. We don't have to spend time with abusive, deadbeat parents. My dp hasn't seen his dad in 16 year and or his mum and brothers 10 years.
Honestly for the sake of both your children, walk the fuck away! You are as much to blame for their pain as he is as you are deliberately (yes it's deliberate as you are refusing to do the right thing) standing in the way of children having access to both parents (yes it's his sons choice not to see him. But that choice is because of you) and a family support network.
Instead of daddy running off to his new fuck buddy and playing daddy to her daughter half the week after a month and a half.
Please honestly answer how any of that is ok?
Really properly think about it. This is all on you and if you continue to deny responsibility and accountability for your actions, you are a truly horrible person and I feel sick to my stomach for your daughter.