You can't do this for her.
You are absolutely not unreasonable to feel that you can't do anything more than you have already done.
You've done so much to help her escape and move on, and now she's placed herself and her children right back at square one.
It's not surprising if you feel you can't join her there to watch him abuse her and her children all over again.
What Turnip has suggested sounds like the right thing to do to me. Tell her you'll be there when she needs you but you can't engage right now.
I might also consider contacting social services in the New Year to let them know he is back in the family home and the children may be at risk.
If that feels like a step too far for you, can you contact Women's Aid instead and discuss your concerns with them? Or anonymously to the NSPCC?
There's a not quite local organisation to me that has advocates for women and children who live in these situations and if you have something similar they might be able to advise you on what to do if you feel you really can't cope when she next needs help.
If nothing else, they might give you the name and number of an advocate who can step into the role you played this year if she gets in touch with you. If you know you have someone to contact yourself when she gets in touch, it might help you to feel prepared and better able to cope when it happens.
You have your own emotional wellbeing to think of. If you really can't cope, at the very least you will know you can contact someone else who can help when she comes to you next.
You sound like you've been an amazing friend this past year but you can't do the work for her, and if you've reached your limit