Honestly op with hindsight I wish I had let dds dad disappear out of her life far sooner than he did.
And yours sounds even worse.
I'm willing to bet he will withdraw from contact at some point anyway, probably early high school years which is what my ex did, partly as that's the point that kids start seeing through stuff and asking awkward questions!
Go to cms and get maintenance properly sorted.
Stop pushing contact, let him know available dates/times and then leave it up to him.
I bent over backwards so Dd saw ex for years, repeatedly reminding him of dates, booking and paying for transport, even reminding him to book annual leave at work...
Eventually dd got wise to all I was doing, it was breaking me financially anyway and so we (dd and i) decided it would stop and we'd let him step up if he wanted to - he didn't! Didn't even bother with phone calls or texts.
With hindsight it was clear as soon as we split that he couldn't be arsed making the effort to maintain a relationship with dd, but for her sake (I thought) I fought for him to see her.
The law as it stood then (and I think its still the case) is that rp can neither withhold nor force contact.
I took a risky strategy of withholding contact as I was pretty sure at that time he would then take me to court to have arrangements formalised (he was still harassing me generally at this point and even though we'd split due to his infidelity he hated the idea of my being with someone else).
I was right and he did take me to court but he never stuck to the arrangements, what I should have realised was taking me to court wasn't about dd it was about punishing me for daring to leave and dating to not crumble in the aftermath of our split.
What I should have done was not push for contact, not let him wind me up when he failed to show up, not let him have contact if he was hours late, not told dd daddy was coming and her be heartbroken when he didn't show, if I thought he was going to turn up late I should have made other plans with dd and not been there when he finally deigned to show instead of rewarding his behaviour...
I suspect if I had done that he'd have gradually withdrawn from contact when dd was young enough to forget him, instead it happened at the worst time for her and there was nothing I could do about it.
It's caused dd immense hurt and pain and made her feel very insecure with Boys/men.