OP, one thing you need to get in your head here:
HE DOES NOT WANT ACCESS.
He knows full well that even with the backstory, if he went to court, he'd get some access. Maybe supervised at first. But he would get it, and he would get the satisfaction of 'getting one over on you'. He can afford it, his solicitor will have told him that he'd ultimately get it. He knows it's what you don't want.
And yet, he hasn't done it!
He doesn't want access.
He doesn't want to have to spend his time with his child.
He doesn't want to give up his weekends.
He doesn't want to have childcare.
He's more happy to see you doing it all than he would be to be able to see his child.
He isn't interested.
He doesn't love or miss her.
He DOES want to upset and annoy you and continue to abuse you, but only if it's actually easy for him, he can't be bothered to put much into it.
So yes to presents and gestures that cost lots, but take no time. Yes to shouting about his rights for five minutes outside the door and harrassing you. No to court appearances and then having to care for a child for a whole Saturday or every other weekend. Jesus no, I'd rather be at the pub...
Stop being afraid of contact. He does not want it.