Apply for CSA anyway. If he got a job at least you would get something.
So you tell him this weekend isnt convenient and you need more notice. Tell the school she has a medical appointment and collect her an hour early that day.
Is he expecting you to provide luggage or does he have hus own clothes, shoes, suncream and toiletries for her? I guess he must do if he is turning up to collect her straight from school. Wonder how that works if he is unemployed.
He won't take you to court if he is unemployed I expect, sounds like a stupid threat.
It is reasonable that you have other plans and he needs to arrange dates with advance warning of minimum two weeks if he refuses to put a regular arrangement in place.
But ultimately at some point he will have to take her I suppose.
So I would right away. Buy a cheap second hand phone and set up Location Sharing in FamilySharing settings. Fully charge the phone, zip it into a little pencil case, turn it on when DD goes to school and put the pencil case at the bottom of her school bag. If the phone is on silent it is highly unlikely her dad will even notice it and you will know where the bag ends up.
If your DD is at school she is anyway most likely capable of having a phone. Get her one promptly, set up all of the settings to limit access to apps etc, and teach her how to use it and charge it. At least then you know she can contact you if she needs or wants to talk to you.