mlp, unless you're working in the private sector exclusively, you do get paid for Bank Holidays. I'm in the same line of work as you, and you just have to arrange your time so your child can be picked up. My dd used to go to a nursery that closed at 6pm, and in 4 years I was only late once, and that was due to an RTA which held me up en route. I was able to phone from the car (stationary traffic) and a nice nn stayed with dd till I got there.
Now she goes to a cm who finishes at 5.30 and I have never yet been late....it takes some organising, but it can be done.
Don't start an operation that might not be finished on time. Have the cm phone number on the wall by the phone, so someone can phone her if things go pear-shaped and you can't avoid being late. Speak to your bosses about flexible working- in your situation they cannot refuse any reasonable requests.
Or, find a cm very near to your workplace so you don't spend ages travelling to get to her. Find a cm who can work into the evenings, if necessary. Look into nurseries, they are often more flexible, due to higher staffing......there are lots of things you can do.
If you are happy with your cm, do everything in your power to keep her happy- don't argue about bank holidays; she should be paid for them, end of story. It is reasonable to charge you for lateness- you are impinging on her free time. I would discuss this with her and make it clear that when your lateness is unavoidable, you are happy to pay her overtime.