So once again he hasn’t kept to the 10am drop off that HE wanted and agreed to. Just keep making a note, OP.
I think with you providing uniform, potties and mattresses etc, and him not bearing his share of expenses, it is clear you are still the primary parent. This isn’t 50/50 at all. And I don’t think you should tell him until you are in mediation and you show you’re still the primary parent and he should pay you money for the children. It’s not just about the number of nights.
It gives you leverage for more time with them as well.