I have been through all of this. My grandchildren live with me under an RO and the order is quite clear to anyone normal.
During the summer holidays, we get 2 1/2 weeks holiday as the primary carers where we do not have to make our GS available for contact. The order states clearly these are "free" weeks each year. They still demand we have to pay back the weekends which would mean my poor GS had hardly any weekends at home during the summer holidays.
They draw up schedules (which are invariably wrong, even getting my GS's birthday wrong) and charts and send them to school etc. They have also demanded extra weekends to which they were not entitled and I foolishly gave in to get them off my back.
They don't seem to realise that if this was the case, I would be demanding the weekends back from their holiday time!
The order is also very clear that the last weekend of the summer holidays is to be spent with us in order that the brothers can return to school together from their own home. They insist this should not happen but cannot provide a reason why other than they don't like it.
However, I eventually saw the light. I made one final reply to a ludicrous letter and told them I did not want to receive any more letters, the order was clear and told them that THEY must comply with the order. I would not entertain any more schedules, charts and demands. I also told them THEY must repay the extra weekend they had demanded and that the order worked both ways.
I understand how frustrated and stressed this is making you. You need to take control now and refuse to be drawn into further correspondence. Do not let him bully you into extra contact. If you do, this behaviour will continue.