He sounds like an arse. And I don't think it will make much difference even if you come up with the best argument in the world - he's going to continue being an arse. You can't negotiate away someone's inherent arsishness.
Mutual agreements thrashed out in counselling only work if both parties are reasonable and commit to sticking to the agreement. If one party decides to make threats to change the basis of the agreement then it all falls apart.
One thing I found that helped was to refuse to discuss this kind of thing face to face or by phone. Instead, do it via letter or email. That gives you time to think about a response before having to give it. It also means you can take all the emotion out of it and just deal with the facts.
If he insists on verbal communication then one tip that might help is the broken record technique. Eg, if he starts banging on about changing the budget, say "We made an agreement in counselling and I think we should stick to it" and then just keep saying the same thing.
But if he insists on changing the basis of the agreement then that frees you to do the same. And one thing I would strongly recommend you think about is going to the CSA. When I had a mutual agreement with my ex for maintenance then I felt responsible for chasing the payments every month and justifying why we needed the money. When I decided to simply go through the CSA it got much easier for me. Sure, the CSA's crap but then so's my ex
and I just need to phone the CSA every couple of weeks to remind them to do something. They then have the responsibility of dealing with all the bullshit excuses and other bluster from my ex.
If your ex is self-employed then I'd not recommend this route as it would be too easy for him to hide income but if he's a PAYE employee then this might be a much better option for you.
You have more options for regaining control over your own life here than you think. It just takes a while for it to sink in that you are only beholden to him for things that you allow yourself to be. You can still break free.