There is a step by step process in the circumstances you have described.
Firstly he must serve you written notice that he has revoked permission to keep pets and state what grounds he has for this, and how it is a breach of your tenancy agreement.
In the first instance he should ask you to resolve the pet issue, not go straight to they have to go.
Which you can then respond to, in writing to disagree with.
If he does not accept your written statement setting out why you have not breached your agreement, which he also has to do in writing, you can either get rid of your pets, or refuse to on the basis you do not believe you have breached any terms of your tenancy, again you must state this in writing.
Then, and only then, can he serve you with a notice to terminate your tenancy on the grounds of a serious breach, that you would not resolve whatever issue with your pets that he has set out in writing.
He would have to take you to court , and the court would have to agree that you have seriously breached your tenancy.
So all this informal nonsense you have described is just that, nonsense.
I not you have asked about paying rent. If you live there you pay rent, break clauses and notices to terminate have no bearing on that.
Finally based on what you say he is trying to get shot of you for whatever reason.