I am based in South Africa, but I lived and worked in the UK for a good while and rented there myself, so this one caught my eye. A few thoughts from someone who runs a letting agency at the other end of the world.
The breakdown here sits mostly with the agent, but not only with them. My first question would be a simple one: who gave permission, and where is it in writing? Start there, because everything else follows from the answer. The tenant asked and was told the owner wanted to see plans first, so the tenant does not walk away from this either; proceeding without that sign-off is on them too. But the greater share is the agent's. They manage the property, which means for these purposes they are you, the owner. A permanent alteration to your asset went ahead without your written authority, and the agent is the one party in the chain paid to make sure that never happens.
We run a very simple operational view on this in my own agency: if it is not in writing, it is not authorised. Our lease carries a clause dealing specifically with this kind of situation. The agent's line should have been "we have plans before anyone proceeds, full stop." So I am sorry this is your experience, and you are quite right to be annoyed, I would be too. Whether or not the work is a gain to me is beside the point. It is my property and I make the decisions on my property, and that is exactly what I would have expected the letting agent to respect.
So I would put it to the agent in writing and ask: who authorised the works and on what basis; copies of everything signed, including anything signed on your behalf; the guarantee and commissioning certificates registered in the owner's name, not the tenant's; confirmation your insurer has been notified; and confirmation of what happened to the old system.
That last point matters more than people think. If this was done under a grant scheme, the property owner is normally the applicant, so it is worth establishing exactly whose signature is on the application. And be careful about accepting the certificate without comment, because that can be read later as consent after the event.
You may well end up happy with the heat pump. But you now own the servicing, the warranty position and the eventual replacement cost of a system you never approved, and that is the part to get straight in writing before you decide whether you are relaxed about it.
An agent's job is to bring you the decision, not the invoice.