@TinySaltLick is spot-on and I would really take this advice on board OP.
The conservatory may not have required planning permission, or it may have passed the time after which you can complain anyway. Whether or not planning permission was sought your neighbour is still within their rights to take issue with any issues your building work has caused- and The party wall process ensures this is done in a fair way.
The party wall agreement exists to protect your neighbour from any damage or issues caused by your building work. It also protects you from unfair or inaccurate claims.
It will have included an independent surveyor who would have taken detailed survey of your neighbour's home and then will return after the building work is completed to assess whether there has been any damage (or during the work if your neighbour raises major concerns).
It is the decision of this surveyor as to whether any issues have been created by your building work. You can appeal against the decision, but they are a professional who knows what they are doing. And any appeal is on likely to be successful.
10 years ago our house extended their house very close to ours. There was a gap of 10cm between their wall and ours, and their foundations were dug using a piling method.
There was a party wall agreement in place and unfortunately after the independent surveyor had taken a look at my house they noticed serious issues had been created in the wall closest to our neighbour particularly in the loft. This required extensive plans from a structural engineer and building work to resolve. The bill was tens of thousands of pounds. Our neighbours initially objected but this was rejected and they had to pay to make things right (I am on sure if they were able to do this through an insurance claim or not).
I have since moved house but I am aware that they have also had ongoing issues with regards to ensuring that access is granted to the new owners of my old house in order to keep the 10cm gap clear amd maintain their wall. In essence the people who did the building work are now required by law to provide access for their next door neighbours to maintain their wall and clean the gap.
Things were also quite tricky when I tried to sell the house, I had to declare any issues that had occurred and give details to the damage caused the remedial works and any ongoing issues. It was impossible to hide any of this because it was all logged with the council by the surveyor, pass a sworn statement is required.
My suggestion would be (hard as it is), that you are as pleasant as possible to your neighbour because if their biggest concern is cleaning of their conservatory you may have got off lightly.