OMG.
Okay following the advice from MN I went out and started taking photos of the mess in my garden and the new door.
Just as I was going back inside I heard people coming in next door. Lo and behold it's the builders (about 6 of them, which seems excessive but may explain how it was done so bloody quickly!!)
Anyway I waited till I heard them at the back and went out CALMLY to explain the situation. I made them aware that this was a private garden for my flat only and asked what the neighbours were planning on doing etc etc.
The older one starts on at me about how they have just been employed by the owner, nothing to do with them etc. So I explained that I'd taken pictures of everything and would be contacting the planning dept for advice.
He became very agitated, started ripping of the stickers on the door (I presume because they had his company name on them) told me it's "just a door, there was a window there any way."
Then they all buggered off!!!!
WTF?? I have googled the company name and it seems to be a very small one man and a van type thing which is based 100 miles away from where we actually live! I don't know what to think, I am awaiting the neighbours return.
Also for those asking yes it is a listed building and the only way the builders could have accessed my garden was to climb through the now long gone window. I would post pics but don't want to completely out myself so I will try to do a diagram!!