Seems likely the new owners next door stupidly assumed it was a shared garden and didn't realise it is your private property
Even if this was the case, they would need permission from the freeholder and the management company, and they should let the neighbours know
Let alone the planning permission AND the listed buildings consent that is required
You can't explain this away as a bit of an oversight! And the builders first question should be 'are all the correct permissions in place for the work?' which should have at least triggered the question 'what permissions?' if it was genuine ignorance