Even if they are installing a juliet balcony / long window tha tcan't be opened into your garden, you need to go around NOW and tell the builders that they are not permitted to be in your garden and you will report them for trespass if they do not leave immediately.
Once they're gone, having secured their mess from the inside and removed every trace of rubble and dust from your garden, leave a photocopy of your land registry plan showing ownership of the garden for the neighbour and requiring (not asking or suggesting) that they immediately cease and desist from any activity on your land.
THEN - phone your local planning enforcement people - ours are amazing and shoudl be incredibly helpful.