The trouble with taking a softly softly approach with a situation like this - where the other party has shown that they are prepared to be breathtakingly cheeky - is that you will come across to them as the weaker party, and they will just keep on being cheeky.
You're thinking of this like a non-cheeky person, and that's a mistake. As in, you'd never do something like this without permission, and if you did have a situation where you'd inadvertently pissed off a neighbour, you'd be eager to get it sorted asap. You're applying that thinking to them, and reckoning that as they'll soon come around and be full of apologies, it's better not to be heavy handed now.
WRONG. They aren't thinking like that. They're the kind of people who think 'oh, I'll fix it to their wall, bet they won't say anything.' They sound pushy, potentially troublesome. Do they seem sorry/worried that they've caused an issue? No! They're either sitting it out and waiting to see what you actually DO, to see if they can get away with it, or they really actually don't care at all, and when you do go to speak to them, they'll probably get arsey.
The best approach with this kind of person, and oddly enough the one most likely to nip trouble in the bud, is to be FIRM. Polite, smiley, friendly, but with a completely non-negotiable rottweiler core.
You need to go around there. Big smiles. Please can I come in to discuss the issue with the building damage to our wall? No? You're busy? I'd like to fix a time now to discuss it. We do need to get it sorted, we're very unhappy with the damage and need to get the structure removed urgently and the damage repaired before winter hits. Can you confirm that you can get the structure removed in the next two weeks, please. We've looked at the extent of damage that there's likely to be once the items hammered into our wall are removed, and our solicitor has advised that it should be a simple case of us approving your suggestion on how you make the damage good, and you obviously funding the repair. Hopefully it won't take too long or be expensive. Unfortunately you need to get it done quickly, as this is trespass.' No discussion,very polite, very very firm.
You waiting for them to take the initiative is putting YOU on the back foot.
What this will achieve is to let them know that you are lovely people but you won't stand for them taking the piss. So in future, they won't try and take the piss, because it's not worth their while. And you might well get on famously. Because they'll respect you. Unlike right now!