Unfortunately, (although it is a very nice picture) that doesn't really help too much.
The main issue is, what work are they doing on their property that requires them to put scaffolding on your property?
If they are building an extension, for example, then, yes, you can tell them to get lost.
However, if they are doing any "basic preservation works", that is to say "the maintenance, repair or renewal of any part of a building or other structure" then they can apply to the court for an order granting them access to your land in order to allow them to do this.
What counts as "maintenance, repair or renewal" seems to be quite broadly interpreted.
If they need to repair their roof, or replace render that has come away from the wall then that would almost certainly be allowed. In contrast, if they just wanted to repaint that white wall to pink or some other colour because of their preference for a different colour, then a purely cosmetic change would not be allowed.
But it is not just a case of them being allowed on to your property in this situation. You would also expect compensation as well.
There was a case in the High Court a couple of years ago (Prime London Holdings 11 Ltd v. Thurloe Lodge Ltd [2022] EWHC 303 (Ch) ). This case involved two houses in a really posh part of London (literally opposite the V&A).
This case, which no doubt cost a load in lawyers fees, was all about repairing some render on the wall of one of the houses.
The court held that they should be allowed access to do this but that the other property should receive compensation under various different headings.
Most of these were more relevant to property developers, but there was one head of compensatable loss that was relevant to most ordinary people who are contemplating having neighbour's scaffolding in their garden, and this was:
"Substantial loss of privacy or other substantial inconvenience"
At paras 158 to 160 the court said:
[158] ... They will also be inconvenienced by the presence of the Claimant's workers on the site and the Scaffolding and there would be a period of weeks during which noisy building works will be held on the property.
[159] Mr Adams-Cairns has found that there would be a number of items which would be relevant to inconvenience of any resident of Thurloe Lodge, including the access of workers and materials on the property; what he considered to be limited noise from the works; an unsightly structure in the form of scaffolding in the Passageway; a very small reduction in light to three windows; concerns about security; and concerns about damage to Thurloe Lodge by the works.
[160] I think it is reasonable under these circumstances to regard both the loss of privacy and inconvenience that would necessarily be involved as being sufficiently substantial to warrant compensation. During the entire period of the Claimant's occupation of the site, the residents would need to suffer having workmen in their garden with the noise and loss of privacy that that involves.
(emphasis added)
The court went on to say at [161] that since this was an expensive property that "it is possible that a high price could be put on this by an occupant"
So, is the work being done by your neighbours an extension or is it "maintenance, repair or renewal"?
There is a whole world of difference in your response depending on what the answer to that question is.
By the way, the case I referenced above is an interesting read (although it is 264 paragraphs).
Prime London Holdings 11 Ltd v Thurloe Lodge Ltd [2022] EWHC 303 (Ch)
It does cover a lot of the nuance around the Access to Neighbouring Land Act 1992, as this was the first time that a claim under the Act had ever gone to the High Court.
I think the judge's final paragraph sums things up very well:
[264] To the extent that they were unable to agree because of differing views on how ANLA is to be applied, I hope that the detailed analysis of the Act that has been provided in this judgment will assist future potential litigants in resolving their differences without going to court and that it will be another 30 years or more before the Act needs to be considered again in the High Court.