Honestly not sure if IABU or not here.
Diagrams and photos attached 
NDN wants to 'finish off' the fence which currently divides our houses.
The fence was like this when I bought my house 2 years ago and I haven't really thought much about it but now I want to build onto the back of our house and realised it appears like NDN has 'taken' some of my land!
What's the MN verdict? And what should I do next?
The house next door is housing association, if that makes any difference.