I'm sorry you felt that you had to leave the thread, OP. I can understand that you feel it may be 'inappropriate' for your neighbour to constantly engage your daughter in conversation, but for him, she probably cheers him up, and they can talk in an uncomplicated manner.
As several people have said on here, people with AS don't "get" hints and the suchlike. He would not necessarily have seen your building a fence as a way to 'block him', but as something you had to put up for your garden. Therefore to be able to see you and be neighbourly, he has had to put up steps.
For AS people, joining clubs and societies can be nerve-wracking. Timetables must be consulted for transport, routes must be known, time and location must be specific. Getting there can be hell as it is not a case of getting on a bus/train. My AS friend tells me that if she is headed out to a venue, she has to allow extra time to find quiet spaces to sit in. Walking down the pavement means noticing every crack, every cigarette butt, every piece of chewing gum, hearing every conversation from people walking by, noticing the glare of light from every shop window, car.... essentially it is a sensory overload. Add to this the stress of meeting new people and it becomes near impossible.
For non-AS people, it's really difficult to imagine the day-to-day because we filter all these out. We don't necessarily notice the plane flying over, the noise of the traffic passing, that someone walking past us has coughed.
The garden is a relatively safe place in comparison. It's non-threatening, and it's likely not to be as over-stimulating.
I agree that you have no obligation to do anything at all for your neighbour, but I thought that it may be helpful if you understood a little more about how he is likely to function on a day to day basis.
If you are able to politely and clearly explain about setting boundaries in terms of chatting whilst in the garden, and why things are not appropriate, then he should, hopefully understand and react positively to those suggestions.