You poor thing - what a horrible man. Some people have no empathy whatsoever and he is playing on the fact you are on your own.
Do you have anyone (friends/family) who could come round and look at the fence and what could be done as far as blocking it up.
The trouble is - it's like a game to him probably. If you fight fire with fire, he could allege absolutely anything. You can't put mousetraps or acid or anything like that down, because next minute you are in court for assault on him. I'm sorry, this probably isn't helping but I am just being realistic and I know someone who was harrassed out of a home. She was a young girl with her first flat, I won't bore anyone with the story, but needless to say, the man is still there and she isn't.
I think you have two choices. If you go to the Police which is the right thing to do, I'm afraid it does have to be declared on legal forms when you come to the sell the house. So, bottom line is, are you strong enough to battle it out, stay where you are, go the Police route, OR before it escalates you could put your house on the market and quietly move without, at the moment, declaring anything because you haven't confronted him or taken it further. Of course, that is not fair, but I'm being realistic.
I'm so so sorry for you, nightmare neighbours like him should be made to live on a rock in the middle of the ocean.
Do you have any support for how you are feeling, this is such a nightmare situation and I can imagine how it is playing on your mind. Behaviour like this becomes all consuming and it rules your life. I speak from experience here.