Firstly, please have another word with your neighbour, and make it very clear to them, politely but firmly, that unless matters improve quickly, you'll be taking further action, as detailed below. Make it clear what you expect - dog to remain securely in their garden and to be well mannered on walks where he encounters other dogs. Depending on response, I'd also go as far as to follow up with a written note, keeping a copy.
I'd advise contacting your local Dog Warden asap. Ask them to have a polite word with the owners as soon as possible, both about the straying into your garden and the behaviour when on walks. This dog will end up with a very short life expectancy if it continues to "pin down" other dogs by the scruff of the neck, that's potentially the sort of thing that ends up with police involvement. Dog should also be neutered asap.
As Kladd has said, this fence needs mending urgently - I'm not a fence expert but usually with hedges it's perfectly possible to have a secure wire fence in front of the hedge without disturbing the planting. Yes, it's a pain that you have to do this, but the security of your garden and your own dog is more important.
It may be that your neighbours are finding the dog a bit of a handful - if you know of any good dog training classes in your area, then let them know about them. Lots of courses are just starting, now that it's September.
If the neighbours are social housing tenants, or private tenants, then have a word with landlord - it's potentially ASB, and for private tenants, they may not have consent to keep pets.
If you feel it necessary to take the nuclear option, then consider legal advice, you may be covered for this as part of your pet insurance. The DogLaw website is probably a good first port of call for specialist legal advice on dog issues, but you may also find that your pet insurance has a legal advice helpline.
The situation sounds as though it needs to be resolved urgently as otherwise the dog is the one who will be paying the price for his owners lack of action.