It’s about time there were controls over cat ownership with people like you around. Cats in gardens, ok, you can’t really stop them. But if someone hasn’t chosen to have a cat it’s horrendously unfair for a neighbour who knows their cat intrudes on other people’s property to not do something to contain it.
I agree. Why would you really want to be known by all of your neighbours as that annoying house whose cat keeps trying to get in? It's horrible when you feel you have to restrict your own family's life and keep your doors and windows closed in hot weather, as if you don't, some cat will keep trying to get in.
You may have cat allergies, crawling babies, smaller pets that you don't want upset/killed, food that you don't want stolen, upholstered furniture that you don't want trashed, valuable and/or delicate ornaments... so many reasons why people don't want strange animals intruding in their homes, as well as that they may just want to be left in peace.
Just because you love your cat and find it endearing, other people have a perfect right not to. This doesn't make them nasty 'cat haters' - just normal people who want to live and let live and enjoy the same rights to a peaceful home life as you do.