I would definitely chuck it back over their fence (the turds, obviously, not the cat).
I would also add that in my garden, it's still an issue even when the neighbours' cats bury their shit, because a) it regularly means that new plants get damaged by them scuffing up the soil, and b) if I'm planting something or weeding, it's really not that pleasant to be digging a hole with a trowel and encountering a concealed, stinking cat turd.
FWIW, I don't really buy the whole thing of 'Oh, it's easy to deter them, just get a super soaker or a sonic cat repeller etc' that cat owners always seem to parrot. First of all, why am I having to spend money on those things because your cat is in my garden? Secondly, I've never found sonic cat repellers to work at all - I've literally watched a cat sniffing one and sitting down by the one in our old garden! And they don't have much of a range. Super soakers are fine if you're prepared to sit there for ages waiting for the cat to appear, and then to take aim and hit it, and repeat that process numerous times, which isn't ideal if a) the cat mostly shits in the night when you're in bed, b) you have other stuff to be getting on with, like a normal life and a job. Any chemical cat repellent might work for five minutes, but it doesn't last because it rains.
I actually like cats - I like all animals. If I see a cat pottering around in the street or at a friend's house, I'm the first to want to stroke it and make a fuss of it. I totally understand that cats who shit in my garden are just being cats and I don't think it's particularly kind to keep cats inside all the time. What annoys me is that some cat owners are just utterly blind to the fact that their pets are a nuisance to others. If I had my own cat, and therefore enjoyed all the benefits of owning a pet, I wouldn't mind putting up with the unpleasant side like clearing up honking shits. But I absolutely do not want to have to put up with the bad side of cat ownership when I don't own a fucking cat!