Negril is lovely, totally different vibe to other parts of Jamaica. I didn't get any hassle there.
I second Rick's Café. Also if you like rum, it's worth a trip to Appleton Rum Distillery.
There was a nature reserve, Royal Palm, near Negril that was a break from the beach, lots of birds to see. It may be closed down now though.
Blue Hole is also meant to be worth a visit but I haven't been.
I'd look to see if you could get a driver for a few days, it's not usually very much, and they can take you to places they recommend. If you get a decent one (look at reviews on Tripadvisor etc) then they will also help you haggle at markets/stalks if you want them to. I had one in Runway Bay, he was great and knew loads of history.
The Juta tours are ok if you would prefer to be with other people. They are small minibus type vehicles.
You'll be fine, just don't do anything that you wouldn't do in any strange city/area. Don't wander off in the dark, go off alone with strangers etc. Negril still has the hippy aura from years ago, lots of Brits and Americans have moved there or spend a lot of time there and have for years. It's lush, enjoy!!