The problem with this, is that even if your horse is "behaving safely", you can still fall off. For example your horse could slip or trip or you could simply loose balance. It is not just bad behaviour that causes falls.
I've also heard a story about someone who wasn't wearing a helmet and knocked themself out when they hit their head on a tree branch out hacking.
I would never get on a horse without a helmet.