Sounds like you need Charmouth, not sure about climbing (but probably is- that sort of thing popular there), however loads of good guided cliff walks / fossil hunts etc on the beach, a nice quiet place and a decent campsite-
we like Wood Farm campsite, its campsite perfection in our opinion , really love it.
HTH