It is far from upmarket, but still worth a visit e.g. Church Ope (walk down to a cove with ruins nearby); Portland Bill (lighthouse, views, tearoom) and there's a Sculpture Park too. Chesil Beach is nice for a walk. I probably wouldn't actually stay on Portland and Weymouth is quite touristy. There are loads of other lovely places nearby to stay instead e.g. Dorchester, Upwey, Bridport, Abbotsbury to name but a few.
Portland is a very beautiful place with intense tourism activity and well organized. Im sure the transport is also diversified and you will easily find a way to travel around. I hope this link will help you http://www.travelportland.com/ .