I quite like the idea of only drying off with a handtowel if it weren't for my hair.
I sit on the jacuzzi seat in the bath after the shower as it is for a couple of minutes, and think a handtowel would take off the rest just fine (I like to airdry on the bed in summer).
But then I have taken the environmental step recently of stopping using soap or shower gel, so I'll probably get burned as a witch.
(For those curious, my routine is shampoo and conditioner in the shower, with face scrub 2-3 times a week. I use hand soap still, and deodorant, but in the shower just give myself a good scrub with a flannel soapless. No smell, no dirt, and no suds down the drain.)