People may not be trying to hide fat, you know.
I have suffered from severe acne since a child. Consequently my back and shoulders are plastered in deep, unsightly scars. And depending on the time of the month, lovely angry zits, too (and I'm 33, FFS!).
On the few occasions I tried to swim in my youth, I would ALWAYS wear a t-shirt. Needless to say I never learnt to swim properly, and am only just this month forcing myself to conquer my fears and get learn to swim!
I now have a tight lycra top which goes over my costume to cover my back. I would never, EVER bare my back in public.
I would love to think that people really didn't give a shit, but the truth is, some people are hideously unkind. I was in the pool last week and a guy with sores over his legs got in the pool (exczema or psoriasis I assume) and I overheard a group of adults sniggering, saying 'if "that" gets in, I'm getting out'. One of them left the pool and the rest moved away from the poor bloke.
Being 'fat' is far more acceptable in a pool imo. At least people don't think you have leprosy!