I'm not sure how I should have behaved. Our local pool has a completely separate toddler pool in addition to a shallow training pool and a full large adult pool. I took 18 month old dd yesterday and we got in the toddler pool which has toys and small platforms etc. Initially we were the only ones in there and then two men in their forties got in and just seemed to be relaxing on the steps and chatting. Being quite reserved, I quietly asked a member of staff why there were adults in the baby pool without children. He responded that he wasn't sure; his colleague let them in and he thought one of then might have a disability.
I'm not sure how I should have felt about this but I felt uncomfortable. I didn't feel able to confront them myself so we just left.
I think maybe the staff should be absolutely sure if who they let in the toddler pool and why. The men certainly did not have any physical needs so could have gone in the training pool which is also warm and shallow if necessary. I'm not sure if this is terribly un-pc though...