It is absolutely exercise to perform salat - and repeated five times a day means that there is gentle, regular movement instead of remaining on a settee or slumped in a crappy office chair for eight hours a day. Add into it the emotional/spiritual aspects along with breathing and you've got something that is a physical and spiritual exercise.
The lovely doctor who lived next door for some years also said that she believed it was also healthier to spend some time squatting as they did to perform household tasks rather than standing up, as it kept the pelvic floor strong, the legs strong and the ankles flexible, thus improving balance compared to people who didn't do those things.
She told me all the major religions had at least at some point in their history a series of positions and movements related to prayer (she specified Islam, Buddhism, Judaism, Christianity, Hinduism) and whilst Christians sang more, that also counted as an exercise in the standing and breathing it required, even if kneeling wasn't as common these days - so in her opinion, the Creator (I loved her way she chose her words to try to be inclusive of me!) had given all people ways to keep their body as well as their soul healthy.
So no, I don't think this is a dodgy or rude question, as it's something I've been told as well.