A fundamental moral issue is that of contempt.
There is a deeply unequal relatonship between males and females due to their different reproductive roles. Deep in the male psyche, I believe there lies a very basic sense for many men that sex is a zero sum game and if he 'wins' by having sex with a woman, she has 'lost'. Lost her sexual virtue and therefore any right to be respected.
Common throughout history is men's idea that good girls don't sleep around and you marry a virgin who is due respect simply because she has not allowed another man to sully her virtue. By this thinking, a woman who simply lets a man sleep with her no strings attached is morally lacking and deserving of less respect. Clearly this has its roots in biology and men's desire to control the female body so that his reproductive success is maximised. The male wants to shag about as much as possible, while the female must only shag him not anybody else.
A woman you can morally corrupt by paying her money to sleep with you is even worse! She 'threw the game' and agreed to be defeated and sell her virtue for grubby money not even passion. (Somewhat like a sportsperson who agrees to throw a game for cash - nobody respects them for it, even if they provide a wanted service by those who pay them)
There is no other 'job' in which a woman is deemed by men to be so morally contemptible for what she does. The irony always being that he is desperate for her to do it. Always the double standard.
This the historical context in which prostitution occurs. The moral aspect of sexual 'work' is to do with how savagely men judge women morally for having sex and especially for being 'buyable' sexually. Sex and sexual control of others is deeply rooted in evolution and biology in a way that jobs unrelated to sex are simply not.
Then there is of course the exploitation aspect - the appalling harm to benefit ratio of the job - but others have written above on that.