Interestingly, the Trolley Problem was invented in its modern form to deal with the issue of abortion, so that's very pertinent.
"Philippa Foot developed the trolley problem in her 1967 paper "The Problem of Abortion and the Doctrine of Double Effect," in which she explored Catholic opposition to abortion. She cited examples in which a pregnant woman's life is uncertain; actions that indirectly kill the fetus are permitted, while those that explicitly kill it are not. For example, a hysterectomy performed to save the woman kills a fetus, but the fetal death is not the intended result, so it is permitted. However, crushing the skull of a fetus to remove it to save the endangered woman intentionally kills the fetus, and therefore is not acceptable, even though the woman will die."
Although these days, the Trolley Problem is generally used to explore whether one believes in Utilitarianism or Deontology.
Personally, I lean towards utilitarianism, and so would advocate that killing one person to save five is the more 'correct' action, although obviously still horrific. It is the least bad option, if one cannot avoid the situation altogether.
Utilitarianism is, I think, the more appropriate position in this kind of situation, as if you inflate the numbers - would you kill one person to save a million - then it becomes very easy to see what action is the most morally beneficial.
Tangentially, the Trolley Problem is also sometimes used to examine whether distancing themselves from an act makes people more amenable to being active vs passive (eg. push vs switch, vs nothing at all), and also what or who people prioritise.
In this case, it seems as though if five women were on the tracks, then many of the people on this thread wouldn't push the fat man, but if the fat man were on the tracks, the same people might just push the five women.