Read https://www.gov.uk/how-to-annul-marriage and then tell me how any of those grounds for voiding a valid but voidable marriage are comparable to the reasons people cite for divorce. These grounds are all things that fundamentally alter the nature of the marriage.
The conversion, in the eyes of the law, of an opposite-sex marriage into a same-sex one or vice-versa fundamentally alters the very nature of the marriage. It goes beyond even abuse and infidelity in this regard because, unlike abuse or cheating, it misrepresents the non-transitioning spouse as someone they are not (same-sex attracted instead of opposite-sex attracted) and misrepresents the marriage that the non-transitioning spouse entered into.
No one has the right to misrepresent another person's sexual orientation unilaterally and with legal force in this way, and that's why it's important for the law to recognise the continued need for non-transitioning spouses to be able to annul a marriage. It's not OK for the non-transitioning spouse to have to pretend to have married and divorced a spouse of the opposite sex to who they actually chose and have to tick that "divorced" box on forms for the rest of their life because of the transitioning spouse's life decisions.