The difference is that someone's sexuality is defined by their subjectively experienced feelings, whereas their sex is defined by reference to objective criteria.
If a dead body is found, whether the person died this morning or 5000 years ago, we will be able to tell whether they were a man or a woman. We will not be able to tell, just from examining them, whether they were attracted to men, or women, or both.
A person's sexuality is entirely defined by how they feel inside. They feel attracted to members of the opposite sex, or the same sex, or both. They are the only person who can tell us how they feel inside, so we have to take their word for it. And if, for whatever reason, they lie about it (e.g. because they are gay but not yet out), it usually has very little impact on others. (The only exception I can think of is where someone knows they are gay and pretends they are straight to the point of marrying and having children, only to come out years later, which is likely to be traumatic for their family members who feel they have been living a lie.)
Not so with sex. We observe and record a baby's sex at birth, because we can see it, and because we recognise that it is important.
Other people's sexualities have nothing to do with us. Another woman being a lesbian has no impact on me. She isn't changing what it means to be a woman, because women are not defined by their attraction to men. (As explained above, we know whether someone is a woman or not even if we do not know whether she is attracted to women, men, both or neither.) If she is married to another woman, this does not change the meaning of my marriage to a man.
But if we accept that a person who identifies as a woman is actually a woman, this changes the meaning of the word "woman". Previously understood to be a word which meant "female people", it becomes a word which includes some male people and excludes some female people. On what basis, it is not clear. Because most of the rest of us do not understand what trans people are identifying with. We do not understand what they think a woman or a man is, or why they believe they are one. If "women" becomes an identity based category, am I a woman or not? I don't know. What I do know is that I am a member of the childbearing sex, and it is important to me to have a word for that, as well as rights, spaces and sports based on this categorisation.
The impact of this on lesbians and gay men is even more severe. Because their sexuality, which is based entirely on how they feel, nonetheless relies on being able to define people according to their sex. The word "lesbian" means a woman who is exclusively sexually attracted to other women. If the definition of the word "woman" changes, so does the definition of the word "lesbian". And if people who were born male or who have sexual relationships with people who were born male are now self defining as lesbians, the original lesbians no longer have a word for what they are.
In short, the difference is boundaries.
If what you are doing is not trampling over other people's boundaries, we should all live and let live. But if it is, there is a problem.