The term trans in your post is an indicator that those people who claim the term for themselves have a philosophical belief about how they identify themselves.
Can you find someone saying that 'everyone can tell' please? I think that it is not a believable statement except that in each person with a transgender identity's case, there will be likely multiple people at some time with various degrees of interaction that will be able to correctly sex that person.
Hence, no person with a transgender identity will pass 100% of the time to 100% of people in the world.
Do you believe that someone with a transgender identity will 100% pass, 100% of the time?
And it is irrelevant that you don't believe that 'trans women or trans men are female or male' because they remain the sex that they were born, whether you or they believe this or not. And expecting any other person to support your personal belief in this in language, law, and policy is unreasonable.