If you believe that the word "man" means "adult human male" then yes, they are men.
If you believe that the word "man" means "adult human of either sex who identifies as a man" and the word "woman" means "adult human of either sex who identifies as a woman" then they are women (but fuck knows what that actually means since the definition is entirely circular).
By claiming it's transphobic to say that trans women are men, they're saying it's transphobic not to believe, or at least pretend to believe. They're saying it's transphobic to use the word "man" to mean "adult human male" (as it has been used for over 1000 years) and not someone with a gender identity. They've commandeered some of the most basic words in the English language and redefined them in accordance with a belief system that most people don't share.
Saying a trans woman is a man is basically modern day heresy. It's completely nuts.