I think that the use of phobia as a descriptor for hatred of homosexuals, immigrants and trans people isn't really the best in general on a lingustic level as it doesn't convey the sense of prejudice, just a fear. A phobia on the other hand conveys an irrationality of thought, rather than pure ignorance. Plus geniune phobias are often anxiety disorders - as in mental health issues - so we have a situation where hatred and ignorance are conflated directly with anxiety disorders in a manner which you could argue is unhelpful to the stigma of mental health.
Phobias, as I say based on my own experience, also convey this concept of irrationality of thought and this allows concerns based on life experience to be more easily dismissed, rather than explored to get to the heart of that reality of experience which might be genuinely traumatic and therefore completely understandable and actually very rational rather than purely irrational.
Homophobia and xenophobia are problematic too, but the issue of the role of women with regard to transphobia is particularly difficult. The problem with transphobia as a term, is that you have this intersection of the history of women dismissed as hysterical, the idea of irrationality of thought and pure ignorance and hatred all being effectively conflated within the bare lingustics.
It might seem like picking at words, but this does matter unfortunately. On several levels.
It allows for the manipulation of the term in political ways and doesn't help the definition of the term by making the definition and consensus on what we mean by transphobia more difficult.
I hope that makes sense.