I mean you are right to a degree.
By way of comparison, on this FWR board on Mumsnet there have been threads celebrating the defacement of rainbow flags, and giving advice to others about how to achieve this defacement more successfully. When I have pointed out to moderators that there are posts giving advice on how to commit illegal acts like this the moderators have told me they will not remove them, as this is about freedom of speech/ expression.
So yes if course, as a gay woman I am likely to find this more psychologically intimidating than straight people as that's points to a culture developing that is happy to attack those symbols that represent me. And quasi authority figures who are happy to support that process.
Those rainbow flags indicate to me and my friends that I am welcome and do not need to feel intimidated, and when I see them defaced and destroyed we all feel more unsafe and worry about what is around the corner. It feels like Putin's ideas are making their way over here. And, as you say, it chips away at our sense of worth, enjoyment of and pride in our own sexuality. Straight people are unlikely to experience this same psychological attack when they see the defacement of rainbow flags.
However, in another sense it doesn't matter if you are gay or straight if 'lesbian cunt' is graffitied on your wall...you still have the same cleaning / wall repair to do regardless of your actual sexuality. And you can still feel intimidated that you have been targeted and might be subject to escalating abuse at your home because of the perception that you are gay.