This isn't the 'gotcha' that you seem to think it is. Not wearing them is the default. Choosing to wear one is doing something deliberately different.
The thread is not about 'policing', it's about exploring the ramifications of this choice, both for the wearer and for everyone they encounter.
I think people need to take responsibility for their choices, including the parts their appearance that they choose.
Every time we choose to wear a certain item of clothing, or jewellery, or dye our hair a certain colour, we are making a choice (even if that choice is simply 'fuck it, I'm too tired to care what I look like').
To an extent, we are accepting that by making this choice, we are accepting the way others will react to us (not including outright abuse, obviously). For example, if i go to a work meeting with people who don't know me, and Im dressed in scruffy jeans because I can't be arsed to dress up, i probably won't get taken as seriously as if I'd dressed up and i could and should predict that. Similarly, if my big, long haired husband goes out in a heavy metal t shirt and leather jacket, people tend to avoid him on the bus, because he looks scary despite actually being quiet and kind! Again, if he is bothered by this, he can choose to present himself differently.
Choosing to wear a pronoun badge, like anything else, comes with its own set of signals to the world. These will be interpreted by others and shape their reaction to the badge wearer. As we've seen on this thread, symbols can mean different things to different people (are pronoun badges a demand? Do they signal political affiliation? People disagree about these things).
But crucially, we don't get to choose how others perceive/ react to us. Everyone needs to take responsibility for the reactions that may be elicited by their appearance choices.
It might not be right or fair, but people will judge other people on how they choose to present, and that especially includes things like badges which are a deliberate symbol, usually of something you believe in or want to happen.
And that is why wearing a badge is not an equivalent choice to not wearing one.