Sadly this situation has shown that there are quite a number of people who like to report their neighbours or make nasty comments on social media without knowing the full facts of a situation. Mask wearing would give them a further opportunity.
Look, you could use that kind of logic to argue against having any rules, ever.
"You can't have social distancing because someone might get the wrong end of the stick and report someone who is actually a nurse in plain clothes!!!"
"You can't ask people to stay in the house when they develop a suspicious cough, because someone might get the wrong end of the stick and report someone who is coughing for a legitimate reason, like long-term asthma!!!"
Well.... yes, they might do, and there were a few cases of this actually happening. But we still have to have the social distancing rules and no-going-out-when-you-are-showing COVID19-symptoms rules, for goodness' sake. They are important rules that save lives.
When the government issues instructions to wear masks (which they will do, believe me, at some point), this needs to be accompanied by clear instructions that there are a few people who cannot wear them for genuine reasons.
People are more sensible than you think. We did have some thread a while back, where a poster was moaning about "OMG there was this coughing man in the supermarket, how dare he go out??" Her post was followed by about 3,528 replies all pointing out that "actually, some people have long-term coughs that have nothing to do with COVID19, they do have the right to go out, you know. Don't jump to conclusions." Most people are sensible and understand the situation.