If I say I can't do something because of anxiety it has zero to do with self esteem, confidence or laziness. I have missed quite a few weddings, events and even struggled with attending work at times because of anxiety, but you wouldn't know it if you worked with me, and as a result I have been disbelieved by some former colleagues and quaintances.
I am an otherwise confident and outgoing person who suffers from cptsd and sometimes things are too much for me, in particular when I've been working through difficult things in therapy.
When I'm triggered and struggling with anxiety, which sometimes comes on like an "attack" then there's not much I can do about it until it passes. (This is not the same as a panic attack.) That can be days, weeks, even months.
I come across as strong, confident and very capable, but there are some things outside of my control and uncontrollable anxiety is one of them. Meds never worked for me and the side effects were horrible.
I won't go into gory detail but sorry, YABU and have no idea what clinical anxiety really is.
Do some people use it as an excuse? Quite possibly, but that doesn't mean your assertion is in any way correct.