YANBU its incredibly bad for the environment. Produces tons and tons of micro plastics that come off in the washing machine and end up in the oceans.
Then they must take decades to breakdown when they eventually end up in landfill (yeah, yeah via the charity shop worn to death, back to the charity rag big, used and binned as an oiled rag).
But at the same time man made, oil based fabrics are used in a lot of technical and outdoor clothing. So have a real advantage over natural fabrics. However I believe for everyday clothing we should be using natural fabrics.
Oh and womens clothes are worse for sneeking in plastics than men's.