Fabric is expensive to buy.
It can be, but as a longtime sewer there are ways and means. I sometimes splash out on something beautiful, and I love a browse at my favourite online shops like Merchant & Mills and Cloth House, but you can also get quite a lot of cheaper bolt end, vintage and remnant fabrics at ebay (eg a vintage silk or cotton sari could be £10-20 and gives you 5m of fabric, and there's a huge range of prints), also etsy, and a lot of online fabric shops have remaindered stock that's cheaper - fabworks is one.
I will also buy old clothes, bedding or curtains on ebay to get a fabric I love, and re-use it. For example you could spend £25 a metre on new linen fabric, or buy an old linen duvet cover off ebay for £20 and get several yards. Sometimes I dye it. (obviously it needs to be the right weight etc so a duvet fabric would work for a floaty dress, less so for sturdy trousers.)
For long-lasting good quality clothes, I like Cos (though I see some PPs have had problems), Toast, Hush (not their knitwear or polyester, but viscose dresses and jersey items have lasted years and years), Arket, American Vintage, and Isabel Marant when feeling super flush. Also shop on ebay a lot for these - it's not only cheaper, but older stuff tends to be better quality and less likely to be polyester.
The best-quality, longest lasting clothes of all IME are surprisingly Superdry. Indestructible, incredibly well made. Not for everyone of course but if you look carefully they often have a few nice things that are not logo-plastered.