On a typical Sunday, I'll turn up in church with the children and I in running gear... it's because we go to the junior parkrun first and just have time to complete it then attend church. No time to go home and change first. I think it would be a pretty lousy Christian who would rather I didn't volunteer most Sunday mornings for the benefit of the community over my sartorial choices based on starting the day in an exposed field. I'm sure God will forgive me 
I do enjoy dressing up but find very little opportunity to do it, not that I have much social life at present anyway. On a rare night out, I feel a bit lost about what to wear as the standard seems to be one extreme or the other. I'm happy to actually go to the effort of actually putting make-up including some tinted moisturiser on at night, but I'm not into the heavily made-up slug eyebrows and war paint contouring at all, and I physically can't teeter around in sky scraper heels, so that look is out. The rest of town is dressed very casually in clothes that would have had them barred entry in most but the roughest venues 15-20 years ago. DM regularly bemoans that I don't dress up, but there simply isn't the reason to and I don't go to places like the theatre or nicer restaurants to do it, and I'm past the wedding stage with everyone now happily married or divorced 
In the daytime it's practicality that drives me. I do wear running clothes a lot, quality gear to actually run in rather than just a casual tracksuity look. Mainly because on an average day it's comfortable and practical for the school run on foot, house work etc before I go for my run. I'll probably finish the day in jeans/ dungarees/ harem pants because it's then easy to change my top for Guiding/ Scouting uniform. I used to wear skirts and dresses much more until I had two lively sons and hit the toddler years of dashing off on sprints at random after them. My smarter clothes hang dusting away in the wardrobe waiting for special occasions and possibly a return to work.
So why is society more casual?
Because we can.
Because it's comfortable and practical.
People don't feel the need to prove a social point through clothing in the same way. Status symbols are cars and mobile phones not necessarily your level of grooming. Being smart can often be the sign of a low level conformist job now, better paid intellectual work had shucked the ties and jackets. DH only meets a client in a suit once, then they progressively dress down to shirt and trousers. Creative industries reflect that in individual clothing.
Most high street fashion is cheaper, stretchy fabrics unless you spend major £££.
People own a larger range of cheaper one-size fits all type stuff or rather one cut. The clothing simply isn't worth customising for better fit (particularly for fabric strength and quality). Loss of personal skills/ time to customise and make clothing.
Generally speaking, if an item of clothing is designed and cut to size 10, at the smaller sizes, the range of fit for different body shapes is more forgiving than when the garment is upsized into plus sizes. It is a greater challenge for a size 18+ woman to pick up a high street item and find it is well cut for her individual body shape. I am not criticising individuals at all, and the best dressed plus size women have had to put more effort into finding the right branding and cut to flatter them. A size 10 can get away with much more laziness.
Because lives are busy spending more time rushing around from one place to another outside the house, not dressed specifically to one activity. Fabrics don't have to be tough enough to withstand the wear and tear of manual work; trades people wear specific items.
Central heating means we don't want to be wearing multiple layers of under garments and protecting our garments with house coats.
I'll stick with the 21st century thanks!