Not sure about trousers as I don't wear them for work, but I wear work dresses and skirts many times before washing - I have loads of layers under them though (underwear, tights, full slip thing) and I suppose dresses have a bit less, erm, 'contact' with intimate areas.
I think we wash things too much though. Apart from underwear, vest tops etc, lots of 'outer' clothes that have been worn a couple of times just need airing, not washing, imo. Probably different if you sweat easily.