flashingnose, you have hit upon an interesting problem. I think my ability to work out whose pants are whose shows that I am really able to do the most complex paid work too - rocket scientist? no problem!
I have three ds's and a dd. Ds1 and ds2 are only 18 months apart so, in many cases, could wear the same size pants, but I try and keep them segregated by at least a size. Ds2 wears ds1's pants when ds1 grows out of them or I buy him new ones. I have to think, a I am sorting the washing, all white pants = ds1, ds1's space pants = ds2, except Star Wars pants = ds1, all red pants = ds2, etc , etc. Dd is obviously wearing different pants, but now ds3 is starting with pants, I sometimes absentmindedly chuck some 2-3 pants in ds2's drawer by mistake!