People who are into fashion do tend to notice these things. I’m not bothered if other people wear something that really doesn’t go with the rest of what they’re wearing - I don’t judge or turn my nose up at it or anything like that, but I generally would notice. Not a big deal though, obviously - no one is obligated to dress ‘well.’
However I personally absolutely won’t go out wearing odd looking outfits - it would just feel too wrong. So I do pay attention to what style jeans I’m wearing with what top, and I’d make sure to wear footwear that fits.