Going to get a pair for DH as they are the perfect style, but was wondering what the sizing is like? I know for some slippers you need to go down a size but they don't seem to be particularly padded inside in so much as it would make that much of a difference?