If your OH said this to you and added an "Only joking,haha" before you went out what would you think?
Sometimes the phrase varies to "Is that what you're wearing?" etc.
My clothes are not in any way out of the ordinary so I'm a bit confused as to why he does this.