It's not just a Caribbean thing. Inside and outside clothes are a feature in a lot of cultures.
I can sometimes change from inside to outside and back again quite a few times in a day. Takes less than a minute. No biggie. I also wash my hands after coming in to the house. Always have done.
It will, therefore, not come as a surprise that we have no outside shoes inside rule.