Sorry, I didn't want to just leave the thread without responding, but I think that if your settings are set to English as the language and UK as the store country and you're still only getting foreign publications, if I were you I would email customer service. They should be able to sort the problem. Sorry I couldn't be any more help. Someone else may come along with better advice. Good luck!
Ah, sorry! For that, you need to log into your Amazon account and under your account settings, go to the Manage Your Kindle page. In the settings menu on the left, there should be a link to change country. That should sort it for you.