"when a country is in debt (ie Greece springs to mind atm) - who is it in debt to"
Basically governments 'borrow' money from banks and other financial institutions, like those who will one day pay our pensions. This is done as people above have said via treasury bonds but in essence its borrowing.
The problem is that this borrowing has got out of hand. If Greece was a person, they'd have been on 'bank of mum & dad' a long time ago and bailiffs would be removing their TV. Greece has been the European equivalent of the spoilt teenager who's gone out spending money on their credit card on whatever crap they could do. To make matters worse they've not bothered with things like tax collection and dodging tax is a national pasttime.
But to compound the problem and this is where the scary part is. The bank lending Greece the money probably didn't have the money. It borrowed some of it from another bank. Who probably didn't have the money, so they borrowed it from another bank too. And so on.
So the biggest fear right now is not whether or not Greece will fall. It's what happens to the house of cards once they do. Directly the UK doesn't have a massive exposure to Greek debt. Neither do the US. But our institutions have lent a lot of money to a lot of banks who in turn may have lent to Greece.
Now imagine it's not just Greece. Greece is just the worst of the errant children. Portugal, Italy, Ireland, Spain and the UK have all been doing the same. 'Luckily', Greece is a small country with a relatively small economy which means the other EU nations can bail out if necessary. If Italy goes the same way, there simply isn't enough money in the pot.
And the only thing which has saved the UK is the fact we never joined the Euro so we can print our own money, we can set our own interest rates and we can devalue. Something none of the other countries can.