OK, three separate concepts are coming out here.
The bail-out of Lloyds and RBS involved the government subscribing for new shares in those banks. Actual cash changed hands, and the government holds its shares in Lloyds and RBS as assets. The government raised the cash to buy shares in Lloyds and RBS by borrowing (in fact, issuing gilts to investors), and it does have to pay interest, but not capital, on the debt.
The problem is that the current value of the Lloyds and RBS shares is £5bn less than the government paid, so if it sold them today it would make a stonking great loss and wouldn't be able to pay back all of the debt it took on to buy them. Its plan is to keep holding the shares until they're worth at least as much as the initial investment.
(As an aside, it's easy to value shares that're traded on a stock exchange like Llyods or RBS - becasue there's a quoted share price for one share at any given time.)
The Asset Protection Scheme (APS) is an insurance scheme designed to help banks with lots of bad loans. Bad debts on the books make banks reluctant to engage in further lending because their capital base is under threat. Once the government insures banks against further losses, the argument goes, they can get back to lending to firms and households to help the economy get back on its feet again. Banks agree with the Treasury how many assets and what type of assets they can insure. The Treasury charges a fee for the insurance it is providing.
I believe that the government is currently making a profit on the APS because it's collecting fees but hasn't yet had to pay anything out.
The deficit is usually defined as governmnet spending, plus interest payments on government debt, minus tax revenues. Note that it's interest on debt, rather than the full capital amount of the debt.
It's helpful to split the deficit into two parts - the cyclical deficit and the structural deficit. The cyclical deficit arises because of the business cycle (that is, the trend for upswings and then downturns in the economy). During the bad times governments tend to spend more than they take in tax revenues (as you say, partly because unemployment benefit claims go up, partly because tax revenues go down) - as Takver rightly says governments typically ride out cyclical deficits because when the good times roll then tax revenues bounce back and benefit claims fall as people get jobs.
A structural deficit arises when a government consistently spends more than it takes in over a longer period of time.
The UK does have a structural deficit, and in my view Gordon Brown has to take responsibility for that. As Chancellor he took the misguided view that his economic policies had put an end to the business cycle - he thought that the good times would last forever. That meant he felt free to keep spending, because there would always be more tax to collect tomorrow. He didn't put anything aside for a rainy day - and as a result Britain went into the recession already saddled with the biggest structural deficit in peacetime history. Not great.
It's the structural deficit that we're all working so hard to pay down. That and the money we actually borrowed (because failing to pay down the debt itself would put the wind up the financial markets so that they demand more in interest payments and might stop lending altogether).
In my view, Labour were right to bail out the banks and keep spending through the recession, to stimulate the economy and prevent total meltdown. The current goverment is right to cut spending, although I wouldn't cut it as hard or as fast. That said, Britain is now one of 12 nations rated "extreme risk" in the Fiscal Risk Index compiled by global analysts Maplecroft.
The UK was ranked 10th in the list of 163 countries because of its high public spending on health and pensions, massive borrowing and shrinking working population. Italy topped the international league table, followed by Belgium, France, Sweden, Germany, Hungary, Denmark and Austria. Japan came ninth, the only non-European country rated "extreme risk".