The contract I'm using was 7.50/month for 5 GB of traffic per month (that's about 150 MB a day on average, given I'm using this from home most days - no landline at all).
They have started offering 15 GB for 15 pounds (which is better value) but there's sometimes a 'sting in the tail' if you exceed the amount allowed in a month. In the case of Three they charge 10p per MB which tots up to 100+ pounds for an extra GB (!) However, O2 may charge double that for 1 GB (200 pounds) if you go over their limit.
T-Mobile seems to just slow the connection down, so long as you don't go over the limit too often. Don't know if they wiykd cancel user accounts, or charge big penalty if it happened every month.
On PAYG there are various costs, not least the cost of the dongle...
Sometimes you might see a 'get started' package which costs say 100 pounds, where there's the dongle (mine is on the end of a 2 metre USB cable so I can see at a glance whether it is active by the blue LED inside - it would be hopeless if it was plugged in the back of my mini-tower PC under the desk!)
Some of the dongles have been sold at 49.99 or 39.99. Three had them at 19.99 for a while (in Maplins I saw the same price, debated on getting one as a spare). However they are also offering a dongle for free (will try to find the website - it was mentioned on MoneySaving Expert, I think).
However, at 15 pounds for 1 GB of data, you can see that's 15 times the cost compared with their contract deal. Some firms offer 3 GB, 5 GB or more. All are differently priced.
For low users the Vodafone PAYG deal may be best. At £15/GB it seems expensive, but you r credit doesn't expire, so if you only use 100 MB a month then it would last 10 months. OTOH if you use 1GB really quickly then it could prove expensive.
Agreed, that Voda deal is probably best for those with low usage needs - perhaps just downloading mail from a POP mailbox.
I think one thing to beware of would be any applications which 'call home' to check for updates (and, of course, MS 'Windows update' which they encourage users to leave 'on' - could gobble quite a lot if there's a big Service Pack that downloads).