I'd second lots of the things mentioned above but things that I didn't spot when I skimmed through the list include:
Microwaveable self sterilise bottles from mothercare - these have been a godsend when I needed a bottle in a hurry and didn't want to wait for an electric steam steriliser (on the box they say it takes 8 minutes or whatever, it's ony when you look inside that you discover you have to wait another 7 minutes for it to finish sterilising in the steam and cool down period). The bottles were also great as it meant you could travel with them and not lug a steriliser around as most people have microwaves that you can borrow.
Join the Boots parent/baby club and buy the pack of nappies to get the free changing bag. Not the world's most glamorous but it holds a pack of nappies, wipes, cream, nappy bags, change of clothes and spare muslins - I tended to leave mine ready packed and it was ready whenever I needed to go somewhere or to have a second set of changing things downstairs etc.
pampers disposable changing mats - I've only used one pack so far (ds is 15m) but very handy to have when changing out and want something you know is clean - or when staying with friends etc. Most of the time they stay pretty clean and you can reuse over and over again but if they do get dirty you can just throw away without worrying.
I had a pack of muslins and a pack of terry nappies to use at for mopping up, protecting, quick blanket, tiny towel, covering the changing mat to start with (when it was cold and ds shrieked on contact) etc etc. both are useful. in fact only thing I haven't used terry nappies for are as nappies .
small blanket made from brushed cotton (got from the baby catalogue) - had a may baby and it was just right as a thin wrap/sheet/blanket, soft towel, soft layer in the pram - now have several and always leave one in the car and one on the pram.
nuk teats and dummies - I know, dummies not good but I had very ill dh who needed lots of sleep and was in and out of hospital, so had to be pragmatic about either making him more ill because baby screaming or let baby have dummy and not scream. the nuk teats were very good for reducing colic and wind but difficult to get hold of - used baby-supplies.co.uk who were very good
also to remember that if you do end up having to use formula at all babies can react differently to different ones - everybody I had known who'd used formula had used sma and so that's the one that I picked up when I had to give up bf but it gave ds bad constipation - changed to aptimil and no more problems.
things I haven't used:
knitted cardigans from mil
avent bottles (and avent stuff in general) - before I had ds the implication was that I had to get avent stuff as it was the best but having now had a chance to look around there is plenty of stuff that is pretty good that isn't avent. not that there is anything intrinsically wrong with their stuff, just that in the shops that is what I always ended up being steered to.