My best buys are Tuppence and Crumble Star wraps...they are great for popping on over just a nappy around the house when baby is newborn and you are doing lots of skin to skin and frequent nappy changes. They are easier to get on and off baby than a cardigan or pramsuit (no wrestling little arms and legs in to sleeves), don't hold the arms out straight, and can be opened up without disturbing baby if you go inside and it is warmer. They can be worn in slings and car seats. We have 2 in the small size (so we can wear one/ wash one) and DD has just started wearing the medium size of which I have bought another 2 (all ebay bargains).
Buy a Thermos mug (metal lined so hot drinks don't taste of plastic) with a sippy lid. It beats microwaved tea any day!!
Other than that...
You need somewhere for baby to sleep, we have an Arms Reach Original Co sleeper, which can be a freestanding bassinet, a co sleeper bassinet, a cot, a travel cot and a play pen, and will take DD for birth to her first bed (which will be a single mattress on the floor rather than a toddler bed).
Something for them to sleep in, I recommend having a swaddle wrap of some sort (we used summer infant swaddleme) on hand in case you need it, DD wore one from the third night until she was about a month old, and is still swaddled now at 11 weeks for some daytime naps (if she has got overstimulated and is fighting going to sleep).
Something to transport them out and about. Car seat (if you have a car, obviously) plus a sling or a pushchair. I have a Moby wrap (loads of people really rate the wrap slings...my DD is not one of them!!!) and an Ergo Carrier (which is much more comfier and supportive, but a bit of a faff with the newborn insert until DD is a bit bigger). I also have a battered travel system, it is handy occasionally but not essential. I use the carseat in it on the rare occasions I go into town, and the carrycot in the garden for daytime napping (and occasionally in the living room).
Nappies, changing bag can be any medium sized bag, I use a 15 litre back pack which is just big enough (we are using cloth so everything is a bit bulkier), a waterproof changing mat, a travel one will do, as babies do tend to pee and poop mid change.
Stuff for feeding, DD is EBF, so from that point of view...Muslins lots of muslins!!! Lasinoh and Breast pads (washable ones for ongoing use, but some disposables at the start as you need loads for a short time). If you are buying breastfeeding tops, get ones with no fastenings, just layers/flaps of fabric...trying to sort out baby and put my boob away without flashing is challenging enough without having to fasten anything more than the bra up.
Baby clothes...unless you have good reason to think you will have a small baby go with up to 1 month clothes. 7 sleepsuits, 7 vests, 2 Tuppence and crumble star wraps, and that is all you need to start. Same in 0-3 months, plus any outfits you fancy, but personally I stick DD in jogging bottoms and short or long sleeve t shirts or rompers, the dresses I have for her barely get used. For sleepsuits ones with built in scratch mitts are good, as are zip ones...I am yet to find the holy grail of sleepsuits...zip with built in scratch mitts!! My top tip, so that you don't go through loads of clothes with poo explosions is to put a reusable nappy wrap over the disposable nappy (I still use disposables at night at the moment, and will use them for camping trips/ holidays), I have only had one small poo leak in 11 weeks!
We have a very bouncy dog...so whilst the following aren't essential they have been really useful...Summer infant pop n play play pen and bouncy chair (placed up high).
Everything except the cot mattress and a replacement mattress for the carrycot was second hand, mostly ebay or passed on from friends/ family. It is very easy to spend a lot of money on baby stuff, but equally easy to spend very little.
Hope that helps.