I had a panic at around 24 weeks that I had no idea what I needed and started making a list, but didn't feel ready to buy anything at that point as I was still worried. I'm 32 weeks now and have bought most things I think, apart from a playmat, and toys etc. I split my list into different sections, started getting stuff for "sleep" to start with.
I haven't bought any bottles yet, as hoping to breastfeed to start with, but have made a list of what I might need if I end up expressing or formula feeding.
I've bought things bit by bit, and have shopped around to try to save money, but you could get everything you need in a day if you know what you're buying.
All the major supermarkets sell clothes, bottles, blankets etc, and lots are open 24 hours. So unless you live in the middle of nowhere, don't stress.
Also, if you start making a "baby wishlist" on Amazon, then you get an extra 15% off a lot of things. I haven't shared my list with anyone, but it's been useful to keep an eye on when the prices of things get reduced and it's helpful to know I can Amazon prime anything I might end up needing.