The baby stage can cost as little or as much as you want it to for the most part.
We used washable nappies, they cost £200 upfront which was a lot but were used for 2 children so works out amazingly cheap. Even when I worked out the additional cost of laundry detergent and extra wash cycles in the machine (both water and electric) it was massively cheaper than disposables.
When we were away somewhere that it wasn't practical to use disposables we usually budgeted on disposables being about £5-£10 per week, depending on brand.
I also used washable wipes, these saved me a fortune. I used shop bought ones when out for ease, but used my own at home. I think it cost about £2 in fleece blanket that was cut up and then I made my own solution with water and baby oil, costing pennies and they were stored in a nice looking lidded tub .
We had a second hand cot and moses basket, but bought new mattresses. We had a second hand travel system that was compatible with the car seat we bought. Later, we bought a new umbrella fold buggy which has been excellent.
We used babygrows on days where we weren't doing much and bought just a few nice outfits for if we were seeing friends or visiting family etc, otherwise it was just Asda babygrows which are cheap and really practical.
We bought the baby sleeping bags from an NCT nearly new sale for very little and we have since sold them on again so they basically cost us nothing overall.
I breastfed so didn't need to buy formula, I did buy a breastpump and it was really useful. Again, I bought it at an NCT sale for very little.
The baby stage was really cheap to be honest, now the toddler stage is far more expensive!!