We didn't get a car until we had our third child. It's fine.
The break-even point for a car is about £8000 miles a year. Or you can look at it in terms of the costs of keeping a car on the road: £200 car tax, same again in insurance (minimum), plus MOT, service, parking permits etc. Unless you have a v. small or electric car you won't see much change from £1000 per year, and that's before you've actually bought the car or paid for petrol to go anywhere in it.
£1K a year will buy you a lot of taxis and hire cars. We used to get the bus and train (family railcard once you have dc over the age of 5). You do need to book ahead for the best rail deals, but you get quite canny at working out how to play the trainfare system. If we went away for a w/e or a holiday to somewhere not accessible by train we'd hire a car. If you find a reliable local hire firm you may be able to negotiate better deals. You could also look into car clubs as well -- they weren't around in our car-free days, but I would imagine the maths stacks up pretty well for occasional use.
Now that you can get buggies on buses life is v. v. much easier. When I had dc1 it was still mainly routemasters on the routes we used in London, and you had to fold the buggy, carry the baby, carry any bags and somehow heave it all onto the bus. Now you just push the buggy onto the bus. How hard can it be? It may also be worth getting a good sling, as buggies are still a pain on the underground and some local rail stations (steps).
For shorter journeys that you can't do on public transport or with luggage etc you can get taxis. It's a good idea to get an account with a good minicab firm, cos they tend to prioritise account customers and it saves having to faff around worrying about having the right money.
For supermarket shopping you get online delivery, and/or one of those old-lady trolleys, depending how close you live to a supermarket. I never quite stooped to the trolley thing but sort of piled it all up on the buggy. It's worth considering the luggage-carrying capacity when you look into choosing a buggy. A veg delivery box is also worth looking into, and working out where the good local shops are to reduce your dependence on supermarkets.
Honestly, there's nothing to it. The equation does change slightly when you have three children as once there are 5 of you, you don't fit in a normal minicab, and friends can't pick you up from the station unless they have a 7-seater etc etc. Plus we couldn't hire eg. a Ford Fiesta, cos we need something with a boot the size of Alaska, so the hire costs go up too. But with one child, or even two, it's a piece of cake. It just requires a bit of organisation, and the ability to balance the upfront cost of taxis against the annual cost of buying and owning a car. Unless you are doing a lot of journeys to out-of-the-way places or live out in the sticks, it's almost always going to be cheaper to not own a car.
Ignore people who say otherwise -- car owners are getting stung more with every passing year, so it makes no sense to get a car unless you have no alternative. Honestly, one little baby, or even a toddler or older child on public transport is not a bit deal. Plus they get quite robust about walking for reasonable distances instead of turning into little weeds who bleat as soon as they have to walk further than 100 yards. If you have a real walking refusenik you can get a pedal-less bike or one of those microscooters. But that's a few years down the line.