There'd be more wheelchair users on buses, if we didn't have this fear EVERY. SINGLE. TRIP we make, that we won't get on the bus, that we will be made to feel awful if we do, that there will be a confrontation, that we will be unable to get off the bus at the right place, that we will miss a vital connection, that we will be stranded and unable to get to where we need to be.
This anxiety the OP is feeling over one bus trip - imagine that was EVERY public transport journey, for the rest of your life.
So if you want to take a pram or pushchair or baby on the bus regularly:
Consider a sling if you can.
If you can't carry in a sling, buy your pram or pushchair with bus travel in mind, and check you can fold down one handed.
Pare down your baby kit so that you can carry that on your back and not rely on it having to be on the pram/chair.
Leave enough time to get off the bus, change buses, miss a bus - factor this into your journey. Yes if everything goes well you'll arrive early, if it doesn't it will be a faff.... but thats what disabled public transport users have to do every single time.
If you must travel with something that doesnt fold, is bulky and public transport is your only option again, factor in extra time.
I have to carefully consider every trip on public transport, what might go wrong, what do I need to account for.
I have to select the equipment I will use appropriately - I use a manual chair on buses/trains because whilst the powerchair is better for me, if there is no assistance to get off a train, I am either stuck on the train or Ill be jamming a front castor in the door and stopping that train from leaving for all of us until I do get assistance.
If booked assistance doesn't arrive to get me on the train, I am stranded on a platform.
In my manual chair which causes me pain, I can at a push, get out and throw it on the train and crawl on, should the assistance not show up.
I have to consider what bags I can carry, on the powerchair I can load tons of bags on the back, they make it more bulky but its heavy so i wont tip it over.
On the manual chair I can't, the more I load the back, the more it tips, the harder it is to push and the more likely I am to go over backwards particularly going up a slope.
Even if I have booked assistance to get on the train, that doesn't mean the train will have a space for me. I could be refused access, I could spend a journey with people knocking into me or sitting on me (has happened) or leaning on me which causes lasting pain.
So yeah if you think it is a faff ensuring you are using the right equipment for the transport you are using, building in extra time for delays, full buses, getting off and getting on the next one... just remember, thats going to happen to you maybe a handful of times, in the brief time your child is tiny.
For me, and other wheelchair users, its for always, forever, for life.