When DS1 was newborn my parents brought over the pram they had bought for him. All excited we put it together and took DS1 out for his first trip in the pram. The first thing I did was carry him down the stairs in the carrycot section holding the handles and the carrycot was top heavy and tipped, DS1 was hanging upside down in the carrycot, thankfully strapped in. Next we attached the carrycot to the chassis of the pram and started to negotiate the doorway, I was shaking by this time, it turned out we hadn't attached the carrycot properly and it fell off and was caught by my Dad, by now I was shaking and crying, hardly the proud showing off my new baby to the world moment I had planned.
This time round with DS2 my DH put the pram together for me and the same thing happened, the carrycot fell off the chassis, I caught him and just laughed, DH did get a ticking off and was re-educated about how to put the pram together.
When DS1 was 7 months old I tripped up whilst carrying him round to our local shop, he ended up on the pavement, as I fell I must have thrown him forward as he slid across the pavement rather than landing straight on it. When we took him to casualty they said there was not even a bump on his head. I am sure it is an instinct thing but I will never forget that feeling.
Anyway as everyone else has said don't let this incident spoil your enjoyment of your baby and if the sling is the best way for you to transport your baby then try it again, I'm sure you'll be fine.