We have adopted 2 children. The first took us about 3 yrs from start to finish, which felt like forever at the time, but TBH we needed that time to learn about adoption properly and get to grips with the whole thing.
The next time it was less than 12 months from making the first phone call to DS moving in with us!!!! Crazy fast!
Do ring around different LAs. Where we live now (different area to when we first adopted) I was told by several LAs that they would not consider us until DD was at school. But then found another one (who turned out to be great and super fast) who are very happy with just a 2 yr age gap.
The first time I felt just like you do now, that all you hear is the bad stuff. And it is true you do, and you will keep on being told the worst case scenarios over and over again. And yes, they can happen to you. But they probably won't and they certainly won't ALL happen to you.
We have been very lucky and have 2 lovely kids who both came to us very young and don't (currently at least) have major issues, so I am very positive about adoption. But I know it is about luck because noone really knows how the experiences they have had are going to affect an individual child.
But you are right to take each step at a time, I felt just like that, and you always say no at any time. But good luck... :)