I think it helps to remind yourself that you want another child 'for yourself', not because your child would like a sibling, or you think a sibling would be good for your child. You want a child, not a sibling. Each of your children will then be the other's sibling, rather than one being your child, the other being your child's sibling. IYSWIM? It's obvious really, but what with the one child being there and real and a huge part of your life, and the other child as yet being nothing but an idea, it is easy and natural to put your existing child first. But if you stop there, then nobody would ever have a second child, as the second always reduces time and attention available for the first. So it helps to think somewhat selfishly 'I want that second child and my PFB will just have to suck it up'.
Yes, another BC could have all sorts of problems too, but I think it is important to recognise that with an AC, the risks are generally higher. And the younger you adopt, the larger the uncertainties.
It's ok to dream of your perfect family constellation, and to hope that you could achieve that dream by adopting. But I think it is important to recognise that things may turn out a little differently, and to think carefully if you as a family would still be ok with that.
That said, if your expectations are realistic, at the end of the day IMO it's all about MAKING happy family life, and that doesn't stop when the children have all arrived. You could have a perfectly happy family life with typically developing BC, and one struggling AC, if you make it so; just as you could have a perfectly happy family life with only your BC, or with two BC, or with your BC and a typically developing AC.
You ask if you are worrying too much about potential problems/the increased risk of problems that comes with AC. I think you need to look at yourself and your family to answer that. How do you deal with problems? With disappointments? Is your relationship strong and adaptable? Are you happy only with the perfect ideal of your dreams, or do you usually happily make the best of things even if they fall short of some ideal? Depending on the answers to these questions, it may be that yes, you are worrying too much, or no, you are not worrying too much, you are right to be worried because maybe adoption is indeed not for you (which is fine, but important to recognise before it's too late!)