I had a feeling for most of my adult life that I was infertile. Literally just a feeling and I didn't know where it came from and then I found out I was (but not psychic with lottery numbers..damn it!). I had the same feeling IVF would not work and based on the other thought and that I was right the first time, I thought trust my gut! I very much focused on the negatives of IVF, there are so many IMO, against the CHANCE of getting pregnant. On the flip side, my life experiences meant I knew I could easily love a child who was not biologically mine and I felt my experiences would mean when that child was older I would be able to relate to missing family members, feeling let down by them and the need to re connect etc. as I'd been through something similar. I had a genuine belief this was meant to be, as if all my life had led to this point.
However, while I'd had this epiphany and was sure as the sky was blue, I had a husband who hadn't had any of those thoughts or experiences. So I gave him time, months in fact and he thought a lot about what it would mean to not have a child that looked like him etc. and almost grieve the loss of that idea and he read a lot, of mainly forums to be fair, and in the end it was him who made the decision to phone our LA. He did this based on the fact he didnt want to do IVF, he wanted to do adoption and thought he was cut out for it. He wasn't 100% sure but figured start the process and we'll find out, after all you're assessed intently, so you'll be tested, questioned and told if you're not, if you don't realise it yourself by that point.
Once we started we took a stage at a time. I'll be honest, it helped that IVF was the last resort because we always had the get out clause of if either of us felt we couldn't do it, we had something else to try, even though we both didnt want to do it and would rather adopt.
Did we ever feel 100% confident? Even with my little epiphany and similar experiences to some of the children adopted, no for me aswell as DH! You will constantly throughout the approval process question whether you can do it and that's a good thing I think. Then the children arrive and be prepared that in those first few months it is so very hard and you will likely question yourself again, did we make the right decision? Am I cut out for this? I think it's just normal to think those things. But then they disappear. I Of course constantly have doubts if I am good enough parent and reflect on what I need to do to improve and tweak things and learn from my bad days (all parents have them!) but that's what makes a good parent I think.
So how do you know?
Be 100% sure you don't want to do the other...IVF, donor eggs/sperm etc. You don't have to start the adoption process being 100% sure you want to do it but you do have to be 100% sure you're done with the other.
Read lots about adoption - books or forums.
Then simply.....go with your gut and always be willing to pull out if your gut changes its opinion! 
We became the parents of sibling toddlers and my gut feeling was right...100% right decision with no regrets. If I found out I was miraculously pregnant now I'd be absolutely pissed beyond belief and depressed about it rather than elated for ruining our perfect family.