oh liahgen, I wish I knew the answer to your question. Do you feel that the doctor who advised you is really knowledgeable on bf and fertility? Can he/she explain why he feels bf is affecting your fertility, and which part of the conception / implantation process is being affected and by what mechanism? Or do you feel it is a more general assumption that "bf affects fertility, but don't know how"
How long is it since you stopped bf? And are you aware of your luteal phase length? (I didn't even know what that was until another poster pointed it out to me.) I charted, and used opks so I knew I was ovulating and my lp was fine. But if I had been trying for a year, even if I knew that bf probably wasn't the the cause, I still might have made the same decision as you. It's that nagging 'just in case' doubt. (I'm not suggesting this was the case for you, BTW!)
I know my dd and I would have been devastated to curtail bf. Everything I've read on the subject suggests that, once pg, unless you are at high risk of m/c and have been advised to avoid sex, then there is no evidence that bf during pg will have any adverse effect at all.
So, if it was me, I might resume bf after a bfp. I have no medical expertise though, that's just what I might do in your position. Bear in mind I sometimes carry a 'lucky baby sock' to keep my pregnancy safe - despite having a full understanding that this is an illogical, nonsensical thing to do, and not actually believing that sockie has any magical powers. It's possible to know the facts and yet still employ hopeful comfort measures at the same time! In fact, I'll dig out the 'lucky' thread that 'helped' several of us conceive!
If this cycle isn't the lucky one for you, (and I really hope it is!) I imagine you will continue to question your decision and whether to resume bf again.
Why not start a new thread in bf topic as there are ladies with more knowledge and experience than me who can contribute.
I really wish you the very best of luck.