It is absolutely understandable to be anxious and be scared to become attached given your experiences. Even without having loss, I was still anxious about becoming attached in early pregnancy. However, once I was out of the riskiest period I have relaxed and bonded. I don't talk to them all the time, but I will tell them it's time for us to go to bed or that we're going for a run, ask them if they're having fun when I can feel them moving a lot, or tell them they're coming to an important work meeting with me today so that they can learn how to win arguments against Daddy...
The midwife isn't talking complete nonsense, but is overstating it somewhat. There is some scientific evidence that bonding in pregnancy can impact on long term mental health for both you and baby, and on neural developmental outcomes for baby post-birth. Just as stress and cortisol can impact negatively on the environmental conditions for baby, love, connection, and attachment results in a neuroendocrinological cascade that as well as assisting with bonding, also promotes wellbeing and reduces anxiety. Think of it like hugging someone you care about - it creates that release of hormones that makes us feel good and happier in general, not just more connected to that person. You can be fine without that, but those interactions make us feel good and warm and fuzzy. However, baby also cannot hear you until about 16 weeks - at which point the neural pathways are formed. It's not impacting on that side of development in the way she has implied.
However, the crux of it is that notwithstanding that, your midwife is being unreasonable to expect you to just be able to switch on those feelings if you are still feeling very anxious about becoming attached, and are emotionally protecting yourself from the risk of greater further heartache, which is absolutely understandable and natural. She should be focusing on reassuring you, rather than admonishing you for how you are feeling, and the impact of what she is doing is just going to lead you to not talk to her about this stuff, which is literally part of her job. If she continues to be unsympathetic, consider asking for a change of midwife.
It may be that once you've got through some (more) scans, tests etc and have some scientific reassurance that little one is doing ok in there that your anxiety will settle a little and you will be able to relax into the later stages of pregnancy. If you continue to feel very anxious, it may be worth considering whether a referral to perinatal services, or speaking to a counsellor, even if just for a session or two, may help you to feel more supported. If it's an option, it's also important to speak to your partner or loved ones about how you are feeling.
And of course as others say, baby can also hear you talking to others and going about your day once they can hear. Whether those have the same impact depends on the types of conversations you usually have: lots of fraught and stressy work conversations with difficult clients, or an environment with verbal abuse or domestic abuse isn't going to be the same as baby hearing lots of soft, kind, gentle and loving conversations between you and your loved ones, or hearing you sing along happily to the radio in the car or the shower or while you're cooking.
Ultimately, if you have the conversations and do the things that make you happy and reduce your stress, baby will benefit from that, so try and prioritize those things: be it exercise or yoga, or having a kitchen dance party to Taylor Swift, those things will help both of you be happier and healthier long term.
I wish you and little one all the best.