This is flatly untrue. It depends on the specific medication. I took a beta blocker for my entire second and third pregnancies because I have hypertension. They're considered one of the safer classes of medication for hypertension.
For everyone saying "the doctor surely knows better": I don't know if OP should have the medication, but I wouldn't rush to assume that the doctor is right. Propranolol does not have evidence of harm (there's the very woolly "it might cause growth restriction", but we don't actually have evidence), but also doesn't have evidence of safety. With medications like this, it's an enormous grey area that is down to the judgment of the doctor. They have to weigh risk vs. benefit and decide whether it's worth it, and they won't all decide the same way even for the same person.
I have a friend who got really rudely questioned by her GP over her decision to continue her antidepressant during pregnancy. The GP asked "how would you feel if there was harm to the baby?" and this is really an awful question because how could she know for sure it was the medication? And what about the risk to her from untreated depression? My friend wasn't shy about saying that to her GP. She stayed on her medication and the baby was fine.
because of the risk/benefit equation, you will have doctors deciding that women should stay on medications (for example mood stabilisers) that are known to have a definite risk to the baby. In my later pregnancies, there was a theoretical risk of harm from my blood pressure medication, but a definite known harm from untreated hypertension.
I will say this though: as-needed propranolol is probably safer than a continuous use SSRI, and it's definitely safer than a benzodiazepine.