YANBU, I had 2 eceptionally large Cats when pregnant with DD, I took precautions with litter cleaning just incaes, but would of been amazed if I wasn't already immune............
One of my Cats by her nature was a bit of a worry, as she was very attached to me, & could be very feisty & vicious when she so chose...she saw off burglars in the past, making a real mess of one of them , & made a mess of DHs arm when we first moved in together.........though usually she was just a big soppy bear of a thing.......BUT as you can imagine, when DD was born she was watched like a hawk, but at the same time I went out of my way to make sure she didn't feel "pushed out"
Sadly the Tom died when DD was only 8 months old, & he just kept out of DDs way..........but my lovely old feisty girl turned out to be amazing with DD, loved her & watched her like her own, & if I went to the loo & baby DD started to cry she would come & get me, when DD got to toddler stage, the Cat always watched her if I left the room for minute & would howl loudly to get my attention if DD ever looked like she even might get into any trouble, climbed out of her cot without waking us etc etc....she almost became like a nanny to her, it was so sweet........when DD was older still & went through some problems with a bully at school, the Cat was her confident, & absolutely doted on DD, they were inseperable
Sadly the Cat died of very old age (20) a few days before Xmas, which was heartbreaking for all of us, but in a weird way also good for DD, in that it taught her about death & grieving
I really wouldn't worry in my experience Cats & Kids are a great combination, for all the reasons mentioned above & more besides......even when the Cat has never had to deal with kids for most of its life.......
my only advice, as above... be extra careful with cleanliness as far as the tray goes, but don't worry too much on hair etc as its been proven early exposure can boost immunity to allergies... & try not to push your Cat away in favour of baby....that might make them jealous, so slightly more of a risk....besides cats are great stress relievers for you