I wouldn't worry too much, I've just had some camembert today myself, and had a runny egg sarnie last week too.
Lysteria can be damaging to the baby, but first of all it has to get in the food, then it has to survive your stomach, then it has to get into your bloodstream, and avoid your immune system and make its way to your womb. Unless you have open ulcers in your digestive system, it's pretty unlikely you'll have trouble.
Lysteria is pretty common, but its usually in low amounts. We all eat loads of bacteria all the time, and most of it does no harm whatsoever. We need certain types of bacteria in our gut for digestion to work properly. So, unless you're prone to getting stomach upsets all the time, or have ulcers, I wouldn't worry about it.
In my opinion, it's a bit like saying don't cross the road because you might get run over. Well, yes you might. Some people get run over, and some of those die. But most people are fine, and you can't spend your whole life looking for pedestrian crossings.