I feel sometimes like there's this psychological thing where pregnant women are almost encouraged to fixate on "banned foods" in early pregnancy - as though it's not really about the foods, but about managing anxiety, and restricting diet is a way of making your intangible worries into something real you can control? Dunno, just a bit of armchair psychology.
I have been reading loads about all the banned foods (just because I'm interested!) and most of the guidelines are at heart about precautions and hygeine. The NHS guidelines are more than conservative, so I wouldn't worry about anything that isn't on their "avoid list", and try and have some perspective about some of the things that are!
For example, avoiding dodgy buffets, service station sandwiches and food out of kebab vans will be more important in terms of avoiding listeria than freaking out about a bit of pasteurised supermarket brie shrinkwrapped to within an inch of its life.
Also some of the advice is about giving a really simple public health message rather than accuracy. Like liver! They just say avoid all liver, but 100g chicken liver (which is an admittedly small portion) is about 5000 international units of vitamin A, and the safe limit for pregnant women is 10,000! So I don't see how a small amount of chicken liver is going to do any harm from time to time, but I guess it's easier to say "avoid liver" than have people eating calves liver by mistake because they thought it was chicken liver?