OP, I understand your concern. It can be quite serious if you catch it in the first trimester so I don't think you're being overly anxious. I work in a country where if you work in childcare with kids under 5 that you are immediately signed off work after becoming pregnant.
The guidelines are to avoid crossing fluids with any young chlid (tears, urine, saliva). So don't change any friend's babies at the party and don't kiss any child or share a cup/utensils/food. Wash your hands after coming in contact with small children, and it's probably just sensible to wash them regularly at a party anyway.
You can be tested for antibodies to see if you've had it before but there are 4 strains I believe (so my gynae told me), so just because you've had it, it doesn't mean you won't get it again.
The risk to your baby decreases as your pregnancy goes on (although the probability of passing it on to the foetus actually increases).
I have 2 under 2 and had never had antibodies for CMV, so I was very wary in my second pregnancy. I never shared anything with my first and made sure my partner did the same. As it turns out, the week before I went on maternity leave, my partner tested positive for CMV! I got tested myself and was negative! So there is hope that even if your nearest and dearest get it (and think about how many pregnant mothers there are with kids under 5 in the house!) that you will get it or have serious consequences.
Wash your hands and enjoy your pregnancy.