CAFCASS are generally rubbish and do not have the best interest of your child at heart. The court will take what they says as gospel and it's very hard to get anything different in court to what they propose. Look at your local CAFCASS ofstead report and you'll probably have your eyes opened very wide. And of course no-one wants to complain and get them off side in case they take offence and so skew their opinion even more! Until you go through the court process you are unlikely to believe some of the decision made. But of course it's all behind closed doors for the sake of the dc's privacy, so we don't hear about the outragous and unbelievable decisions made on behalf of children in their apparent best interest. And that applies to both parents with care and NR parents too! Learn to play the game, like youe ex has. Document everything, keep records of missed contact and any issues. And if you or your dd don't feel happy or safe don't be afraid to change the contact and have him take you back to court over it.
Saying all that I think you'll find it's ok for your dd, at least to start with. Dad will be wanting to make a good impression, and she will get to find out something about her dad and in time be able to make up her own mind and see that you didn't stand in the way of her having a relationship with him.
Make sure you fill your time and plan what you are going to do. It will be tough, but your dd will realise in time that you were always there for her and dad bought things and palmed her off on his new partner. Don't try and match him in terms of 'fun' and gift buying. Keep doing what you've been doing and being the strong, stable parent she can rely on.
Rather than thinking of them as playing happy families, try and take some comfort from the fact that there will be another woman there who will hopefully look out for your dd and ensure that she is well cared for despite your ex. She probably has rules and boundaries for her own dd so your ex won't be able to go completely wild and let your dd have free reign (ie stay up all night, eat nothing but sweets and happy meals etc).
Also be prepared for things not working out between them and that contact may well cease if that happens. He's probably told his new partner that you have been difficult and made it impossible over the years. She's likely to be the one pushing him to have contact and sort it out because she wants to believe he is a good dad and will be there for her and her dc too. If it's a fairly new realationship then he probably wants to please her and show he can be a good dad so is doing what he has too to prove it and make her happy. It won't last I suspect. And if it does hopefully that means he has grown up a bit and is taking responsibility now.
It won't be easy but try and enjoy the free time you will now have. Use it to keep yourself sane, strong, and to enjoy the things you've had to put on hold these past 4 years (such as your hobbies and time to be you rather than just 'mum').