hiya,
(just to answer your last post) if he stops money - well either way see your sols. If you don't want ot see him just yet, is there a 3rd person (neutral if poss) where you can leave DD and X picks her up from there?
you won't be damaging her emotionally at this point. you're asking for space - he's trying to have things all his way thou, and trying to control the situation - but obv he wants to see his dd - which is fair enough.
I'd say drop her at her nan's house (his mums - or ask your mum to drop her at his mums or something??) or a friend, and say to him you ahve no probs him seeing his dd, but for now you do not want to talk to him about things, as you some space. and to that end the plan re your dd will be for him to see her on XYZ (set days for you) between XZY times) to be collected by X and dropped back off where she's picked up. maybe a little confusing for DD in the short term, but in the longer term it maintains the contact whilst giving you your space.
if possible do it all via email, if not keep all texts you both send and receive.
he'll prob come back with some diff dates and times - give a little compromise if you can, but not too much - show willing, keep calm, be factual, and stay reasonable. try to keep it all matter of fact about your DD.
if he comes back with any more crap about emotionally damaging your DD - tell him, quite simply that you are not prepared to use your daughter as a pawn nor are you prepared to be emotionally bullied & manipulated, by him using your daughter against you, as you are a good mother - if he chooses to follow that path then you will have no alternative but to seek legal advice.
(advice I suggest you seek either way - knowing your rights is half the battle - these 'men' think they have the upper hand but bullying us when as the mothers/main care providers/resident parent however you dress it up, we have a lot of power and the upper hand - just most of us are to upset to exhurt it.