You are certainly not imagining it and yes,this is all symptomatic of an emotionally abusive relationship.
He is still trying to reel you back,and you are certainly not alone in experiencing this.
Why does he visit dd at your house.This must be terribly difficult for you,and is giving him the opportunity to continue with his abuse of you.
This is abusive - yesterday,in your house,during his visit to dd, he was "pretty nasty, accusing me of affairs, slagging off my father etc" then ,this morning,he sends text messages "begging me to move to where he is, saying he still loves me, wants to make a life for us where we both grew up." -well,read that back - all about him for a start,apart from being written and delivered potentially anywhere,and with any one else present....ie,not evidence that his sentiments are in any way genuine ,or motivated by consideration for you and dd.
It is damaging and hurtful for you to listen to all his stuff "about what he is telling his friends and family". Is there any way you can avoid him completely?.He knows the effect this is having on you.It is deliberate abuse,and you need to protect yourself.
I recognise that feeling of wanting "to shake everyone and tell them the truth".That will fade over time ,although it seems difficult to imagine,now.I really couldn't give two hoots what anyone believes about me and my experiences with xp these days.I know the truth,and me and my dc are living our lives just fine.But 18 months ago,I could not imagine feeling this way.
But,while you are still getting the crap,it is very,very difficult to break away.This is what you need to do.
Do not see him.Do not listen to him.Block his number - get legal support re contact if you have to,if it is drawn up and agreed,there should be no need for any other communication,you can take "emergency" arrangements into consideration at the drawing up stage,and stick to them - is there a close friend/family member who could be emergency contact,for example.
The button he is pushing is the one about your shared history and the plans and hopes you had together about your future.It is more difficult for you since you are living away,in a place you moved to for him.Any chance you could move back?You need to give yourself time and the opportunity to grieve for what you did not have and for those hopes and dreams.