Sadly a break up is likely to be hardest on the dc as they're the innocent party in the middle.
If it acrimonous it's likely to affect them deeply and for years.
If you're not going to allow the dc to sleep at the nrp's are you then willing to allow the nrp to spend days on end with dc at your home ? Or consign them to roaming around outside in all weather ? Or just stop contact ?
There is no ideal solution.
I do sympathyse with the OP, but if the dc isn't at risk at dad's unfortunatly I don't see what you can do other than hope dad modifies his behaviour for the sake of his child.
I'm coming at this from the other side.
Despite being a gentle , loving and empathic human and dad my XP was very much against dd staying with me. I was accused of all kinds of stuff.
It was 'outragous' that dd should stay with me. And despite only proposing ( in her eyes 'demanding' ) alt weekends and a weekly visit, she went from solicitor to solicitor I guess hoping to find one who agreed with her. She didn't.
Eventually she had to agree but felt I'd bullied her into agreement. I just bite my Tongue and get on with loving dd.
Hand over, and her attitude at times to dd being at mine can be problematic.
Dd does of course love her mum, and to shorten a long story, if XP is around dd has on occassion been reluctant to leave home, egged on by XP, if XP isn't around it's the oppersite , she'll be enthusiastically asking me what toys to bring and where my car is.
And in my case I recognize that dd has split loyalties.
I could rabbit on all night about details and things that have happened. But essentialy I think you have to be as positive as you can and certainly not give a child that young a choice in the matter.
It was paramount to me to make dd feel safe, secure and happy in my home. A lot was common sense, but being dad and not involved in mums groups and dd's day to day stuff I had a few hills to climb and even now after a year I have very little in the way of support. You don't tend to get nrp play groups, and unless you have friends with same aged children it's very hard to get involved in chilrens groups let alone mums groups.
I think a lot of men do find some aspects of parenting isn't first nature, and I can understand an RP's anxiousness .
But like I said. It's not an ideal situation. You had a child together and now you're apart. If there's no risk you shouldn't stop contact or overnights. You can attempt or hope the nrp listens to suggestions but little else IMO.