Sorry but have read OP again and I am still with Yurt. He has no concept of integrating as a family and wants to divide and rule.
Sorry Aimsmum.
Basically he wants to be the most important person in your life. He never will be.
Stay friends by all means, he sounds nice in other ways, but what he is demanding/suggesting now can be taken as a portent of things to come - almost a warning.
I can barely believe he has the gall to even mention something like this.
I can only think that perhaps he is afraid it will NOT be the way he wants it to be, and is trying either to get you to say , Oh well we;'d better not live together then' so he is off the hook, or else to 'warn' you that he will expect to be top of the tree and no less, so if he ever complains about DD, he will be able to say 'well I did say before I moved in' bla bla.
Quite possibly he knows deep down he can't cope with being a parent full time and will be relieved if you say no.
To my mind he evidently can't cope with it as if he was the right bloke for the job, he would be loving the prospect of being in a family with you and your child.
My ex was another who really didn't want more kids - I thought naively that I could have a boyfriend for me, and he wouldn't have to be a dad to my child, but it was impossible.
One example text from him:
'Here's honesty: Kid's a pain in the arse, but you are so the one'