He has made it clear that he does not want to live with you, op. It's easy to say, "We may do so way in the future", because that is a long way off and much can happen in intervening years. He also does not want to have children.
You want to set up home with a man and have another child.
Why are you hanging on to this man? He is obviously not right for you, nor you for him.
There are people happy to live apart, enjoy each other very much but have their own space, don't live in each other's pockets. That's great but it is not what you want for your future.
You can hopefully explain this to him, calmly, and he will have to accept it but there will be no cause for any bitterness. You can both look back at the relationship with happy memories. That is my wish for you, op, but only you can make it happen that way and he has to understand. I
I realise I am an 'outsider here', am not giving advice but suggestions. I don't think it would be easy for me! However, I would not want to live with someone again, of that I am sure, I value my independence and own space so your specific situation would not arise.
I must add that you are blessed to have a child, which I am sure you already know, and you could meet someone who feels the same as you about a relationship, in the not too distant future.
Just think it over and I do wish you lots of luck.