I wouldn't rush into living together, from a practical point of view as it hard to rewind. Have a good read of the Step-parenting section on here. You may think living together will make your life easier. It may not.
What do you perceive living together would bring that you don't have currently? If you think it would mean him being around more for you and the baby, he could do that now, by living with you and just going back to his flat when he has his DC? If it would mean more commitment, talk to him about that. If it's for financial reasons, talk to him about contributing whether he lives with you or not.
Try not to worry about what other people think, and do what's right for you and your DC. Prepare some stock phrases for curious people if it would make you feel more comfortable - "oh yes, it's what works for us, it's great how he manages to be such a great dad to all his kids!" (if he is). "Things may change in the future, life's full of surprises!". It's no-one's business whether the pregnancy was planned. It's all very well people asking curious questions now but I would hate my DS to think he was unplanned when he's older, so I focus on the "happy surprise" angle.
Do you work? Is he supportive and helpful? Do you like where you live? If it ain't broke, don't fix it! You are in a good position at the moment really so don't rush out of it.
Do have a look at the Step-parenting board, if you haven't already.
Good luck 