Help me set a sleep routine for us parents

[dashoflime]

DS aged 3 months sleeps from 10pm to 4am, then settles again till 8am (if I'm lucky).
We have him with us in the evening and put him down at 10 in his moses basket. Me and DH have a bit of TV time then I go to bed at 11.30-12.00.
DH has always been a bit of a night owl and likes to stay up late.
DH is now getting into the habit of staying up until DS wakes at 4.00, then carrying him up to me and waking me up so I can breastfeed. He then sleeps to between 10am and midday.
I sleep till 8am and get up with the baby.
I am very pleased with DS sleep pattern but dismayed that DH and me seem to be sleeping in shifts. Its lonely and I miss the intimacy of being in bed at the same times.
Can anyone suggest a better routine for us? I have said to DH that there is no need to stay up till 4.00 waiting for DS to wake.
He has replied that he doesn't want to wake the baby by moving him upstairs earlier and doesn't want to leave him alone downstairs as too much bother to get up again and bring him upstairs. What do other people do?
It seems like the baby has a great routine, but the parents are all over the place!

[dashoflime]

Actually, no, don't answer this here. I've just noticed the Sleeping forum. I'll post it over there.

[ZombTEE]

I'm going to answer it here anyway.

Keep what you're doing for now. It won't be forever and you're both getting some sleep. At this point, that's the most important thing!