Mine went to sleep late and woke up late at that age, but only because it was the time she wanted to sleep naturally. Personally, I don't think there's a perfect time that works for all, go with whatever works for your baby and your household. If he's cranky from 5pm and wants to settle down by 6 and it works for all of you, I don't think that's a problem. If you are wanting it to be later so he wakes up later, you can just adjust it slowly and adjust day time naps accordingly to make it work better for you.
I am exactly the same as you. My 4 month old does it sleep anymore than 3x30 mins naps in the day and come 5-5:30 is tired and ready. I end up putting him down between 6-6:30 most nights and he can take up to 20 minutes to settle and still wakes 2-3 times at night. I have also been thinking about about a later nap about 4:30-5 to give him a later bedtime which might help with the night wakes but I can't get him to go later. I think we may need to just read our babies and if they are ready at 6:30 then that's how it has to be and I will sort out night wakings when he's a tad older x
DD is 5 months and goes down at 7. Sometimes 6.45 if she's not napped well in the day. It used to be more like 7.30/8 but is creeping earlier. She'll be pretty cranky from about 6 ish and would likely prefer an earlier bedtime, but I have DS to work around. We keep her going with cluster feeds, nappy off play and a longish bath. On a good day she'll wake at 6.30am, on a bad day she tries to get up at 5am
I can't remember the exact timings now, but it was whenever she was ready to sleep for a long stretch. At 3 months this was about midnight, by 6 months it was about 7, and she brought it forward bit by bit by herself.
3mo ds is a bad napper so is generally in bed asleep 5.30-6. He's still up 2-4 times a night though. Hoping he'll go a bit later once he has decent naps and is sleeping better at night. My 2 older dc were always very early in bed too though until they started school as they were just too tired.