Contact is for the children. It's irrelevant what is fair for the adults.
Routine and structure are incredibly important for children this age.
Has he openly said he keeps him up to spend time with him? If so I would send a friendly email quoting that and saying obviously it's wonderful he wants to spend as much time as possible with his son and you welcome that, but can you try to increase daytime hours with him instead so the structured routine all children need can be implemented - the lack of sleep is hugely detrimental to his development. Write it in your own style and way so it doesn't look like you are creating a paper trail. But when someone is neglecting their child - and keeping a toddler awake so they can meet your emotional needs is neglect - you need that paper trail.
It's great that his father loves him and wants to spend plenty of time with him but I would just alter this pattern now so it doesn't get set in stone. It's not serving your child. And also, when does he get to see his sibling (if you each care for one at nights) once they start school, and are in different houses all weeknights? How will that work?
Mediation sounds positive. Best way to talk all this through. But I would get the email done first, frankly, because if he denies it all you need the evidence there.
What's the child support situation? If he has the children every single night then your care in the days isn't counted - you are expected to pay him child support. Did you know that - does he?
My DS went to bed at 6.30 when he was 2 - 3, btw. Sleep is essential for all of us, but especially a toddler.