Take the wind out of his sails by pre-empting him... and do it with aplomb.
Tell him you already know about his other woman and that whatever love and respect you once had for him has been destroyed by this knowledge.
Tell him your ds is struggling to process his df's absence and that his contact with his son must take place outside of your home to avoid further confusing him.
Tell him you expect him to have ds every other weekend from early evening Friday to Monday morning and one overnight stay in the week, say Wednesday early evening till Thursday morning.
Tell him that you will celebrate ds's birthday on the day and he can make arrangements for his celebration the following day or other date.
Tell him that in future ds will have two celebrations for events such as Christmas, birthdays etc and that school/nursery holiday care will be equally shared between you.
Don't rant and don't beg. Be sure to sound unemotional and detached and do your utmost to give an Oscar winning performance - you can go to pieces after the conversation but on no account should you let him know that you are anything other than in control of this situation.
Don't give his ego the satisfaction of having two women fight over him; the fact that you don't intend to be brought into the fray will astound him and it could be that the shock will make him realise what an utter fool he's being - don't hold your breath for this awakening as it may take some time to permeate his dense brain.
You sound so very down and, while this is understandable, please know that when you are emotionally on the floor the only way is up and you WILL rise again.
Am I correct in assuming that you're not married to this callous piece of work? Is the tenancy of your rented home in joint names and why isn't the landlord making good the damage caused by water ingress?