You partnered and had children with a man who has a track record of not being proactive and not taking full responsibility for his children and sorting out his parenting arrangements and commitments. He is just doing it again with a second mother.
I get it, things we think we can handle and let slide when life is easy ie. pre-kids, become much bigger problems when we have much bigger responsibilities. I dismissed my ex-partners small red flags too.
He had kids before he met you, unfortunately for you, they are a higher priority than you. For their sake he needs to re-establish a relationship with them, and due to the age gap, it does seem most sensible he does not bring along his younger children.
He now has an additional commitment to look after his younger children while you work which is a timing conflict. It seems like he needs to organise childcare for while you are at work, can he ask family or friends? Perhaps he can look for suitable local childminders?
I get it, it's annoying for you, but it's none of the kids fault your partner is irresponsible, so focus on the actual problem at hand, your partner taking accountability and responsibility for his obligations.