Hi,
You currently have an 'informal' arrangement so yes you can object. However, is it wise to do so?
As both of you are separated you are both entitled to move on with your lives independently and that means the emotions involved in new partners, particularly in relation to children.
The fact is, that when he has the children, they are under his guardianship he has parental responsibility and he can leave the children with any person who he may deem fit, as long of course that they are not under or subjected to any 'significant harm'
The bottom line is that if you object, then he can take out a Child Arrangements Order which will give him and you parental responsibility on set days and during that time he can again leave the children with who he pleases.
Unless you can come to an arrangement of what you both consider to be in the best interests of the children, he ultimately will be able to leave the children with his current partner with either your consent or via the courts without your consent.
Perhaps not what you wish to hear but I hope this helps clarify his parental responsibility and rights.
Kind Regards
McKenzie Friend Phil