My DH's two sons (16 and 18) no longer come to stay with him on weekends. Both stopped doing it when they reached 16. No problem with their relationship, just concentrated on friends and girlfriends.
My DH still goes to visit them in their house every week (EOW, it's twice) and stays for an hour or two. He's close to his ex who grew up in an unstable environment and decided that my DP is the closest person she has. So they are like "brother and sister", so I guess because my DH can come anytime he wants, SS don't have the motivation to come to our place, unless there's a special invitation.
That means that SS rarely get to see our toddler.
I'm curious about opinions. If your DH had a good relationship with ex, would you be ok if he took your child to his ex's place to connect with his kids. Of course, the ex would be playing the nice aunty.