I was raised in a situation a bit like this including the various girlfriends. It was also 70s and 80s so Mom by default had full custody and we saw Dad once a month. Today I suspect that was a good thing, because although he had a good career and paid what was ordered by the court, he would not have done one bit more for us. I love my Dad and get along with him, but he never put us first. Dead last, we were, right behind himself and his vitriol for my Mom.
My Mom never said he didn’t love us, of course. She actually never said a bad word about him and even today at 80 is pretty tight-lipped. But his behavior spoke for itself; he was selfish and acted like a victim when instead we 4 kids were the ones suffering.
Mom did what she had to do, including the traveling but she didn’t do more than was ordered by court. That wasn’t because she was annoyed by the girlfriends (although of course She must have been), but because she knew the environment was not good for us.
I’m a fairly unbiased person here, and a stepmom myself, after 6 years as the girlfriend.
I don’t believe you should drive 2 hours with a newborn to enable these overnight visits. I don’t think 4 hours in the car over a two day weekend, every weekend, is good for any of your children. I’d tell him it’s not possible for you due to breastfeeding, and that you worry about the kids having to do so much traveling. Perhaps agree to once a month instead, or times when it’s possible to do at least a 3 day visit.
Sounds like this situation is unlikely to last long with his track record.