Actually I agree with a lot of what's written on here, and the usual mumsnet men haters jumping on a situation, to put the boot in to someone unsure.
Firstly OP, if you're not happy about the situation yourself, then act on it (which it sound slike you have). However do that for you, not for anyone else. Just because other people can't understand a situation, doesn't make it a lie.
When I split with my ds dad when he was 20 months old, the guilt was immense. I'd had a child to be the best mother I could, and to give my child the best family upbringing I could. I felt I'd failed, I felt horrendous he would grow up split between two households and the whole thing just filled me with guilt. So I used to have his dad round, we went on days out, we had meals, we spent time together so neither of us missed moments with our child and our child didn't miss out on a 'normal' family atmosphere. Don't get me wrong, I couldn't stand the guy, he had treated me badly but absolutely worshiped his child. So I put up with it for the sake of my child.
Whne I met my OH, he had 3 children with his ex, who they shared custody of, but it was just easier for the kids to stay at their home, and he visited, or spent time there, as well as with his own place. They still did (and do!) family dinners, birthday dinners, special occasions etc and stayed amicable for the sake of the kids. We've been together years now, and as the kids have got older, the households have become more 'seperated' but just because two parents are trying to deal with a seperation of a family unit and the guilt they'll no doubt feel, doesn't mean there Re still romantic feelings involved or that their feelings for a new partner are any less authentic.
Now don't get me wrong, that situation is not for everyone. Of course there were times I was frustrated with either my own situation or my OH's but both of us were just trying to do right by our children. However this may not be for you. That's fine, and understandable. But make thta decision based on your feelings not of strangers on an Internet forum that jump straight to the cheating or shit dad answer. Neither have to be the case.