I don't know why, I wish I did. My ex husband wasn't a great father from the start but after we separated (due to many things, lots of which involved his hands around my neck), he couldn't have been worse. He bad mouthed me to our mutual friends resulting in them backing away from me, told them and my 2 eldest children (his biological sons) that I wouldn't let him see them (absolutely not true, every line of communication was open he just refused to use them and made them feel guilty for not contacting him instead) and when we moved overseas, with his full permission and after signing a letter with the notary (he almost ran to that notary) he stopped paying maintenance. Ok, he wasn't legally obliged to but morally, this was a kick in the teeth for my sons. It really took me about 5 years to get to the point where I thought, fuck it, they don't need this shit and neither do I. It was such a difficult time before that though. When both myself and the boys would ask him to spend more time with them he would say that he had a life too and that he needed to 'sort himself out'. I went through years of trying to convince my children that their father did love them but showed it in different ways. Even after several years, he refused to come into our home (I had now remarried and had 2 other children) and would only eat cake with them on the front step on their birthdays, refused to come in to their home. They eventually realised for themselves how selfish he was.
On his last weekend with them, before we moved overseas, he told them that he couldn't afford to come and see them that year, that he had lots of bills to pay (they were 9 and 12 by now). He then took them, with his girlfriend, into a travel agency and booked a holiday for himself and her for a couple of months later. She has since been in touch with me (they are no longer together) to tell me how he always thought it was a massive effort to see the boys on his one weekend a month.
Anyway, to cut an extremely long story short, my ex husband didn't see our second son for 3 years untill he saw him in the mortuary after he took his own life aged 15. We don't know why and I'm not blaming my ex husband, in fact I blame myself more than anything for not realising how he was feeling but I did think that it would make him realise what he had lost and what he could still have with our eldest son who is now almost 20. He did, for about 2 weeks, even though he tells friends that he calls him and Skypes him regularly - not true. But his life hasn't involved our children for so long and they never did mean a great deal to him. I hope to god that he rots in hell for all that he has put them through, even though they have always felt sorry for him for me leaving that abusive relationship. Ah yes, I'll always have to live with the fact that I broke up our family because that's what he made them believe. Narcissistic and toxic ex husband. Tick. Selfish twat of an ex husband. Tick. Ex husband that thinks more about himself than his children. Tick.
In answer to your original question OP, I think that it's very much a case of out of sight, out of mind. For men and for women although it's usual, in a vast majority of cases, for mothers to look after the children after separation and so this tag line is often applied to more men than women. I know that there are a lot of fathers out there who want to spend more time with their children and their ex wife's won't let them. This is not the case with my ex husband and with a lot of people that I know.