WishIWasRimaHorton, I don't like to do this, but here goes ...
As a soon-to-be female NRP who will have shared parenting and residency (at the moment, OH and I still live together), I have never read a thread that has made me so angry and incensed. I work later than my husband, my hours are crazy compared to his. Yet, at the moment, it is me that DS is closest to. It is me he runs to when he is upset. It is me he asks for over and over and over again, and cries for when I'm not in the room. It is me he is closest to. I'm the one who buys him clothes, cooks his food, sorts out care with his childminder, gets up with him at night. This is despite the fact that he sees my husband more than he does me. Based on your logic and the supposed incomparable awesomeness of being a mother, my DS ought to live with me primarily.
I'm sorry, but face-time does NOT (and should not) necessarily dictate to whom the child is closest. And when he does cry for me, as upsetting as it is, I have to let him be with his father. He is a parent also, he loves his son as much as I do, and they deserve to have a close relationship which need not be at the expense of mine. Taking this attitude means that DS may miss me, but he soon forgets and spends so much enjoyable time with his father who cares for him in a different way to how I do (eg he may not be as quick as I am to wipe DS's nose or cut his nails), but more than adequately.
If his home is as filthy as you say, then refer the matter to social services. Unfortunately, unless he is seriously risking their helath (which it sounds like from what you've said but does not tally with the inaction of the authorities), then his house need not be as spick and span as yours. I do not agree with children living in a filthy environment, but I do not subscribe to the view that a completely spick and span house is non-negotiable (if it were, so many of us would fail at the first hurdle!)
As for the kids being "dragged backward and forward", that is your take on things and if you show them how upset you are, of course they will feed off that. There is no need for it to be as traumatic for them as you are making out. If you and your ex can afford a nanny, then you can afford for the kids to have enough clothes and belongings at each household, removing the need to drag large/multiple items between both of your places.
Don't martyr yourself - it is not fair to you, your ex, or your children. Rather, see what you can do to make things easier for them instead of insisting that their lives will be a misery now and forever after if they are not living with you full-time Monday to Friday.