I sympathise. But I'll just give you the other side.
I have a daughter, she's 2 and a half. I love her dearly. When I split with my partner I went from seeing her every day but two in the week, to an afternoon in the week for a few hours and alternate weekends. It took three months before my XP 'allowed' me to have DD to stay overnight. There was no reason not to, I wasn't abusive or anything like that.
Finally having my daughter to stay was one of the great moments of my life. There were a few brief occasions where she asked for mummy's home, but overall we had a great time together. It was wonderful. My daughter really loved her time there.
When I see DD in the week, I chirpilly as possible say I need to head back to daddy's home, with out fail her little face drops and she says ' daddy go home not ' . It takes a lot to reassure her I'll see her again, and not just burst into tears, which I'm not really prone to do as a grown man.
I spend entire weeks rattling around the house wishing my little girl was jumping around on the sofa, or tucked up in bed upstairs.
I do sympathise with you, I know it's hard for mums to not have their children around. But be positive. You'll find you appreciate the time, and you're doing the best for your DD