It never gets right but mostly I can let it go.
My 'kids' are now 40 and 42. I split with their dad when they were 6 and 8. For a long time I bit my tongue when they 'felt sorry for dad 'cos he doesn't have any money'. As teens, they got his measure. As adults, he remarried and became more stable and wanted to spend more time with them.
Now, most of the time, his wishes override mine because 'he gets upset' when they don't jump to his tune; buy his wife a Mother's Day card. See him on his birthday, her birthday, Christmas etc. etc.
I know they care about me more than him. I know I am the one they want when they are upset, angry, stressed or whatever. AND they tell me first when something to share.
However;
They do love him and they go along with his wishes because it's easier not to upset him, whereas they know I don't make demands or have expectations.
They know my love is unconditional - they still feel that if they don't pander to him, he will let them go again.