I get this too Mosschops, my DS likes Daddy and Nanny waaaaay more than me. Why? Because I'm the one with broccoli, and bedtime, and teethbrushing, and time out, and all the other stuff he hates. They have Haribo and falling asleep in front of cartoons. They're a treat, I'm a parent. And hopefully in twenty years time he'll realise this and know I did the best for him.
Play the long game OP. If this is the same sort of situation, they'll thank you eventually.
It does depend a lot on how old your son is though. Over the next few days, you will probably find out a bit more of what went on, what was said. If ds is pretty young, try not to take it too seriously; two weeks is quite a long time for small children, and - from what I've seen with friends in similar positions to yours - it's as if the child is blaming the mum for not being there. It can take a while for a child to get used to being with the other parent, and not having the resident parent around.