We went through exactly this stage with DSS1 when he was 11. Every time he didn't get his own way, he was horrible to all concerned and demanded to go to his mothers. Three days later he would be back, when he didn't get his own way at her house.
What worked eventually for us was a combination of a united front - both parents refusing to tolerate bad behaviour at the other house, and ignoring the horrible things said, refusing to rise to them. He possible feels that life is a bit confusing, and is trying to assert some power over the situation. He's also trying to punish you all for his feeling that way. So don't let him. Be firm, and any painful things said, respond with a mild "yes dear"
Is it possible to sit down with your ex and get him on board? The "I want to go to my dad's, you don't love me" routine should stop if his dad won't tolerate bad behaviour at your house. A united front between all parents is important. If your ex is unreasonable, try to sell it to him as a way of improving DS's happiness, and ensuring that ex isn't next in the firing line.
I think letting him go to dads, while a welcome break from the stress isn't the best stratagy in the long term, as it is letting your son control things far too much. He needs to learn the boundaries, and the consequences of being horrible to people.
DSS1 grew out of this after about six months, after both parents were refused to allow him to control things. He returned to his lovely self, and was never trouble again, throughout his teenage years. He is 19 now, and a lovely considerate young man. So there is light at the end of the tunnel! Good luck.