Sorry, but WHY can't you talk to the Hosting Mum? It's not like you are gossiping, spreading lies, this is not just a playground scuffle, this X is a danger!
AT 11, you can't be there all the time for him, he HAS to be able to stand up for himself, if that is what he wants to do.
My gut feel (but I'm not 11 ) is that he shouldn't put himself into a situation where he may be subjected to more death threats, more violence, and without any expectation of back up from his peers.
Unless the hosting mum can be told the truth, and your fears, and she assures you that a very close eye will be kept on X, then tbh, I don't think you can rightly let him go, it does seem hypocritical, and sending confusing messages. If you decide NOT to allow him to go, make sure you tell the host why, X doesn't need anyone enabling or covering for him, which is what you would be doing if you flannel over why your DS won't be attending.
I take it that the parents of X are not at all bothered about the fact that their precious little thug darling is going around battering other children??, and that perhaps as a punishment he ought to skip the party?????