Your own son must come first. Whatever the reasons behind it. If he has been shouting at a 2-year-old and especially if you have reason to believe it has been physical, then yes, you are entitled to say that your home must be a safe environment for your toddler.
I think it would be good for your SS to have the Saturday morning 'alone time' with his dad - I would say the same if he was not a stepkid - I think it is important for all kids to have one-on-one time with each parent. BUT that has to be in balance and it should never be presented as a reward; I would also strongly suggest that your DH make sure their 'alone time' is spent doing normal sorts of things, not special treats and ever-increasing activities! They could go for a walk or a ball game, or even just running errands together - it should be about the time to interact with dad as he would if they were together in a household, not about your husband entertaining him.
I don't think, though, that this is the whole solution, in fact it may be a completely separate issue! Your SS also needs to know that his behaviour is not acceptable, so DH needs to be giving him consequences when he acts out with anyone in the family. He is old enough to understand, and he is old enough for it to be something you and DH address within your home, regardless of how his mum deals or doesn't deal with it. Have you tried things like reward charts? What punishments does DH give him now when he acts up, swearing and being loud?
If your DH can't resolve it through ordinary means, and the behaviour continues to escalate, then by all means you can insist on other arrangements - but that would mean perhaps something quite extreme, like DH having to spend weekends in a hotel or at least agreeing to keep the children separate on SS's visits, with your DH being the one to make the adjustments rather than you or the small children being made to feel they cannot be safe at home.