No you are not over reacting.
Imho, and I am sorry this sounds preachy but I do feel rather strongly about this.
You need to come down hard on him NOW and in a way that he will not mistake the message.
This what I would say -and have said to my son...although I said it to him when he was 5 and his sister was 4 (they are now 16 and 15):
-
This behavior to his sister has to stop now! Absolutely prohibit abusive behavior under your roof.
She is a member of this family and has every right to live here in peace without anyone harrassing, bullying, teasing, or any other form of abuseing her over anything. The home is her sanctuary just as it is for him and you.
If he is feeling frustrated or bored or angry he will dispense with those feelings in a way that does not involve his sister in any way, shape, or form.
If he persists, and he will not take direction from you on this reformation of his social behavior, then you will seriously find someone who will: military school.
You really need to stick up for your daughter. It is very very important that you do.
It is also doing your ds a huge favor in informing, teaching, making him understand that treating girls like objects (just like dad) is terrible and horrible. This isn't degrading his father, it is discussing the reality that his fater's behavior towards women/girls, and thought processes that governed the behavior, are very hurtful therefore very wrong. Thus your divorce-have you explained the divorce to them yet? Maybe you should.
If he wants to go live with his dad, I would seriously consider it to spare your daughter. But as you have already guessed, he probably would develope into an emotional abuser as well-it does sound like he is on his way. I know you don't want to give up your son, but if he is EA then you come to a point of just not having those kinds of people in your life- even if they are your family.
Read Lundy Bancroft's book "Why Does He Do That?" and have your daughter read it so she won't think her dad's/brother's behavior is "normal" from guys thus get herself into relationships with abusers.