You're not "overreacting" at all; that comment doesn't even make sense. You're not saying you're going to report this man to the police or start a social media campaign against him or stalk him and exact revenge, just that you won't see him again. I wouldn't either.
But I expect you won't hear from him as he's most likely a one-and-done, conquest-driven, bedpost notcher type - and if you do, I'd assume he's hit a dry spot and you're better than nothing. But you'd be setting yourself up for a miserable time if you see him again knowing how unsettled and disappointed you felt the last time. The absolute nicest and most benign interpretation of this man's behaviour is that he doesn't give a damn about you and simply does what he wants. It doesn't sound like that's what you're looking for from dating, so put yourself first and don't give him any more mental energy, whether he contacts you again or not. You don't owe him anything.