I don't think the Slytherins were the only mean ones though.
Luna had her stuff stolen by the other Ravenclaws and was called Loony by kids from all houses.
The Hufflepuffs in particular were horrible to Harry first when they thought he had opened the chamber of secrets in Book 2, and then when he was chosen as the other hogwarts champion for the triwizard tournament.
As for the Gryffindors, Hermione literally disfigured Mariette Edgecombe when she ratted them out to Umbridge, Fred and George who were about 16ish at the time) hissed at first years getting sorted into Slytherin, plus James, Sirius etc bullying Snape when they were younger.
Basically they all have the potential to be dickheads in some ways, because they're teenagers!
OP: re Hermione being a genius, I don't know, she just reminds me of the kids in school who did well because they studied a lot - they obviously were clever and also put a lot of effort in, so were good all rounders but there was nothing that made them exceptional.
The people I know who are what I think of as genius level talented
a) tended to excel and I mean really stand out, in one or two particular disciplines and
b) didn't need to put loads of effort in, they could sleep through lessons and then knock out an essay in ten minutes that got amazing marks. If the boundary for an A* was, say 85% they tended to not see the point in putting an hour's work in and getting 100% when they could put ten minutes work in and get 87%.
All the really clever kids in school never bothered volunteering to answer questions etc because they were so far ahead and found it so easy they weren't paying attention. A lot of the time the cleverest kids were the ones misbehaving because they were so bored. Or they'd come alive in the lessons they liked and not care about the others. They were often quite quirky as well - like Luna.
For example when Harry gets better marks than her in potions because he follows the Half Blood Prince's changes rather than the way the book says to do it - it's the difference between getting decent marks because you follow the rules exactly and experimenting/being passionate about/exceptional in a particular field (like Snape in potions).
Obviously she is clever, it just comes across as just normal highest set in school clever to me, whereas I imagine Ravenclaw to be where the musical prodigies/people who can do mathematical theorems in their head/speak ten languages/create new spells, etc. type of people go.