I think native speakers are more likely to make these kind of mistakes because they learnt to speak the language before they learnt to write it.
I, as a non-native speaker, learnt to write it first so the difference between have and of really stands out to me - and yes, it does make my eyes bleed.
(obviously as a non-native speaker I make other mistakes that native speakers wouldn't make).
Would of isn't the only one, there's also then/than, they're/their, etc.
I have noticed that, as I grow more and more comfortable with the language, I am more likely to mistype you're/your, etc.
But personally my pet peeve isn't even a mistake, it's the use of & instead of "and". I mean, it's a three-letter word, how lazy do you have to be to not be willing to actually write it down ?!
(to clarify : yes, I know I'm being unfair and there's nothing wrong with using the symbol. But it still makes me grind my teeth every time I see it).