but sometimes they ARE in the wrong!
Sometimes people do take offense where none was meant, either because they over-reacted, or misunderstood, or have some communication difficulties (for example autism can sometime make it harder to understand sarcasm or nuance) or there was a different cultural context/language barrier, or because they want to make a point, or whatever.
You can't say that the person making the comment is ALWAYS, "in any situation" automatically in the wrong no matter what, regardless of how innocuous the comment was.
You can't say you'd agree you were in the wrong if you said 'Good morning' to a colleague as you walked into work and they burst into tears and made a complaint about you because they'd just heard a family member had died so it wasn't a good morning for them and so you shouldn't have said that?
Or what about if you used a word that sounds offensive in other languages and a colleague of a different nationality took offence? e.g 'shabby' apparently sounds like the term for 'stupid cunt' in chinese, or 'cookie' 'small penis' in hungarian. If you said to your colleague 'Do you want a cookie?' and they made a sexual harassment complaint about you, would you really hold your hands up and say 'Of course that is completely my fault, I should have known how every single word could be interpreted in every other language in the world, I will never use the word cookie again, in fact I'll never speak in front of anyone whom I can't guarantee might not have the exact same understanding of language as me.' (which means I can never speak).
If offence isn't intended then the offence does originate from the person who has interpreted it as offensive. It might be an honest mistake but that doesn't mean that the speaker has to take responsibility for it.