"I think some words become offensive simply because of the way that it has been developed by our culture/society."
Sassure's theory of linguistics says exactly that.
A word is made up of two components - the signifier (the spoken/written/signed word) and the signified (the concept/image that relates to). No word has an intrinsic meaning, it is all contextual based on the constructs of the society within which it is used.
So, for example, if I said 'ulotrichous', you might not know what to say, because you (may not) don't know what it means. It means "having very woolly hair". Now you know, if I say "The clown is ulotrichous", you have a preconceived image in your mind that the subject of importance is the clown's woolly hair. It doesn't matter that he had those big red shoes on, or that he has a squeaky nose. He has very woolly hair.
Now, depending on whether woolly hair is a positive attribute in your society, the sentence "The clown is ulotrichous" will either be met with an increase in your opinion of the clown, or an increase in your negative feelings towards him/her.
Same goes for some of the older words to describe people who are living with a disabling condition. "Invalid" - root word is Invalid, meaning not legitimate, null and void.
"Retard" - "A BBC survey in 2003 ranked retard as the most offensive disability-related word, ahead of terms such as spastic (not considered offensive in America[13]) and mong."
"# Retarded comes from the Latin retardare, "to make slow, delay, keep back, or hinder." The term was recorded in 1426 as a "fact or action of making slower in movement or time." The first record of retarded in relation to being mentally slow was in 1895. The term retarded was used to replace terms like idiot, moron, and imbecile because it was not a derogatory term. By the 1960s, however, the term had taken on a partially derogatory meaning as well.[16]"
Quotes From Wiki