The primary purpose of porn is to excite sexual arousal. The primary purpose of a fiction book is entertainment. Even one with graphic sex scenes, those scenes are part of the wider plot, and in most books will be one or wo scenes, something the book has built up to over the previous 300 pages, or less than 5%, maybe 10% of the text and that's not even true for something we commonly associate as being a sex focused book like 50 Shades of Grey. More time is devoted to the emotions and problems associated with the sex, than the sex itself.
While it might not be true for erotic fiction, for romance novels, romantasy, 'chick lit' etc is not typically used for arousal, the sex scenes are part of the greater plot and usually fairly realistic in terms of how someone in real life might build up to intimacy, having no sex scenes in books or years or yearning, prolonged eye contact and hand brushing isn't reality for most contemporary relationships and it would be jarring if books were written like our sense of chastity was still 200 years in the past.
Coupled with the fact that everyone who reads that book will experience the scene differently. If I asked everyone reading this to picture a old, slightly crumbling castle in an open space none of you would have an identical image. Some people won't even have an image, just a vague outline or see the words in their mind. The description has been used as a prompt for your own imagination, some of you may have pictured a different state of decay, different size or age of the castle, different features such as gardens, a moat, how many windows, details such as a courtyard, farm animals, horses, plants, trees, maybe your castle has a wood store or a blacksmith, maybe it's under attack or has archers stationed on it, characters in the castle scene or maybe it's completely ruinous and abandoned and so on and it's the same for every scene in a book.
I really don't think we can object to sexual images created in our own imagination, even if there is an external prompt such as a line in a song or a scene in a book on this basis. Even considering a film with a sex scene vs a porno in the same way, one has sex as an element of plot, a natural progression of the character interaction and development, sure it might be arousing for some people, but they're also unlikely to be playing Pretty Woman over and over again to get their rocks off, and the other doesn't have a secondary purpose like plot or education, it's just for arousal and is being used only for a sexual crutch.
Others have already covered also that there's no exploitation of women from written fiction, you know no one has been trafficked or has had their image put online against their will but I do think it's an important point in the arguement.