I have a mental Literary Shitlist just for conversations like this 
There are spoilers in some so I've bolded the titles in case people want to avoid them.
The absolute worst book I've ever read is The Fountainhead - I would be quite happy to see that book slowly and methodically tortured to death before being cast into some kind of space-time slip that erased the possibility of anything remotely similar ever being conceived of throughout the entire span of humanity's existence. I once made the mistake of accepting a friend's offer to lend it to me, only to discover that she refused point blank to take it back before I'd finished reading it. I had that book for seven years before I gave it to one of those free book places, just so I could rest in the knowledge that it wasn't taking up precious oxygen space in my house anymore. On reflection I wish I'd recycled it, at least that way it might have been useful to someone, somewhere.
Before I discovered that one my least favourite book was The Road by Cormac McCarthy - the only other book I've ever literally thrown across a room through sheer infuriation. It was highly recommended by someone whose taste I normally love, so I really wanted to like it, but the plot - at least of the first 70 pages, which was as far as I got - can be summarised as 'father and son use 500 synonyms for grey with made-up words and pretentious lack of punctuation to symbolise post-apocalyptic dystopia; occasional cannibals'. However, after reading The Fountainhead I feel slightly less inclined to hate it just by virtue of the fact that it's only enormously irritating and not a giant steaming turd-maggot of narcissistic literary masturbation, so there's that.
Unlike the two above I actually finished The Dice Man - I couldn't imagine a book where the 'hero' rapes his downstairs neighbour within the first 30ish pages (it's ok though, she secretly wanted it all along!) being quite so popular, so I kept going in order to reach the point where he was revealed as a shitdick and got his comeuppance. Turns out he was actually just a straightforward hero all along and this was being presented as a valid life strategy. Ok then. This one isn't quite as bad as the above, but it sticks in my mind because it has the worst concept:execution ratio of anything I've ever read - even among the people who like it there seems to be a lot of 'well ok so it's a bit shit, but you have to admit that the premise is interesting'.
Most (not all) of Ian McEwan's books are mildly infuriating, mainly thanks to the fact that they're actually quite good until he ruins them with shitty contrived endings that he seems to think are clever. Amsterdam is a decent book until the two protagonists simultaneously get it into their heads to murder one another by manipulating Dutch euthanasia laws, all because of a paragraphs-long misunderstanding of where exactly the emphasis goes in a sentence on a postcard. Saturday is similarly good until the psychopathic would-be murderer is spontaneously reformed by some middle-class twonk reciting poetry and the beauty of scientific progress. Sometimes I wonder whether McEwan's ever met anyone who's not an old white male literary genius, because NOTHING HIS CHARACTERS DO MAKES ANY FUCKING SENSE.
The only other book I disliked enough to remember now was West of the Wall by Marcia Preston - I feel a bit bad about that one as it felt far more well-intentioned than any of the others, but it was full of characters called Rolf and Wolfgang who spent every other paragraph being poor, wearing lederhosen and mournfully eating sauerkraut, and I just couldn't get into it.
(That was a LOT longer than I intended, but I feel much better now 
)