It's funny, I was thinking about this the other day – I mean how we have a sort of relationship with a writer. Nabokov gave a lecture on Dickens once and he said "to enjoy him, you must surrender to the voice." It always stuck in my mind. I believe he said the same about Jane Austen. By voice, I guess he meant the personality behind the words.
I'm fond of Bertrand Russell's essays, for example (not so much his heavy philosophy, more his popular, lighthearted essays, on things like the joys of laziness, or what makes people boring, etc). Whenever I dip into one of his books, he's there – that clear, funny, tender-hearted voice. I feel the same with other writers. Whenever I pick up P. G. Wodehouse, Patrick Fermor, George Orwell (more the non-fiction), Bill Bryson, Douglas Adams, etc, they're there in the room, and it cheers me up. It's a cliche, but it really is like meeting up with a much-loved friend.
That said, there are other writers whose work I enjoy but whose voice/personality I dislike. Evelyn Waugh, for example, was a superb writer but a horrible man. I love his books. In fact, I'd have him in my top five or six. But every so often his nasty, sadistic little personality peeps through the writing. I like Virginia Woolf, but can't say I love the person. I feel the same about Anita Brookner, Anthony Burgess and Philip Larkin – love the writing, don't love the voice.