I've done a few literary pilgrimages - Haworth (for the Brontes), Chawton (Jane Austen), Rye (E F Benson - annoyingly, the house was closed), Newstead Abbey (Byron) and Corfe Castle/Isle of Purbeck (ahem, Enid Blyton). Love it.
All these, except Newstead. Have also been to Batemans. And came off my bike (push rather than motor) just by Clouds Hill, too.
Grew up in the heart of Hardy country, so most of the Dorset ones aren't really literary pilgrimages - going past Max Gate was just part of going into town, and involved with stuff to do with William Barnes just because of being local. And now also lots of Austen stuff because of where I live now.
Have been to Hay a few times - have family nearby, who are Friends, so I just let them know which sessions I want to go to, and they put me up and sort out the tickets, which is jolly good.
When I went to Australia, I was determined to go to the Snowy Mountains because of reading Elyne Mitchell's Silver Brumby books - I did see wild brumbies up Mt Koscuisczko, though not silver ones.
I went to Louisa May Alcott's house near Boston. And Anne Frank's House in Amsterdam, more because of reading it than the history, I think.
I'd also like to go to Prince Edward Island for Anne of Green Gables, and do a US trip for Laura Ingalls Wilder.