Read the story of the founding of the abbey (in v small print on left-hand side of big arch way inside). I've forgotten the exact details, but the founder built it in penance after punishing his brother for some crime by setting him out to sea without food or water to perish alone apart from his servant. Feel that was hard on the servant.
Another great walk - from the top of Bulbarrow, through DWT reserve called Green Something, then down into Milton Abbas. Bit of a hike back up, though.
Have you ever been to Salisbury Cathedral if you like abbeys? I love going round with the volunteer guides and listening to their stories. There's a great one about a skull, a rat, poison and a crusader. They didn't want to tell it to my class of 8-year olds in case it upset them (They loved it, of course)