Been an English Heritage member for 20 years. Not many castles or abbeys scare me, although my husband saw a faceless woman wearing a long cloak, at Sudeley Castle, or rather in the woods there, in the 1970s.
Although one of the places mentioned several times here, I can't name because it is too outing, I was in the garden there very early one morning before it opened to the public, with my late friend, (we were doing living history) and it definitely felt like we weren't alone. But very benign and not 'creepy' at all. We both felt utterly at home there - at the same time, I think we both felt there was something slightly off kilter about the place. But garden, not house.
Another vote for Bolsover Castle. One of the few that crept the bejaybus out of me - one particular room, with, IIRC, a window-seat. I have been back recently and the second time, felt nothing.
Another place that always shook me up for no really obvious reason - Witley Court, near Birmingham. Just felt 'alive' to me.
Years ago, not doing living history, but we went a lot to a certain castle on the English/Welsh border. I had this horrible feeling near the woods nearby. Just always felt like I was going to stumble on a body, there. Family went to a re-enactment there, some time later and a young girl hung herself in the same woods, that weekend.
Copper beech tree in the gardens at Lotherton Hall, near Leeds. I saw an Edwardian looking little boy under that tree. Got closer. No-one there.
Clifford's Tower has no effect on me. Husband knew someone who used to work there and she told him when they closed up at night and the public gone, they often hear running footsteps on one of the spiral staircases and knew there was nobody else in the building.
There is a church in the parish where I live so I can't name it, that I can't step foot in.