Don't worry, it's not catching and you're not offending or insulting anybody by being there.
Stay clear of any Alpha courses, though. They're fucking high pressure sales tactics hidden by 'we're not putting pressure on you, it's all you making your own mind up' and some free food.
I've worked in a lot of churches as part of my job, happily sing religious music in choirs and DP has worked in quite a few cathedrals. I'd be very tempted to get involved in the bellringing if I didn't like having lie ins on Saturdays. We don't feel the need to say 'This is all a bit bollocks, isn't it?' in the middle of prayers and, tbh, I love the architecture, the stained glass, the history and the feeling of being there.
One of the local Priests also says it would be stupid to remove people who didn't believe from church, as they're hardly likely to feel God's Grace whilst being ejected from the premises.
Go, enjoy a bit of singing, a bit of calm, something different from the day to day. make sure you have a cracking Sunday Dinner to come back to, though, as coming home to the smell of a chicken roasting in the slow cooker is amazing.