The 'people' factor is the biggest deterrent for me.
I like to 'look forward' to trips out but any pleasure from plans is eliminated by the 'what if there's a screaming child?', 'what if there's a selfish twat who talks all the time?'. It's not cheap to goto the cinema now so I'm not going to gamble on my experience being ruined by others' selfishness.
It's the same as going out for dinner now. I LOVE going out for dinner with my other-half but the stress I experience when I see there's a table with more kids than adults immediately makes my heart sink and I wish I'd stayed home*
*Yes, yes, I know! Some (many) children can behave in pubs and restaurants (and it's a topic as old as the hills) but the fact that so many cannot (or more importantly, will not be made to) makes the expensive act of going out for dinner a bit too much of a gamble for me.