I get it - but honestly, it's just not been my experience at all - I've played in pubs (admittedly, a naice pub, not the one known for fights), a church hall, castles (that was LARP), obviously university as well as people's houses. My ex never played any of this stuff, but he's wandered around with the kids into board game shops and been invited to whatever they have that's child appropriate. Now he was a bit judgemental about the dressing up for LARP, but didn't care about the tabletop, and would have quite enjoyed Warhammer if you didn't have to paint the figures first (after all, kids have been playing with toy soldiers forever - it's just a riff on Risk if you think about it).
When I told my friends and colleagues that I was off for a weekend long game with my uni buddies they were all either amused (that a grown adult still did this) or interested - not one had any issue with it.
I've seen posts on the local facebook group asking for club ideas for more geeky children (where I live now most people are more involved with local sport clubs) - no Games Workshop here, but things like lego are suggested
I guess that's what makes life interesting and makes this all the more complicated - not only is everyone different, but the society around everyone is different.