The main issue is just being the mad bra and pants lady next door, but you might now care about that. I wouldn't be offended if. Saw my neighbours in their underwear in the garden, but I would find it hilarious and I would tell people. You'd go in the bonkers but harmless neighbours category.
Years ago we used to have odd neighbours who were obsessed with preventing us from seeing inside their house. If we happened to arrive home at the same time as them, they'd stand outside their door (not talking to us, because they wouldn't so much as smile and wave at us) and not make any attempt to open it while we busied ourselves getting the kids (and sometimes shopping) out of the car and into our house. They'd just stand there in the pissing rain waiting so we'd make sure to take our time. They were bonkers in other ways too.