We used to have this problem with a very difficult neighbour. The trees were ours, were over 100 years old, his house was built right next to them 25 years when they were only marginally smaller. He used to rant and rave about light, leaves etc, and basically wanted us to grub up the only privacy there was between us and him.
I should add that as they were so old, and were fighting for space, they had grown very tall and with straight trunks, so were not significantly overshadowing their modern split level already womb-like dwelling. Furthermore, the trees all had protection orders on them.
His suggested solution was to cut off all the branches on his side, from where they started around 10 metres up. We decided that would be a bad thing as it would destabilise the crown, and almost certainly lead to snapping an death of the tree.
We were at loggerheads over the damned things (pardon the pun). When we sold the house, and new neighbours moved in, he became inexplicably pleasant and chatty even though they have also declined to butcher the trees. I think that people too often fixate on trees, and or fail to notice them when they move in.
At the risk of sounding like a hippy, I don't see why people's short-term interests should override the tree's much longer term ones. So I say YANBU.