I grew up in a house in a much cheaper area. My parents got divorced when I was a kid and my dad ended up with the house (long story). A couple of years later he met someone else, started spending less time there until he just never bothered going back. No one did any maintenance and at one point there were squatters in there. Someone ended up buying it, knocking it down and rebuilding. So that's one way something like this can happen.
I'm surprised it would happen in such an expensive area but when a house isn't lived in or maintained, it can fall apart surprisingly quickly.