How often have they been back to visit your house over the years?
The one who used to live their said that you wouldn't have let them take a photo had they asked and that "the people who lived their before you were much nicer than you are" but how does he know?
Does he make a habit of coming back for a photo every so often? This can't be the first time if he can comment on the previous owners before you.
And I'm sorry, but that's weird. I still pass my childhood street every now and again, but I don't make a habit of driving down it to take photo's of my old house and make judgements on the new owners. This bloke sounds like he makes a habit of it.
And since they took the photo without coming to ask first, he has no idea how nice or not nice you might have been in comparison to the owners before you or how likely you would have been to agree to it had they asked first.
I can understand why someone taking a picture of your house might upset you, especially as you are pregnant and feeling emotional. My PILs spent months effectively stalking me, waiting outside the house, watching it from further down the road, driving around and around it in circles etc, checking where we were and what we were doing. I ended up being too scared to open the curtains in case they were out there looking in. Having some strange group of people taking a photo for no obvious reason would probably worry me as well.
But ignore what he said. He doesn't know you and he was the one being horrible.