Apple is watching you!
You just go to your iPhone Settings, then Privacy, then Location Services, then System Services where you'll see a very curious option to see your Favorite Locations. It is exactly what it sounds like: a map of the places you've visited most frequently, down to the (thankfully slightly inaccurate) street address.
Thankfully you can turn it off!!! Creepy!