In my experience, at 3yrs old, they probably won't be that bothered about the little buildings and extras - they just like to put the track together. Things like tunnels and hills are good, lots of interesting bits of track to make good layouts.
Having said that, you will be spending a lot of time playing with it too, and if the London buildings make you happy, then why not buy them! The bigjigs London set is lovely.
Bigjigs is compatible with Brio, but I would check that the Le toy van tracks are compatible with other brands.