OK - here are some ideas.....
For coffee, there are the obvious Starbucks. Think there are two between Sydney Street intersection on King's Road and Sloane Square.
You could also try the cafe in Partridge's located in Duke of York's. I've never seen it crowded in there, though must confess have also not been during peak hours. Think there is a Patisserie Valerie at Duke of York's too.
You could try the ground level cafe at the Bluebird (not the upstairs restaurant), which serves decent food, and should be fairly laid back about buggies.
Another place I've never tried, but have heard Mums talk about, is a place called Munch. It's located upstairs (either above Heal's or Habitat, can't remember which). The entrance is marked and is on the street level next to the movie theatre near Waitrose. Think there's a lift too (possibly inside the shop), so that you don't have to negotiate stairs with a buggy.
There is also the Pizza Express just across from M&S.
Hope that gives you a few ideas. Don't know much about Fulham or Parson's Green, so hopefully someone else will respond about those areas. Have fun.