Hi mamachris. I have always found Brighton to be very baby friendly- there are so many babies and buggies around, everyone's very used to us so I don't think you'll have trouble finding somewhere to eat.
As for things to do, bit tricky with a 4mo but there are some lovely parks- there are gardens around the Pavilion which are very popular, and there's also a cafe there so you can sit out and enjoy the sunshine (hopefully). If it rains the museum is just there too and that's free entry.
You can catch the electric train along the seafront for a few quid, and that stops in 2 places- the first stop has mini golf and a water play area and then it continues to the marina which has a load of cafes on the Waterfront, cinema, bowling etc. The train station is just east of the pier- also a good place to wander around! By the way, it also takes you past the nudist beach so if you're lucky you might get a glimpse!!!
This Sunday is the London to Brighton bike ride so if you hang around on Madeira Ave (same place as the electric train) you can watch loads of sweaty people arriving!!!
Oh, and also you can take a number 77 open-topped bus to visit Devil's Dyke - this is a good place to visit if the weather is fine. (beautiful views of the downs, and you can even see gatwick if you look carefully) You can join at the main bus stops, including Brighton Pier, Churchill Square and the train station. You can enjoy excellent views from the top of the bus. The journey takes about half an hour and runs about once every half an hour during the busiest times of the day. There's a pub up there if you want to get lunch.
Well, I wrote more than I had planned! Hope that helps- have fun!!! I'd say watch out for a woman with hair like a yeti, but I'm going to be in London for the weekend!