We just got back from there with our 3 year old and 9 month old. Our DS had his 3rd birthday there.
We did Central Park and the zoo which he loved. He also enjoyed looking out for all the squirrels in the park.
On his birthday we went to the Natural History Museum as he's really into dinosaurs just now and he had an absolute ball!
We also did the Staten Island ferry (yes it is still free) and went up the Rockefeller Centre though to be honest he didn't get too much out of that.
You must go to Toys R Us - we were staying right next to Times Square so were in it several times. They have an enormous moving T Rex and of course the ferris wheel.
We were given a tip from a friend for a place to take DS for his birthday meal - Mars 2112 at Times Square. Standard American food but the whole place done out as if you're on Mars with guys walking round dressed as aliens etc. Pretty naf for adults but DS loved it and his eyes were out on stalks when the 'Martians' were walking around.
Mostly we just walked around taking in the atmosphere and he seemed to love the hustle. We ventured on the Subway once but we were with locals so it was easy and there were plenty of spare hands to help us with the pushchair etc.
Obviously our 9 month old DD didn't get a whole lot out of the trip but then again she didn't complain much either so I guess she enjoyed the experience.