We've just done 5 days with kids and it was amazing. Our itinerary was
Day 1. Times Square, Ellen's stardust diner, shopping, Bryant park which is amazing for chilling and playing all the free board games
Day 2. Tenement museum, highline, breakfast at the end of the highline, meat packing district, soho, south street sea port, Madison square park
Day 3: Empire State Building, Bloomingdales, bike hire, 2nd avenue deli and riding round Central Park
Day 4: trip to jersey gardens mall and dinner in midtown
Day 5: ground zero, freedom tower, 9/11 museum, staten island ferry to see Statue of Liberty.
We could easily have managed a couple more days and would have spent more time in the village and gone over to Brooklyn. It's an incredible holiday to do with the kids