When are you going? With or without kids? The autumn months are lovely and it doesn't usually get too cold until late in December. Although I believe this winter is expected to be quite snowy.
I actually like the statue of liberty/ellis island crossing, although a lot of people think it's over-rated. It's true that you can get just as good a view of the statue from the staten island ferry but on liberty island you can really get a sense of the scale of it. And the Immigration Museum on Ellis Island is very interesting. The Top of the Rock is indeed much better than the Empire State Building, with significantly shorter queues, but the ESB is a wonderful building. If you do do the ESB, if it's at all windy make sure you have warm clothes for the top. Otherwise you will spend 3 hours queueing for the top only to come down again after 5 minutes because you're freezing!
If you like art galleries, MoMA and the Metropolitan Museum of Art are both amazing, and the Frick Collection is also well worth a visit (doesn't allow kids though). The Guggenheim is less good, although it's a wonderful building, unless it has a special exhibition that you're interested in. Both the Met and the Guggenheim are right by Central Park so combine well with a walk around the park. Turtle Pond is one of the park's nicest features imo, and don't bother to spend much time finding strawberry fields.
Also don't bother going for a walk along Wall Street. Unless you are fascinated by the finance industry, but even then, there's nothing to see really. Imagine if you went to London and went for a walk along Fleet Street... like that.
If the weather is okay the walk across Brooklyn Bridge is well worth doing, and there are some nice green spaces being developed along the East River in Brooklyn that you can explore, and lots of nice places for lunch. And a carousel.
The tenement museum is fascinating, I strongly recommend it. You have to do a tour, and you can book online, but they have additional tours available if you just turn up I think.