I lived there for a while when we didn't have kids - so sadly I'm not as familiar with the kiddy elements. But, if you plan on living in Manhattan, then go for the Upper West Side. Loads of people with kids live there, it's close to central park and most everything to do with kids is located in that area.
If you can't afford Manhattan, consider Brooklyn. Also very family oriented, lots of lovely big parks. But you'll need to get the subway into the city (and if you're only there for a short time, I would recommend living in the city - you can't beat it!!)
Rentals in NYC are ridiculously expensive and you have to pay enormous agents fees on top of the rental. Trying to find something with a garden will be trickier but there are loads of people in NYC with dogs so they must be out there.
Applying for social security numbers: if you/your husband have a job, then you usually get a letter from them saying that you're there for x amount of time and you apply to the social security office. I can't remember how you do it but you'll get help from your employer. Get this first. Without a social security number, you can't get anything else - no rented accomodation, no bank account, no utility bills etc. Also, if you've rented here before, take references over there with you - sometimes they're picky about that.
Just about no-one cooks in NYC. They shop from small delis. Whole Foods is a fantastic supermarket for fresh, wholesome food. There are too many great places to eat out to mention but buy a Zagats guide and you'll be set.
Shops you have to go to:
Crate & Barrell
Pottery Barn
William Sonoma
Bed, bath and beyond (to kit your place out cheaply)
Warning:summers are ridiculously hot and humid and winters are so bloody cold you'll feel like your ears are going to freeze off (you don't go out without a coat, hat, scarf and gloves).
But NYC is just so fabulous - I am so jealous. You'll have a fab time. If I think of anything else, I'll post again.