Despite my earlier post, I'm going to go against the no-sayers and say $170k with only 2 kids is absolutely fine. No, you probably won't be able to live in Manhattan or Brooklyn, especially if you are sending your kids to public (ie State) school, but there are a gazillion and one places you could live that would give you a great experience and you could afford.
I personally think the commute from Millburn and Summit is life suckingly depressing, despite the fact the towns themselves being lovely, so somewhere out there would be up for your DH to decide if he could face doing that twice a day. The Westchester County river towns would give your DH an easy commute, and have a great lifestyle, restaurants and amenities. Take a look at Dobbs Ferry, Tarrytown (check the school allocation of properties there), Ardsley, Hastings on Hudson. If your DH is prepared to suck up the travel time, Pleasantville is lovely, has a great vibe and great schools. Also, Armonk. Your DH would need to commute to White Plains North and then drive the last 15 minutes so he would need a car. You could also take a look at Croton-on-Hudson.
I'm not going to lie, you're not going to live excessively on $170k. My eldest was 10 when we moved here and it's the little things that REALLY add up. DS used to go to soccer club in the UK (obviously it was called football club though!) and it cost us about 10 pound a week. Here we pay about anything from $300 to $1k, depending on competition level, for a season (ie spring, winter, autumn, not a soccer season). Your youngest won't be old enough for Kindergarten so you'll have to pay for his pre-K program. And then OBVIOUSLY, you have to suck up your health insurance. We have great insurance but it still cost us $2000 when DS broke his leg and $1500 when the school sent one of my kids to A&E because they thought he'd broken his leg and he'd only twisted his ankle....and another $500 for the ambulance they sent him in! not bitter at all
Anyway, I feel like I'm going on a bit now! Feel free to PM me if you want any more local info.