I lived in America (Texas, then briefly DC) for many years, and I have spent quite a bit of time in New York, so I hope I can offer helpful info. Cars are not handled by the federal government, so laws and requirements will vary by state, county, and city, but they are generally quite similar across the nation. Pretty much everywhere, however, obtaining and registering a car are both complicated and painful processes.
You will need to consider where you will store the car when not in use (if you are in or near the city, or go into the city) and that can be very expensive. In general, simply having a car in NY is prohibitively expensive.
Parking a car, wherever it is that you drive it, will always cost something if you go into NYC. It's very difficult to find free parking anywhere. You will have to pay to park at home (unless you live in a detached with a driveway and/or garage), you will have to pay to park at work, and you will have to pay to park everywhere else you go.
Don't buy anywhere near NYC. The price of a car itself is much higher in and around NYC, as most people there simply don't have them. If you can manage it, drive well outside the city (perhaps upstate New York or even Pennsylvania or one of the surrounding states) to buy or even lease a car. The further you get from NYC, the lower the prices (which goes for just about everything as well), and IMO it would totally be worth a 3-4 hour sight-seeing drive in a rented car to save several thousand dollars. Depending on the car, you can find them for 5 or 10k less outside of the city.
There is always the option to lease, rather than buy outright, which is something like a 3 or 5 or 7 year term of making payments that are basically what you'd pay to buy, only you trade it in for a newer model at the end rather than pay it off and keep it. Those who always want to have a brand new car often go that route, as they'll always have a car payment anyway. The prices on leasing will be higher nearer the city as well, so if you can manage it at all, get far away from the city even to lease.
I also recommend buying USED rather than brand new. The second you drive a car off the dealership's lot, it's worth thousands of dollars less. You can find fantastic deals on new, reliable, low-mileage cars from car hire companies like Hertz, Enterprise, etc., because they sell as soon as they reach about 30-40k miles for a nicely reduced rate. There are also reliable resale dealerships like CarMax which do extensive testing, offer warranties, and are much more affordable than brand new cars. Also, if you buy a used car, the inspection is good until the sticker runs out, so that's one less thing to do right away. Used car dealers have a LOT of wiggle room on price, so never, ever pay the full sticker price. Ever.
---> If you buy from a dealership (new or used) you must haggle, HARD. It's absolutely harrowing. Car dealers are extremely slimy and will take you for everything you've got. They will pressure you, they will be rude to you, they will belittle you, they will laugh at you, they will intimidate you, they will frighten you, they will pass you around, they will outright lie to your face, they will keep you in a little room and refuse to let you leave, they will leave the room repeatedly to "talk to their manager" while just trying to make you sweat, they will use police interrogation methods on you, and they will just generally bully you into paying the most they can squeeze out of you. They're notorious for this! The MSRP or sticker price is ABSOLUTELY NOT the price you pay for a car, ever. You should expect at LEAST 10% off that price, if not more, and the more cash you have to put down, the more they should take off the top.
---> In fact, I HIGHLY RECOMMEND taking a local friend who has experience buying cars WITH YOU, and letting them do as much of the talking as possible. Honestly, act like you don't speak English or something, as car dealers (especially used car dealers) are some of the pushiest people in America, and buying a car is genuinely one of the most harrowing experiences you'll go through over there!! I really cannot recommend strongly enough that you take somebody with you who has experience with car dealers!!
You are in for a bureaucratic nightmare. Whether you lease or buy, new or used, you will have to carry insurance, register the vehicle, obtain licence plates, pay property tax annually (may coincide with registry), undergo inspection, and obtain a driving licence. Insurance can be bought from any company that will cover you. You can have the inspection done at any garage that offers inspections (they will have a sign outside). The rest, however, cannot be done anywhere outside the city or county in which you live. Also your licence will double as your identification card and is accepted as such everywhere.
Carrying liability insurance that covers any damage to others' property that you cause is compulsory by law, and you will be ticketed or fined or have points removed if you are caught without insurance, but if you can afford it, full coverage is always best. If you buy a brand new car, or if you finance at all, the lender will almost always require that you carry full coverage insurance. You will be responsible for finding an insurance company on your own, within [a certain number of days] of the purchase.
Insurance can be obtained online fairly easily, but it can be a crapshoot. If you carry home owner's or renter's insurance, ask them if they will do a bundle. Otherwise (and even if you do already carry insurance), you'll want to do lots of research on insurance companies. They will all offer basically the same coverage, but their prices, discounts, etc., will vary. You are also able to haggle with these people somewhat and ask for discounts, etc., however you may get shafted because they "cannot" access your UK driving records. It could be a pain in the arse, due to just having arrived there.
Your car will need to be registered and taxed with the city or county annually, and additionally you will have to take it for an emissions/safety inspection annually as well. It's expensive, and it can even be more painful than the buying process. You may need to take and pass a driving test to obtain a licence, though it's possible that you could simply "transfer" if you've got one now. Check with the city and county websites to find out what you need, and where you've got to go to complete it. Find out how much you can do online, and DEFINITELY see if you can "check in" or "get in line" online, because... well... see below.
Renewing your licence, as well as renewing your registration and tax, can be done online subsequent years, but I'm pretty sure you will have to go in person the first time to get the licence (and possibly take the test), register the car, prove the inspection, anything else required and to prove your identity for each. Each of these services may require you to visit different departments in different locations, as well. The DMV is another notorious butt of American jokes as it's the picture of bureaucracy and red tape, so expect to be there all day long, just sitting and waiting, and make arrangements for the DC as such. It is a long, slow, totally unforgiving process that often requires you to start at the very beginning for one paperwork mistake, and the people who work there are usually rude and terse, indifferent to your troubles and aggressively apathetic, so read everything as carefully as you possibly can, and genuinely plan to be there for several hours.
Look online to find out which offices you need to go to, check their office hours, AND check with the county AND city well in advance to make sure that you have all the necessary documents in your hands and every scrap of information and identification you'll need before you go to the building. Then ask locally if there is more than one location, and, if so, whether any particular office location that is quicker or less crowded. There may be only one place in a county where you can get each of these things done, so it's always overflowing with people, but even if there are more, they will still all be crowded. Some people actually line up hours before the place opens, because if it happens to be particularly crowded their number may never be called before closing and must return the next day. Some locations have a longer day each week, where they stay open an hour or two later. ALSO! Depending on the city and county, you may have to visit two (or more) different offices to complete the process. In Texas, for example, you go to the county DMV office to get your licence, the county tax assessor to register the car, another office entirely to obtain licence plates for the car, and a garage to have your car inspected. It really is a nightmare!
Perhaps I missed it, but are you 100% absolutely positively CERTAIN that you really and truly need a car? You are moving to the only area in America with good public transport, where most people do not drive, so it's really a massive expense. In fact, in the City, they actually (passively) discourage car ownership. Less than half of New Yorkers even own a car at all, as the public transport there is world class and it's a city built for walking and there are plenty of other options like day-rentals and as-you-go borrowing companies. It's entirely possible that you simply won't ever need a car in that area, and if you're not absolutely CERTAIN that you do, I would highly recommend trying your best to get by without one.
I apologise for the long post, but I do hope it's helpful. I'll post again if I think of anything else, and I'm happy to answer any specific questions, too! I wish you the best of luck (and the best of patience!), and congratulations on the move! I hope you and the family enjoy it!