When you arrive in the US, you need to go to the local social security office armed with your work visa, passport and anything else you can think of. You fill out the forms there and they process for them in a few days (you can phone up to find out the number before the actual card arrives).
You will need to sort out a driver's license really quickly - the moment you know your Social Security Number. You will have to take a written test and a road test; you will be fine to use your British license until then.
Once you have a Driver's License, you are almost a real person - it is much easier to open accounts, get a mobile phone, etc. with a license.
Expect some difficulties trying to set things up without a SSN, such as renting an apartment and getting utilities connected. If your employer is willing to do anything like this for you, jump at the chance. Banking also requires a SSN, so make sure you have a source of funds to tide you over for a couple of weeks. It's a good idea to make sure that you have internet access to your UK account, and that your bank card will work in the machines over there (there shouldn't be a problem). You can always withdraw money and use cash, which is always welcome.
Taxes are all linked to the Social Security number. You have to pay Federal, State and Local (city and school district) taxes. Taxes are all based on the calendar year, and are taken directly from your paycheck. You have to do tax returns each year, with the Federal one being due 15th April. The first year can be really hard when you are doing your tax return because it is split year and includes your relocation costs. Keep receipts for anything you think will be tax deductable. A lot of Americans pay someone else to do their taxes, but it's getting more popular to use a software package - I use Turbotax for our US taxes and it works well (note to self: 15th June deadline for overseas residents looming fast).
The other major bit of bureaucracy is health insurance. You need to know what your employer will give you and if you need to buy insurance yourself. It is not cheap, but there are ways where you can just buy emergency cover, and pay for individual doctor's and dental visits as you go, etc.