I guess this is partly due to the wide range of terrain in the USA so you can ski, visit the beach, climb mountains or kayak etc as well as being able to access Canada and Mexico without one (I believe).
you absolutely do need a passport to enter Canada and Mexico.
The majority of Americans don't live in a city, so for most of them leaving the country is unlikely. It's more like saying that as a Brit you've never left Europe/Northern Africa, than saying you've never left the country.
Really? It's hardly a rural country. States which border the atlantic ocean make up about 40% of the population. My guess is most of those people live within a three hour drive of a major airport. That's just the eastern seaboard. Throw in California and Texas and you quickly get well passed majority.
Distance also isn't the only issue. If you were to drive from NY to Canada, as long as you know the difference between km/h vs m/h, you are not going to experience any cultural issues. Unless you are going to Ireland, you will face having to know a few words in a different language if you leave Britain.