First off, dogs these days break down into two major camps - working breed (collies, gun dogs etc.) who are bred to work and are only really happy doing it, and "companion" breeds, who are kept as pets to provide companionship.
If you have working lines dog, and do not keep it active and engaged (mentally as well as physically) you WILL get "problems" - though the issue is not with the dog, it is with the owner stopping it from BEING a dog as it has been bred to do (terriers will frequently want to dig and kill small stuff, collies will want to herd stuff, etc.). Stopping them from doing this will cause distress and "bad" behaviours (i.e. the dog gets frustrated normally) from the dog.
Companion dogs are almost a breed apart - the work ethic of a working lines Lab Vs a companion line (or shudder a "show line") Lab is dramatic and very visible. They are bred as canine "friends" first, and often for looks than for capability (thus the impact of line breeding, or major single source stud dogs, in the genetic makeup of the more popular breeds makes them a lot less robust)
The reason for the pet insurance is driven a lot by the companion breeds - if you have a "canine friend" (and let's be honest, dogs can be incredibly good at getting inside your emotions), you want to do all you can to look after them. There is SO much you can do these days that you could not do "in the old days" - pet MRIs for example, ongoing treatments where previously you would have the dog euthanized, etc. However, it takes money to do this, and thus pet insurance. Mind you - it is an incredibly good view of what the US health market looks like... ;-)
Overall - a) technology has improved, so you can DO a lot more, thus the need for insurance and b) the rise of companion dogs/lines as opposed to working lines, means how we treat (and what we expect from dogs) has changed quite a bit.