My concern would be that headphones, by their very nature, deliver sound directly to the ear with an intensity that the human ear has simply not evolved to cope with. Sure, you can limit the risks by lowering the volume and limiting the exposure, but in practice that is not how most people use headphones.
Sounds below 70 decibels pose no known risk of hearing loss. That level is usually described as similar to the noise level you get when driving an average car down the motorway with the windows shut. Any sound above that level has the potential to cause hearing loss, depending on the length of exposure and the intensity of the noise. There's also some level of individual variability, which isn't fully understood. Essentially what happens in this type of hearing loss is that the tiny hair cells in the inner ear, which pick up sound waves and transmit them to the brain, are forced to vibrate beyond their natural limits. With repeated and prolonged exposure to damaging levels of sound, some of them will die off, much as if you stretch a metal spring beyond its natural elasticity. There is also a connection between headphone use and tinnitus, although the mechanism is not well understood.
I would be very surprised if the decibel level delivered by a set of headphones, particularly when listened to against any kind of competing background noise, eg. in a car, did not exceed the safe level by some margin. The 'in-the-ear' iPod type ear-buds are known to be more damaging than conventional headphones, but I suspect that in the real world there is in fact no safe level for headphone use, whatever SONY and co. tell you in their publicity material.
This is a bit of a hobby-horse of mine (and of most other people working in the field) simply because there is so little accurate information available to consumers. Compared with how much publicity there is about eg. the very small and poorly-understood danger of vaccine damage in a child with no known risk factors, I am constantly amazed by how cavalier people are about exposure to hearing hazards where the probability of damage is much higher and the mechanisms are very well understood. Just because the damage is not immediate and not dramatic doesn't make it any less real.
I have to use headphones for work some of the time, but I am very careful to limit my exposure, and I would never use them for recreational use, eg. to listen to music. Given that children are probably potentially more vulnerable than adults, I can't see any reason why I would want my children to be using headphones.