Orlando - it sounds as if you are arguing that, because some distractions are unavoidable (children in the car, for example), we shouldn't bother to avoid any distractions.
I would argue the opposite - since there are some unavoidable distractions, it is our responsibility as drivers (in control of 1+ tonnes of lethal weapon, as Magrat says), to avoid those distractions - like using a phone, or fiddling with the sat-nav - that are avoidable^.
The bottom line, for me, is that I don't want to have to live with the guilt of causing an accident, and hurting other people, if I could have avoided it by a simple measure such as not using my phone.
I have been involved in an incident on the motorway. Luckily the traffic was moving slowly (40mph, or thereabouts), and I was paying close attention to the traffic around me - because I a) noticed that the exhaust on the car in front of me was flapping around in the breeze in a very hazardous way and b) had time to reduce my speed when the exhaust and catalytic converter fell off and came bouncing down the road towards me!
I ended up with it jammed underneath my car - the car behind me stopped in time (because the driver was paying attention), drove round me and went on his way. The van behind him also stopped safely - and then was rearended by the van behind. I have been told by the police that that driver was either too close, too fast or not paying enough attention to his driving - he will be appearing in court later on this summer, and I have been called as a witness for the prosecution.
If I had been on the phone, or fiddling with the sat nav - it could have been a much worse outcome. As it was, there was minor damage to my car (I needed a new tire, and wheel arch liner, and dh had to rebuild the front indicator housing with Meccano and superglue!