I do know of one family whose children are strictly limited with regard to 'screen time'- I believe it's 30 minutes per day. Those children are incapable of amusing themselves outside of that time- their lives are as strictly managed and the result has been that they now expect to be entertained, and will follow you about the house whining that they're bored and asking what you're going to do for them.
No doubt they're a rarity, but it isn't a strategy that's ever appealed to me. The TV is on a lot here, and ds2 (now 11) has always had free access to it. He learned colours, words, foreign words and all manner of other things from children's tv when small. Obviously it wasn't the ONLY thing he ever did- days out, walks every day, family meals at the table, games, chores etc were all done as well, but outside of those things I don't believe limiting screen time is that harmful.
However, as with most things, it's how you use it. If it's the ONLY thing your kids do, then it's going to be harmful, no doubt, and my guess is it's those parents that the guidelines are aimed at.
Like with a lot of things (chocolate/sweets; soft alcohol for teens and so on) I think if you strictly deny children those things they can develop an unhealthy fervour for them in later life.