Milk is not unhealthy at all. Most adults could drink full-fat milk and not suffer from it, unless they either consumed large amounts of milk or had underlying problems with cholesterol etc and so needed to really reduce saturated fat levels.
If you actually look at how much fat milk contains, for many foods that would be classed as low in fat. For example, a pint of whole milk gives 3.9g fat, of which 2.6g is saturates. The government recommends women eat no more than 76g per day of fat (20g saturates), and men 100g (30g saturates). So if the rest of your diet is good, whole milk is not bad for you.
I think it's really sad to see the 2 year olds at our playgroup eating low-fat yoghurts etc - then lots of chocolate and crisps! That balance just doesn't seem right.
In our house we don't eat much saturated fat, so I'm happy to give my children, and indeed ourselves, a mix of whole-milk and semi-skimmed milk and milk products. Whole milk also naturally contains Vitamin A, which semi or skimmed milk doesn't unless it's been fortified.
I have read lots of views by 'experts' claiming it is partly a con by the dairy industry to get everyone to buy the skimmed and semi-skimmed products which are actually a by-product of the cream industry. Don't know if that's true, but it's an interesting theory!