DS read the first four at age 6.5-7.
Strangely, he finds books two (Chamber of Secrets) the scariest.
Having talked to many children, I realised that kids tend to not find the dementors as scary as grown-ups do (not in the books, at least). The whole 'sucking all hope and love and joy out of your life' thing feels very dark to us, but for kids they are often 'just another monster' IYSWIM.
Also, in book three there is no Voldemort. And there are some really good things that happen to Harry (e.g. discovery of his godfather). So, overall it is not too bad.
I wish I'd held off on The Goblet of Fire for a while longer. This (book four) is where it really starts gettin dark, with the first death of a 'goodie', and a child to boot. Also, there are some more teen-relevant themes such as budding romance etc which just passes the younger readers by.
So I'd say, book one is fine at age 6, book two can be quite scary (basilisk, Voldemort trying to control people through Tom Marvolo Riddle's diary, persistently high level of threat, Harry being blamed for things, also topics such as 'mudbloods' - discrimination and hate crimes), book three is less scary for kids than for adults, but book four really steps it up with violence and senseless killing. After that (books 5-7) it gets darker and more grown-up by the chapter.