Areas I personally think are rough, or just not very nice looking, are: New Whittington, Grassmoor, Grangewood, Mastin Moor, Derby Road Area (though there are some nice housing estates off of Derby Road), Stavely, Holmewood.
Nice areas: Wingerworth, Walton, Old Brampton, Cutthorpe, Hollymoorside, Ashgate, I quite like tapton, parts of North Wingfield are nice.
Mixed bags: Brimington, Newbold (upper Newbold is nice), Calow, Hasland, Hady
Ashover is lovely and very close to Matlock and is truly rural, but it will be more expensive. It depends whether you want a true rural experience in a little village, or whether you're happy with a larger town which is near the quaint places. Barlow again lovely but more expensive.
As for surrounding areas, I personally hate Clowne. Bolsover is cool as it has a castle with great views, but I can't comment on living there. I personally like the look of it, though some say it never recovered from the mines shutting.
Harthill which is Sheffield way is nice. I really like Dronfield and think it's a cute little place, but can't comment on the specifics of living there. Eckington/Mosborough areas are quite nice.
Like anywhere, it has nice bits, and it has some true dumps lol. There are some very run down bits, but they're not really comparable to the run down areas of major cities.