I've lived in Greater Manchester for 20yrs. Previously lived other parts of the UK (including London) and overseas.
Manchester is not a particularly good looking city, nor does it have much 'touristy' stuff. It's not somewhere most people would go for a trip to have a mooch around in the way that one might in Edinburgh, London, York, Bath or even Liverpool. It's similar to Leeds or Birmingham in that sense.
What it does have is a fascinating and proud social and industrial history, brilliant universities, fabulous cocktail bars and eateries, some excellent museums, a deep culture of popular music, sports, creative outlets and projects, some of the finest Victorian architecture in the UK and some of the most exciting urban regeneration in the world. It's a city that brings together people from all over the world and welcomes them as Manc. It's a city for those who live here - or want to live here - rather than for visitors. The Peak District, Bowland, the Lakes, Snowdonia and the North York Moors and Yorkshire Dales are all within a couple of hours' drive (parts of Greater Manchester are basically in the Peak District albeit not in the National Park).
It's a brilliant place to live for the reasons above and more and it objectively isn't awful. Certainly not to everyone's taste - but not awful.
Unfortunately you're in an area of the city that's frankly a bit shit. Your experience is a bit like getting off the train at King's Cross in 1995, walking over to Dalston and as a result thinking all of London is awful (I say 1995 as London has changed beyond all recognition in the last 25yrs or so with many of the historically shoddy bits being gentrified so there's not too many contemporaneous examples).
I'm glad you enjoyed Elizabeth Gaskell's house. If you have time, head over to Ancoats or the Northern Quarter or even over to places like Beech Road in Chorlton where you'll see a bit more of the vibrancy of the city