Everyone's situation and perspective is very different, but I have recently gone through a situation of myself and my neighbours causing a big fuss a couple of years ago about my living situation due to a downstairs neighbour taking on the 1 bedroom flat that adjoined to my nice little maisonette in a lovely area.
It was bad and both myself and others did complain a lot about the neighbour and drugs, ASB etc, but truthfully I was also motivated to complain because I was sick of the property. I'd had a surprise third pregnancy, the house had no in built storage and one room was downstairs which we didn't make good use of so it felt pokey. We didn't have adequate storage furniture and we weren't making best use of the space at all, and the property needed repairs and new flooring desperately.
We were eventually moved and it was awful. I spent every day of the last year and a half wishing I had seen the benefits, realised what I had instead of focusing on the negatives.
After much begging and pleading and several strokes of good luck, I've been told I can return to my HA house and I'm so extremely grateful.
I know this post is very me me me, but I can't express in words the grief and loss I experienced giving up my secure and safe home. Yes it needed work, yes it wasn't ideal, but I've realised I could have done so much more to improve what I had. It's only actually going through with leaving and dealing with a much worse situation that made me realise how good I had it.
OP, given the current climate and given that you are in a densely populated area, and with the uncertainty the rent reform bill will bring for a while, price increases and such, I would be ever so careful about giving up a secure tenancy.
What if you were given a really similar property in a high rise but 16 floors up instead of being on the ground floor? What if you can't afford your new also mouldy flat and end up in a B&B for 3 years?