Op you have had a bit of a kicking here by some posters. Poor show, especially in light of what you are going through.
I am a landlord, and yours sounds v unprofessional, especially the comments about getting the regulars to spy on you.
I do think that a pre end of tenancy inspection can be beneficial as it can head off problems and set expectations, but if this is what he intended, he hasn't gone about it well if he is making up bs about checking rain goods (although that is definitely something I check regularly - gutters, downpipes etc as huge amounts of damage can be caused if not cleared).
In terms of scuffs on the hall wall, it depends how bad they are. A couple of minor ones are likely to be classed as wear and tear - huge scuffs going up the length of the stairs are more likely to be classed as damage. I would suggest getting wall erasers, like Doktor Power, and very gently trying to remove. I wouldn't try to touch up with paint as it usually looks awful.
Re the carpets - I don't think it reasonable to suggest you get them cleaned, although if there is a stain, I would suggest you spot clean it.
I was at a recent event held by one of the deposit scheme companies in which they explained their decision making around deposit disputes. They differentiated between reasonable wear and tear (not deductible) and cleaning, which is deductible. I wouldn't give the ll any ammunition to withhold your deposit. Clean thoroughly and take your own photos and video - many more than you think you need. Review what is stated in the tenancy agreement.
It may be worth checking what documents you were provided at the start of the tenancy - EPC, gas certificate, inventory all come to mind. If you haven't been given these it would strengthen your negotiating position if he tries to withhold the deposit. Are you based in England? There are slight variations in legislation if you are in NI, Scotland or Wales.