I completely agree the current position is discriminatory.
However, I also see the risk to the landlord of having a tenant on benefits if they need to sell/don't pay rent as they are more likely to sit it out for months or even a year to be made officially homeless, as well as the issues of non-direct payment of the housing benefit part of UC. This makes benefit payment households far less attractive for landlords and I don't see any moves to fix this.
It's the same for pets- I would take pets if I could take increased deposit because cats and dogs do sometimes cause damage, but given there is now a cap on deposits, it's not such an attractive option.
The whole marketplace is such a mish-mash and often these new regulations aren't put into place with any thought into other changes that could benefit the market, so everyone just does a work-around to save themselves from the worst excesses of other people's behaviour (both tenant and landlord).
If you make it too cheap to rent, and impossible to recover deposits, then people just prefer to have their properties standing empty- I have experienced this abroad where the damages done to the property exceeded the rental income for the year. No point in renting out, this won't solve the shortage of supply problem- that would require sustained social housing building programme (now would be an ideal time to put that in place, Boris), changes to the council criteria for being homeless so you don't force everyone to go to court (also unpleasant for the tenant), and ideally direct to landlord payments on the request of the tenant (as it makes a DSS tenant way more attractive). Pets- meh, I guess we'll have to put up with them (or charge a pet rent like they do in the US).
Instead there's just a lot of tinkering around the edges. I don't see what forcing small landlords out of the market will do in the absence of social housing, the bigger companies are not nicer or more understanding to DSS tenants whatsoever, nor do they keep all their properties in good repair.