Err, I'm an accountant and have been doing small business accounts/tax returns for over 30 years! It's EXACTLY how it works. Most will continue to put the expenses through the books, so the cash IS straight off the profits as they're not just failing to declare the income, they're continuing to claim the associated expenses.
I've been involved in dozens, if not hundreds of tax enquiries where this is exactly the scenario. It's hard to believe that so many are so blatant to do it, as common sense would say you're more likely to be caught, but lots of traders buy on trade accounts so monthly statements show all their purchases, so not easy to hide anyway.
Obviously, some make cash payments to casual staff (thus also evading employers NIC, etc), or to other "cash" traders especially skip hire, scaffolders, etc., so in those cases, yes, you'd be right that it's only the profit being evaded by the trader in question, but the cash paid to their workers/subbies etc is also being used for tax evasion, which just multiplies the amount of income tax, NIC and VAT evaded over several different traders.
Having said that, some traders are a lot more cunning, and have, say, different trade accounts with different suppliers, so buy goods for "declared" jobs from one supplier with costs put through their books, and goods bought for "cash" jobs from a different supplier who they pay in cash and where the invoices don't go through the books. Of course, when the cunning ones get caught out, their punishment (i.e. tax penalties) are higher because they can't claim innocence or accidental poor book-keeping, and they face the highest tier of penalties for deliberate and concealed tax evasion.
The black economy, which includes this kind of thing is the largest component of the official UK tax gap - far higher than tax avoidance by global firms, millionaires etc, simply due to the numbers game, i.e. huge numbers of "normal" people evading a few thousands of pounds against a very small number of "big" people/firms evading millions.