I don't see why it couldn't work the same as when you buy petrol - they put a pre-authorisation on your card to a certain amount. Then the actual amount you spend is what is ultimately deducted. That tech has already been in place for a number of years so wouldn't be hard to roll out, and isn't giving any more data away than you would already do if you pay by card anyway.
You'd have to scan your receipt or card to exit (a system which a lot of self-service tills already have). There could be a separate exit by the security guard for people who don't buy anything - yes technically someone could still stuff a pack of biscuits down their jumper but they wouldn't escape with whole trolleys worth of stuff, and they would have to go through a security gate first so anything with a security label would beep.
The only disadvantage would be people who prefer to pay with cash, but how many people still do that? And there could still be a workaround if you wanted to pay with cash at the till as long as you had a card to use for the pre-authorisation. perhaps you could get pre-pay cards etc.
The amazon grocery stores in the US pretty much already work that way, with the added tech of them being able to calculate what you owe as you walk out with it without you having to queue and pay - that could be another option.
While people have concerns about tech, something not working properly is a possible negative outcome, whereas constant shop lifting isn't a hypothetical but is actually happening.
At the end of the day shops aren't going to do anything until the easier option (charging the law-abiding 95% more to cover the loss of the thieving X%) stops working, so it will be on us to change it. If one supermarket successfully implements an alternative it will be up to consumers to vote with their feet - if it means cheaper prices for them and a nicer experience without scabs nicking stuff so feel that 'giving away' data supermarkets already have access to via CCTV, clubcard and bank cards is a worthwhile exchange.