I've worked for the Inland Revenue, and Social Services and had access to huge amounts of personal data in both of these jobs.
With a bit of work (and an absolute absence of moral considerations), I reckon I could have caused a considerable amount of mischief with all the information I had access to. Much like this guy who worked at the DVLA and passed personal records to animal rights activists. You can multiply that 100-fold if I had access to all of your life's data, rather than just the tax bit. Get it right and I reckon you could be off to somewhere high temperature, low tax and zero extradition treaty before anyone noticed they'd been cleaned out.
I'll happily accept a biometric passport. I suspect it might have a fatal encounter with a microwave soon after it reaches me though. .. See more on the dangers of ID here .