You were not being unreasonable. TV Licensing has an extremely bad track record (up to and including assaults and rapes on householders). Any TV Licensing person must know this and should therefore expect to be treated accordingly.
Starting the interaction with you by reading you 'your rights' is a harassment technique, nothing more. There is no need to do this and it wouldn't mean anything in the magistrates' court (which would be where any case against you would end up). TV Licensing 'officers' are simply employees of a private company, nothing more.
You don't need to interact with these people, and you really shouldn't. If you give them any information at all, they are likely to use it to harass you further. For proof of this, Google 'TV Licensing harassment' or suchlike - and I certainly know this myself first-hand. Foolishly, when a (male) TV Licensing/Capita employee turned up at my (very rural and isolated) cottage one evening, after dark, I let him in. He was extremely unpleasant to me, despite the fact that I was alone, I was painting (as in decorating), and the television was tucked under the stairs, without an aerial, and when switched on showed only 'snow'. I later received an official letter from TV Licensing telling me that, as the 'officer' had found a television 'installed' I was going to be taken to court.
I fought back in no uncertain terms, by letter, and was finally awarded £50 in compensation and was asssured that I wouldn't be troubled by them again for a period (three years? I can't remember now). A week or so later the whole thing started again. I fought back again. Another £50 compensation. Ridiculous nonsense.
When I moved I telephoned TV Licensing to inform them I didn't need a licence, and was treated with great rudeness by the woman to whom I spoke. Her attitude was "You're lying, and we're going to come and visit you unless you pay a licence fee". I complained very strongly (by letter) and received a letter of apology from 'TV Licensing' (Capita).
Since then I have refused to deal with TV Licensing at all and, when they come to the door, they are given very short shrift. If they try and bully me (and they do), they get the full-on screaming harridan bit. If they are ever unwise enough to stand their ground after that, I am not going to be responsible for the consequences.
I follow various blogs on this subject, and occasionally Write Letters (MPs, the BBC Trust, newspapers, etc.) because I think the way TV Licensing are allowed to act is setting a dangerous precedent. I for one certainly don't want to live in a world where people can come to the door and demand entry and demand proof that you are innocent. Ask anyone who lived in Berlin in 1939 what that kind of thing leads to...
OP, you might find this interesting:
tv-licensing.blogspot.com/