TV Licensing is a commercial name used by the BBC and the body is used for collection of the licence fee from those people who don't pay up 'voluntarily', i.e. upfront, by Direct Debit, etc. The BBC awards the business to a commercial company (usually Capita plc). So these people have no rights at all as regards rights of entry to your property, rights of interview, rights of inspection, rights of search, etc. If they want to enter your property then they either have to be granted access by you, the householder, or they have to apply to the courts for a warrant and enforce this warrant (usually by asking the police to accompany them). The courts will not grant such a warrant without proper proof that some offence is being committed - and in the case of watching 'live' broadcasts on equipment enabled to receive such broadcasts, this is extremely difficult to prove; warrants are rarely granted.
As other people have said, it is not up to you to prove a negative - i.e. that you do not watch television - but rather it is up to the 'accuser' to prove that you are breaking the law. This is the whole spirit of English Law and is commonly enshrined in the phrase 'innocent until proven guilty'.
The way TV Licensing works is, in a nutshell, to harass people until they either give in and pay for a licence (which, as you can see from some posts here, are actually sometimes paid for when they are not required) or admit that they need a licence and have not got one - and sometimes this 'admission' is not kosher because people say something like 'Well yes, I do have a television' and this is put down by the TV Licensing caller as an 'admission' - whereas of course the mere possession of a television (or any other receiving equipment) does not in itself necessitate holding a licence.
TV Licensing people are not officials, they are like any other cold caller to your home. To let them into your home is pointless because there is no way of proving a negative in this case: whether or not you have a television (or other receiving equipment) is irrelevant and whether or not that equipment is capable of receiving a signal is irrelevant - so how can anyone hope to prove that you are or are not watching 'live' broadcasts by gaining entry to your home?
Unfortunately employees of Capita etc. are not properly vetted (see the various media accounts in this respect) and so you would be very unwise to let them into your home. Some of them may be pleasant enough people - but some are not and there are no guarantees, so err on the side of caution and do not let them in. Given that they are also earning commission on each sale of a television licence, they have a vested interest in getting you to say something which can be held against you in your claim not to need a licence - and some of the less scrupulous employees do twist words and even state that householders have made admissions where they haven't. As there is no point in allowing a TV Licensing employee into your home, be wise and don't do it.
Letting a TV Licensing employee into your home may or may not stop the harassing letters - it's the luck of the draw (but the odds are against it).
As for the principle behind all this, even if you are not bothered about the spirit of English Law being broken down (innocent until proven guilty), then it might be worth considering that setting a precedent whereby any private company can demand access to your home and demand that you prove a negative might be a very dangerous thing indeed...
If I sound hard-edged about this, it's because I have been harassed by these people for decades. I have let them in - and been treated very rudely in my own home as a consequence (I did finally get compensation, but it took a lot of fighting) - and I have telephoned them, and I have written letters. It seems not to matter, the letters keep on coming, and so - periodically - do the Capita employees; I'm afraid they get very short shrift these days.
Yes, I have a television; no, I do not watch live broadcasts. I need a television to watch DVDs. However as I do not expect Sainsbury's to come to my house and enquire why I don't shop in their stores, or the dog licensing body to ask why I don't have a dog licence, or HMRC to enquire why I don't have a distiller's licence, so I don't expect Capita or any other BBC employee to come round and enquire why I don't have a TV licence.
Currently I am saving up the harassing letters which come to my house every month - and which make the postman think that I am a licence evader, which is slightly embarrassing in a very rural area - and, if I receive just one more visit from a Capita employee I am going to take them and the BBC to court under the Protection from Harassment Act. I, for one, have had enough.