And yet I've never got ill from failing to rinse my washed dishes. Never had food poisoning, partner never had food poisoning (at least, not from eating at home), never had anyone get flu or a cold sore that wouldn't be better explained another way… in fact no apparent ill effect at all from all those germs and viruses. Those ones that cling on when scrubbed with a detergent-filled sponge then rinsed/rubbed in hot, reasonably clean washing-up water, but which will succumb to a subsequent quick sluice under a tap.
(None of this water is nearly hot enough, for long enough, to kill germs anyway — your tap water shouldn't be coming out much above 50°C. Dishwashers are the way to go for optimal microorganism removal, as they can use water hot enough to damage living things. And all those people routinely using teatowels, instead of letting things air-dry, are probably smearing on far more bacteria than ever get removed by rinsing. But even if there's living microorganisms clinging to my crockery, they're not going to multiply on a dry plate, so, meh 🤷🏻 They're probably no more dangerous than when I ate the food that dirtied the plate in the first place.)
Of course, I do have some common sense. Food waste is scraped off first, incredibly grubby things are soaked and the soaking water chucked, and as much fat as possible is wiped and binned rather than washed off, to protect my sewer. Things are washed in order from cleanest to dirtiest. Items washed later (and therefore in water that might be a bit greasy or have small amounts of food dissolved in it) tend to be those that will get very hot anyway — pans, baking trays, etc.. Visibly dirty water is ditched, as is water that's cooled enough that grease is harder to shift. My cloths and the sink are bleached and my sponges replaced regularly. A raw-chickeny chopping board, or something that's touched gluten, gets washed last, and the water thrown away — I wouldn't want to wash anything else in that water whether I was planning to dibble it under the tap afterwards or not, but that's probably largely awareness-campaign-induced over-caution.
And nothing on my draining rack has bits of food hidden under mounds of foam, as a PP suggested 
I mean, okay, maybe there's more invisible living/dead germs and microscopic food particles on my dishes than a rinser's. In fact, yeah, I think it's likely. But a) is there any evidence showing significantly increased germ removal, with measurable health effects, from a quick extra rinse with warm water in a home environment, and b) if these potential extra germs don't actually harm those of us who eat from unrinsed dishes — which they don't appear to (I mean, it's a common way to wash up in the UK but I've never seen the FSA put out advice/campaigns urging people at home to rinse their washed dishes) — then why would I care about extra germs at all?
I might take extra-special care if I had an immunosuppressed family member, but I don't. If I were handwashing in a commercial context, I'd immerse the washed dishes into extremely hot water as recommended by the FSA in such settings (if handwashing is unavoidable), but then there's a lot of things restaurants need to do that aren't considered necessary in homes.
That leaves detergent residue. If there are people who really can taste it, or get films they don't like floating on drinks or whatever, then fair enough, I can see why they'd rinse. Hell, maybe microscopic Fairy residue that I can't detect is silently doing terrible cumulative damage to my insides. But as I said earlier, it's not enough that I or anyone in my family can taste it, it's not enough to trigger DP's SLS sensitivity, so I really can't bring myself to give enough of a fuck about it to start burning extra fossil fuels, draining away extra treated drinking water, or standing in front of the sink for any longer than I absolutely have to 
I do find it funny how utterly horrified and disgusted some Brits and very many non-Brits are by us death-defying non-rinsers, and the nauseating filth doubtless lurking invisibly on our hazardous plates. Something's going to kill me, and maybe it'll be un-rinsed washing-up, but I doubt it.