Kitty do you live in a flat? I think that the situation is slightly different if you do and it's your only bathroom - you've got all your luxury items in there and some oaf of a bloke comes in, lifts up the lid of the loo with a bang, has a long held, stinky pee and flushes with the lid up (I hate that - boak - the particles spray up everywhere all over everything and it's not even your own piss)and then he most probably doesn't bother washing his hands. And the toothbrushes are sitting there above the sink and he could......No! It feels like mild violation
If he's oversized, clumsy looking, dirty or rude, then I'd say 'No, sorry, there's someone in there right now' or I'd tell him that some floor tiles have just been laid and they're still drying or simply tell him 'no'! You're under no obligation to let him use your bathroom if you don't want him to. I don't think Tesco would condone them using customers' bathrooms. They have an hour between each delivery anyway and how much do you already pay for the delivery charge? When we used to have Tesco deliveries they were always late and mostly rude men I'm afraid. One of them, like your's, moaned non-stop that he had a bad back, couldn't wait to get the job over and done with for the day, blah, blah, it wasn't done in a charming way either. And he stank to high heaven of cigarette smoke too. No, I wouldn't have let him anywhere near my loo because of the attitude.
Why don't you try Ocado? The men are more polite and friendly. They never complain, the deliveries are usually on time and the goods are always the ones you've chosen. Heck, I'd probably even let the Ocado man use the bathroom if he asked me, they're better trained men but I don't think he would. You're the customer, you're paying, you decide.