To think this salesman was incredibly rude?

[Stripysecrets]

Someone knocked the door today with double glazing leaflets. I was on the phone but took the leaflet. He asked if I was interested in a quote. I said yes stupidly and he noted down my address then asked my name and phone number. I assumed they would ring at some points but he said he had the designer in the next road and he would come and measure up in a minute.
I said no it wasn't convenient and he was really angry. Made a frustrated anger noise and stormed off down the drive. I said I'm sorry and he turned round and shouted really horribly, 'no you're not'. I was quite shaken stupid as that sounds.
Am now worried as he knows our name, address and phone number.

[Stripysecrets]

I can't find it but I know they were a local company so I could probably google.