xstitch - do you have repeat patients (like the Dr's) or are they only there the once (A&E type thing)?
Would it make a difference if they did call the police a couple of times - could they build a reputation of meaning what they say??
OBTW - of course YANBU, you are only responsible for your own actions. However, a certain group of people will always see you as 'one of them' and 'all of them' are incompetent. Try not to take it personally and take the insults/threats/crap off when you take your work clothes off - leave it at work as best you can x