I went to the Mumsnet Take Five event and it was quite humbling, tbh. I've never had a fraud against me so I was quite smug. They went through all these examples of what can happen and by the end, I was feeling really stupid for thinking I could avoid it.
You can get a text message that arrives inside an existing thread which are legitimately from your bank, or internet provider, etc etc, and which gives a number or link that looks equally correct, and yet it's a fraudster sending you to hand over your data in good faith. If they have your phone number, then they can do this. That's all they need - bonus points if they have your name, too. A guy at the event sent a text to a Mumsnetter, and it looked legit and did indeed concertina in an existing bank message thread. That's one way. There are loads more. He said 95% of us, if called by someone claiming to be from our bank, answer the security questions. We don't hang up, wait a while, then call back on a number from the back of the card. Yet we have no way of knowing that person is legit. An especially grim fraud: someone called and started asking security questions, and the person wasn't comfortable, refused to answer, and called their bank. They were right and it was a fraud attempt. A few hours later the bank called back and said they'd identified a pattern of fraudulent spending and needed to go through it, so the person went through all these bonkers transactions they hadn't made.
Only the last call was not from the bank. It was by someone associated with the first fraudster, knowing that the target had sussed them and would have called the bank, and would now be very receptive to an approach claiming to be the fraud department. So they'd handed over a ton of data on that last approach.
I had a text from someone who knew my name and had my phone number, saying they were Netflix and the account was suspended if I didn't check my details and make a payment, as this month's had failed. I almost fell for it, but the Netflix account is not in my name, but my husband's... I was literally digging around in my bag to find my purse, to go and pay when I realised that. I have no idea where they got the number from, coupled with my name. I don't hand it out that much.
They also said that fraud is really commonly from cloned cards, from fake websites, and from cashpoints that have been doctored to obtain your PIN and card details. It might not be your bank at all. They said the best protection against that is always to use cashpoints physically inside banks, because they are a lot harder for fraudsters to access, and a lot riskier for them to mess with.
It can happen to anyone. It probably will do, at some point. It's a question of risk mitigation, rather than avoidance, I think.