You're already better equipped - you've dealt with angry people in person, so you might find that them being on the phone means it really doesn't bother you because they are just voices, rather than people who might actually be able to do something - you're already de-escalating situations in person, on the phone can be far easier (and at the end of the day, you can hang up on them if need be).
I did insurance/independent contractors rather than banking (you get access to training and qualifications through banks, though, which is much better) - dealing with anybody/any role outside the call centre environment in the future usually means that people are amazed how easily you deal with people that would have them in tears; you get to a point where, when there's a really difficult caller, you volunteer to take those calls off others 'Oh, them. Put them through to me' or can tell when somebody else is on the receiving end of one of those calls and provide support.
It can really increase your confidence quickly - and be good for developing good boundaries/assertiveness all around, along with being able to respond quickly to situations confidently because you're used to having to switch on to a completely new situation every 3 minutes, for example.
DP's done similar in financial, telecoms and Local Authorities. He can take anything that's thrown at him over the phone - he was worried about going into a job where stuff happened in person, but once he'd done it, he found that the phone skills had been really useful, so he was then able to deal with both for later roles.