It's also actually something that's taught in psychology - if you touch someone on the arm they're much more likely to notice / remember you. Especially in flirting!
Of course some people don't like to be touched and if they ask you not to then you try to avoid it. But it's not offensive!
Sorry, but psychologists and you absolutely do not get to tell somebody what they do or don't find offensive when it comes to who is or isn't allowed to touch their body.
I realise that this scenario is right at the opposite end of the spectrum, but in principle, this is also what groomers/abusers do - cross boundaries and then react hurt and (ironically) offended when their target object and ask them to stop doing it. They will often go on to tell their targets/victims that they are 'mistaken' to think that they don't want to be touched like that and therefore that they are the one causing the problem and not the perpetrator.
I realise that you aren't deliberately doing it, OP, and I'm most definitely not suggesting in any way that you're an abuser; however, you need to deliberately NOT do it. If you feel that you're being friendly and tactile but somebody else finds what you're doing handsy and unpleasant, this doesn't mean that you need to stop doing it altogether but it DOES mean that you need to stop doing it to them as soon as they have asked you to stop.
'Trying not to' is not enough and it is not their fault if you 'try' but not to the extent that you actually succeed in respecting their explicitly stated boundaries.