In town? Or in the supermarket?
I find it doesn’t seem to matter who it is either - it could be a close friend or my own mother and I still find we are making awkward stilted conversation even though we usually get on well!
DH says it’s because you aren’t expecting to see them and you both have a task to complete that you are acutely aware has been interrupted.
YABU - it’s not awkward
YANBU - it’s awkward