do I challenge him on where he sees this going or do I just back off, get on with my life and if he’s really interested, his actions will show me?
I'm sure it's just the language you chose rather than meanijg it literally, but i wouldn't 'challenge' him as much as just have a conversation to see where he thinks things are going (if anywhere). The conversation itself doesn't need to be heavy.
People will tell you to 'bin him' but, in reality, that's not how people work and most people would just have a conversation! He hasn't actually treated you badly, you're just in a bit of a tailspin about it.
You might have fallen for a love bomber or your emotional response to the situation might just be out of kilter with reality given your personal circumstances. I disagree with a pp who saod it it were good, you wouldnt be feeling like this about it. Normally, I'd agree but your subconscious can go into overdrive to protect you and this might jsut be a period of adjistment.
I wouldn't be giving him the benefit of the doubt though. As you say, he's intelligent and exprienced at life. If he says 'keep it casual', well then you know. If he says he wants something more, then you decide if you do and see if his actions going forwards match up with his words.
(Or you decide you cant be arsed with that and put a stop to it yourself!)
Personally, I'd have a conversation (also in my 50s) because I can't be doing with uncertainty and I'm also an adult. But I'm also well past making excuses for men and just hoping it will get better.
It sounds like you have a pretty good lifestyle and are happy with life. So anyone you're with needs to add to that and not detract from it.
So... no help at all, really!