My opinion (a bit different to some):
It's good he and you are known people in the dog-walking group. This gives you some groove to slot back into.
You say you shouldn't have agreed to meet him in the first place. ... ? Do you actually find him attractive? Or just didn't want to be mean and say no?
He sounds possibly naive/definitely lonely/has had a crush on you for some time.
I agree you need to work on boundary definition and defence. You've given him the wrong messages.
BUT
We still don't know if he has the keys.
His behaviour so far has been OTT and kind of inexperienced/uncool/also boundary-blind. Taking your house keys is totally different. And he may well not have.
For THIS reason, I wouldn't tell work. Because I wouldn't create irreversible drama around myself based on an unproven assumption. It actually IMO makes you seem a bit rash - I only mean re: telling work. Of course if he had taken them, fine to share it with your manager. But not if not. And you don't know.
You sound like you very much need reassurance and someone to talk to. I'm not sure your manager is the right person for this private matter. For me, that would be a professional boundary I wouldn't cross.
It's unfortunate you have the tickets. I'd honestly probably say you can't go because ill/work issue/something else he can't challenge, and say can he find someone else to go with and have your ticket.
I agree with others that careful withdrawal here would be wise.
You'll be able to judge more how to do it by his eventual response to the key whereabouts issue.