OP, it sounds like you are trying so hard not to overreact or be paranoid or unfair that you’ve turned yourself inside out trying to be reasonable. But to be blunt, where has that got you?
Going back to his reaction to your raising the situation:
he said he does find her attractive and gets the vibe that it's mutual.
He admitted the eye thing and that they both sort of try to avoid each other because of it.
He did say that he didn't accept her [on instagram] as he didn't want her seeing family pictures.
And he was immediately clearly not avoiding her and shutting things down, but photographing her etc.
This is not OK. I’d call it an emotional affair. They might not be having long, heartfelt chats alone, but they are communicating to each other re their mutual bond regardless. Yes, it’s a self-indulgent fantasy, but it’s disrespectful towards you, and obviously so hurtful.
That he should carry on with this after you’ve told him you’re aware is just totally unacceptable.
He’s getting a kick out of this. It may well never become physical. I think there are many men with partners and kids who have no intention of leaving, but who love to get an ego boost from connections like this. Classic having his cake and eating it.
Shamefully, I’ve been the OW in an emotional affair. I didn’t know the term then and minimised what was happening as being OK as long as it didn’t become physical. I’m now disgusted with myself. I finally lost all respect for him and saw that he had behaved appallingly. I think he dealt with it by extreme compartmentalisation - his wife and I, as a close friend, were simply different relationships. He never let himself worry about the potential effect on her (or on me).
But, as far as I know (and I’m pretty sure) she never called him out on it. So he never had to face it head on. When I raised it with him he got (uncharacteristically) cross, and I backed off.
tl;dr: he may have no intention of making it physical, but he is still prioritising his ego-boost and work-place frisson over your feelings. And he doesn’t seem to feel any compunction. I’d attempt to break down that compartmentalisation.