You'd think someone who believed that he was doing the right thing being with his OW, and what he said about the DCs being happy to see him happy, would be glad that you did not seem upset. Alternatively if he were a decent man at heart who knew he had really fucked up, it would be a massive relief to see that at least he had apparently not fucked up your life along with his. (Although he has, of course. But there would be a certain comfort in knowing you could cope and weren't totally devastated.) One might disagree with his actions but would perhaps be able to dredge up a certain measure of sympathy. This, though - this is not either of those men, is it? This is a pathetic whiner who can't understand why he's not the centre of everyone else's universe. Worse, I suspect he might prefer you to be pale and miserable because it would make you easier to manipulate. Not that he wants you to suffer, as such, but your suffering would be a means to his end.
When I log out of one of my computer games I expect to find my characters where I parked them last, unchanged, ready for me to pick them up and march through their paces again at my convenience. In real life, though, people do all sorts of stuff when you're not interacting with them; they go on existing in their own space and having self-determination and stuff. The day I lose sight of that distinction is the day I'll be carted off to a loony bin, dribbling. Yet here is not only an apparently functioning adult, but a psychologist, who seems to expect people to behave like game avatars. No wonder he's getting in trouble at work, and no wonder he blew an excellent marriage and family out of the water.
Sorry, rambling, haven't quite woken up yet...!