It wouldn't occur to me that cooking, chatting, drinking wine and watching TV would cross any normal boundaries, because those things are all so... platonic to me. I've done them with almost all my friends, male and female at some point. All my flatmates, male and female, too. None of them have ever had any inherent frisson beyond the quotidian activities they are. The fact that I now do them all with DP is because I live with him, not because they're inherently coupley or romantic. The only difference is I might actually cosy up to him (physically, not metaphorically) in a way that I wouldn't with my friends.
Socialising but maintaining boundaries - yes, of course. Boundaries around things that are inherently romantic or sexual like flirting, kissing and so on. Chatting and watching TV doesn't even nearly fall into that category.
"Attractive woman" gets to the point I suppose - that this is actually just territorialism. How many years older, or how many more grey hairs, or how far outside conventional attractiveness, would she have to be for you to be OK with this exact same arrangement?
That indicates that it's not the arrangement itself that's inappropriate. It's her looks. An attractive woman is, it seems, innately inappropriate for a married man to be around. How... misogynistic.