when we are there alone, is it wrong of me to not want him to think of her
Sorry, to be blunt, but yes, in my option that is wrong of you. He's going to think about her when he's with you. Not all the time, and less over time. But he still will. The feelings of loving/ missing/grieving for a deceased spouse isn't a tap that the surviving spouse can switch on or off at will. It's more of a leaky pipe, that drips all the time, and suddenly, often without warning, gushes out.
He says 'he can't be with her anymore' so it's okay to talk about her, but I can't/shouldn't about XH because he's still here so always going to be a threat?
Does he say the second part of that? Because if so, that is at least very insecure of him, if not rather controlling. And definitely double standards. You're just as entitled to talk about your XH as he is about his wife. I imagine the kind of things you'd say would be different, but particularly if you share a child and therefore have ongoing communications, it would be totally unnatural for you to not talk about your ex.
My boyfriend talks about his (long -term, live-in) XGF much, much less than I talk about my husband. But that's because there is less to say - his good memories were soured by the mess of their break up; which I imagine happens when most relationships split? But of course not to widow/ers; we're usually keeping on going after a great relationship, rather than moving on from a bad one.
BUT that doesn't mean that he's not allowed to talk about her. I'm here if he needs or wants to talk, the same as he is for me. And little memories and anecdotes crop up the way they do about friends or family members.