Daisym, hope you're doing ok. Like the fact you are now taking charge and calling more of the shots.
At the moment it's just an intangible suspicion. It's the uncertainty of not knowing one way or another that is painful. Knowing one way or another would be a relief. And you hate them for putting you through that - are they really that insensitive? are they the man you married?
My dh's bf who we love dearly had an 'emotional' affair despite being married to the most beautiful woman I know. The OW was so obviously predatory in a way that women, including his wife, instantly recognise. He was then flattered by the attention and it was ever so gradual, but slowly he was investing his emotions in her and not his family. They would socialise as a works group but she'd always be by his side and they would be the last two left together. The OW would negatively comment on his wife, saying things like, 'I hope I'm not getting you into trouble texting so late' etc.
His wife even deliberately met her when inviting colleagues for a meal. I think to see what the 'competition' was and, although she'd never admit it, because she is so stunning to make the OW feel intimidated. The OW then did the pathetic "I don't think your wife likes me..did I upset her in any way?" conversation the next day, grrr.
What sorted it was actually bringing it into the open, so it wasn't the elephant in the room any more. His wife confided in me and then he confided in me so I felt I could be blunt. The OW was so obvious I could actually tell him what she had done and said before he did - the poor sap. Portrayed her as the bitch she was. Asked him what his parents would think if they knew. Where did he think this was heading? The D word - divorce - was a shock to him. But had to tell him he owed every loyalty to his wife and none to the OW - so far down the line he'd gone that his priorities were warped; he was thoughtless and behaving in a way that we just didn't recognise. Dh, who doesn't do relationship issues ,simply said, "You still seeing that woman?" and shook his head and said "You're a twat."
Sometimes an outsider shining a light on the behaviour can be a wake-up call. I'm not sure I'm explaining it too well but sometimes ime the 'secrecy' of it allows the hb's behaviour to continue.