Hi OP, my close friends are a combination of men and women: they’re all like family: uncles and aunties to my kids and vice versa. In some cases, our friendships have lasted decades, as various partners on both sides have come and gone.
Personally, I couldn’t be in a relationship with someone who felt uncomfortable with me having friends, male or female. It’s none of their business who my friends are.
There’s a big difference between being friends and a flirtation. The key here is does your DH behave like this with his male friends, i.e. message or chat at weekends? Even if he views his colleague as a friend, she may be interested in him romantically. Most importantly, do you trust him? And is that wise?
As to what you do about it. Make it clear to him what your boundaries are, e.g. if you discover anything romantic/sexual has happened, it’ll be the end of your relationship which will impact the whole family. Stay alert to his behaviour, so you more easily spot if he’s being deceptive..
Then, get on with your life. When kids get older, couples have the chance to have more of a life of their own outside the family: both together (days/nights out and even holidays as a couple) and individually. Enjoy the chance of have a life outside your family, to pursue your own interests and make new friends.