Emmalouise2babies, I am very sorry this sounds awful. It seems very clear that if something is not going on already it will most probably be, before long.
What you choose to do about this is really up to you.
The area I am most worried about is what your kids may end up witnessing while you wait for your less than 'd'h to dig his own grave.
I am also very worried that your comments include quiet violent images of what you will do to either or both of them (your dh and this woman) if you discover your suspicions are true.
If you hit your husband or this woman you could land up with an assault charge, you may also find yourself facing them both denying anything and making out you are as being aggressive and saying they are not having an affair, and of course they may not be doing so yet.
In additional to this any aggression on your part may lead to unexpected consequences, such as aggression from either of them, and may be witnessed by any child in the house too.
Lastly, this other woman has a boyfriend who you describe as a 'total bastard'. If your husband does begin an affair with this woman then this total bastard may come to your home and make trouble. You may feel your husband deserves this but do your kids deserve to see this.
Although I do feel very sorry for your situation I also feel sorry for this younger woman, in a crappy relationship, who is perhaps looking for solace and an exit from her bad relationship and may see your husband as that chance. I am not excusing what she is doing but I also wonder if she really knows what she is doing and the risk she is placing on your marriage and perhaps most of all, if her partner is jealous and violent, on herself!
Whether you should leave him or seek counselling or try and salvage your marriage is totally up to you but please do remember that although you may be the main 'injured party' there are several other people who may be affected if this inappropriate situation continues.
Good luck, OP, I hope it works out OK whatever you decide to do. Take back the control and do not allow this to continue, please.