I think people should be careful to jump in and say "he is having an affair," because really nobody knows. There is every chance he could be headed in the direction of an affair or even in the midst of one, but just because he has had a couple of texts and has become close friends does not mean he is having an affair.
these threads are always emotive and people post from their own experiences. But not every experience is the same and not every relationship is the same.
Op I think there are several issues here tbh, but that they are all potentially leading towards the same thing.
It sounds from your posts as if you potentially don't have much in common and don't spend much time together as a couple. You say that he's out rehearsing while you're at home with the kids, and that even when he has gigs it's his mother that goes to them and not you.
It sounds as if you are frustrated by this (and rightfully so), because he is spending his time elsewhere and you're at home. But if you're feeling like this, perhaps he does too?
Currently it seems that you don't do anything together as a couple, and that he's having fun playing in this band (which is perfectly ok), except that there are others in this band who share his enjoyment of the music, and consequently friendships have developed. Music is a powerful tool. Playing music can involve a lot of emotion, and I can totally see how close friendships would develop between people who play together.
So it's possible that he's developed a close friendship with this woman because of their involvement with this band, and he's sharing things with her that he's not got with you at home.
I can see why you don't like that. I wouldn't either. But telling him to end the friendship isn't going to change what's missing in your own relationship.
You need to try and make time for each other. Get a babysitter and go out for a meal. If you can't get a babysitter cook a nice meal and wait till the kids are in bed and spend the night in. But actually I think that you should try to get out together, because it seems you yourself don't actually go out, so he sees you as the wife at home iyswim. If you bring your own relationship back on track then he will have less insentive to talk to this other woman.
You should still express to him how you feel about this other woman, but do it in conjunction with "we seem to be drifting/I'm always home and you're out so we never get time to spend together," (don't opportion blame, make it a thing you have to work on jointly).
I wouldn't be inviting her round, and I certainly wouldn't be going round to punch her in the fact as per suggestion below. It's your dh that you need to work on, if you can get him to come back to you the other woman will be irelevant.