A big red flag for me is the continued messaging to the female after you told him how it made you feel & asked him to stop. This is an unpopular opinion but both me and my husband have agreed having close friends of the opposite sex is a no no for us. Although we didn't have any whilst we got together anyway so that helps.
For me this shows, that the relationship with the female is more important and valuable to him than you as he chose to continue speaking with her despite your feelings.
It would feel a bit like emotional cheating/micro cheating to me. Also the checking your phone/deleting numbers is weird, maybe a guilty conscience and thinks you are doing the same as he?
The jealousy is a big red flag. He maybe just very Insecure, or its because he is cheating & scared you will too. Either way, it isn't good.
You've been together a long time & have children so I understand you may want to try & work on the relationship. For me , I'd feel like it would only get worse, but if you feel the good outweighs the bad with him then it may be worth trying.
For me the biggest indicator of whether to get rid or fight for the relationship is , is he required to put in the effort needed?
He may need therapy for his jealousy and controlling behaviour. Is he required to do this?
The discarding of your feelings/lack of care you feel from him is another thing. You can try talking to him about it, but if you already have & he knows how you feel and has made 0 changes or efforts to this, then it isn't looking good really.