Oh dear. I'm just going to ramble a bit, and hope that in the midst of the haystack, there might be something useful for you ... . sorry in advance if there isn't.
I think that you sound a bit shocked by what your counsellor has suggested. And that's interesting. They're usually quite hesitant about saying anything quite so abrupt. Are you sure that's what s/he said?
If so, here's the thing: You are in the position where a counsellor is telling you your marriage is abusive, but you aren't at the point where you can see that. It suggests to me that you probably need the aid of a counsellor (I would also recommend good, helpful, thoughtful friends, but not everyone has that,) to work out what is going on/has gone on that you can't see that. I suspect that will be a process, and one that will have to deal with your relationship pattern, probably from childhood models, and your self-esteem.
I suspect your counsellor is trying to get you to see that it is OK to demand, for yourself, that your need for intimacy is recognised as legitimate, and should be met by people you share your intimate life with. I suspect there's more, that you can't "see" but your counsellor is trying to get you to see.
All of this suggests to me that you need to do this work before entering another relationship - just in case. It's not necessarily the case that if you walk out of this current relationship, straight into another, that you'll end up repeating the same pattern unless you intervene. But the fact that your counsellor is effectively telling you that you don't "see" stuff, and that your boundaries and sense of rights and self isn't as firm as it could be, does, frankly, augur a little ill.
Wrt "friend". Well, on the one hand - he's propping your marriage up, isn't he? Without him meeting your emotional needs, I guess your marriage would be even less tolerable.
On the other hand - his friendship is restoring to you an idea of how human relationships ought to work, and that you may have value. Important and necessary things in restoring your self-esteem, and getting you to a point where you feel your life is worth fighting for.
Because that is what this is all about really - what are you prepared to do, how many people are you prepared to make uncomfortable for you to have your life? How valuable is your life? I'm guessing you are going to need to do some serious reflection and work (with the help of others) before you get to the point of being able to adequately answer that question, and adequately value your life.
That's why it's so hard to answer your initial question. On the one hand, of course you can leave at 50, people leave and start over all the time. And people find love at all ages - if that's what they want. On the other hand, it's not easy to leave, and to do that weighing up of your needs, everyone else's, and the (disappointing) present versus the (terrifyingly) unknown future. I suspect you're going to need more help to do that.
Wrt a relationship with this man. I honestly don't know. I wonder what you'd think about him if you'd gone through the process of self-recovery? He might be a truly special person but i worry a little that you might not be in the best place to judge that just yet. There again, maybe you are. I don't know. On the plus side, he's got you thinking about what life outside/in a transformed relationship might be like. That's a good thing. Whatever else, you should hold on to that thought.
Wrt leaving your marriage. Well, you sound as though you're not there yet, if you're still asking. I suspect that if you take the path to self-awareness, you might. You'll certainly insist on changes. Which your dh may not be able to (adequately) make.
Wrt abusive relationship. Here's the thing, if both of you were the type that isn't big on lots of emotion and intimacy, your relationship would be fine. but the fact that it is making you unhappy, and you are having to actually go out and find affection elsewhere means it's not working. I'm guessing that that is what your counsellor is picking up on? Is your counsellor suggesting that your dh witholds on purpose, as a form of control? Are there other forms of abuse going on? You don't need to answer any of this, but they are questions you should ask yourself.
OK - all of that is just rambling. all of it is hugely conjectural. I don't know your situation, so it could all be wide of the mark. but perhaps some of it may be of use - to spark off thoughts - to say "no", or "yes", or "maybe" to.
good luck.