I've tried about a million different antidepressants and mood stabilisers (bipolar type 2). I've also known a lot of other people who have.
'They affect everyone differently' is not just something people say. Results really can vary wildly between two people with very similar symptoms. The same drug can overstimulate person 1 and zombify person 2. Person 1 may get every side effect in the booklet while person 2 may just feel 'a bit sketchy' for a few days.
In all honesty I haven't tried a single antidepressant that didn't affect my libido/ inhibit orgasm in the short term. With some it has improved in time, with others it hasn't. But, like I said, that's just me.
If the symptoms you are taking it for are improving/have improved then it all comes down to the trade-off. Which affects your quality of life more? The symptoms or the side effects? Sometimes it's worth staying on them a while and seeing if things improve (the usual 4-6 weeks time frame given is nowhere near enough to assess what the long-term affects will be), sometimes it's worth trying a new one.
Another thing to remember is that if you are under GP care, they don't have specialist training in mental health issues. You could ask for a referral to a psychiatrist if you think you need more help with meds, or you could do your own research to aid you in discussions with your GP. Just don't necessarily assume they will know how best to treat you if you don't actively get involved in that decision.
If you are thinking of changing your meds, I would suggest asking your GP for printouts of the information sheets on several different drugs that might be appropriate and reading them fully in your own time (just don't get freaked out reading the list of side effects list, remember they have to list every side effect that has ever been reported basically!
Good luck. It's not easy finding the right meds (and that sucks if you're not dealing with things well atm) but it can be a real game-changer when you get them right.