Well that's a valuable realisation. In your position I would do a great deal of work on self esteem first. Whether you stay in the marriage or not, have an affair or not, making these decisions with healthy rather than rock-bottom self esteem would be very useful to you.
Can you sign up for six CBT sessions free online, funded by NHS. You don't need GP referral if you are in UK. Many companies offer this. They give you lots of self-esteem worksheets, self compassion and self care worksheets. No counsellor is perfect or wholly 'gets' you but as long as they seem to have integrity, working with them can untangle some real emotional knots. It's a good starting point.
You could make a list of other things that would bring you joy, pride, self esteem. Are they worth a shot?
Does your marriage lower your self esteem? often on here people say 'I'm unhappy even though my husband is great' and then describe someone who isn't?
Do you have kids?