OP, I've been there, feeling like something wasn't quite right in my marriage. DH was my first boyfriend and after a year of marriage, I had a breakdown.
I spent months and months weighing up the grass isn't greener/ life's too short seesaw (reading thousands of posts on mumsnet), and none of us can tell you what to do. It's really, really hard having your head turned and I think you need to be kind to yourself and acknowledge the feelings to help you avoid acting on them.
You have to work through this, and pay attention to your life. Try counselling, individual and couples, explore the other facets of your life - work, hobbies, friends, family, to recognise if this is an issue with your marriage or something wider. Take responsibility for where you are in your life and take a few months to be very honest with yourself.
If you decide to leave your marriage, then you are entitled to, but I think you know that looking elsewhere is a coward's way out of a challenge and not the answer, otherwise you'd be shagging the OM, not talking to us.