OP, have you ever asked yourself how you would feel if your partner/husband cheated on YOU? Do you realise that if he finds out about this, you are likely to loose everything that you actually value in life, your home, your friends, possibly even your family?
I am ashamed to admit that I was you at one point, (although not quite so naive) I was bored at home and felt neglected, no excuse, but that was behind what happened. I had more than one affair, and got away with it, but then suddenly I got caught!! When my DH found out, he was devastated, but having talked it all through, we decided, or so I thought, to stay together and give it another try. I had been lucky! Then, a few months later I discovered that he had turned the tables on me, and had an affair himself. I will never forget the sick feeling in my gut, when his girlfriend phoned one day, not expecting me to be at home, and I realised that he had been cheating on ME!! I confronted him, and he said 'now you know how it feels!' He then told me that I had killed the love that he had for me, and he was now in love with her, and was leaving to be with her. I was DEVASTATED!! All this time I'd been playing with fire, but never REALLY took on board what I was risking, just enjoyed the admiration and attention, without ever giving any serious thought as to what it would do to my DH if he found out, or what I was putting at risk. It took me years to get over what I had done, and to build up a new life for myself.
So my advice to you, would be, stop being so blind and naive, be polite to your lover at work, as you should be to your boss, but absolutely NOTHING more. Do your job, but start looking for a new one as a matter of urgency, and whatever you do, DO NOT see this man outside of work EVER again. If you do this, you may just be lucky, and not lose the life that you are currently taking for granted.
However, if you no longer want that life, then at least have the honesty to admit that your relationship with your partner/DH, is over, and let him find someone who REALLY loves him and WANTS to be with him. Then, you will be free to do whatever you like.