Mascara if I'm going to be honest it sounds like neither of you behaved especially professionally, but it sounds like he was worse, plus obviously as the boss the onus is on him to set the tone of the discussion and maintain professionalism.
You say nobody knows about your relationship. I would doubt that tbh, whatever impression you have got. These things are almost impossible to keep secret.
Don't I remember you looking for another job? If so, if this relationship is serious, I would advise you continue to do so. Even leaving aside what has happened today, even if utmost professionalism is maintained at all times (impossible), it is difficult and fraught with problems working so closely with someone you are in a relationship with.
I think you need to address that issue more long-term, how your relationship will work in the office, and, if it's serious, decide whether you think you can actually make it work while working so closely. I have never known anyone who has...
In the short term, obviously you need to deal with this immediate problem. I would advise having a conversation at work about that particular issue, say you did not think he behaved appropriately or professionally, apologise for your part in it, and sort that out. Then say you think you both need to have a more in depth conversation about how things are going to work, which is something you could do outside work.
Good luck, I hope you sort things out and I hope you have calmed down a bit as well