I would say this is an excellent opportunity to have a very open, honest, but CALM conversation about trust. You must be honest about how you came to see the messages, apologise for the manner in which you did it, however, use it then to explain why it's important that you discuss and agree guidelines and rules regarding supervision/checking of his phone. You can then discuss trust further - eg. if he wants you to trust him with the phone, he has to build up that trust by showing he can be trustworthy with it. Talk about how if he wants friends, future girlfriends etc... to trust him, he has to behave respectfully towards them etc.
You can do all this in a calm manner, talk about use of language, ask him what he feels is appropriate etc... get his views on it and talk to him like an adult.
Yes he's only 12 and yes you have a right to check his phone and monitor his online activity etc. The difference is in how you approach it; if you go in shouting and laying down rules in a draconian manner, you are likely to meet resistance, and he will become more secretive and less likely to have open discussions with you. If, on the other hand, you involve him in agreeing guidelines and rules, and invite him to have open discussions with you about it, that will give him a greater sense of responsibility and maturity about it. Ask his thoughts and views and invite him to make suggestions about how to go forward.
The important thing, I'd say, is to make sure he feels he can come to you and talk to you about ANYTHING he is concerned about or is unsure about without you flying off the handle. When I was growing up I didn't feel able to talk to my mum about anything because of her attitude, closed mind and her way of turning everything into a major drama. To this day, we do not have a close relationship and I'd go so far to say that she barely knows me because I could never be open and honest with her. I couldn't tell her anything, however minor, without her going crazy and overreacting.
So with this, I'd be inclined to say that yes, ground rules need to be in place, but don't go mad, remain calm and try to involve him in taking responsibility for his online conduct, his behaviour with his friends, this girl etc... his language, understanding how things can get out of hand online.
Finally - be prepared for him going mad when you tell him about reading his messages. I'd suggest allowing him time to calm down before you attempt a proper conversation. He's unlikely to hear you properly if he's angry. Everyone needs time to process things and calm down before being expected to act reasonably.