Well I would say, don't assume anything, but ask him.
In the end, asking is the only way to find out. Yes he might lie andbyoubwill have to judge that, but unless you ask, an innocent explanation will never make it to your ears. Snooping for evidence won't provide conclusive evidence and you will be left wondering.
I would approach this relatively light heartedly - 'look what I've found in your pocket' type comment.....and then just wait to see what he says.
If you don't feel comfortable with the explanation, say so. If this is clearly the end for you, say so. If you aren't sure what to do, say so.
Decide before you have the conversation, if he has cheated or been planning to, is that a deal breaker for you. It isn't for everyone and only you can decide if you will put up with it.
What about if he says there's an explanation but it's not convincing - are you willing to say this and walk away?
And if there is an innocent explanation which seems perfectly valid, are you willing to admit that you felt suspicious. I don't think you particularly need to apologise for this, in light of a condom being a suspicious thing for a non condom user to have, in an early stages relationship.
Don't delay, torturing yourself with the unknown and your worst fears. Get in and ask. It's better to know.