Yep, fuck-all to do with him. He is being invasive, controlling, and nosey. And if you had slept with 100 men, there is a 100% guarantee he would judge you on it, and think you were a bit 'easy.' If it was one of his mates saying he had shagged 100 women though, he would probably buy him a 'congratulations' card. 
I remember reading once (and I believe this to be true!) that women 'downgrade' how many men they've shagged, and often will keep it to single figures (so they'll say it was 8 or 9, even if it was 30 or 40 or 90.)
But MEN will 'upgrade' their magic number. So if they have shagged 5 or 6 women, they'll say it's 20 or 30. If it's 15 or 20, they'll say it's 50.
I sometimes see men on tv (on shitty reality tv shows,) who are only in their mid 20's, who claim to have had sex with 1000 women, and I think 'Oh do fuck off! No-one believes you!'
My friend has a brother who works in a big London hotel on a late shift, (like 5 til midnight,) and he speaks to a man at a different hotel up the road some nights. This man claims that every single time he is on late shift, he gets women coming up to him asking for sex, offering him a blow job, and asking them to 'go down' on them etc. At least once a week this happens apparently!
My friend's brother said in 10 years of working in his hotel, that this has never ONCE happened to him. And he is waaaaay better looking than the guy at the other hotel. And the upshot is, no woman on he planet would waltz up to a male night-time receptionist and ask if he will fuck her. It just wouldn't happen.
@Drangoli
It sounds like your partner is very insecure. Just say 'yeah I'm a virgin, you're the first one!' When he looks a bit
just say 'well that's what you want to hear isn't it?!!!'