An alternative might be to say something like: "Are you sure you want me to answer that question?" wait for an answer and then be very, very matter of fact/slightly TMI about it: "I've been in x relationships over the years and have had sex with y people. The first relationship I had started when I was z years old, the most recent one ended $ months ago. We don't really know each other well enough yet for me to give you a lot of detail about those relationships, but if there's anything specific you'd like to know, feel free to ask and I'll try to answer."
If you're feeling particularly irritated by the question, you could go for some variation of "$partner1 and I had great sex, though the relationship ended for other reasons", "partner2 loved anal" etc.
I'm coming at this from the angle of someone who has polyamorous relationships; one important communication thing that's generally applicable across relationships is "only ask questions you want to know the answers to" - if you don't want to know a bit of detail, then don't ask about it. If he's going to ask questions like that, then he's only got himself to blame when he gets more of an answer than he's expecting/might actually want.
Poly/Mono regardless, people enter into relationships as they are, formed by all of the experiences that have got them to where they are - previous relationships are likely to be a part of that (previous sexual encounters, perhaps less so). I feel strongly that it's important not to apologise for being who you are, with your life experiences. You are you. You might not choose to share the more complicated/distressing details with a very new partner, but they're still a part of how you got to where you are.
If he isn't motivated by something innocent in asking about this (and I think other people may be right, given he's been asking repeatedly), then call his bluff - be open and unashamed about previous people, but don't give away more detail about those relationships than you feel comfortable doing with someone you don't yet know and see if his reaction tends towards an inappropriate level of possessiveness or not.