Yes, this happened to me too. I have no idea what website I went to, but I certainly don't recognise the BritishDrive one. I DO recognise the Gov UK and DVLA one. So they cleverly redirect you from the government site to their own site somehow.
I was lucky enough that my bank blocked my card as a result of a suspicious payment. I found out 2 weeks later and contacted the bank who asked me to check the payment to BritishDrive. That was the first time I saw their site. That's when I also found their confirmation mail in my junk with reference number.
So yes. I must have given them my card details, assuming DVLA charge for the service. Not at all stupid if you to got a government site you follow the instructions on screen. If it says there's a charge, you pay, rather than google first if this is a legitimate charge.
I was apparently clever enough not to sign up for a contract as on contacting BritishDrive with the reference number they told me I don't have a membership.
My bank refuses to block them as they 'offered a legitimate service' and I received my driving license.
I still think it's a fraud. If I walk into a post office to post a card and I then get funneled through a hallway into a different room that looks exactly like a post office and the people in it are dressed in post office uniforms and then tell me that in fact the postage on my card is wrong and I need to pay double, I wouldn't dispute that if I have no experience with posting cards before.
It's a very clever fraud as technically you paid and got a service.