Our bodies can choose where to take the calorie deficit from. So, they might decide to drop fat. This is their least favourite thing to do, though, because fat is your body's 'energy savings account', and nobody wants to drain their savings account.
So, more likely your body is using less calories to do other stuff, like keep your skin fresh and keep muscular tone and regulate your hormones and keep your bones hard and fight infection etc etc etc.
The body doesn't just use calories to gain/lose fat. It's doing a million other things, too. This is why people go on calorie controlled diets, maintain their weight, and feel more and more rubbish until they give in and return to their original calorie consumption. But by then, their body is needing less calories for all the functions, and so stores more of the consumed calories as fat.
In a nutshell, that's why calorie controlled dieting doesn't work for everyone. Some people's bodies will choose to burn fat, some won't. That's why some people say it works, and for others it won't.