Calories in/calories out (CICO) has been debunked. The only way to access stored fat is to lower your insulin production as you cannot access stored fat in the presence of insulin. By reducing carbohydrate you drop your insulin and in order to run, your body turns to your fat stores, (once it has used the glycogen in your liver first and your muscles second). This is why ketogenic diets work. This is why the paleo diet works. This is why the carnivore diet works.
The second you eat carbs, your body runs on carbs instead of ketones/stored fat because the circulating sugar that it is turned into by the liver, is damaging to our cells.
If you keep topping up your carb load all day and half the night, your body cannot ever access your fat stores because of the insulin in your blood stream and this is why CICO doesn't totally work. You can be in caloric deficit but if the calories you have had is carbs, insulin is present and nothing will change. You just feel ropey af.
OP if you are still reading. Your DH probably used the 'royal we' to be nice. He is being nice. His thoughts and feeling are valid as are yours. He's not shallow or horrible or a woman hater. He knows his own mind is all and on threads on here where posters have said that they no longer fancy their DH because he has become massive/stopped washing/ stopped going to the dentist or started drinking five bottles of vodka a week, those posters are told that they are right to feel that way or at least that their feelings are valid.
He has been decent enough to raise it and by the sounds of it, in a kindly way. You have a choice of what to do with that information.