I've probably got a bit of an overview of this issue. I do long-distance open water swimming, and pool training. Say, 3-6 hours in cold water or 6 miles in the pool, and some speed sessions inbetween.
I've given up running last year as i got injured, so just do swimming, weights, spinning and yoga. I'm bigger and fitter than ever before and always have a layer of fat than doesn't shift. My diet hasn't changed much at all and i don't think Im hungrier than when i was a runner (I'm certainly more tired though). When I was training recently up to 5 hours a day in the sea, I didn't feel any hungrier than I would have done normally.
I think my body 'holds onto' fat more, probably because it panics everytime I try to freeze myself
. Psychologically, I wouldn't want to be less fat as I'll get colder, so maybe I eat more than I think. I always eat every half hour during a long swim anyway, so maybe that makes a difference to what I eat afterwards.
Sorry - not very helpful but it is something I have thought about. I think if you really cut down on your calories and trained hard, you would lose fat. But I have never seen a skinny open water swimmer (unless in a wetsuit).