Wow we are wearing our extra large judgey pants today aren't we? 
Those declaring size 14 to be an unhealthy size must be utterly surprised that I, as a size 22 am still standing. 
I have recently lost a lot of weight (6 stone) this is about half of what I need to lose to reach my 'healthy BMI' target. I've gone from having a BMI of 53 to 39. So officially I've gone from being morbidly obese to just plain old obese. :) My weight loss is on the back burner at the moment as I've become pregnant after years of TTC (possibly because of the weight loss) so I'm keeping up the healthier eating mostly but not actively trying to lose more. Though I do weigh less at week 20 than I did at booking in helps that booking in appointment was just after Christmas
I would probably still call myself fat if asked to describe myself. I do have quite a bit of it under my bump. But I find comments like 'Wow, you are looking good/slimmer/healthier' more motivating than 'Gosh, you're still big/fat/obese'.
I hope I haven't been insensitive to my DH since I started losing weight. He's just in the obese category (BMI 32) and weighs a little less than me got me pregnant just soon enough to stop me from overtaking/undertaking him I do sometimes get annoyed when he complains about his weight and wanting to lose weight without doing anything about it. I understand, because I used to do the same, but he must be able to see that I had to change loads to lose weight, and he's annoyed because he hasn't managed to lose more than a few pounds over the same time - but he's still eating much more than I do. I think its one of those things though that you have to be in the right frame of mind to change.
A lot of our eating is/was emotional, and in order to lose weight I had to really internalise the lesson that food could only make me feel 'better' momentarily. At the moment, DH feels he needs that momentary comfort even though he knows it doesn't help in the long term. I just have to be ready to support him when/if he's ready to lose weight or if he decides he's happy where he is.
Obviously our baby will be the focus of our attention once it's born. But once I've recovered from the birth and established breastfeeding, I'm going to get right back into losing weight so that my body is more healthy for any future pregnancies and so that I can model a better relationship with food/exercise for baby.
It's up to you how you feel about being a size 14. I know that fom my perspective it seems slim because I can barely remember fitting into size 14s (have a strong memory of my sister and I going clothes shopping and not fitting into a size 14 and the sales assistant saying loudly 'your friend needs a size 16' and my sister replying 'shes not my friend'
I would have been a young teenager). However, others feel that a size 14 is big for them and they'd start losing weight at that size.
If the comment made you feel bad then please tell him that it did and that you didn't realise he saw you like that. I'm sure he'll be more sensitive in future if he's normally a good guy.
I think the problem is that tendancy to associate uglyness/unattractiveness with being called fat. He probably didn't mean it that way at all, he was probably making an offhand comment about size (perhaps even thinking of you before you lost the weight) and not about your attractiveness to him.