Ouch - but how very male, to say something they think is honest or even helpful and then not understand that it's incredibly hurtful. STBXH used to come out with "I don't need to think of anyone else because I remember how nice you USED to be"... and then expect me to be happy about it. At least yours apologised!
I do strongly agree with sallysparrow that he should get some help for his sexual hangups, but alas, that's usually the last thing a man will get help for because he would have to admit there was a problem. Obviously there cannot be anything wrong with him in that department because it would hit right at what he feels makes him a man, so it must be your fault. Silly, I know, because anyone can get a problem with any part of themselves and it doesn't make them less of a worthwhile human being, but men's love affairs with their bits is something I'll never understand. Perhaps a sex counsellor would, if it's not something the GP can help with.
One thing your H seems to have a fair point about is that the counsellor does seem to have taken sides. It is not a counsellor's job to judge between partners. He may fancy you or liken you to his younger sister or something, but whatever it is, and however good for your ego, it's worryingly unprofessional. OK, your H wouldn't be happy with any counsellor who doesn't agree that you are being silly and what a wonderful partner he is, but perhaps you should meet his concerns halfway by trying someone else.