Polly,
I second what weewee said. I don't think that blokes really understand that it's not just about not getting pregnant month, but there are also concerns about getting pregnant, especially for us BESHies, as every month that passes takes us statistically further from our goal.
I started this game whenI was 34...within the recommended age range for ttc #1, and now I am 38, FFS!
Also, we have all got a life and a job, and frankly, TTC and the possibility of pg affects decisions that one might make...(e.g. me and the boyf bought a big 'fuck-off' house 3 years ago, thinking it would be full of kids by now, when in fact it just equates to a massive mortgage and heating bill!)
Your man is probably secure in the knowledge that he has many years of fertility, so can't what you feel about it.
Do bear in mind that, for him, he might see you getting really upset, and one of the solutions is to lay off the TTC for a while. He clearly feels that this is consuming you and affecting family life, and just wants the 'old' you back. There's nothing wrong with that, but he is a bit insensitive in the way he expresses it.
Go ahead with your FC appointment - if anything they might give you Clomid or something, which would not affect him anyway. Also, your chap might be anxius about being dragged in to get his sperm quality tested...
I suppose, what I am trying to say, is that you have a family who clearly love you, but don't share the yearning you have to extend their family unit! I am sure that you will feel better in a couple of days' time...you know what it's like in this ttc game!