I'm so sorry this has happened to you. I found out I had lost my baby yesterday at 9 weeks, and I am dreading someone saying "be grateful you have 2 already" as I think I think I would slap someone! To be honest, much as I was terribly sorry for friends who have gone through this before, and would have loved to have known the right thing to say/do, I can't say I understood properly just how horrible it is until yesterday.
I understand the feeling a failure; even though my rational mind is telling me this is just a random awful thing that happened, I still apologised to my husband for losing the baby.
Like Lucky said above, having children already is really hard, as one look at them is a reminder of how much you have lost.
The other thing I feel, is just tremendous sadness that there is no proof the baby was there, and that it will be forgotten. I am going to make a little book of the few things I have (photos of the empty sac from the private scan I had yesterday, and the report the consultant wrote, a poem I wrote for the baby, nice e-mails I had from my friends offering their sympathy, etc). If I want to talk to the baby, I will write down what I want to say and put it in the book - maybe this will help when it comes to my due date for example. I think we will have a memorial type thing just in the garden too, maybe release a balloon or something just to say goodbye. I think these things will help me personally, I'm not suggesting they would be right for everyone.
I guess what I am saying is that you have every right to grieve in your own way and for as long as you like, and no-one has the right to deny you this. It would be unhealthy to do otherwise.
Maybe the "relief" your OH seems to show is his reaction to his own grief? Obviously I don't know him or you at all, but could you talk with him about it? My OH always copes with things very differently to me, and sometimes it can be hard to understand, especially when things are traumatic enough already.
I do hope it will become less painful for all of us as time goes on, though of course we will never forget. x