Wow, what a year... I have a lot of respect for you actually. Why would anybody judge? You weren't in a relationship at the time (for many people that in itself is where they would judge I guess but plenty of people make mistakes), and for the foreseeable future you weren't in a relationship.
I think you are feeling bad because you feel sad for your fiancé. Obviously the circumstances of the break up etc are none of my business but you make it sound like it was mutual and the sad bit is you wanted to make it work this time around. Then you found out about your pregnancy... And feel you have messed up again when you just wanted it to be right and forever. Of course, you can tell me I'm wrong, but that's how interpret it. The bad thing about guilt is that it eats away at you, and two things can happen:
- You change the way you behave towards your OH - perhaps trying too hard or making sacrifices because you feel you owe it to him
- You become very defensive about things which can affect your relationship.
Look out for these. (Btw I have no experience nor am I qualified to offer advice but I'm just writing what comes into my head and seems to make sense to me! Take or leave or believe what you like!). Your fiancé wants to be with you because of who you are - don't change that, just be yourself.
So, it's early days and I imagine you will always feel guilty, but start getting over it as much as you can and get back to being you. You can't change the situation. If your fiancé decided to walk then that would be his choice and I don't think you could blame him. But he didn't. He's told you what his decision is and what he wants to do and you have to respect that. He loves you, and he will love the baby you have made because it too is part if you, I'm sure. He loves the fact that you feel so strongly for your bean and can't face aborting it - and if you care about something he will too. It sounds like you have a pretty awesome guy there...
Yes, he will probably always feel a bit sad - but he has kids from a previous relationship too. Granted your intimate relationship with your friend wasn't exactly long term... But things happen. You both accept that you have lived and loved before, and this is your new chapter. It's not unusual to have step children that you love like your own. And apart from the genes, everything else about this child will be his - he will bring it up, teach it to ride a bike, take it to school, change its nappy, hold its hand, kiss its grazed knee - it's his, the other guy disowned it. And its yours, and he will see you in the child. If he can accept all this then you have to as well, and make it any harder on yourself, or for him.
I don't think I can suggest that you stop talking about it - it's too early for that, but it might soon be time to start sweeping up any egg shells you are walking on. There will likely be times when he gets sad and down and you will need to let him have some space, and times when he needs a hug and reassurance that he is the one for you - and how his amazing behaviour and support makes him the best thing that ever happened to you - but so long as you show him every day that he is the most important person in your life then that is all you can do.
You can't predict whether your relationship will stand the test of time, or whether things will work out, but that isn't your fault. Both of you may have more decisions to make that come about in the future and you will have to tackle these at the time and respect each others' feelings. You only have the here and now, and the decisions and situation you have now, and you have to live for now. Who knows what might happen tomorrow anyway? Take every day at face value. If you want to make it work, respect your current decisions as a couple and embrace them. It is no help to worry about the future, it might never need worrying about. And if it does, then worry about it then!
Good luck with everything - and stop blaming yourself quite so much! You aren't the only one to have made a mistake with contraception... Which, ultimately, is what happened. And your "mistake" could well be one of the best things that ever happens to you!
Look after yourself and your fiancé, just be yourself 
PS The hormones won't be helping... So keep things in perspective and ask for help from your GP etc if you need it. 