A bit late to the party here but just chiming in to say that I'd never wanted to do IVF and was fairly horrified by the prospect of it. DH and I had promised ourselves we'd draw the line way before IVF. We'd always said we'd accept that it wasn't meant to be, and bow out gracefully. But that was before we'd had a full-term stillbirth & a couple more miscarriages.
Eventually, as I got towards the end of my 30s, we grudgingly decided we would try IVF. When the time came for my first appointment, I found I was pregnant naturally. Hurrah! Then I had another miscarriage at 9 weeks. Made another appointment to start IVF 6 weeks later. Bottled it - just couldn't do it, it made me too sad and frightened. The following month I forced myself to go through with it - and what do you know, it resulted in another miscarriage. We will probably not do it again, just because I can't take any more loss and I am not convinced it can help with our situation.
I think if your fertility problem is mechanical or hormonal (sperm not getting to egg for whatever reason / ovulation problems or egg maturity) IVF is very good at overcoming those issues and more likely to be successful. Less so if your trouble is 'unexplained', or due to poor quality sperm or eggs (which is likely the case for us) - that's when it becomes much more of a numbers game - trying to get as many eggs fertilised as possible in the hope there are a couple of good ones left. This often means several more cycles than you might have if your problem is, say, blocked tubes or low sperm count.
As others have said, physically it's not so bad. It's the emotional side that is the hardest (and knowing you're paying so much money for so much heartache). For me, after so many losses, I couldn't bear the sense of failure and disappointment at every step of the process: not responding well enough to the stims, an unexpectedly terrible fertilisation rate, not having enough embryos to do genetic testing, etc. It felt like every other day I was told of some novel way in which things were going badly.
Something I didn't initially realise about IVF is that you need to be prepared to do multiple cycles. I had a consultation at one clinic who said I had a 56% chance of success - I queried this, as other places had said my chances were much lower. They clarified they meant a 56% chance over 3 cycles. It's just what they assume you are going to commit to at the outset, as the first cycle is often experimental - where they discover where your particular problems are and how you respond to the drugs.
It sounds as though it's mainly your DH who is unhappy with the idea of it. The risk of twins/multiples was initially a worry for my DH, but it's by far the least likely outcome of a cycle IVF (when I had treatment age 38 we were told we had an 8% chance of twins from our 2 embryos - a 12% chance of a singleton, and an 80% chance of nothing.). At 36 you'd have to push for them to put back more than 1 embryo anyway - I think these days they won't put 2 back in until you're 37 (but I could be wrong).
My sense from talking to others who have done IVF and feel happier about it than I do is that it's not so bad if you're OK with multiple cycles, and you know the risks, and feel confident that it gives you the best chance of having a baby - and you're feeling fairly robust in general.
Good luck.