If you want to get a good feeding relationship established, I'd strongly suggest taking a couple of days out and retreat to your bedroom with your baby. Take whatever you need to keep you sane - Box set DVD / computer / box of chocolates etc, and just hang out in bed with your baby, do lots of skin to skin and feed him on demand. Ditch the formula (as long as he's feeding, of course) as formula can really mess up breastfeeding.
Housework, visitors etc can wait for a couple of days, this is important :)
For a couple of days, don't worry feeding too much, just let him feed when he wants to (so long as he's feeding and producing wet nappies).
From your post, if he's feeding a lot - (especially in the evenings), this is not because he's unsatisfied, this is "cluster feeding" and is totally normal. Your body produces milk on demand. He is feeding lots to let your body know it needs to produce more milk. But for the cluster feeding to work, you need to stop using the formula or giving milk in a cup, just breastfeed him.
Also, feeding 40 minutes then falling asleep sounds normal. Your boob can't be "empty", your body makes milk on demand. You can mess your supply up however if you supplement with formula. (If you're happy to end up on formula, that's fine, but if you want to breastfeed, you have the best chance if you avoid formula AFAIK).
Also, a baby will automatically take a bottle if you put it in their mouth, won't they? I don't think this is a sign that he's still hungry necessarily. If he's breastfeeding for 40 minutes, that sounds good. Perhaps he needs something else after? Winding? Or maybe if you feed him in bed for a couple of days, you can just let him stay asleep as you won't have to "put him down" after he's fallen asleep, you can just leave him where he is.
The signs you should be looking at to see if he's getting enough breast milk are that he's putting on weight and having plenty of wet nappies. I don't read anything in your post to suggest he's not getting enough milk from you. Not sucking vigorously is normal, and falling asleep at the breast very normal indeed. Sounds like you're actually doing just fine :) All you need is a bit of faith in yourself and nature. And to ditch the formula!