Sorry, but I too just wouldn't have let that explanation by your DH cut it, I'm afraid.
When your BIL was actually insulting you to your face, in your home - 'Well, you could be more solid' etc., what was your DH doing? Just sitting there, head down? He was being insulted by his PRICK of a brother too, right? Did he just smile and nod and watch you well up? Make a few 'oooh now hang on' pathetic little noises?
I would be really really angry. And - sorry - but you are NOT. You are meek. Politely asking if you can just point out that he didn't stand up for you... and then taking it completely at face value when he then launches into a big description of how when you weren't there, he did?
That isn't good enough. I think you know that although I'm sure your DH did speak to your SIL, it will have been just as meek and conciliatory a conversation as yours with him. And that is not what is needed. WHY did he not speak to his brother DIRECTLY, as in - 'You've really upset us both, I am PISSED OFF with you and your RIDICULOUS attitide - it'll be a long time before we repeat this invitation'.
There's no way any sex would have been happening in that aftermath, let me tell you! And I think that that totally sent out the wrong message to your DH - that essentially you are fine with things, which you aren't.
I hope I don't sound harsh, it's just that the progression of events here has shown me EXACTLY WHY this situation happened. You really, really need to let yourself show to your DH that when you are pissed off, you actually ARE pissed off. You need to stand up for yourself more. Having a peaceful, non-volatile relationship is obvously really good, but that can go too far in that if you are both pretty passive, what happens is that when there's outside conflict afoot, such as Arse Bil, it's easier and nicer for your DH to let you down than let down/upset the other person - because he knows that you will sit there and take it, and that if you get upset, you will still do that from a point of not blaming him - you may cry, but you will let him comfort you. You will talk it through, but you won't make life hard for him. And the basics - sex etc. - are all still there.
There has been NO COMEBACK for your DH here. And as he seems an equally passive person - ok, a bit spineless, at least as far as his brother is concerned - then this won't have been solved. Your DH is sat firmly on the fence, I'm afraid. If you'd heard that conversation, I'm sure you wouldn't have felt stuck up for at all. I'm sure it was full of 'I'm sorry, but it is a little bit upsetting for lightsandshapes at the moment, she is feeling very sensitive...'
So far on this trip, you have been insulted and upset and BIL has felt no comeback, I can assure you. There will have been no indication from your DH that they are now not welcome to visit, and the moment will have been lost. So next time? Well, they want to come and see the baby, don't they? and spend a nice few days with BIL telling you everything you're doing wrong and then next summer, well, baby's first summer so let's get the kids together, yes? and they can swan in and spend a week upsetting you while you have a small child to look after
So right now, I would be saying VERY PLAINLY to your DH - 'So, right, you stuck up for me. So I take it that means that they know that for us it isn't going to work having them to stay again, yes?' And if he hmms and harrs about that, then you say pleasantly, 'Well, this isn't happening again in my home, so I want you to speak to BIL and make it clear that this hasn't worked, and if they wish to visit again, they will be staying elsewhere and making a day visit only.'
I'd like to see your DH's face at that point.