I really hope you might still be reading OP.
This is intended to be helpful, not unkind in any way.
If your husband has served time for a crime that involved a child, it would serve you much better to expect that people might be wary of that fact and to have developed some strategies between the two of you to cope with any negative comment or fall-out. Especially if he then makes some kind of dodgy remark to a child (even in jest) which could easily be misinterpreted. This would help prevent further upset for you. Do you think you could handle any similar situation without escalating if you talk openly about how to deal with it better if there is ever a next time ?
He may have served his time OP, and now be a really lovely, trustworthy, hard working man, but even your own mum thought he should apologise for the misunderstanding caused by whatever it was he said. Don’t you trust and value her opinion OP?
I assume he didn’t apologise or make attempt at least to explain himself or to communicate reasonably but instead you immediately took offence and went home so upset, raging and nursing your wrath between you that you then made the grave mistake of calling your brother about arrangements to get together at Easter time as usual, except you would have to do it separately away from his wife.
Do see how that may well be perceived as antagonistic on top of their own upset at the situation. Why remind him to co-ordinate easter arrangements in the same breath as saying you would keep separate from his wife. You were lashing out in anger to punish SIL and that never goes well. It could almost be taken as you trying to split the couple’s marriage up by asking your DB to choose between you, his sister, - and his wife. What were you expecting him to do or say?
Your SIL then also responded in anger herself, by talking to a friend about what had happened and it snowballed into gossip with the result that you have become alienated from your mum group.
I can quite clearly see how upsetting and impactful that must be for you to live with. It’s very sad.
Please take time to try to see this from all angles.
You could then ask to talk to your closest friend in the mum group to explain how the misunderstanding came about, including any mistakes you made in handling the entire situation. Owning the consequences of your own actions often goes a long way in allowing the truth to be seen. And by your account there is nothing terrible to hide so you can be quite open.
Hopefully, you will be able to resurrect your standing as a good, reliable, calm, honest, up front member of the group without any further drama and can gradually get back to normal life over time. Wishing you well.