"Oh, so sorry but it's going to be very small. We do plan to have a big party sometime later on though."
If anyone wants to argue, just repeat, "Oh, I'm sorry," accompanied by a sad face. Don't engage with any further explanations or answer any challenges or respond to drama. If they don't accept it after being told a couple of times, get off the phone or change the subject or find a reason to leave or turn your attention to something else.
It doesn't matter if the cousins invited your husband to their weddings; they obviously had much larger weddings. Also, this isn't even the cousins themselves arguing.
People like to push but they will get used to it not working if it doesn't work. Just don't play and they won't have any choice.
With such a small gathering, I certainly wouldn't want 20% of the people there to be ones who were nasty to me anyway.
If you do have a large party later on, I'd try to get rid of the cousins then, too. I don't want people around me who don't treat me well. Cousins typically have a very small part in your lives once everyone moves on with their own families anyway.