I think it really depends how close a friendship you have with them.
1)Do you already regularly socialise with them outside work for non work reasons, eat lunch together most days, know the names of their partners, children and pets, do you have their personal phone numbers and email addresses, are you on a WhatsApp group with them?
Or 2) are they just the people you like best at work because the others are toxic, but really your relationship starts and ends at the office door and you have no contact with them outside work.
if it is 1 then chat to them about the invitation and explain your problem, they will understand and either decline the invitation to cut off the gossip, or may accept and offer to have your back against the bullies.
If it is 2 then, yes you like them, but they are not really your friends, and although they probably appreciated you sending the save the date they will not be offended if you withdraw the invitation, because let’s
face it, most of us have better things to do than to go to a virtual strangers wedding.