Sometimes these surveys really are anonymous, as in the responses cannot be matched to individuals.
Sometimes responses could in theory be matched back to people; but if promises of anonymity were made, it's down to the integrity of those involved not to drill down to that level of information.
If anonymity (as opposed to confidentiality) has been promised, they would be putting themselves in an awkward position to follow up in a way that makes clear this was not honoured.
Most likely scenario in relation to what you describe: as a result of issues you (& perhaps others) have raised, they are following up at a team/ department level. i.e. it's very possible everyone closely linked to those colleagues is being asked to discuss things.
Don't assume they know what you said, let them tell you what they want out of the meeting.
The badmouthing you were warned against is very different from responding to a survey they asked you to participate in, so dont worry too much about that.
Although a degree of professionalism would ideally guide the way you express issues - something to bear in mind in the meeting and future surveys.
Good luck!