Could you tell all my [male, incidentally] colleagues that, given that they're making all sorts of "fidgeting and silly repetitive little movements" in said meetings by answering their emails/checking the news/reading Twitter on the laptop, opening their paper mail, reading their papers for the next meeting, fiddling on their phones, clicking their pens, slurping tea loudly, doodling and making disgruntled sighing noises, often while whispering to each other? Yet these all seem acceptable to the powers that be.
I often knit quietly and fairly inobtrusively in the meetings, and I started doing that because their goofing off bothered me. Knitting helps me ignore all that and concentrate on the content and contribute.
Basically, 'knitting in meetings is bad' (and apparently, judging from some responses, causing the Downfall of Civilisation and such) is a very blanket statement. Depends on the workplace, depends on the meeting, depends on the attendees and the business being done, depends on the physical space arrangements, depends on the knitting project, depends on the needles.
It's like the person last week who kicked the passenger in front's seat instead of just asking her to put the seat up during the meal. Can you not just say, "I'm sorry, I'm finding that very distracting/uncomfortable, would you mind holding off until later?"