DD1 not ASD but social/communication processing centres shot to sh1t, and has some visual issues that will get worse if she doesn't wear them, so I'll tell you how it is for us and you can see if any of it rings bells.
DD1 seems to be oversensitive to her glasses (I mean doesn't get "used" to them, always aware of them on her face) so we are always taking them back to the opticians' to make sure they are comfortable. We find that she tends to snuggle up to things, leaning on the glasses and compressing the nose pieces which then means they need to be adjusted.
The next thing is they have to be difficult to get off - curl sides essential but I'm guessing you've got those.
Lastly, pair them indivisibly with any motivator. We started with: "glasses, food. No glasses, no food". So they are always on at dinner and the moment they come off the plate is taken away. Repeat the mantra very calmly, replace the glasses and insert food straight after. The down side of this of course, is that "I'm full" is signalled by hurling her glasses across the room, lol! Then we moved on to other motivators, so now she has to wear them to watch TV or have a bath. If she's watching TV I have the remote in my hand. The minute the glasses come off the TV magically "dies".
All of this doesn't fix the problem of course but it improves it a bit because it means there are chunks of time where she does keep them on reliably, so she doesn't get into an ?escalating habit of removing them. I find these phases of glasses-hurling come and go and you just have to keep on top of it, but as you say without making a fuss. Nursery (SLD nursery so high staffing thank goodness) are very good and send a note home in her book saying how often the glasses have been taken off, so I know how she is doing there.
Don't know if any of that helps - obv you have to interpret for your different situation and just take any bits that are helpful!
I wonder also if maybe he is using his "no glasses" time as a retreat from the world - maybe he sometimes prefers the sensation of being out of focus? In which case, ghastly as it is, maybe patching some of the time would actually be tolerated better?