It never goes away does it, that craving? 
You are further along than me, my two (both with autism) are only 13 and 11 and I know it's different because they are never likely to be independent in any meaningful way, so I can't totally relate and don't pretend to, but what are the consequences to him of treating you this way?
My youngest in particular doesn't really give a shit how he makes you feel. He doesn't get it. He certainly doesn't factor in the feelings of those around him when he does stuff!
So he needs painful (not physically!) consequences for unacceptable behaviour, because I can't rely on him ever not doing something because he doesn't want to hurt or upset me! He won't even think of me, iyswim.
Your son's behaviour needs to affect him. It needs to bite. If he's like my youngest, you can tell him that he's hurt you until you're blue in the face and he'll do the automatic "sorry" because he's learned that's the required sound - but he won't actually give a shit!
Is there anything you've got? anything at all? I realise at 20 it's going to be difficult if not impossible.
There is also the other option. At 20 he is an adult. He is entitled to support as a vulnerable adult, independent of you. You could hand over his care to social services, take a step back and have a different relationship with him - ie not his carer / financial advisor.
Easy to say though my eldest is only 13 and i'm already having the cold sweats at the thought of him being 'out there'. Even though 'out there' in his case is likely to mean supported flat.
It's a tough one. In many ways, it's easier when they're children, isn't it?