It is quite common for children of many ages to steal money, tho' he is quite young, it's normally slightly older children that do it. That might be in your favour; you can tackle it gently, as it looks as if you have and possibly nip it in the bud.
One point though that I'm wondering about. Why is it so wrong of him to take 'things of yours' like paper clips and cotton? Is it a bit draconian to have a lock on your office door? Harmless bits like paper clips and cotton 'to make things' sound as if they should be freely available and not be 'owned' as such.
Maybe some regular pocket money, rather than a 'virtual' account might help? My DH has always been a fan of an account (which you write up in a booklet) which he had as a child. I've always felt that it should be real money that children can handle and accumulate and count that is important, esp at a younger age.
If he is stealing to give, it might be worth looking at whether he feels inadequate in some way compared to his peers. Does he want to compete, is he buying friendship/popularity? He sounds a sweet little boy so I'd tread carefully and not threaten him with 'outings' to teacher.
I, by the way, regularly stole money for years from my parents, sometimes to give, sometimes to spend, often on things I knew I would NEVER be allowed (sweets etc..). I feel I'm pretty responsible with money now and would certainly never dream of stealing! So don't condemn him as a putuative robber!