Surely you have some redress here. I agree, down to the solicitors with you. Make a list. You can't be expected to have him in the house at all if he is going to steal from you.
I would take him to the cleaners if at all possible. These people must not be allowed to think they can get away with it.