I know exactly how you feel. Iāve had a holiday home for about 30 years and let my close family & friends use it for free or for a small amount to cover water/electricity.
Some are more generous than others, as they know I also pay council tax, ground rent, club fees etc. One friend who has used it scores of times, sometimes for free, has let other people use it without my permission. Once saying it was just her & her boyfriend, I later found out it was also her cousin & aunt!
Then another time let another cousin & her husband use it, again without checking with me if was OK first. Added to that they were both really big people, so Iām surprised that the ancient bed (at that time) survived.
Unfortunately she still has a key but itās unlikely Iāll let her use it again. I have ways of seeing that she doesnāt on the quiet, as I have a water and electricity meter that I can check. Sheās the only person whoās abused my trust but it does make you wary of being too generous. Maybe people appreciate something more if itās not for free?
Other people have been great, often paying a little bit more than I suggested or buying me gin because they appreciate my generosity.
Probably best that you have a word with him and set some boundaries, in case you get taken advantage of like I have.