I hate having anyone in too @Whyohwhy99 - I dread it. I anticipate the arrival of the notification letter weeks before it arrives. I have had warning letters in the past and my house is a grubby hovel of which I am profoundly ashamed. But it is a condition of my HA tenancy that they must do these checks once a year (as must all landlords); I don't want to risk my tenancy or any evictionary actions, which are a real risk if access is repeatedly denied. I am ashamed of my home - but I am very lucky and very thankful to have it.
Let them in, hover outside while they do their thing and get it done before the warning letters start. The engineers are not there to chat or to pry; they have to get as many checks properly carried out in as short a time as decently possible and they need to crack on. It's not a "home" to them; it's a place of business. In - job done - gone. I do make sure my bedroom door is closed and blocked, but that's the only outward manifestation of my own fears.
They don't know you; it's a short necessity; it's done and that's it for another year. Think of all the angst and despair you save yourself by just keeping the first appointment and not putting yourself through the torment of delays, more, build-ups and warning letters. The first visit avoids ALL that and, as you say, you will let them in eventually, in the end. So snip off the dread and cut it off at the first stage. In - out - done. Quick as that. As @maslinpan says above - you can let them know your concerns in advance. But do let them in. They will not judge you - they're there for a specific, short-lived, task and that is all.
The saving grace for me is that my dog invariably loves them and is always pleased to see them (the last one said that he had to do a 40 minute return drive to his previous client because their cat went and sat in his van while he was working, nobody noticed and the cat was only discovered, purring in a tool box, when he arrived at his next job)!
It will be alright. Honest. x