Some letting agents know the law and are very professional - e.g. my current agents are scrupulous about giving a fair amount of written notice for access / inspection.
Consequently I've developed an excellent relationship with them, and have (for instance) no problem with their trusted maintenance man letting himself in while I'm at work, if that happens to be the best time for him. If the landlord needed to get something out of the shed, I expect he'd be a tad apologetic & I'd say 'Don't be silly' & let him in.
I've also had a dreadful, unprofessional letting agent / landlord combination who let themselves in when I was out (or when they thought I would be), amongst other deceits such as lying about having put the deposit in a protection scheme, faking gas safety checks etc. That particular letting agent had no qualms about breaking the law or harassing a tenant when he thought he could...
I changed the locks 
and moved on as soon as I legally could. From what I gathered on Landlordzone at the time, changing the lock to protect your privacy is fine (though it would probably sour the relationship a bit if you did it when you didn't need to), provided you change them back without damage at the end of the tenancy.
But if you don't give a set of keys to your landlord or agent for emergency access, then they may say you are liable for any damage caused if there is an emergency (like fire / flooding when you are away & unable to grant them immediate access) - that's a chance you have to take, and was one reason I was glad to get out of that situation ASAP before anything like that happened. (Place was badly maintained, making that a genuine concern).
If your letting agent is as dodgy as your landlord, then there's no point lodging a complaint with them.
ccpccp - the OP doesn't say she is behind with the rent 
just that the landlady tried to evict without due procedure & that she stood up to that. There may well be 2 sides to the story but that's a bit of an assumption.