It’s great that so many people can stick to their set hours, but certain jobs are output based, meaning in order to meet those particular tasks you need to put in some extra time. If you work in one of those jobs you either become a lot more efficient at getting things done (which some people are good at) or, most often, accept that you need to do extra but are flexible about when you do that, and know it will vary.
OP accepts some extra hours are inevitable in their job, but the galling part of the situation is that their boss seems to be claiming the start and finish hours differ to the contact, and is saying they are late when they are in fact early. This needs to be dealt with, as this kind of attitude is what makes employees feel unmotivated and want to leave an otherwise enjoyable job.
It sounds like boss is a bit of a bully, so I can see why it feels difficult to tackle it, but you have to do it. If there are enough staff willing, the best thing to do is band together and perhaps have a group meeting with the boss, or individually send emails setting out your concerns on the same day. Do you have a HR manager you could use as a buffer?
Otherwise, you will have to just set out your own expectations - say that you have always gone above and beyond in this role, but going forward you will need to choose when any extra work beyond your contractual hours is performed, to suit your schedule. If the boss says it’s essential to do more every day then you need to say that they have therefore set an unrealistic workload for your role.
This is why unions are helpful, but you need a certain percentage of staff to be in the union for the workplace to formally recognise it. They could still give you advice if you join though.
Good luck - let us know how it goes.