At least teachers actually have those holidays. University staff do not all have the whole summer off. Dh has been at his university for years so get the maximum of 35 days a year leave. Generous by many standards but it spread across the year. Plenty of academics take 2-3 weeks in August as leave but certainly not the whole summer. DH often loses some of his leave as he can't take it all as there is so little space in the academic calendar.
The summer work involves
marking and exam committees, attending summer open days and graduation. Setting and marking resits and resit exam committees, research bids, papers. Conference attending, preparing /updating lectures and student materials, for the new year. Liaising with admin staff (who have a max of 25 days of leave a year) on timetables for the new year, and induction preparations for new students. Dealing with clearing applications in mid August is a big one for DH.
That is just off the top of my head, I am not an academic so I am sure there is more. Yes it is less work than in semester time but by no means is it 'free time'
I think university teaching staff have it worse than teachers in terms of public opinion as to how 'easy their job is'
I am by no means saying that teachers and university teaching staff have harder jobs than others, they definitely dont - ,just that everyone and his dog seems to have an opinion about them both, based on very little knowledge of what the job actually entails.
Everyone thinks they know what the job is as most people have some contact with teachers and also a lot have contact with university staff, whereas in reality they only see maybe 30-40 percent of what the full job entails.
With other jobs people either place more value on them ( doctors etc) or know less about the jobs so make fewer assumptions.
As for teachers moaning too much. There is a teaching crisis. They are leaving in droves. Maybe we should believe than when they tell us how stressful it is? Surely they can't ALL be moaning Minnies