How often do universities get the equivalent of Ofsted?

[eatyourveg]

Been looking at the QAA reports for a few places and some are as old as 2009

Does anyone know if there is a schedule similar to Ofsted schools and colleges inspections whereby if you're doing well they leave you alone for a while and if you're not doing well they come back a lot sooner?

[titchy]

Don't forget also that every module of every course will be marked and then a sample externally marked so lecturers who mark inconsistently, or whose student scripts are consistently lacking the required quality are picked up pretty quickly.

[eatyourveg]

I asked the question because ds3 is now pouring over various prospectuses as well as unistats and the Guardian, Times and Complete university league tables. So far he's narrowed it down to 20+ courses in roughly 14 different institutions.

All of them are throwing out statistics about being in the top 10 for this or top 20 for that and although a fair bit of it is traceable, neither of us can find anywhere to verify some the claims being made.

Its particularly the teaching excellence rankings, I'd like to find. Are they available anywhere?

[titchy]

The league tables and uni stats both use data provided by the higher education statistics agency so it is genuine and traceable. The methodologies and sources of data are on the league table websites, but there is no actual measure of teaching quality, although percentage of 1sts and 2:1s and maybe student satisfaction will act as a proxy for teaching quality.

[JeanneDeMontbaston]

There are student satisfaction surveys, but they're worth taking with a pinch of salt. Students are asked to participate, but often don't, and inevitably, since their responses are self-selected, you can get very unreliable data. There's also some evidence that students are less satisfied with tutors for whose courses they end up getting higher marks (IIRC this is US research, though, so might not really describe how students are in general).

There is an external board that deals with unresolved complaints by students, FWIW, so universities can't just do what they like.

Personally I think it is a pity that the REF is so much more famous than any measures of teaching. You probably know, but the REF (mentioned above) actually measures quality (and quantity!) of research output, not teaching. So, some people contributing to a university's high REF scores will be doing little of the teaching, although obviously, it's nice to be taught by someone who is able to give the inside scoop on new research.

Most places will tell you their stats for student employment - that might be useful? And numbers who drop out?

[AllMimsyWereTheBorogoves]

The most reliable external evidence on standard of teaching you will find is the National Student Survey, as far as I know. As others have said, you have to take this with a large pinch of salt, but the higher the participation rate the more reliable the results are likely to be.

Universities will do their own internal monitoring of student satisfaction but they don't usually publish it. They should also be trying to get lecturers to improve their teaching by putting them through some basic training, getting them to do peer review and monitoring marking standards/timeliness/student feedback and so on. This is patchy, though, at least in my limited experience.

The QAA doesn't observe teaching at all - at least they didn't when they last inspected my employer. As far as I could judge from my lowly position as an administrator, the inspection/audit/whatever it's called seemed to involve lots and lots of paperwork, often requested/collected at the very last minute, and various meetings with high level people and a few students.

[UptheChimney]

University teaching is constantly monitored. Every department in the country was reviewed via QAA every 5 years up to that point. QAA periodic reviews are no longer so prominent, partly because HEFCE found them so expensive. But departments/schools are still subject to periodic external review.

As well:

Our teaching is monitored, and peer-reviewed internally every single time we do any summative assessment.

We also do a module/course review for every single module every time we teach it. At my place, this has to show evidence of second marking and response to student feedback. Student feedback & our responses are published to our students.

That's then checked by internal committee processes, at both Departmental and School/Faculty levels.

It's then checked and remoderated if necessary annually by External Examiners.

There are annual Departmental reviews, answerable all the way up to the VC, and available for QA periodic review, which is organised by each university now.

There is the NSS, which -- in terms of a properly designed questionnaire & survey is not worth the paper it's written on, and that's the opinion of anyone properly trained in consumer focus group & research techniques.

This all is organised as if we actually don't know what we're doing, which is just so much BS, frankly. I know far more about what my 1st years need in order to understand the discipline they're reading, but every single administrative procedure is based on the assumption that 18 year olds know better than I and/or that I must be 'cheating' them somehow.

And most of the younger academics I know who are mobile are actively looking to leave the UK system because of excessive monitoring.

The league tables are produced by newspapers, not HEFCE, QAA or the like. So take with a huge tablespoon of salt. Look at the individual scores, or go to the source: HESA and HEFCE stats.

[titchy]

Unistats is produced by hefce, but uses the same data as the league table providers.

[eatyourveg]

NSS is purely student's rating their own experience - it could be a rubbish place but the students think its great, are incentivised to fill in the survey and so it then gets a good rating.

I like the QAA reports because it mentions the uni's strengths and areas for improvement

So far ds's refining criteria seem to be graduate employment, drop out rate, % of firsts and upper seconds, guaranteed accommodation in first year, industry placement opportunities and erasmus/study abroad options. Would be nice to have a neutral/external quality of teaching review to help cut it down a bit - it will be expensive if he decides to go to 14 open days

[AllMimsyWereTheBorogoves]

Well, it could. But by final year, which is when the NSS is completed, most students who get round to filling in a survey like that will do so with more in mind than just how good the beer is in the student union. If they've found that most of their teaching comes from lectures with 400 students crammed into one not-quite-big-enough lecture hall with the lecture delivered by a Ph.D. student reading from somebody else's powerpoint, they will know by this stage that that is not good enough. If they have constantly had to wait months for feedback and then found it wasn't detailed enough to help them improve, they will have views on that too.

Before Up The Chimney says this, I do know that not all students make best use of the feedback they are given! But I also know that the faults are not all on one side. There are places which teaching as seriously as her university/department does. But there are also places that pay it lip service. It's not that difficult to have good policies about what should happen. However, they are not always observed in practice and there is not always any real impetus to sort out problems. That's the kind of thing that is not easy to find out about before you start.

Has your son looked at The Student Room, eatyourveg? Lots of info there. More salt needed (in great handfuls, not just pinches, at times) but it might round out what the NSS says.

[JanineStHubbins]

I know of one institution where the HoD personally telephones students to ask them to fill in the NSS.

There is a huge problem developing in the UK around a consumerist attitude among students (where 'good teaching' = 'easy classes'), and the NSS merely reinforces that.

[JanineStHubbins]

And I would be very wary of interpreting the number of 1sts and high 2:1s as indicative of teaching quality.

[UptheChimney]

I'm not thinking of the students' answers to the NSS, but the kinds of questions, the way they're expressed and, the way the survey is put together -- that's what my stats & questionnaire exert colleagues object to. It's a survey that wouldn't pass muster if submitted as part of an undergrad social sciences assessment according to my colleagues. The irony ...

OTOH, that s not to say that even by third ear, students still have some odd ideas about what is "good teaching" -- some of them are rather lacking in self-reflection. For example, one year we had several of them tell us that they didn't learn anything by doing their dissertation, and they didn't know why they were paying fees for the supervision on that assessment.

It's not unreasonable to judge from that that any student who says s/he didn't learn anything by doing a Final Year dissertation has only him/herself to blame. Yet that sort of thinking is what's recorded by the NSS and has knock-on effects for staff.

[UptheChimney]

yes yes yes, JanineStHubbins!

The NSS is the National Student Satisfaction survey. Clearly, students are (generally) more sensible than to fill it in depending on the beer in the Union. But the survey is not about quality of learning, it is about "satisfaction." It's not a serious attempt to have a student reflect on his or her learning and how they got there, that is, how their lecturers facilitated their learning .

But to have students reflect on this would require qualitative research, which can't be done by a tick-the-box survey.

Too many students and their parents, I'm afraid equate teaching and learning. They are related, but they're two different processes.

I can be a brilliant teacher; if a student resists learning, there's very little I can do. Likewise, sometimes an apparently "bad" teacher can inspire excellent learning.

This is a really fundamental difference between school & university, and not enough people understand it.

I think of myself as facilitating my students' learning, by offering clear structures within which they learn about the topic. The structures I set up (eg a schedule of reading and discussion in seminars) guide and frame their learning, but it's not exactly me teaching them.

[uilen]

"There is a huge problem developing in the UK around a consumerist attitude among students (where 'good teaching' = 'easy classes'), and the NSS merely reinforces that."

Yes. In my field high NSS scores generally reflect easier courses and easy exams. There are also institutions whose valued added scores partly reflect inflated numbers of firsts. In the league tables produced by the newspapers there seem to be "fudge factors" whose origin is hard to understand.

Strangely student satisfaction is high for some departments at very well-known universities, even though these departments are considered by academics in the field to be teaching very poorly (poorly structured modules, poor lecture notes, few assessments, little feedback). So do students rate the teaching highly just because they think it must be good, as it is a "top" university?

[JeanneDeMontbaston]

It is understandable, I think.

Students fill in those surveys before they have their final results, don't they? It's natural to think if you found a course hard and frustrating and differently taught than the easier option you took, that it must be badly taught.

Does NSS correlate student reports with student results? I'd love to see how the two match up.

[slug]

Institutional reviews, as they are known, should take place around once every 6 years by the Quality Assurance Agency. My institution went through one two years ago. The results should be on their website somewhere. Or you could try looking here

I agree with all the points made about the NSS. These can be skewed in small institutions and small departments. One of our depts rarely ends up in the reports as it's small with a notoriously reluctant student body. Last year our students boycotted the NSS in protest at whatever it was fashionable to protest about that week. The result of which very few departments passed the threshold for inclusion in the results.

Having said that, drop out rate is a good indicator of how well recruitment went. Graduate employment rate is,IMHO as good a criteria as any for rating courses.

[PiratePanda]

Every freaking year.

The NSS which measures student satisfaction very very badly is every year; the Times Higher Education World University Rankings which measures everything better than most other similar rankings ditto, and the REF which rates just research is every 6-7 years.

The Guardian guide should be taken with a MASSIVE pinch of salt because it does not include research in its rankings, when research is a massive part of what makes university university, and university teaching on a whole other plane from school. Use the other newspapers.

That being said all the university league tables are really only rough guides. It's rare for a university department to entirely move out of top, middle, or bottom, but it is very common for it to move several places up and down, depending on those two disgruntled students whose single lecturer for one module in their final year was absent for two weeks with the flu and neglected to give the substitute proper instructions because they were at home vomiting .

So look at the cluster of universities that are consistently in, say, the top third of the table for your subject of interest. Then look at their offer - if they say three As, they mean it and you're unlikely to get in otherwise so no point prioritising it if you're not a 3 A student.

Then look at the departments you like the look of and see if you like the profile of the lecturers and senior lecturers (who generally do tend to teach undergraduates) and the elective modules on offer in years 2 and 3 (and 4 if it's an MSc) to see if you like the subjects and think they might lead on to something interesting career wise (or otherwise!). You should also be able to glean from those modules who the lecturer is and therefore whether or not the world-leading professors advertised as a big draw do, in fact, teach.

Cost of living is likely to be a factor; London is obviously expensive but has lots of part time work and other career boosting opportunities; big cities like Manchester, Leeds, Newcastle, etc are much cheaper to live but also have plenty of PT work. Oxford and Cambridge are both a LOT cheaper to live in for students than the usual surveys that are done by the likes of HSBC - some of the cheapest rents (and much shorter terms), heavily subsidised meals and alcohol, no need to use public transport or drive, etc, etc, etc.

And then there are all here subjective things - big, small, nightlife, urban, country, etc.

But the most important thing is that you choose the best quality of course for the grades you can attain.

[eatyourveg]

All really interesting responses - thank you.

AllMimsyWereTheBorogoves will direct ds towards tsr though it seems rather top heavy with accommodation and freshers posts at the moment

PiratePanda The Guardian seems to rank the institutions for ds's subject in roughly the same place as Complete and Times. Unfortunately his subject is not a traditional one and so whilst his top choice sits firmly within the top third for the subject, in the overall tables, its in the bottom third and almost all the others that he has short-listed, whilst also in the top third on the subject tables, all seem to be overall in the lower half - only 2 make the top 60! :(

Whilst that would not bother me if he secures a job within the subject area as presumably the institutions are well known/respected within their sector - if he chooses to follow an alternative career, I would be wary of him being penalised for not having gone to a RG or similar hence wanting to make sure wherever he goes, the teaching is quality is well regarded.

[titchy]

A degree that gives him an industry placement should balance out not having an RG degree if future employability is a concern.

[UptheChimney]

Oh and I forgot to add, every single fecking year of my career (25+ years in 3 different countries' systems) my own career progression (getting a job, keeping a job, getting tenure, being promoted, getting a pay rise) has involved reporting term/semester student satisfaction/feedback scores. Anything less than an average of 70% positive scores is looked into, or seen as a reason for blocking promotion, progression etc -- at my current place averages of 3.5 (on a 5 point scale) are considered the baseline of acceptable.

[PiratePanda]

Have you tried to ascertain the grounds on which he is making his decisions?

Don't worry about the overall table; look at the subject table. Is there a reason he's not considering, say, two of the top five in the subject table?

RG isn't everything. If what he wants to do is vocational and has a particular career path preferred by employers, then non-RG universities may be absolutely fine.

[eatyourveg]

Yes its vocational Pirate and he looked at numbers 2, 6 and 9 of the top 10 but rejected them all for being too heavy on theory and thin on work placements/practicals. Also not many of the theory based courses offer external professional qualifications which quite a few of the vocational ones do.

The leading name in the field is in the top 10 ( 4, 5 or 6 depending on which table you look at) but again the course looked rather bland in comparison to number 11 and number 18 in the subject tables which are 83 and 87 respectively overall.

I think he would be happier at a post 94 place but I would like to be sure he has as much info as possible. Unlike ds1 who is doing a traditional subject at a well known institution, I feel with ds3, I don't know enough about his subject or the reputation of the places running it, to be able to help him much.

[PiratePanda]

So he wants to go to a place that clearly spends more on marketing than teaching? ;-)

Look at the graduate recruitment figures; they will tell you if those work placements actually turned out to be useful.

[eatyourveg]

Will do - thanks