Head of AQA: exams could be online within 3 years

[noblegiraffe]

Colin Hughes, head of AQA has been interviewed:

Mr Hughes said he would support the launch of “national pilots” for “on-screen assessment”, and that it would take just three years to go from the today’s “totally paper-based” system to having some modules on computers…

Computer-based exams would have a range of benefits, he said, such as making it possible to set “adaptive” tests that tailor themselves to each student – getting harder or easier depending on how the student answers questions, to more accurately pinpoint their ability.

Because pupils would all sit slightly different tests, adaptive tests could eventually revolutionise the school calendar, removing the need for a summer exam season.

Instead, students could take the tests on a “when-ready basis”, which would be “arguably fairer”, said Mr Hughes.”

He also says that issues with lack of computers was something that could easily be sorted. I’m not sure how, tbh.

I read stuff like this and think ‘fgs another overhaul of the exams system is not what we need right now.

I also really don’t want kids sitting maths exams on a computer.

inews.co.uk/news/education/gcses-a-levels-2022-computer-based-exams-online-tests-aqa-1308135

@BungleandGeorge If you look at the world of work almost everything is now on a computer

Except of course people whose primary function is customer interaction face to face, or providing a practical service.
So hospitality, retail, care, animals, plumbing, construction etc.

We've already removed BTECs at GCSE level from almost everywhere. We need to make sure that any changes to how exams are run don't disadvantage the lower 50% again.

Instead, students could take the tests on a “when-ready basis”, which would be “arguably fairer”, said Mr Hughes.”

I can tell Mr Hughes that it might seem fairer, but I would have taken my maths (and possibly Physics) several years early, and would never have been ready to do my English.

I can also say that the only way to really do maths online would be have a touch screen you could write on, in which case you might as well have paper.
Otherwise you'd be spending too long formatting and getting special characters.

I think it also might disadvantage the poorer students disproportionally. So parents with the money will buy better computers, or be able to buy equivalent ones for them to practice and get familiar ones at home. Or are they planning on sending the computers home with the pupils, in which case they're going to have fun (read lots of time) checking that they haven't uploaded anything that can cheat with it.

Are the schools providing the computers? If it's anything like the covid ones, good luck with getting those in 3 years. try for 30. Our school got less than 10% of what they were offered about 6 months after apparently they'd gone out despite being a school considered in need.
They have just got the promised CO2 monitors in a few classrooms though. Apparently with all windows/doors open it takes less than 5 minutes for them to turn red.

[quote BungleandGeorge]@TheLovelinessOfDemons my kid had a scribe for sats. Our secondary do not provide a scribe, allow computer or anything else because they don’t want to. I don’t think there’s an argument that these things don’t help level up the field a bit[/quote]
I wouldn’t consider that school then tbh. If a child needs a scribe/computer/reader etc they should get one. We have loads without echp with support for exams.

A switch to most public exams being online/digital is inevitable. The only question is the timing. It will probably take more than three years due to inertia, underfunding and Luddite resistance.

The International Baccalaureate middle years programme (equivalent to GCSE) is already assessed through online assessments
ABRSM recently switched music theory exams to online, on-demand . They also offer digital performance exams
AP exams - the closest equivalent in the US system to A levels - are now offered in digital format.
If all these exams boards can already offer online/digital exams, then it's fairly obvious that GCSE and A levels could be done online as well.

Electronic assessments offer several advantages over pen and paper based assessments:

1. Adaptive assessments can quickly hone in on a candidate's true level, whereas if the questions are fixed in advance then a large part of the questions are either too easy or too hard for a given candidate.

1. A test made of questions selected randomly/adaptively from large databases of questions can prevent cheating because each candidate will be answering different questions, which can't be known in advance by the candidate or teacher/administrator (see e.g. the recent scandals where pre-U papers were leaked by the teacher involved in setting them to their own students). It also discourages some forms of "teaching to the test" e.g. where teachers or students gamble that a certain topic won't come up because it came up the year before.

1. Electronic assessments can hugely level the playing field for students with SEN and some medical conditions such as hypermobility and dysgraphia. which impact handwriting speed and legibility. Access arrangements such as use of a computer are currently given to a subset of such students, where a parent or teacher has been astute enough to suspect the condition and where they've had either the luck to be seen on the NHS, where waiting lists can be years, or the parents have deep enough pockets to go private. If everyone uses a computer this inequality is removed.

If all these exams boards can already offer online/digital exams, then it's fairly obvious that GCSE and A levels could be done online as well.

In theory, but others have pointed out that the scale of these large public exams and limited funding not only for the actual technology but for rooming, support and supervision makes it unlikely for a long time to come. The extent of extra funding would be enormous - and as I pointed out earlier, we have a Government that doesn’t provide enough funding for the basics in schools (let alone adequate ventilation or extra staff in a pandemic).

It’s unfair to characterise resistance as Luddite when at least some of it is the result of previous failed attempts to introduce tech into education.

Anyone who has, as I have, seen students requiring assistive technology struggling to use slow, outdated equipment - all the school could afford - and software blatantly not fit for purpose (a screen reader that rendered text devoid of any meaning) will know that these sorts of aids are not always as helpful as they could be (and I write as a mother whose daughter was able to use such aids successfully when at university, so I’m not dismissing them out of hand).

If all these exams boards can already offer online/digital exams, then it's fairly obvious that GCSE and A levels could be done online as well.

Driving theory tests and teacher numeracy and literacy tests are also online.

The difference is that you don’t have huge numbers of pupils taking them at the same time in schools. Going to a test centre to do a test or having a handful of kids sit something at the same time is a very different kettle of fish.

I used to work in 'change management'. One of the things we had to do was assess whether previous attempts at change had been successful, or not.

If previous changes had worked, people went in with the mindset that new ones would too.
If previous changes had failed, or been replaced soon by other changes, then people resisted changes or delayed making them until it was clear they were here to stay.

I think teaching is in the latter camp. So many initiatives and fads. So many times the governments make changes but don't fund them, or realise they are going too fast.

With an open mind that anything could be possible, why do all exams have to be taken simultaneously?

Adaptive tests could mean this is no longer necessary, as currently happens with online driving theory tests.

I am over simplifying. And the move to online will be phased over a long time. But issues can be overcome.

why do all exams have to be taken simultaneously?

Because if they’re not, then what on Earth do you do with the kids who have sat their exam while preparing the rest of the class to sit theirs?

At the moment exams are marked on a curve rather than to absolute standards. So they have to have the whole cohort sit the same exam / together to be able to do that.

If we moved to more pass/fail system that was standard based it would be easier to sit things in a more ad hoc manner.
But given that the government have been moving to more fine grading not less, I can't see that happening any time soon.

For the IB middle years programme about 40,000 kids took the on-screen exams in the same exam session. For the AP exams around 4.9 million took the digital exams. That's not a handful. For many of the schools involved it would have been their full cohort in the relevant year groups.
For AP, the large numbers across different sessions / days were handled by preparing multiple papers. That's not unlike what is currently done for on-paper IGCSE's, e.g. the Cambridge CIE board has for many years prepared 3 completely different sets of papers for 3 different time zones in each IGCSE exam series.
There's no reason it couldn't be done for GCSE and A level.

THE MONEY.

No one is saying that online exams aren’t possible, but that schools don’t have the facilities.

@extrastrongmints

For the IB middle years programme about 40,000 kids took the on-screen exams in the same exam session. For the AP exams around 4.9 million took the digital exams. That's not a handful. For many of the schools involved it would have been their full cohort in the relevant year groups. For AP, the large numbers across different sessions / days were handled by preparing multiple papers. That's not unlike what is currently done for on-paper IGCSE's, e.g. the Cambridge CIE board has for many years prepared 3 completely different sets of papers for 3 different time zones in each IGCSE exam series. There's no reason it couldn't be done for GCSE and A level.

The same for CAT tests which cover English maths and science. Hundreds of thousands of tests delivered globally across multiple time zones, session style, online.

It can be done and very successfully. It's not a quick and easy thing to implement and I think the 3 yr timeline is overly optimistic but it is absolutely feasible to deliver these tests online and it's unrealistic to think that all awarding bodies won't be heading in that direction.

Kids who sit early can then spend their time boosting up their weaker subjects. If they are strong in all subjects then they can move on to their next step.

Utopian ideal, but it would be good if people can progress when they are ready rather than because they are a specific age.

Kids who sit early can then spend their time boosting up their weaker subjects. If they are strong in all subjects then they can move on to their next step.

Where? There won’t be any classes available for them to be boosting up their weaker subjects or to move onto A-level.

It's not that expensive.
The exams can be run using free proctoring software which lock down the computer and ensure nothing else is running (that's what the ABRSM online exams do) so there are no software costs. A cheapish laptop is around £320 - I've seen these used in dedicated exam centres. Even assuming they're used for 3 students' exams and nothing else, the basic hardware cost over the long term would be about £107 per pupil.
The current entry fee of a pen and paper GCSE entry is around £40 so for average students taking 8, schools already spend about £320 per pupil on GCSE entries. Yet nobody complains that this is so expensive that it can't possibly be done. It's simply accepted as the status quo and factored into budgets accordingly.

Oh god no.
The maths website they use at dds school I s shit. It misses points if you don’t type the exact keyword it’s looking for, or spell it wrong, or use a different phrase.
Sometimes it marks right answers wrong.
It’s the cause of much frustration!

Have you any idea how out of touch your post is about the current reality in schools, extra?

However flawed the examination system is (within an education system that’s flawed in itself), it’s in fact astonishingly reliable and efficient given its scale (over four million GCSE subject entries per year).

In theory anything is possible. Practically, not so much.

For the past eight years my (retirement) job involved the delivery of Maths, English and ICT specifications to a relatively small number of learners in a flexible and accommodating fashion so as to ensure they got the learning and support they needed, and sat on-demand exams when they were ready, not according to an external timetable. It worked, but only just. It was a logistical nightmare, and though the use of online exams helped in theory, if anything technical went wrong on the day the repercussions were very long lasting. I would love to see an education system where everyone learned at their own pace with targeted teaching, intervention and support, and suitable tests at suitable times. But that’s very unlikely to happen without significant funding and a complete change of mindset - not necessarily from teachers, who are sometimes almost painfully open to new initiatives, but from society, higher education, the professions and Government.

£40 pays for the paper paper - but mainly for the setting, marking and admin.
So now we're paying £40 + £320.
And we need at least ten times the number of invigilators.
The costs are prohibitive.

I am terribel at typing.

I also tecah Englisha nd scoiolgy which people on ehre have opined as 'working well' for computers. I teach many students with eg dyslecxia. They make even more mistakes tyoing than handwriting. They wil be makrked on spellign and punctuation and this will inevitably eman they are marked on typing errors rather than actual spelling or grammaticla errors.

I have left my typos in to show the magnitide if the situation for me!

Timing would be an issue too. Many young people type at a diffeent speed form writing. Lots of them offerd laptopts ofr assessments ultimately reject them.

Someoen also emntioend annotation fo texts ,w hich si an impoartnat issue in some subjects.

I agree computer assessments may be the future but handwriting is actually very importnat to master.

Hope you cna read my mess!

Piggy

Also a dual system is a pain. When DD has MyMaths or similar and has to work things out on paper, the first thing she has to do is correctly transfer the question / figures, and then at the end correctly transcribe the answer. Given her working memory issues, just doing that is slow and painful.

The same for CAT tests which cover English maths and science. Hundreds of thousands of tests delivered globally across multiple time zones, session style, online.

It can be done and very successfully. It's not a quick and easy thing to implement and I think the 3 yr timeline is overly optimistic but it is absolutely feasible to deliver these tests online and it's unrealistic to think that all awarding bodies won't be heading in that direction

I mentioned the CATs tests earlier in the thread but not in a positive way! Each year, the CATs tests throw me out of my classroom for three weeks, in order for the Year 7s to sit their CATs tests (3 weeks because pupils are always absent etc. so often need to sit them later), which isn’t good for my pupils’ learning.

There are also frequent technical issues with the CATs tests as they are online.

Pupils pretty much can see each other’s screens (the wooden screens between each do bugger all) so not exactly ‘cheat’ free either.

And you can’t move on to the next test until the time runs out so quick pupils have to sit there for ages staring in to space, before they can press submit.

Ah, I remember CAT tests! All ICT/Computing lessons delivered in classrooms without computers for two weeks minimum, including exam classes. What a wonderful introduction for Y7 to secondary education (and what a comedown after the thrills of induction days, with wall to wall fun lessons - in non-Covid years, anyway).

@noblegiraffe

You do maths by hand. Stylus, wax, chalkboard, tablet. It’s all your hand drawing stuff on a surface.

Typing stuff onto Equation Editor is not how you do maths.

Stephen Hawking didn't get your memo. According to your definition he stopped doing maths some time in his 20's. He was a patron of the charity Nasen and supported their work on DME which is wholly relevant. Maths is done with the mind, not the hand. It's the concepts that are important. A blinkered requirement to produce written work by hand is discriminatory. Technology can level the playing field.

There are many fabulous ways of learning/doing maths, from elementary to professional, without putting pen to paper. e.g :
Dragonbox
Geogebra
Geometer's sketchpad
Maple
Matlab
Mathematica / Wolfram alpha