Can you question the 'Review of Marking'?

[itsnotaremark]

OCR Geography. DD used the extra space at the back to continue her answers. On both Papers 1 & 2 there appeared to be a question where the extended answer at the back had not been marked (no annotations/comments etc.). All other extended answer questions had annotations.

We submitted them both for review. One paper came back acknowledging that the extended answer appeared to have been missed and awarded DD an extra mark.

The other paper came back unchanged. Fine. But this morning we received the reviewed paper and that page is still totally untouched - no annotations even though the rest of the paper is covered with them? Could this have been missed twice?! I did consider maybe there was nothing in the extra bit to award marks but Chat GPT begs to differ and I would still expect some annotation to show it had been seen given the extensive amount of annotation on the rest of the paper?

I know you can appeal but from what I understand that is appealing the process? Would this fall under that? What can we do to check if it definitely has been seen? Does anyone know or has anyone successfully appealed a review?

You would need to appeal and the school can advise you how and on what grounds.

It's not my board or paper but when I do reviews, I am not allowed to annotate except in very specific circumstances, which would not include this one. I would be expected to comment that I had seen the additional pages but nothing more, and if there was nothing additional to credit, I wouldn't even have to do that.

Agree with Mrs Hamlet
You can appeal but the lack of annotations does not mean the extra paper hasn’t been seen, in fact our software doesn’t even allow to click through until you have a) acknowledged the existence of extra material and b) used your cursor to actually see it all.
Reviewers are a selected subset of senior examiners so they are all fully aware of all potential issues and challenges.

Thank you both for your responses - it is helpful to know. I did a bit of digging and according to OCR reviewer guidelines, 'markers and reviewers must apply at least one validation annotation (such as a tick, cross, or specific mark scheme symbol) to every single page containing candidate text'.

The examiner had even marked 'seen' on a blank page (again as required by their guidelines). So it does seem weird that a whole page of text has absolutely nothing on it? It looks even more odd given the extensive annotation on all the other pages. So it really does appear to have been missed but I just don't get how it can be missed twice?? Or how it would have bypassed the 'submission block' also mentioned in their guidelines? There is a small red paperclip looking symbol on all the attached Extra Answer Pages - I wonder if this small annotation gets it past the submission block? To add, this symbol was also evident on Paper 1 but that reviewer noted that there was an administrative error and that although the addition page was linked, there were no annotations to show the extra work had been marked, and they went on to mark it and awarded a further mark.

DD emailed the exam officer and popped by to see her - she agreed it looks strange and will query it with OCR (DD said the EO said something about being able to query it before going to an official appeal?). So we'll wait and see and keep everything crossed that this page has 1 mark on it given DD is only 1 mark away from the higher boundary.

Electronic Marking (via RM Assessor)

• Active Responses: Markers and reviewers must apply at least one validation annotation (such as a tick, cross, or specific mark scheme symbol) to every single page containing candidate text. 1, 2]
• Blank or Additional Pages: For any extra blank pages or unlinked sheets attached to the script, the examiner must explicitly use the "Flag as seen" or "Seen" tool.
• 1]
• Submission Block: The electronic system (RM Assessor) will actively prevent an examiner or reviewer from submitting a completed script if any page has been left

If it has missed twice it sounds like a scanning issue. I can only see pages tagged to a paper - I can see the whole script as well as the question I mark or review, but if the scanning hasn't flagged it as belonging to that paper, I won't see it at all.

It's certainly a mystery.

I’m a senior examiner who reviews papers and I physically cannot submit a script on RM unless I’ve flagged every page as ‘Seen’, which stamps a little ‘Seen’ annotation on it. This sounds very odd if that page has no annotation on it at all - RM wouldn’t have let it be submitted.

I have also marked for OCR in the past and the minimum requirement is for the "seen" annotation to be entered on the extra pages. There is usually a warning for examiners that the script contains extra sheets of paper before you even start marking.

Have you checked the marks entered on this question before the extra sheet was used. For example if it was a 6 mark question, do the annotations for this question ticks, crosses and carrots i.e. the following symbol with the arrow going upwards > add up to the maximum number of marks?

My understanding is that extra sheets are scanned in and then added manually to the marking system.
What you are describing is that perhaps this process wasn't done correctly.

This was an 8 mark question asking to "evaluate.... ". The answer in the space provided has some reviewer/marker highlights a couple of DEV annotations and then a box with L2 at the end of it.

However, her answer continues on the extra sheet (it's a page long and there are zero annotations). It's on this page that she presents her counter argument to the points made in the original first paragraph and thus 'evaluating' the point.

From googling/AI, it seems as if the fact we can see the little red chainlink annotation on the top left hand corner shows that process wasn't followed. Apparently that red chainlink is a flag to the examiner to link this answer to a question - and once linked it should disappear. The fact it is still there and viewable to us means it hasn't been linked. This is according to AI so not sure how true this is?! Apparently examiners can override the link by just viewing the whole paper and scrolling down to the extra pages but it seems as if this particular examiner did not do that. I'm not too sure tbh - would I be allowed to upload a picture of her answers to this question - not sure if that breaches any exam guidelines?! Would love to get your opinions as I can believe that mistake happening once but twice seems a bit ridiculous! And makes me wonder if I'm msising something!

Had to name change to post pictures - if not allowed I will delete! But hope it helps show what I mean 🤞🏻. I’ve included the question and answer in the available box and then the continuation on the extra page.

Appeals are about the process but this is a procedural issue so I would 100% recommend appealing. This has to be supported by the school so you can’t do it yourself but it sounds like the exams officer is helping you. Good luck.

For my subject the maximum mark for levelled response questions is 6 marks. But I would expect the extra page to have the "seen" annotation as a minimum, even if the examiner awarded no extra marks. I think the highlighting on the script indicates the 4 points which met the criteria for level 2.
Extended writing questions are a bit more nuanced and it's not impossible that a specific term or phrase was needed to obtain level 3 marks which was not used and hence this was not a level 3 answer. But it still could be the case that this part of your dds answer has been missed and the review has not made it clear which scenario applied, so I would say that it is worth appealing.

Thank you both for your advice - it is reassuring to know that an appeal is reasonable in our circumstances. We have left it with the exam officer who says she will deal with it towards the end of the week as she is prioritising students who have not finished their initial reviews yet. So hopefully we'll have an answer next week - I will make sure I come back to update.

Do please let us know.
from your photos I think there may well be a scanning error.
I mark for a different board but as I said we cannot move on until we acknowledge the presence of extra pages if they have been “attached” to the response and flagged on the system.

However if they have not been scanned in and flagged, or just scanned in and not flagged there is no indication on the script so potentially ( likely) then they would be missed.
I am most definitely not blaming your dc, but the initial response finishes before the end of the allocated space ( i can see why, she had loads more to write) and there is no star/ cont/ XP/ EP etc etc to alert any marker to the need to open the whole script and find anything extra so fingers crossed the full review finds these pages and they do get marked.

UPDATE - the exam officer emailed to say that she'd had a useful conversation with one of the Compliance Managers in OCR's Appeals Team who explained the process and looked at her paper. This is what we've been told

"Continuation pages are linked by the original examiner to the relevant question - this is shown by the red chain link symbol in the top left corner of the additional pages that you can see on all the continuation pages you used including the page your Q2(c) from Paper 2 is on. Completely unused continuation pages are not linked in this way and instead have the SEEN stamp on them - again as shown on your exam papers. The Compliance Manager noted that, while there are not any individual marking annotations on the page for Q2(c), the page was linked to the relevant question and he can't see anything of concern to suggest to him that that part of your answer has not been taken into account and considered".

I am still not convinced though - there is so much annotation on the rest of her paper - including crosses where statements she's made are wrong/not relevant - this whole page just looks totally out of whack. Surely something on that whole page would deserve a tick/cross given much smaller paragraphs have multiple annotations?!

Further, another one of her papers had a similar issue (although it was just one question, not the whole page) and that reviewer concluded that that question looked like it had been missed! So the fact the red chain link symbol is there does not seem to prove that it has actually been marked by the examiner? I will attach a screenshot of the other reviewed paper in my next message - annoyingly I need to name change to attach pictures!

If anyone has any advice I'd really appreciate it - as our exam officer seems to be putting us off appealing given the cost/hassle but I really don't feel anyone has actually read or marked this page and we're just getting fobbed off!

The other paper that had the same issue - but this reviewer noted that the lack of annotations implied it hadn’t been marked and awarded an extra mark.

I also feel the fact that the L2 box appears at the bottom of the original answer page and not at the bottom of the extended answer page (i.e. at the bottom of the whole answer) implies that the rest of the answer wasn't actually reviewed - L2 is the final score so shouldn't that be noted at the end of the whole answer and not part way through?