ALGORITHM CAUSES WIDESPREAD DISTRESS
This year (2020), because Covid-19 prevented school examinations from taking place, the English government used a mathematical device (with the impressive-sounding name of ‘algorithm’) to determine student grades at A-level and GCSE. A helpful description of how this was done is on the internet at: https://www.wired.co.uk/article/alevel-exam-algorithm.
Based on a school’s previous examination results and teachers’ ranking of students in each erstwhile examination subject, grades were determined for every student. As a result 39% of teachers’ grades for A-level were downgraded. Not only was it judged unfair to students, but demeaning to the teaching profession. There was a national hue and cry and, as a result, government ditched the algorithm and allowed teachers’ grades to stand.
ALGORITHM REJECTED
This attempt to standardise school assessments needed to be rejected and there was widespread relief when this was done. But in the process of adoption and then rejection it has caused distress to hundreds of young people, their parents and their teachers.
TEACHERS KNOW BEST
Teachers work with their students for months, if not years, and become familiar with much of their academic ability. In contrast examiners see the work of a few stressful hours. Some students perform brilliantly under stress, others succumb and perform poorly, most hope that they’ve swotted up on the questions to be asked and some find they haven’t. This year, of course, examiners have had a Covid-19 holiday!
Teachers are professionals and their judgements should be trusted to the same extent that we trust the judgements of doctors.
WHAT SHOULD BE ASSESSED?
Take the idea of assessment back to first principles. What should be the outcome of a successful school education? Here is a list, of academic abilities, not in any special order of importance, which employers and university selectors might look for. Beyond these, of course, are social attributes, personal issues, sporting skills - which school education supports, but which are not examined.
> Descriptive ability: reporting of knowledge received ;
> analytical ability: sorting out ideas;
> logical ability: structuring argument;
> problem solving: devising and testing hypotheses;
> creativity: devising unique ideas and artefacts;
> judgement: evaluation of evidence;
> discernment: assessment of values;
> ability at synthesis: integration of any of the above; and
> effective communication in speech and writing.
CAN'T ASSESS MOST OF THESE IN EXAMINATIONS
Reflecting on this list it is certain that achievements in most of these areas cannot be assessed effectively in three-hour examination papers. Most examinations (mathematics and art being exceptions) test little more than memory and the ability to write coherently and quickly. These are the specifics for a career in journalism: while needed for other careers they may be less important.
[Cynics may say that the main purpose of school examinations is to keep otherwise restless young people focussed on lessons! I reject this view!]
BUT TEACHERS CAN MAKE BETTER JUDGEMENTS
Can teachers assess these abilities better than examiners? Yes. We can expect teachers to make a much better stab at it than examiners because they have seen and assessed much more evidence over a year of relaxed contact than the examiners’ few hours of evidence provided under conditions of stress. Nevertheless it is unlikely that a teacher, although working with students for several hours a week for a school year, and often longer, talking with them and regularly marking their assignments, can make accurate assessments of all of these abilities for every student. Indeed it can be argued that accurate assessment is illusory.
THE DUBIOUS QUALITY OF A SINGLE GRADE
Putting a single grade, say “B”, or more so “B+”, on the work of a student in a subject, say history, or art, or mathematics, is problematic. How can anyone summarise performance over the nine attributes listed above with a single grade? Yet because it is what nearly all of us experienced when in formal education we assume it is a valid and proper way of describing performance. It is ingrained, but deserves to be challenged!
THE PROBLEM FOR SELECTORS
Of course, a selection device for future careers is needed. When an employer or a university selector has a few places but a large number of applicants, picking according to grades may be the only way of choosing whom to interview. Teachers’ grades are likely to be more reliable than examiners’ grades but even so a measure of variability is to be expected. One teacher’s “B” may be “C” to another, but is very unlikely to be “D”. Thus selectors should be guided, but not directed, by awarded grades when interviewing applicants. Selection is, of course a minefield, where even interviewing is judged problematic. Perhaps all that selectors can expect to achieve is to reject those whom seem unsuitable rather than to select the best!
WRITTEN REPORTS MUCH MORE USEFUL
Of much greater value than a grade is the written report on a student’s progress and prowess, showing what ground has been covered in a subject and to what extent the student has developed personally in some of the abilities listed above. As with grade awards, one teacher may make slightly different judgements to another.
ASSESSMENTS ARE NOT INFALLIBLE
Perhaps the most important point about assessment is that it must not be cast in stone. An assessment is a judgement made by one person (the teacher, examiner or selector) about another person (the student, examinee or candidate) on the basis of whatever evidence is to hand at the time. At another time, in a different situation, a different judgment might be made.
Teachers, examiners, and interviewers, can be expected to acquire competence at judging ability, although no one deserves the hallmark of infallibility! The idea of algorithms which attempt the impossible task of trying to standardise teachers’ judgements should be abandoned, as has now been done belately.
...