Degree Algorithms and the Law

2017 saw the publication of two major reports on UK higher education degree classification systems. The jointly authored report by Universities UK and Guild HE explored the policies and guidance which provide the framework against which degree classification decisions are arrived at by the 120 institutions which participated in their survey (2017a). The Universities UK/Guild HE report was complemented by David Allen’s analysis of the effects on student degree outcomes of varying polices and guidance relating to degree classification decisions across the higher education sector (2017b).

This blog post is concerned with an aspect of the degree classification process which appears, at best, in the form of marginal notes in both reports. Through these passing references, each report confirms what academics, in all disciplines, have long come to realise. Their expert judgement plays a limited (or non-existent) role in degree classification decisions. Such was not always the case. The judgement of expert examiners used to be decisive in cases where a student performance was on the ‘borderline’ of two possible degree results. Today, this traditional space of academic judgement in the degree classification process is fast disappearing, and it is now “expected that the number of institutions using an automatic algorithm to decide on borderline cases will increase (2017a: 39).

Here it is suggested that a degree classification process which does not permit the exercise of academic judgement in ‘borderline’ cases is incompatible with administrative law.

According to the Universities UK/Guild HE report, 65 out of the 120 higher education institutions surveyed dealt with borderline cases by “automatically” applying the degree algorithm, or by simply not considering borderline cases (2017a: 38). These institutions’ degree algorithms, and degree classification decisions based upon the model the algorithms provide, are unlikely to withstand the scrutiny of the UK Administrative Court, Even those institutions who submit borderline cases to the judgement of members of an academic board may prove to be vulnerable in administrative law terms if judgement is constrained by inflexible rules pertaining to when and how that judgement is exercised.

An example of when an algorithm design which factors in academic judgement may nevertheless contravene the law is provided by Sinclair, Wright, Edwards and Keane in their study of how different degree algorithm designs affect degree awards. They note that a common practice across the sector in borderline instances is to limit academic judgement to cases where a student is “within 1% or 2% of the next band” and only “on the basis of specific criteria such as requiring in excess of 50% of the final year marks to be in the upper band, or specific modules to be included in the final calculation” (2017c). Provided that rules of this kind retain for the decision maker a residual discretion to actively set them aside in favour of a decision based on the particular circumstances of the case in hand, the rules will be safe from administrative law challenge.

The Universities UK/Guild HE report is silent on the question of how tightly bound by an algorithm academic judgement is in those institutions in which it is permitted. However, its overall findings and observations – not least the implication that academic judgement is inimical to “fair” and “transparent” decisions (2007a: 4) - make it doubtful that such residual discretion exists.

Administrative law imposes many obligations on public authorities on whom is conferred the power to make binding decisions which alter an individual’s position. Arguably, the most important of these principles is that the authority in question must not fetter its discretion through the rigid and inflexible application of the rules and policies operating on the authority’s decision making processes. The principle that a public authority must not fetter its discretion when exercising its public functions has been affirmed time and time again at the highest levels of the UK courts, most notably in R v Secretary of State for the Home Department ex parte V and T [1998].

There is no more compelling an example of an individual whose position is altered by the binding decision of a public authority than that of a university degree candidate, whose life chances often turn on the decision of where along the degree classification scale an examining body (which is a constituent part of the public authority) will assess the outcome of his/her years of study.

Degree classification decisions are made by algorithms which increasingly accord a decisive role to a mathematical calculation of the percentage scores gained by students in the modules studied. In nine cases out of ten, the mathematical calculation coincides exactly with the judgement of the expert examiners on the question of what degree classification an individual has earned.

Administration law should intervene in the relatively small percentage of cases in which academic judgement and mathematical calculation are at odds. In such a situation, administrative law decrees that academic judgement must prevail. In such a situation, the higher education administration mandates that the mathematical outcome prevails. One might conclude, therefore, that the administrative courts and higher education examination boards would be in conflict. Paradoxically, however, although no court would deny that the principles as espoused in cases like Ex parte V and T apply to university examination boards when making degree classifications decisions, the UK Administrative Court has consciously interpreted its rules on standing in such a way as to preclude any chance of the principle being applied in any concrete degree classification dispute.

The question of how borderline degree classification decisions should be made has real consequences, as Allen’s recent report confirmed. His findings attracted the attention of the mainstream media. According to Allen, there is a “real risk that different algorithms could result in different classifications given on a student’s mark profile”(2017b: 8). As illustration, Allen informs us that “in the case of the individual set of marks, the degree outcome ranges from an upper second (66.69%) to a 1st (70.72%)” (2017b: 1).

Although Allen sees potential inequities in the current degree classification system, he does not attend to the question of whether those potential inequities, (as well as other perceived problems, such as ‘degree inflation’ and disproportionately low attainments of black and minority ethnic students) might be addressed by an exercise of academic judgement. For instance, expert judgement might conclude that in relation to a particular student profile, a 66.69% aggregate score merits a first class degree award, Conversely, a particular student profile might indicate to subject experts that a 70.72% aggregate score merits an upper second class degree award. None of the institutions who responded to the Universities UK/Guild HE survey appear to have degree algorithms which would permit a judgement so far away from the mathematical outcomes used in Allen’s example. I suggest that Allen’s example reveals why administrative law is right to insist, as it does, that a realm of discretion must always be preserved in connection with decisions as momentous as those attending the degree classification process.

All involved in higher education should be concerned by the fact that higher education institutions are among the few (if not the only) public authorities which are insulated from the effects of the rule against fettering of discretion when exercising their primary public function of awarding degrees.

Notes

This blog post is an extract from an article titled, Academic Judgement and the Force of Law, which is an open access publication, available at patriciatuitt.com.

The article is arranged in four parts. The primacy of public law, explores the legal implications of the fact that the award of a degree is the oldest and most entrenched public function of higher education institutions, especially universities. Academic judgement framed as a legal right, explores three ways in which academic judgement can be said to be justiciable. The Judicial Review action, examines how the Administrative Court at once retains the integrity of the rule against fettering of discretion, whilst giving higher education institutions free reign to implement the most restrictive of degree algorithms. Finally, a question of vigilance? argues the case for legislative intervention in order to ensure appropriate levels of vigilance over the degree award process.

References

Allen, D. (2017b) Degree Algorithms, Grade Inflation and Equity: The UK Higher Education Sector. Economics Working Paper Series 1801.

Sinclair, N., Wright, C., Edwards, G. and Keane, P. (2017c) Degree Classification: Does the Calculation Model Affect the Award?, London South Bank University.

R v Secretary of State for the Home Department ex parte V and T [1998] AC 407,

Universities UK/Guild HE (2017a) Understanding Degree Algorithms.