Márton VidaAs part of the TK MILAB - Institute for Legal Studies research seminar series, Ljupcho Grozdanovski held a seminar on the legal issues of proving discrimination as caused by the use of AI. Focusing on proof and evidence, his main argument was that current EU law fails to satisfy the requirement of effectiveness on the side of the claimants, and that of fairness on the side of the respondents. As a result, he advocated the lifting of algorithmic opacity and a revision of the notion of human agency, so that the requirements of a fair trial and due process are met.
The seminar started with the introduction of Márton Varju, a senior research fellow at TK. He explained that the event is a result of collaboration with the Artificial Intelligence National Laboratory (MILAB), and therefore it concerns the regulation of AI. Then he proceeded to introduce the speaker, Ljupcho Grozdanovski, who is a researcher both at the Université de Liége and NYU Law School.
In his introduction, Grozdanovski outlined the principal questions he is interested in. First, he discussed whether the origin of AI bias bears any relevance for law; that is, whether we should shift accountability from users should artificial systems be responsible for discrimination. Second, whether there is a need to re-define EU law to better meet the requirements of fairness and effectiveness in cases of algorithmic discrimination. At this point, he highlighted human agency and algorithmic opacity as the most burning issues to be tackled.
Having identified the core dilemmas examined in the talk, he proceeded to provide a brief outline of AI recruitment technologies. He explained that these systems are both fast and cost-effective, creating a huge economic incentive for their use. However, as the famous case concerning the HR algorithm used by Amazon highlighted (where it started to discriminate against female applicants), these systems are by no means bias free. Then, he distinguished between two types of biases, the ones contained in the data set initially used, and ones developed through machine learning. As nor the data can entirely be neutral, nor can we safely predict that the system will not develop any discriminatory selection criteria, Grozdanovski concluded that there is no bias-free operational model for AI.
Given this alarming fact, how can the law respond? Grozdanovski highlighted that the principal barrier is algorithmic opacity; that is the reasons why an AI arrived at a given decision are not transparent. In discrimination cases, this causes grave problems for both claimants and respondents. While the former will be unable to prove that they were discriminated against - being barred from accessing evidence -, the latter will be unable to defend themselves - being forced to prove that the algorithm’s “reasons” for difference in treatment were legitimate, proportionate and necessary. Having established this, Grozdanovski outlined three possible alternatives of response: (1) a general presumption of discrimination in all cases where algorithmic recruitment takes place, (2) a reinterpretation of EU law so that it meets the standards of fair trial and due procedure, and (3) sticking with the status quo.
Examining the three alternatives, Grozdanovski rejected option (1) as it would simply be too harmful to businesses wishing to utilise this otherwise useful technology. He then admitted that current scholarship and court practice mostly prefers option (3), but he made two serious objections to it. First, the principle of feasibility of evidence is harmed under the status quo as claimants are unable to access evidence necessary to prove that they were discriminated against. This is especially troubling, considering that from European Court of Justice’s case law it can generally be inferred that recruiters (unlike employers) have no obligation to share reasons for arriving at a given decision. Grozdanovski emphasised that this would essentially make algorithmic discrimination fall entirely outside the scope of the EU non-discrimination directives. Second, current EU law regards AI systems to be objects, making their users liable for any harm they cause. This makes respondents unable to prove that the algorithm arrived at discriminatory decisions autonomously, putting an unfair burden of proof on them (which is, as already mentioned, the establishment of the fact that the difference in treatment by the AI resulted from a legitimate aim). Having rejected both options (1) and (3), he proceeded to establish his preferred solution, option (2).
Outlining his proposed remedies, Grozdanovski addressed each of his two objections to the status quo separately. On the side of the claimants, he called for a joint reading of the GDPR with EU non-discrimination legislation. In his reading of Article 22 of the GDPR, it creates three separate obligations for recruiters to lift algorithmic opacity. I quote them from his research paper published under the same title: “(1) An obligation ex ante to inform the job candidates of the nature of the recruitment and provide an explanation of the functionality of the HR algorithm. (2) An obligation ex post of explanation and human intervention, aimed at revealing the rationale behind a specific decision. (3) An obligation to process personal data in compliance with EU law, which of course includes non-discrimination principles.”[1]
On the side of the respondents, he considers the main issue to be human agency; that is that under the current status quo solution only humans can be made accountable for discrimination. As already mentioned, this would leave respondents in a position where they would have to prove that the difference in treatment resulted from a legitimate aim. However, in algorithmic decision-making that assessment would amount to mere guesswork. Therefore, Grozdanovonvski proposed the revision of the otherwise historically irrefutable notion of human agency, creating a new ground for defence. Therefore, should they be able to prove that discrimination was developed autonomously, respondents may shift the liability to producers.
During the Q&A session following the talk, a clear dilemma between business interests and the principle fairness was highlighted, out of which Grozdanovski admitted that he favoured the latter. He also stood by the necessity of implementing his proposals should we wish to pursue this ideal. However, he re-emphasised the lack of court practice on the matter, meaning that many of the nuances will only be clarified when courts start to apply these interpretations of the law. These include the exact obligations in Article 22 of the GDPR and the legal consequences of shifting the liability from AI users to producers.
__________________________________________________________
[1] Ljupcho Grozdanovski, In search of effectiveness and fairness in proving algorithmic discrimination in EU law, Common Market Law Review Volume 58, Issue 1 (2021) pp 123-124.
__________________________________________________________
It was supported by the Ministry of Innovation and Technology NRDI Office within the framework of the Artificial Intelligence National Laboratory Program.
__________________________________________________________
The views expressed above belong to the author and do not necessarily represent the views of the Centre for Social Sciences.