Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation

Introduction

In recent months, researchers,¹ government bodies,² and the media³ have claimed that a ‘right to explanation’ of decisions made by automated and artificially intelligent algorithmic systems is legally mandated by the forthcoming European Union General Data Protection Regulation⁴ 2016/679 (GDPR). The right to explanation is viewed as a promising mechanism in the broader pursuit by government and industry for accountability and transparency in algorithms, artificial intelligence, robotics, and other automated systems.⁵ Automated systems can have many unintended and unexpected effects.⁶ Public assessment of the extent and source of these problems is often difficult,⁷ owing to the use of complex and opaque algorithmic mechanisms.⁸ The alleged right to explanation would require data controllers to explain how such mechanisms reach decisions. Significant hype has been mounting over the empowering effects of such a legally enforceable right for data subjects, and the disruption of data intensive industries, which would be forced to explain how complex and perhaps inscrutable automated methods work in practice.

However, there are several reasons to doubt the existence, scope, and feasibility of a ‘right to explanation’ of automated decisions. In this article, we examine the legal status of the ‘right to explanation’ in the GDPR, and identify several barriers undermining its implementation. We argue that the GDPR does not, in its current form, implement a right to explanation, but rather what we term a limited ‘right to be informed’. Here is a quick overview.

In section ‘What is meant by a right to explanation?’, we disentangle the types and timing of explanations that can be offered of automated decision-making. The right to explanation, as popularly proposed, is thought to grant an explanation of specific automated decisions, after such a decision has been made.⁹

In section ‘Why there is no ‘right to explanation’ in the GDPR’, we assess three possible legal bases for a right to explanation in the GDPR: the right not to be subject to automated decision-making and safeguards enacted thereof (Article 22 and Recital 71); notification duties of data controllers (Articles 13–14 and Recitals 60–62); and the right to access (Article 15 and Recital 63).

The aforementioned claim for a right to explanation¹⁰ muddles the first and second legal bases. It conflates (i) legally binding requirements of Article 22 and non-binding provisions of Recital 71 and (ii) notification duties (Articles 13–14) that require data subjects to be provided with information about "the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject" (emphasis added).

Having challenged the legal basis for a right to explanation, we then consider whether the right of access in Article 15 provides a stronger legal basis. Following our analysis of the implementation and jurisprudence of the 1995 Data Protection Directive (95/46/EC), we argue that the GDPR’s right of access allows for a limited right to explanation of the functionality of automated decision-making systems—what we refer to as the ‘right to be informed’. However, the right of access does not establish a right to explanation of specific automated decisions of the type currently imagined elsewhere in public discourse. Not only is a right to explanation of specific decisions not granted by the GDPR, it also appears to have been intentionally not adopted in the final text of the GDPR after appearing in an earlier draft.

In section ‘What if a right to explanation were granted?’, we consider the limitations of scope and applicability, if a right to explanation were to exist. We show that a ‘general’ right to explanation, applicable to all automated decisions, would not exist even if Recital 71 were legally binding. A right to explanation, derived from the right of access (Article 15) or safeguards described in Article 22(3), would only apply to a narrow range of decisions ‘solely based on automated processing’ and with ‘legal’ or ‘similarly significant’ effects for the data subject (Article 22(1) GDPR). We examine the limited cases in which the right would apply, including the impact of a critical ambiguity of language that allows the broader ‘right not to be subject to automated decision-making’ (Article 22 GDPR) to be interpreted either as a prohibition, or right to object.

The last section concludes the article with recommendations for a number of legislative and policy steps that, if implemented, may improve the transparency and accountability of automated decision-making when the GDPR comes into force in 2018.

What is meant by a right to explanation?

Before examining whether the GDPR specifies a right to explanation, it is necessary to examine what one may mean by an ‘explanation’ of automated decision-making. Two kinds of explanations may be in question, depending on whether one refers to:

• system functionality, ie the logic, significance, envisaged consequences, and general functionality of an automated decision-making system, eg the system’s requirements specification, decision trees, pre-defined models, criteria, and classification structures; or to

• specific decisions, ie the rationale, reasons, and individual circumstances of a specific automated decision, eg the weighting of features, machine-defined case-specific decision rules, information about reference or profile groups.¹¹

Furthermore, one can also distinguish between explanations in terms of their timing in relation to the decision-making process:

• an ex ante explanation occurs prior to an automated decision-making taking place. Note that an ex ante explanation can logically address only system functionality, as the rationale of a specific decision cannot be known before the decision is made;

• an ex post explanation occurs after an automated decision has taken place. Note that an ex post explanation can address both system functionality and the rationale of a specific decision.

An example may help clarify how these distinctions interact. Take an automated credit scoring system. Prior to a decision being made (ex ante), the system provider can inform the data subject about the system functionality, including the general logic (such as types of data and features considered, categories in the decision tree), purpose or significance (in this case, to assign a credit score), and envisaged consequences (eg the credit score can be used by lenders to assess credit worthiness, affecting the terms of credit such as interest rate). After a decision has been made (ex post), an explanation of system functionality can still be provided to the data subject. However, the provider can also explain to the data subject the logic and individual circumstances of their specific decision, such as her credit score, the data or features that were considered in her particular case, and their weighting within the decision tree or model. In other words, the provider can explain how a particular score was assigned. Further, when pre-defined simplistic or linear models are used and fully disclosed, predictions about the rationale of a specific decision are possible in principle ex ante. However, in both cases the provider’s ability to offer an explanation of the rationale of a specific decision may be limited by several legal (see section ‘What if a right to explanation were granted?’) and technical factors, including the use of complex probabilistic analytics and decision-making methods.¹²

These distinctions between two kinds and two different timings of explanations are implicit in the GDPR. Their importance will be highlighted as we examine the possible legal bases for a right to explanation.

Why there is no ‘right to explanation’ in the GDPR

Three distinct possible legal bases for a right to explanation of automated decision-making can be found in the GDPR. A right to explanation can possibly be derived from: safeguards against automated decision-making as required under Article 22(3), and commented upon by Recital 71; notification duties under Articles 13–14 commented upon by Recitals 60–62; or the right of access under Article 15, and commented upon by Recital 63.

These bases are respectively referred to as a right to explanation derived from (i) safeguards, (ii) notification duties, and (iii) the right of access. We will assess each in turn. On the whole, the claim that a right is granted by the GDPR to an ex post explanation of specific decisions (at a minimum) that seemingly applies to any instance of automated decision-making is based on a combination of safeguards and notification duties. It combines non-binding Recital 71 with binding provisions of Articles 13–14 and 22 to argue that "The law will […] effectively create a "right to explanation," whereby a user can ask for an explanation of an algorithmic decision that was made about them."¹³ This claim is incorrect for several reasons, explained below.

A right to explanation derived from safeguards against automated decision-making

Starting with the claim¹⁴ for a right to explanation derived from safeguards, Article 22 (see Figure 1) and Recital 71 of the GDPR address a data subject’s right not to be subject to automated decision-making. Article 22(3), which addresses safeguards against automated decision-making, states that:

the data controller shall implement suitable measures to safeguard the data subject's rights and freedoms and legitimate interests, at least the right to obtain human intervention on the part of the controller, to express his or her point of view and to contest the decision. (emphasis added)

Critically, a right to explanation is not mentioned. Rather, after a decision has been made, and assuming the decision meets a condition specified in Article 22(3)a (to enter or fulfil a contract) or Article 22(3)c (with explicit consent), data subjects are granted additional safeguards to obtain human intervention, express views, or contest a decision (Article 22(3)), but not to obtain an explanation of the decision reached.

In all of the GDPR, a right to explanation is only explicitly mentioned in Recital 71, which states that a person who has been subject to automated decision-making:

should be subject to suitable safeguards, which should include specific information to the data subject and the right to obtain human intervention, to express his or her point of view, to obtain an explanation of the decision reached after such assessment and to challenge the decision. (emphasis added)

If legally binding, this provision would require an ex post explanation of specific decisions, as Recital 71 addresses safeguards to be in place once a decision has been reached. To show why Recital 71 does not establish a legally binding right, a brief aside into the legal status of Recitals is required.

Recitals provide guidance¹⁵ on how to interpret the Articles, but are not themselves legally binding.¹⁶ As Klimas and Vaiciukaite explain, "Recitals have no positive operation of their own" and "cannot cause legitimate expectations to arise."¹⁷ Baratta further expands:

In principle the ECJ does not give effect to recitals that are drafted in normative terms. Recitals can help to explain the purpose and intent behind a normative instrument. They can also be taken into account to resolve ambiguities in the legislative provisions to which they relate, but they do not have any autonomous legal effect.¹⁸

Jurisprudence of the European Court of Justice (ECJ) shows that the role of Recitals is to dissolve ambiguity in the operative text of a framework. The ECJ has commented directly on the legal status of Recitals, clarifying that: "Whilst a recital in the preamble to a regulation may cast light on the interpretation to be given to a legal rule, it cannot in itself constitute such a rule."¹⁹

Returning to the GDPR, Article 22(3) lists the minimum requirements that have to be met for lawful automated decision-making. There are no ambiguities in the language that would require further interpretation with regard to the minimum requirements that must be met by data controllers. As long as these requirements are met, automated decision-making is lawful and in compliance with the GDPR. With this said, future jurisprudence (see section ‘What if a right to explanation were granted?’) can still interpret the meaning of ‘suitable measures to safeguard’, and establish future mandatory or case-to-case requirements to be met by data controllers, including a right to explanation. This is, however, only one possible future. A right to explanation is thus not currently legally mandated by the requirements set in Article 22(3).

In addition, rights have to be explicitly legally established prior to their enforcement. This idea stems from the relationship between legal rights and duties. The scope of a right can be subject to interpretation; a legal basis for its existence must, however, first be beyond doubt. Rights of data subjects typically correspond with a duty on the side of the data controller.²⁰ Negligence in relation to legal duties can be punished through fines and other procedures. It would be highly controversial to impose fines on data controllers without having previously clarified explicitly and beyond doubt what duties must be met. Doing otherwise would conflict with the principles of fair trial (Article 6 of the European Convention on Human Rights and Article 47 of the Charter of Fundamental Rights of the European Union) and the rule of law.²¹ Criminal and administrative procedures have to be laid down precisely.

It can be concluded that data subjects will not be granted a legally binding ex post right to explanation of specific automated decisions on the basis of legal safeguards in Article 22 as it currently stands. That this is the case does not appear to be the result of an oversight or fiddling with subtle interpretations (eg the meaning of ‘suitable measures to safeguard’ in Article 22(3)). On the contrary, the omission of a right to explanation from Article 22 appears to be intentional. The safeguards specified in Recital 71 are almost identical to those in Article 22(3), with the significant difference of the further inclusion of a right ‘to obtain an explanation of the decision reached after such assessment’ in Recital 71. The purposeful omission of this text from Article 22 may not be an oversight but suggests that legislators did not intend to implement a right to explanation of specific decisions in the GDPR. What happened?

Looking at previous drafts of the GDPR and commentary from the trilogue negotiations,²² one can see that legislators had stricter safeguards in place on automated decision-making and profiling, but that these were eventually dropped, including a legally binding right to explanation of specific decisions.²³ An early indication of the debate around the right to explanation can be seen in the November 2013 report of the European Parliament (EP)²⁴ and the December 2014 report of the European Council in response to the original GDPR text proposed by the European Commission (EC)²⁵ in 2012.

The EC’s proposed text did not contain a right to explanation. The EP proposed the following amendment to Article 20 (now Article 22 in the adopted version of the GDPR), paragraph 5:

Profiling which leads to measures producing legal effects concerning the data subject or does similarly significantly affect the interests, rights or freedoms of the concerned data subject shall not be based solely or predominantly on automated processing and shall include human assessment, including an explanation of the decision reached after such an assessment. The suitable measures to safeguard the data subject's legitimate interests referred to in paragraph 2 shall include the right to obtain human assessment and an explanation of the decision reached after such assessment …. (emphasis added)

The EP’s preferred text mandated a ‘right to obtain human assessment and an explanation of the decision reached after such assessment’. These safeguards would have been part of Article 20, meaning that they would have been legally binding. However, the proposed safeguards were not adopted in trilogue. This change suggests that legislators intentionally chose to make the right to explanation non-binding by placing it in Recital 71.

The European Council’s 2014 draft,²⁶ on the other hand, only required that:

the data controller shall implement suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests, such as the right to obtain human intervention on the part of the controller, to express his or her point of view and to contest the decision. (emphasis added)

The Council suggested to add the text:

to express his or her point of view, to get an explanation of the decision reached after such assessment and the right to contest the decision, (emphasis added)

to Recital 58 (equivalent to Recital 71 GDPR).²⁷ The Council thus suggested to place the right to explanation added in the EP’s draft in a Recital. This approach was eventually taken in the final text adopted in 2016.

Interestingly, despite years of negotiations, the final wording of the GDPR concerning protections against profiling and automated decision-making hardly changed from the relevant Articles and Recitals of the Data Protection Directive 1995. As with the GDPR, a ‘right to explanation’ does not appear in Article 15 of the Directive (see Figure 2), which addresses automated individual decisions.

Although Article 22 GDPR has not greatly changed from Article 15 of the Directive, a few changes are still noteworthy. First, the only safeguard against automated decision-making mentioned in the Directive is the opportunity to express one’s views. Article 22(3) additionally names contesting the decision and the right to obtain human intervention as suitable measures. Secondly, explicit consent is included as a case in which automated decision-making is allowed (Article 22(2)c). Finally, as opposed to the provisions in Article 15 of the Directive, it is no longer necessary that the data subject requests the contract in order for automated decision-making to be lawful.

A right to explanation derived from notification duties

Articles 13 and 14 GDPR specify notification duties for data controllers concerning the processing of data collected from the data subject (Article 13) or from a third party (Article 14). In the aforementioned claim, these Articles are cited as a basis for a right to an ex post explanation of specific decisions. The claim starts with Articles 13(2) and 14(2), which state that data controllers need to:

provide the data subject with the following information necessary to ensure fair and transparent processing.

According to Articles 13(2)f and 14(2)g, this information includes:

the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject. (emphasis added)

This duty applies in cases of automated processing meeting the requirements of Article 22(1) or 22(4) (more on this later).

It has been suggested that the notification duties in Articles 13–14, in combination with the safeguards defined in Article 22(3), grant an ex post right to explanation of the ‘existence of … logic involved … significance … and envisaged consequences’ of automated decision-making.²⁸ This claim is mistaken for two reasons.

First, only an ex ante explanation of system functionality is explicitly required by Articles 13(2)f and 14(2)g. These notification duties precede decision-making. Notification occurs before a decision is made, at the point when data is collected for processing. This holds true even if Article 14 introduces some ambiguities when data are collected from third parties rather than data subjects (insofar as the controller needs only to notify the data subject within 30 days of collection). As explained in section ‘What is meant by a right to explanation?’, only an explanation of system functionality is logically possible prior to decision-making. Therefore, Articles 13–14 cannot be used as evidence of an ex post right to explanation of specific decisions that can logically only be given once a decision has been made (timeline problem).²⁹

Secondly, the claim links Articles 13(2)f and 14(2)g to the safeguards in Article 22(3). This link is not made in the GDPR. Articles 13(2)f and 14(2)g apply only to Articles 22(1) and 22(4), which do not address safeguards against automated decision-making. The supposed link—between notification about the logic involved, significance, and envisaged consequences of automated decision-making in Articles 13–14, and the ex post right to explanation incorrectly attributed to Article 22(3) (which only features in Recital 71)—is therefore untenable and can be dismissed. The claim also conflates the legally binding notification duties, specified in Articles 13–14, and the non-binding right, specified in Recital 71.

It follows that the claim for an ex post right to explanation of specific decisions³⁰ is not correct. Any suggestion to the contrary fails to distinguish between (i) the legally binding duty to notify the data subject of the logic involved, significance, and envisaged consequences of automated decision-making system before decision-making occurs (timeline problem) (Articles 13–14), and (ii) the data subject’s non-binding right to an explanation of specific decisions (Recital 71) after decision-making occurs.

The language used in Articles 13(2)f and 14(2)g also supports the interpretation that only an ex ante explanation is required. Data controllers must inform the data subject about the

existence of automated decision-making, including profiling … [and provide data subjects with] meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing.

The language used suggests that data subjects must be provided with information about how an automated decision-making system works in general, for which purposes, and with what predicted impact, before automated decisions are made. Notably this cannot include any information about how a specific decision was made or reached, but rather addresses how the system itself functions, eg its decision tree or rules, or predictions about how inputs will be processed. For fully disclosed simplistic or linear models, this may show how specific decisions would be reached in the future.³¹

A right to explanation derived from the right of access

In contrast to prior claims (see ’Introduction’ section), it may also be possible to derive a right to explanation from the right of access established in Article 15 GDPR. Article 15(1)h is identical to Articles 13(2)f and 14(2)h: data subjects are granted a right to be informed about the existence of automated decision-making and to obtain meaningful information about the significance, envisaged consequences, and logic involved. Specifically, the subject should be informed about the existence, purposes, and logic of data processing, and the intentions and legal consequences of such processing. By having this information, the data subject should be able to examine the lawfulness of data processing and invoke legal remedies.³²

Together, Articles 13–15 form what has been called the ‘Magna Carta’ of data subject’s rights to obtain information about the data held about them, and to scrutinize the legitimacy of data processing.³³ Articles 13–14 create notification duties for data controllers, while Article 15 establishes a corresponding right of access for data subjects.³⁴ In contrast to the notification duties of data controllers in Articles 13–14, the right of access has to be invoked by the data subject. The articles are a unit, insofar as they provide the data subject access to identical information, and use the same language.

Although seemingly insignificant, the change from a notification duty to an access right has important consequences for the timing of explanations required from the data controller. Given that the phrasing of Article 15(1)h is identical to Articles 13(2)f and 14 (2)g, one could assume that the right of access similarly only grants access to an ex ante explanation of system functionality. However, the right of access is dependent upon the request of the data subject and has no deadline; the ‘timeline problem’ of Articles 13(2)f and 14(2)g does not apply. At first glance, the data subject can request this information at any time, including after an automated decision has been made, making an ex post explanation of the rationale of specific decisions plausible.

Nonetheless, it is reasonable to doubt that the right of access grants a right to ex post explanations of specific decisions already reached. Consider the semantics of Article 15(1)h. The phrase ‘envisaged consequences’ is future oriented, suggesting that the data controller must inform the data subject of possible consequences of the automated decision-making before such processing occurs. This interpretation follows the timeline constraints of identical provisions in Articles 13(2)f and 14(2)g discussed above, which only allow for ex ante explanations. Data controllers are required to predict the possible consequences of their automated decision-making methods. The term ‘envisaged’ limits these predictions to ex ante explanations of system functionality, for instance, concerning the general purpose of the system, or the type of impact to be expected from the type of decision it makes. For instance, a credit agency could predict that the scores they produce will impact on credit worthiness (eg interest rates). If applied to decisions already made, the phrasing becomes incoherent.³⁵ It would seem to require data controllers to predict the personal consequences of decision-making for individual data subjects after an automated decision has been made, including how the decision could be used by other data controllers and processors.

The semantics of the German translation of Article 15(1)h GDPR provides further support. The German Article 15(1)h states:

Tragweite und angestrebten Auswirkungen einer derartigen Verarbeitung für die betroffene Person. (emphasis added)

This sentence translates to ‘the scope and intended consequences of such processing for the person concerned’ (authors’ translation, emphasis added). This indicates that the data controller must inform the data subject about the consequences the controller wishes to achieve with automated decision-making. According to this phrasing, the data controller is not asked to predict consequences but rather explain the scope, intention, and the purpose of such processing. This suggests that the right of access is not addressing how an individual decision was reached, but rather the duty of the data controller to provide information about the existence, aims and consequences of such processing. This equates to an explanation of system functionality.³⁶

There are similar reasons to doubt that Article 15(1)h grants an ex post right to explanation of specific decisions. Data controllers are required to provide information about the ‘existence of automated decision-making’ (emphasis added). This phrase does not suggest an explanation of how a decision was reached. Rather, the data controller is only required to inform the data subject that automated decision-making methods are being used to process her data.

The phrasing of Article 15(1)h, as with Articles 13–14, points to an explanation of system functionality. However, data controllers are also required to provide ‘meaningful information about the logic involved’ in automated decision-making. As noted in section ‘A right to explanation derived from notification duties’, this phrase, as used in Articles 13–14, has been argued by others to grant an ex post right to explanation. If correct, Article 15(1)h would grant a right to explanation of specific decisions, not only system functionality, as the data subject can request the relevant information both before and after a decision has been made. However, there are further reasons to doubt that this is the case.

For Article 15(1)h to be coherent as a whole, ‘meaningful information about the logic involved’ must be interpreted in connection with the other terms (existence of, meaningful information about significance and envisaged consequences) used of Article 15(1)h, which are limited to explanations of system functionality. Interpreting ‘logic involved’ to grant an ex post explanation of specific decisions would mean the other terms of Article 15(1)h would be incoherent, if the right of access was invoked after a decision was made. This interpretation is further supported by a comparison of the language used in Article 15(1)h and Recital 71. Data controllers are obligated to provide information about the

existence of automated decision-making … meaningful information about the logic involved, as well as the significance and envisaged consequences of such processing (Article 15(1)h), [as opposed to] an explanation of the decision reached (Recital 71).

The phrasing of Article 15(1)h is future oriented, and appears to refer to the existence and planned scope of decision-making itself, rather than to the circumstances of a specific decision as suggested in Recital 71. If an explanation of specific automated decisions was intended to be granted by Article 15(1)h, as in Recital 71, the usage of different language between the two would be odd.

Nevertheless, given the lack of an explicit deadline for invoking the right of access, one cannot be certain, on the basis of semantics alone, that the right of access is limited to explanations of system functionality. Despite this, we argue that, as with notification duties in Articles 13–14, and regardless of when it is invoked by the data subject, the GDPR’s right of access only grants an explanation of automated decision-making addressing system functionality, not the rationale and circumstances of specific decisions. This conclusion is supported by implementation of the 1995 Directive’s right of access by Member States, which has mostly limited informational obligations to system functionality. If interpretation of the GDPR follows historical precedence, its right of access will be similarly limited. To articulate this claim further, it is necessary to examine in detail Member State implementations and interpretations of the Directive’s right of access.

Right of access in the 1995 Data Protection Directive 95/46/EC

It is important to note that a right of access that grants data subjects some explanation of automated decision-making is not new, and has not proven an effective transparency mechanism.³⁷ Rather, this right has existed since the 1995 Data Protection Directive, and has been implemented in national law by most European Member States.³⁸ Similar to the scope of the GDPR’s right of access, the Directive’s right of access provides means for data subjects to discover whether a controller is processing personal data. If so, the data subject is then entitled to know the extent of data being processed. This shall enable the data subject to scrutinize what data are used and take appropriate action such as requesting rectification or erasure.³⁹ Notably, the Directive’s right of access has generally not been interpreted as granting a right to explanation of specific decisions already reached, as it is not part of the safeguards at the time automated decisions are made in Article 15(2)a of the Directive; this distinction is comparable to the difference between Articles 15 and 22 of the GDPR. The Directive names only one safeguard against automated decision-making, namely the right for the data subject to ‘put his point of view’. A right to explanation of specific decisions as a safeguard to ensure lawful automated decision-making was not envisaged.

The implementation and interpretation of the Directive’s right of access varied across the Member States. Despite much debate,⁴⁰ consensus has not emerged concerning the type of information data controllers must disclose to provide data subjects with ‘knowledge of the logic involved in any automatic processing of data’ per Article 12(a).⁴¹ A report published in 2010 on the implementation of the Directive across Member States suggested that it was left to the Member States to define the scope and requirements of the right of access. The report urges clarification of the requirements and limitations on the right of access concerning information about the ‘logic involved’ due to the growing importance of automated decisions.⁴² In part, the lack of consensus over the meaning and requirements of ‘logic involved’ owes to the relative lack of jurisprudence on the right of access. Despite the Directive having been in force for over 20 years, the requirements and limitations of the right of access applied to automated decision-making have not been extensively clarified or tested in courts across Europe.⁴³

The limited jurisprudence available reveals limitations on the Directive’s right of access. Several overriding interests and exceptions have been identified that significantly limit both the scope of applicability and content of the explanation. In general, data subjects are entitled to receive some information about the general functionality of an automated decision-making system, but little to no information about the rationale or circumstances of a specific decision. The 2010 report reflects this, noting that the language used in the Directive reflects a very narrow scope of applicability for the right of access due to a number of exceptions and limiting or overriding interests.⁴⁴ Recital 41 of the Directive clarifies that the right of access can be limited by trade secrets and intellectual property, especially relating to software.⁴⁵ These interests have proven strong limiting factors on the right of access as implemented and tested by Member States.

Several examples can be offered. French data protection law⁴⁶ grants data subjects a right to receive information about the ‘logic involved’ as long as it does not contravene copyright regulations. To allow data subjects to challenge decisions, information must be provided about the general logic and types of data taken into account, ‘but not (or at least not fully) of the weight that is attached’ to specific features.⁴⁷ The full code of the automated decision-making system or algorithm does not need to be revealed.⁴⁸ A similar approach is taken in the UK’s Data Protection Act 1998, which also limits the right of access to protect trade secrets.⁴⁹ As with French law, data controllers

must inform data subjects of the factors which they take into account in the "evaluation" underlying the decision, but without having to reveal the exact weight given to each of these factors (i.e. the copyright-protected algorithm used in the automated decision-taking process).⁵⁰

German data protection law has similarly recognized a distinction between explanations of system functionality and specific automated decisions in section 6(a) of the Bundesdatenschutzgesetz, which jointly implements Articles 12 (right of access) and 15 (safeguards for automated individual decisions) of the Directive.⁵¹ Notably, Germany implemented a right allowing data subject’s to request an explanation of automated decisions that are not made in their favour. The right is implemented as an explicit safeguard against automated decisions in section 6(a)2(2) of the Bundesdatenschutzgesetz. The right was voluntarily enacted as a safeguard beyond the requirements set in Article 15 of the Directive, which grants the right to express views as the only safeguard against automated individual decisions. Interestingly, the right to an explanation as an extra safeguard provides some insight into how the Directive’s requirement to explain the ‘logic involved’ was interpreted by German legislators. Section 6(a)3 of the German Data Protection Act separately extends the right of access enshrined in sections 19 and 34, allowing data subjects to obtain information about the ‘logical structure’ of automated processing, which refers back to Article 12(a) of the Directive.⁵² If ‘knowledge of logic involved’ in Article 12(a) was intended to establish a right to obtain an explanation about decisions reached, it would not have been necessary for German legislators to enact separately a right to explanation (section 6(a)2(2)) in the same Article containing the extended right of access (section 6(a)3), especially considering both rights must be invoked by the data subject. Even if one wishes to argue that sections 6(a)2(2) and 6(a)3 refer to the same type of explanation (ie of specific decisions), the use of different wording across the articles—‘main reasons for the decision and have it explained’ in section 6(a)2(2), ‘logical structure of the automated processing of the data that concerns [the data subject]’ (authors’ translation) in section 6(a)3)—suggests that the two mechanisms entitle the data subject to different types of information.⁵³

Following this, German legal commentary and jurisprudence⁵⁴ addressing the extended right of access (section 6(a)3) suggest that the information it requires is limited mostly to system functionality. The data controller does not need to disclose the software used, as the software is considered to be a trade secret.⁵⁵ Some German commentators believe that some (or the ‘top four’) features factored into a decision have to be disclosed, but not the algorithm used due to trade secrets.⁵⁶ Data controllers are not obligated to explain how the software is working or, especially, to give any details about its code. The data controller is only obligated to explain the logic of the ‘decision tree’. The ‘weighting’ (authors’ translation) of specific features and the parameters used to make the decision do not have to be disclosed. This is meant to protect trade secrets and manipulation of the decision-making system.⁵⁷

This interpretation of the right of access as being limited to system functionality in order not to contravene trade secrets is also reflected in German jurisprudence. According to several commentators,⁵⁸ the German SCHUFA⁵⁹ judgments⁶⁰ show that data subjects do not have a right to investigate fully the accuracy of automated processing systems (in this case, credit scoring), as the underlying formulas are protected as trade secrets. The protected formula would consist of, for example, statistical values, weighting of certain elements to calculate probabilities (eg the likelihood of loan repayment), and reference or comparison groups.

The judgments indicate that all three elements of the right of access enshrined in Article 12(a) of the Directive aim to provide general information about the usage and purpose of data processing. Concrete elements of the screening procedures do not have to be disclosed.⁶¹ The data subject is entitled to know which data and features were taken into account when the decision was made, in order to be able to contest the decision or demand that inaccurate or incomplete data be rectified. However, the weighting of these elements, the method (scoring formula), the statistical values, and the information about the reference groups⁶² used does not have to be disclosed.⁶³ The judgments state that jurisprudence, academic literature, and legal commentary commonly agree that the abstract methods used to define credit scores do not have to be disclosed, and that this position is in accordance with the intention of German data protection legislation.⁶⁴

It is worth noting that the SCHUFA judgments do not explicitly address automated decision-making, as the court decided an automated decision was not made because automated processing was only used for preparation of evidence, while the actual decision was made by a human being.⁶⁵ The judgements are nonetheless insightful insofar as they demonstrate a strong tendency to protect trade secrets in relation to the right of access. As discussed below, this case provides an example of an important limitation on a right to explanation established on any of the three legal bases in the GDPR identified above. Automated decision-making is defined in both the Directive and GDPR as decision-making based solely on automated processes.⁶⁶ Quite crucially, this creates a loophole whereby even nominal involvement of a human in the decision-making process allows for an otherwise automated mechanism to avoid invoking elements of the right of access (both in the Directive and GDPR) addressing automated decisions.

Finally, Austrian legislators similarly implemented the requirements of Articles 12a and 15 of the Directive in section 49(3)⁶⁷ of the Austrian Data Protection Act. As opposed to German law, the right to obtain an explanation about how an individual decision was reached was not implemented as a safeguard. Only the right to express one’s view is named as one of the mandatory safeguards, as mandated by Article 15 of the Directive. Section 49(3) establishes an extended right of access (section 26) which is the data subject’s right to know, upon request, about the logic of the process of automated decision-making.⁶⁸

Austrian jurisprudence⁶⁹ is very vague on the right of access and automated decision-making. Existing decisions do not fully explain how much the data controller is obligated to disclose under the right of access, and are in some sense contradictory. In most decisions, an obligation was recognized to explain how the system in questions functions.⁷⁰ In contrast, one decision stated that the right of access according to section 26 and the right to know about the logic of the process (section 49(3)) also includes the criteria and the weighting of the criteria which would then allow the data subject to understand how a decision was reached. However, the Austrian Data Protection Commission simultaneously acknowledged that trade secrets can limit this right. The Commission concluded that the extent to which the data controller needs to disclose decision criteria and weighting must be determined on a case by case basis.⁷¹ In another case, the Commission denied the existence of an individual automated decision because the criteria used were based on a large group rather than on the individual. Therefore, the rights of access and to know about the logic of automated processing do not apply, if the basis of that decision is a group (‘peer group’ [authors’ translation]) rather than (data about) the individual.⁷² This distinction highlights a tension in the definition of automated decision-making and profiling in the Directive, insofar as automated processing of data describing groups, rather than individuals, does not allow for invocation of the right of access.⁷³

From the Directive to the GDPR: the right to be informed

The Directive’s right of access provides an explanation of the system’s functionality which has been heavily limited by trade secrets. The loophole—through which automated processes that merely produce evidence for decision-making (rather than actually making decisions) are not subject to the right of access (specifically, the provision to disclose information about the ‘logic involved’)—has also proven to be a significant limiting factor. A relative lack of jurisprudence across Member States has not helped clarify and unify the requirements. This is problematic given the current and emerging growth in automated decision-making and data processing.

The GDPR appears to offer less protection to data subjects concerning explanations of automated decision-making than some current data protection laws in Europe based on the Directive.⁷⁴ In particular, the GDPR's right of access appears to not offer more protection for data subjects’ interests than the Directive's right of access.⁷⁵ The use of future-oriented semantics in the GDPR (unlike the Directive which did not explicitly acknowledge a decision-making timeline), as well as its terminological overlap with notification duties, suggest that the GDPR intends to further limit the right of access regarding automated decision-making to explanations of system functionality.⁷⁶ The phrasing of Article 15 GDPR in particular points towards a general explanation of the existence and functionality of automated decision-making systems. Article 12(a) of the Directive grants data subjects a right to obtain ‘knowledge of the logic involved in any automatic processing of data concerning him at least in the case of the automated decisions referred to in Article 15(1)’.⁷⁷ It is interesting to note that this phrase is open to greater interpretation than Article 15 GDPR, ⁷⁸ which requires only information about ‘the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject’. As argued above, this phrase in the GDPR requires that the data subject be informed merely about the usage and functionality of automated decision-making methods. The change of wording indicates that the intention of the right of access in the GDPR is to grant access to information about the ‘usage’ and functionality of such automated decision-making. Again, this suggests an even stronger intention to limit the right to explanations of system functionality, not the rationale and circumstances of specific decisions.

Legal scholars are already debating the scope of the right of access in the GDPR. According to German commentary⁷⁹ on the GDPR, it is sufficient to be informed about the envisaged consequences in a very simple manner. For instance, an explanation of how a low rating of creditworthiness can affect the choice of payment options would be sufficient.⁸⁰ The type of explanation recognized in prior German jurisprudence⁸¹ and German commentary⁸² on the GDPR is limited by overriding interests of the data controller, eg protection of trade secrets, or prevention of ‘gaming the system’ by users. The process that the algorithms use does not have to be disclosed.⁸³ Furthermore, the rating of similar groups has historically not needed to be disclosed.⁸⁴

These recent commentaries on the GDPR follow the general interpretation and prior jurisprudence on the right of access in the 1995 Directive. According to commentators, data controllers do not need to explain fully the rationale and circumstances of a specific decision to provide data subjects with ‘meaningful information about the logic involved’ (Article 15(1)h GDPR). Rather, the information offered by data controllers will address general system functionality, and could be heavily curtailed to protect the controller’s interests (eg trade secrets, intellectual property; see Recital 63).⁸⁵ It is worth noting that additional limitations can also be imposed to protect the interests of other parties via Union or Member State law.⁸⁶ Paal also notes that the purpose of Article 15 GDPR is to allow data subjects to be informed about the usage and functionality of automated decision-making. As the scope of information data controllers are required to disclose in Article 15 is the same as in Article 13, Article 15 similarly requires only limited information about the functionality of the automated decision-making system. Paal also notes that “meaningful information” does not create an obligation to disclose the algorithm, but only to provide basic information about its logic. Schmidt-Wudy argues that if necessary to assess the accuracy of data processing, information about the algorithm could be given, with appropriate limitations to protect trade secrets.⁸⁷ However, the type of information to be provided is not specified.

As with the Directive, the practical requirements and utility of the GDPR’s right of access will similarly only be revealed through testing and clarification via jurisprudence and expert opinion, such as from the Article 29 Working Party, the new European Data Protection Board established by Article 68 GDPR,⁸⁸ the European Data Protection Supervisor, or its Ethics Advisory Group⁸⁹ (see ‘Conclusion’ section). However, the implementation of the Directive’s right of access strongly suggests that the GDPR’s right of access will be far from the ex post ‘general’ right to explanation of system functionality and specific decisions, which we have argued it is mistakenly attributed to the GDPR. Rather, through the right of access, the GDPR will grant a ‘right to be informed’ about the existence of automated decision-making and system functionality, limited in applicability along the lines above and those described in the following section.

What if a right to explanation were granted?

Although a meaningful right to explanation of specific automated decisions will not be introduced by the GDPR, the contribution of such a right to the accountability and transparency of automated decision-making may provide compelling reasons for legislators or data controllers to introduce one in the future. It is possible to envisage at least four main scenarios that may lead to a right of explanation of specific automated decisions in practice:

• An additional legal requirement is enacted by Member States, separate from the GDPR, granting a right of explanation of specific decisions (similar to actions taken by German legislators under the 1995 Directive) (see also ‘Conclusion’ section).

• Based on GDPR Article 22 and Recital 71, data controllers voluntary choose to offer a right to explanation of specific decisions as a ‘suitable … safeguard’. The right would be an additional and voluntary safeguard to those already required by Article 22(3). Controllers could do this on the basis that an explanation is required to invoke one of the three legally required Article 22(3) safeguards, ie express their views, obtain human intervention, or contest a decision.

• Future jurisprudence broadly interprets the safeguards against automated decision-making (Article 22(3)) to establish a right to explanation of specific decisions. This could occur, for example, on the basis that an explanation of the rationale of an automated decision is required in order to contest it or express views. Future guidelines of the European Data Protection Board could support this interpretation.

• Future jurisprudence establishes that the right of access (Article 15 GDPR) provides a basis for explanations of specific automated decisions, as a requirement to provide information about the ‘existence of … logic involved … significance … [or] envisaged consequences’ of automated decision-making (Article 15(h)1). This interpretation could also be supported in future guidelines of the European Data Protection Board.

Of these scenarios, the third and fourth seem to be the most plausible at the moment. Concerning the third, Article 22(3) guarantees that human intervention is available for automated decisions rendered in fulfilment of a contract or with explicit consent (see below). On this basis, one may argue that, although it is certainly not explicit in the phrasing of Article 22(3), the right to obtain human intervention, express views or contest a decision is meaningless if the data subject cannot understand how the contested decision was taken. The right to contest has already been interpreted by Member States, in enacting the 1995 Directive, as merely a right to force a controller to make a new decision. This interpretation is found in the UK Data Protection Act 1998 (Article 12(2)b): subjects can demand a new decision to be made, albeit without any way to assess the reliability of the old decision. A broad reading of Article 22(3), according to which an explanation is required to contest a decision, would strengthen the right to contest. In this case, the argument for a right to explanation of specific decisions could be further buttressed by drawing on the rights to fair trial and effective remedy enshrined in Articles 6 and 13 of the European Convention on Human Rights and Article 47 of the Charter of Fundamental Rights of the European Union. Without an explanation of how the algorithm works, both rights are hard to enforce, because the decisions/evidence used will be impossible to contest in court.⁹⁰

Concerning the fourth option, implementation of the right of access in the 1995 Directive has shown the need for interpretation of vague provisions by Member States and national courts. As noted above, consensus has not emerged over the meaning or requirements implied for data controllers when explaining the ‘logic involved’ in automated individual decisions. Austrian jurisprudence has demonstrated that the scope of ‘logic involved’ is sufficiently broad to include that some elements of the rationale or circumstances of a specific decision be explained along with system functionality, albeit limited severely by data controller’s interests (eg trade secrets). Despite aiming to unify data protection law across the Member States, the GDPR’s right of access will need to be similarly interpreted and tested. Given that the reference to ‘logic involved’ occurs in both the Directive and GDPR, it is plausible (but unlikely) that future legal interpretation of the right of access could establish a right to explanation of specific decisions.

Limitations on a right to explanation derived from the right of access (Article 15) or safeguards against automated decision-making (Article 22(3))

Assuming one or indeed a combination of the previous four scenarios occurs, and hence that a right to explanation of specific decisions is granted, other provisions in the GDPR may still limit its scope significantly. A ‘general’ right to explanation as proposed elsewhere (see ‘Introduction’ section), seemingly applicable to all types of automated processing, would not exist. A primary limitation is the narrow definition of automated decision-making in Article 22(1),⁹¹ defined as:

a decision based solely on automated processing, including profiling, which produces legal effects concerning [the data subject] or similarly significantly affects him or her.

An automated process must meet this definition for Articles 15(1)h (right of access) or 22(3) to apply, and thus for a future right to explanation established on either basis to be invoked.

Automated decision-making must have ‘legal or other significant effects’, with a decision based ‘solely on automated processing of data’ (Article 22(1)). The latter requirement opens a loophole whereby any human involvement in a decision-making process could mean it is not ‘automated decision-making’.⁹² While the required level of human involvement is not clear in practice, the phrase ‘solely’ suggests even some nominal human involvement may be sufficient. There is still uncertainty as to whether the usage of automated processing for the preparation of a decision ultimately acted upon by a human constitutes a decision ‘solely based on automated processing’, if the human does not interfere, verify, or modify the decision or decision-making rationale.⁹³ Preparation of evidence for a decision, and making the decision itself, are not necessarily equivalent acts.⁹⁴ Martini believes that automated processing of data for ‘assistance to make a decision’ or ‘preparation of a decision’ is not within the scope of Article 22.⁹⁵ Decisions based predominantly on automated processes, but with nominal human involvement, would thus not invoke Article 15(1)h (right of access) or Article 22(3) (safeguards against automated decision-making), and thus would not require an explanation of system functionality or the rationale of specific decisions, assuming that such a right to explanation of specific decisions was established on either basis.⁹⁶

Interpretation of Article 15 of the Directive, which was also limited to decisions ‘based solely on automated data processing’, does not provide clarification. The strict reading of ‘solely’ by Martini was reflected in the SCHUFA judgements already discussed (see section ‘Right of access in the 1995 Data Protection Directive 95/46/EC’). In contrast, Bygrave argues that a relative notion of ‘solely’ is required for the phrase to be meaningful. According to this position, decisions formally attributed to humans, but originating ‘from an automated data-processing operation the result of which is not actively assessed by either that person or other persons before being formalised as a decision’, would fall under the scope of ‘automated decision-making’.⁹⁷ It is not clear how this provision in the GDPR will be interpreted in the future.

The scope of data processing to which Article 22 (and Recital 71) applies was narrowed in the adopted version of the GDPR compared to prior drafts. The phrase ‘a decision based solely on automated processing’ proved a point of contention between the EC and EP drafts. Article 20(5) of the EP’s proposed amendments⁹⁸ to the EC’s draft⁹⁹ adds the phrase ‘predominantly’ to the measures to which the Article would apply (‘Profiling which leads to measures producing legal effects concerning the data subject or does similarly significantly affect the interests, rights or freedoms of the concerned data subject shall not be based solely or predominantly on automated processing and shall include human assessment …’ (emphasis added)). Following this, the EP wanted to restrict automated decisions on a broader basis than the EC, ie those predominantly and not only solely on automated processes. With ‘predominantly’ not being adopted in the final text of the GDPR, it would appear the strict reading of ‘solely’ was intended.

Questions can also be raised over what constitutes ‘legal effects’ or ‘similarly significant effects’¹⁰⁰ required for Article 22 to apply. Recital 71 provides some guidance, as it describes certain situations of ‘significances’ eg online credit applications and e-recruiting practices. Where a decision has no legal or significant effect, Article 22 does not apply. For an automated decision to have legal effects on the data subjects, it would need to affect their legal status.¹⁰¹ Since in most cases the data subject has no legal right to be hired or to be approved for a credit application, cases of being denied an interview or credit by an automated process would not fall under these categories.¹⁰² Admittedly, such cases could be considered to have ‘similarly significant’ effects. However, the term ‘similarly significant’ is itself vague and requires interpretation; significance varies on the perception of the data subject (effects of receiving a rejection letter will depend on the economic situation of the data subject, for instance), whereas impacts on legal status can be determined according to the letter of the law.¹⁰³ Further, in practice it may cause a burden for the data subject to prove that processing affects them significantly.¹⁰⁴ Alternatively an external standard for what constitutes significant effects could be defined.

As these constraints demonstrate, the definition of automated decision-making in Article 22(1) significantly narrows the scope of any future right to explanation. Automated decision-making that does not meet the definition provided in Article 22(1) would not be constrained by provisions of Article 22, or the additional measures required as part of notification duties (Article 13(2)f and 14(2)g) or the right of access (Article 15(1)h), including information regarding the ‘logic involved’ (see section ‘A right to explanation derived from the right of access’). A right to explanation implemented through any of the four paths specified above would similarly not apply, still significantly narrowing the right’s potential applicability to a very narrow range of cases meeting all the requirements in Article 22(1) and discussed in this section.

A further factor would constrain the information offered as part of an explanation. As indicated in the discussion of the right of access in the 1995 Directive, any future right to explanation would likely also be limited by overriding interests of the data controller. Recital 63 of the GDPR similarly establishes that the right of access should not infringe upon the rights and freedoms of others, including data controllers. The right can be limited for the sake of trade secrets or intellectual property rights, especially regarding copyright of software. As with the right of access itself, the specific disclosure requirements of Recital 63 require interpretation.¹⁰⁵ The Recital notes that:

the result of those considerations should not be a refusal to provide all information to the data subject.

Jurisprudence and legal commentary concerning the Directive’s right of access (see section ‘Right of access in the 1995 Data Protection Directive 95/46/EC’) suggest that the balance between the data subject’s right of access and data controllers’ rights and freedoms will require limited disclosures of the ‘logic involved’ in automated decision-making, primarily concerning system functionality rather than the rationale and circumstances of specific decisions.

Limitations exclusive to a right to explanation derived from safeguards against automated decision-making (Article 22(3))

In addition to the above limitations on a future right to explanation, a number of further limitations are exclusive to a right derived from Article 22(3). In the first instance, Article 22(2) states three conditions that, if met by an automated decision-making process, cause Article 22(1) not to apply:

(a) is necessary for entering into, or performance of, a contract between the data subject and a data controller;

(b) is authorised by Union or Member State law to which the controller is subject and which also lays down suitable measures to safeguard the data subject's rights and freedoms and legitimate interests; or

(c) is based on the data subject's explicit consent.

Article 22(3) specifies that safeguards (ie the rights to human intervention, expression, and contest) only apply when automated decision-making meets Article 22(2)a or c. The scope of any future right to explanation enacted in relation to the safeguards specified in Article 22(3) is therefore limited to cases meeting clause (a) or (c), ie those necessary for entering or performing a contract,¹⁰⁶ or with the subject’s explicit consent. It is worth noting that the safeguards in 22(3) do not apply when a decision is made in accordance with Union or Member State law (Article 22(2)b). In the latter case, explicit and specific safeguards are not described. Rather, ‘suitable measures to safeguard the data subject’ must be laid down in the relevant Union or Member State law. This clause potentially excludes a significant range of cases of automated decision-making from the safeguards in Article 22(3) and any right to explanation derived thereof. German commentary on the GDPR has suggested that the ‘suitable measures’ called for in Article 22(2)b do not include the disclosure of the algorithm used due to the risk posed to trade secrets; however, measures to minimize and correct discrimination and biases should be implemented.¹⁰⁷

The exemption for automated decisions related to contracts raises a further limitation. Article 22 does not define when automated decision-making is ‘necessary’ for entering or performing a contract, which runs the risk of ‘necessity’ being defined solely by the data controller. Additionally, it is important to note that Article 22(2)a envisions a situation that is different from explicit consent (which is listed as a separate exception in Article 22(2)c). Legislators were contemplating a situation where data controllers make automated decisions that are necessary for a contract, but without seeking consent first. If consent would be necessary, it would have been enough to list the contractual exception under Article 22(2)c. This structure suggests that there can be situations in which the data subject does not consent to an automated decision and, apart from the general notification requirements and right of access in Articles 13–15, does not know about the decision. Data controllers are therefore allowed to decide that automated decision-making is necessary for contractual obligations, while the data subject is unable to object to it. In this case, the data subject retains the right to contest, express views or obtain human intervention for a decision reached under Article 22(3), but not to object to it being made in the first place.

Two interpretations of Article 22

Several other restrictions on Article 22(3) and any future right to explanation derived thereof depend upon whether Article 22 is interpreted as a prohibition or a right to object. Article 22(1) GDPR states that ‘the data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her’. Due to its language (‘a right not to’), Article 22(1) can be interpreted in two ways: as a prohibition¹⁰⁸ or a right to object to automated decision-making. The two interpretations offer very different protection to the interests of data subjects and data controllers.

The first interpretation reads Article 22(1) as a prohibition, meaning that data controllers would be obligated not to engage in automated decision-making prior to showing that a condition in Article 22(2)a-c is met. The second interpretation reads Article 22(1) as establishing for data subjects a right to object to automated decision-making, which will not apply if one of the requirements in Article 22(2)a-c are met. These interpretations are differentiated by whether action is required by the data subject to restrict automated decision-making. The action in question, a formal objection by the data subject, requires both awareness of the existence of automated decision-making and a willingness to intercede, both of which require intentional effort on the part of the data subject.

Notably, this ambiguity has existed since the Data Protection Directive 1995.¹⁰⁹ The wording of Article 15 of the Directive¹¹⁰ allowed the ‘right not to be subject of an automated decision’ referred to in Section 1 (‘Member States shall grant the right to every person not to be subject to a decision which produces legal effects concerning him or significantly affects him and which is based solely on automated processing …’ (emphasis added)) to be interpreted as a prohibition or a right to object.¹¹¹ The ambiguity led Member States to implement this right and associated protections differently.

Article 15 of the Directive has been implemented by Austria, Belgium, Germany, Finland, the Netherlands, Portugal, Sweden and Ireland as a general prohibition,¹¹² with some exceptions. The UK has a different model: data subjects are entitled to request that no automated decision is made about them, but not in the case of so-called ‘exempt decisions’. In cases where data subjects have not lodged such a request, data controllers have to inform them about the fact that an automated decision has been made as well as about the outcome.¹¹³

Due to the similarities of language and content between Article 15 of the Directive and Article 22 GDPR, the varying implementation of Article 15 as a prohibition or right to object by Member states supports the interpretation that Article 22 is ambiguous and can be read as a prohibition or right to object. Resolving the ambiguity prior to 2018 is critical, as the two interpretations have very different consequences for data subjects and data controllers.

Impact of the interpretation of Article 22 on a right to explanation

If Article 22 is interpreted as a prohibition, data controllers will not be allowed to make automated decisions about a data subject until one of the three requirements specified in Article 22(2) (necessary to enter or to perform a contract, authorized by law, or explicit consent) is met. Data subjects do not need to act to prevent automated decision-making, but are rather protected by default. Supervisory Authorities would shoulder the burden of enforcing Article 22 by ensuring automated decision-making is carried out legally, and could levy penalties and fines in cases of illegal decision-making. Data controllers, when making automated decisions under Article 22(2)a or c, would need to enact safeguards as specified in Article 22(3). As explained above (see section ‘What if a right to explanation were granted?’), these safeguards could be voluntarily or legally extended to include a right to explanation.

If Article 22 is interpreted as a right to object, automated decision-making is restricted only to cases in which the data subject actively objects. When an objection is entered, decision-making must be shown to meet Article 22(2)a-c. For automated decisions that meet a requirement of Article 22(2), the data subject cannot object. However, when Article 22(2)a or c is met—meaning that the decision is made under contract or with consent—the safeguards specified in Article 22(3) would also apply. In these cases, the data subject would be able to request human intervention, express her views, and contest the decision and, if enacted in the future, demand a right to explanation (see section ‘A right to explanation derived from safeguards against automated decision-making’). Critically, if Article 22 grants a right to object automated decision-making is legally unchallenged by default, even if it does not meet any of the requirements set out in Article 22(2), so long as the data subject does not enter an objection. This limitation increases the burden on data subjects to protect actively their interests relating to profiling and automated decision-making by monitoring and objecting to automated decision-making.

With this comparison in mind, interpreting Article 22 as a prohibition grants greater protections by default to data subject’s interests, at least in the cases in which Article 22(3) would apply. As a prohibition, data controllers would be legally obliged to limit automated decision-making meeting the definition in Article 22(1) to the three cases identified in Article 22(2) (contract, Union or Member State law, consent).

In contrast, a right to object would not pre-emptively restrict the types of automated decision-making undertaken by data controllers to the three cases defined in Article 22(2). Rather, these restrictions would only apply when a data subject lodged an objection against a specific instance of decision-making. At that point, processes not meeting a requirement of Article 22(2) would need to stop, and the safeguards specified in Articles 22(2)b or 22(3) would never be triggered. Article 22 as a right to object would thus circumvent a right to explanation introduced through Article 22(3) by allowing automated decision-making not meeting a requirement in Article 22(2) to occur until the data subject enters an objection. Such ‘legal’ but pre-objection decision-making would not be subject to a right to explanation derived from Article 22(3). With that said, a right to explanation derived from the right of access would not be similarly circumvented. In this case a data subject’s right to explanation would apply to any decision-making meeting the definition provided in Article 22(1), even if the decision-making proved to not meet a requirement of Article 22(2) following the data subject’s objection.

To summarize, if a right to explanation is enacted in the future, at best, data subjects will only deserve an explanation when automated decisions have (i) legal or similarly significant effects, and (ii) are based solely on automated processes. Further, if a right to explanation is derived specifically from Article 22(3), explanations will be required only if automated decision-making is (iii) carried out to enter or under contract, or with explicit consent; and (iv) when overriding interests of the data controller (eg trade secrets) do not exist, as specified in Recital 63. Further restrictions on a right to explanation derived from Article 22(3) depend upon the prevailing interpretation of Article 22 as a prohibition or a right to object.

To disambiguate this limited type of right to explanation from the ‘general’ right to explanation in future discussion of the impact of the GDPR on automated processing of data, and to reflect accurately the scope of limitations on any such right, we recommend addressing instead a ‘right to be informed’ about the existence of automated decision-making and system functionality. The right to be informed addresses the information provided to data subjects about automated decision-making, taking into account all of the limitations on the scope of applicability and type of information to be provided by data controllers as described in the preceding two sections. The right to be informed further accounts for precedents set in the 1995 Directive, and the impact these precedents will likely have on future interpretation of the GDPR’s notification duties (Articles 13–14), right of access (Article 15), and right not to be subject to automated decision-making (Article 22)

Conclusion: future of the accountable automated decision-making

Despite claims to the contrary, a meaningful right to explanation is not legally mandated by the GDPR. Given the proliferation of automated decision-making and automated processing of data to support human decision-making (ie ‘not solely’), this is a critical gap in transparency and accountability. The GDPR appears to give strong protection against automated decision-making but, as it stands, the protections may prove ineffectual. However, transparent and accountable automated decision-making can still be achieved before the GDPR comes into force in 2018.

A right to explanation of specific decisions is not legally mandated by the safeguards contained in Article 22(3), or notification duties in Articles 13 and 14. As proven by the 1995 Directive, the right of access is ambiguous. However, the GDPR's right of access provides a right to explanation of system functionality, what we call a ‘right to be informed’, restricted by the interests of data controllers and future interpretations of Article 15. Any future right to explanation will further be constrained by the definition of ‘automated decision-making’ in Article 22(1), which is limited to decisions based solely on automated processing with legal or similarly significant effects for the data subject. As it stands, a meaningful right of explanation to the rationale and circumstances of specific automated decisions is not forthcoming.

Analysis of prior drafts of the GDPR has revealed several tensions between the EP, Commission, and Council. The placement of the right to explanation in non-binding Recital 71 appears to be a purposeful change deliberated in trilogue. The EP generally sought stronger protections for data subjects against automated decision-making than the EC or Council. Specifically, the EP wanted to include a right to explanation in Article 20,¹¹⁴ whereas the Council would have preferred to have the right to explanation in Recital 58.¹¹⁵ The EC did not include such a right at all. Further, the EP wanted to protect citizens from automated decision that have legal or significant effects when predominantly,¹¹⁶ and not just solely,¹¹⁷ based on automated processes. Human assessment would also have been required.¹¹⁸

As the GDPR is intended to unify data protection law across all European Member States, the interpretation of Article 22 as a prohibition or right to object is critically important. Which interpretation will win out in the implementation of the GDPR in 2018 is not yet clear. Both are viable as suggested by the split in the implementation of Article 15 in the Data Protection Directive 1995 by Member States. Without clarification prior to enforcement, Article 22 will allow for conflicting interpretations of the rights of data subjects and controllers concerning automated decision-making across Member States. Conflicts may soon become inevitable because the two interpretations protect very different interests.

Article 22 interpreted as a prohibition offers greater protection to the interests of data subjects by prohibiting all automated decision-making not meeting a requirement of Article 22(2). In contrast, when interpreted as a right, Article 22 creates a loophole that allows data controllers to undertake automated decision-making without meeting a requirement in Article 22(2), unless the data subject objects. Once an objection is entered, decision-making must be shown to meet one of these requirements or must stop altogether. As a right, the data subject’s interests in not being subjected to automated decision-making are undermined, insofar as significant effort (ie entering an objection) is required from the subject to protect her interests. Article 22 therefore roughly favours the interests of data subjects when interpreted as a prohibition, and the interests of data controllers when interpreted as a right.

The ambiguity of the right not to be subject to automated decision-making (Article 22), and the loopholes and weaknesses it creates, shows that the GDPR is lacking precise language and explicit and well-defined rights and safeguards, and therefore runs the risk of being toothless. Several actions may be recommended to correct some of the weaknesses identified in our analysis. The following recommendations are intended as guidance for legislative and policy steps to correct the deficiencies we have identified in the protections afforded to data subjects against automated decision-making.

Legislative progress can be achieved by modifying the GDPR prior to its enforcement, or passage of additional laws by Member States. Additional legislative steps by Member States are highly likely, as seen with the UK’s House of Commons’ Science and Technology Committee’s recent inquiry on ‘algorithms in decision-making’,¹¹⁹ which was inspired in part by informal consultations by ‘Sense about Science’ (a UK-based charitable trust) with the authors of this article. The inquiry gathers expert opinions on how to achieve accountability and transparency in algorithmic decision-making, including identification of barriers (eg trade secrets), mechanisms for oversight, and requirements to make decisions explainable. As evidence that the recommendations made here can be the starting point for new laws, the inquiry explicitly refers to the rights and duties laid out in the GDPR. On the policy side, the recommendations can influence future guidelines issued by bodies such as the Article 29 Working Party, the European Data Protection Board, the European Data Protection Supervisor, and its Ethics Advisory Group.

We make the following recommendations:

1) Add a right to explanation to legally binding Article 22(3)

If a right to explanation is intended as suggested in Recital 71, it should be explicitly added to a legally binding Article of the GDPR. Such an implementation should clarify the scope of applicability of the right with regard to the impact of Article 22 interpreted as a prohibition or right to object. Alternatively, Member States can be encouraged to implement law on top of the GDPR that requires an explanation of specific decisions. A right to explanation of specific decisions could be considered a suitable safeguard necessitated by Article 22(2)b and 22(3) if an explanation is necessary to contest a decision, as already prescribed in 22(3). The rights to contest a decision, to obtain human intervention or to express views granted in Article 22(3) may be meaningless if the data subject cannot understand how the contested decision was made. To this end, a right to explanation can be introduced requiring data controllers to provide information about the rationale of the contested decision. Clear requirements should be introduced stating the evidence to be supplied by the data controller. Evidence regarding the weighting of features, decision tree or classification structure, and general logic of the decision-making system may be sufficient. However, the risks for innovation and beneficial processing posed by a right to explanation that requires automated decision-making methods to be human interpretable should be seriously considered.¹²⁰

2) Clarify the meaning of the ‘existence of … significance … envisaged consequences … [and] logic involved’ in Article 15(1)h.

The language and meaning of core concepts in Article 15 is ambiguous. This leaves open the possibility of a right to explanation of the rationale of specific decisions (see section ‘From the Directive to the GDPR: the right to be informed’). However this interpretation is implausible for a number of reasons. As explained in section ‘A right to explanation derived from the right of access’, the semantics and history of the right to access, and the duplication of provisions in Articles 13–14, suggest that the right of access is intended merely as a counterweight to the notification duties of data controllers, and not as a means to introduce a new right (ie a right to explanation of specific decisions) beyond the scope of Articles 13–14.¹²¹ Critically, interpreting Article 15 to introduce a right to explanation of specific decisions would not match the intended purpose of the right of access, which according to Recital 63 GDPR is meant to allow the data subject ‘to be aware of, and verify, the lawfulness of processing’. Language should be added to clarify that Article 15 is intended either as a counterweight to Articles 13–14, and thus provides a ‘right to be informed’ about the existence of automated decision-making as well as system functionality, or as a right to explanation of specific decisions. The intended meaning of the five core concepts of Article 15(1)h should be made explicit, and their impact on the information required for data controllers to communicate to data subjects under the right to access (and, similarly, Articles 13–14 notification duties).

3) Clarify the language of Article 22(1) to indicate when decisions are based solely on automated processing

Article 22 is limited in applicability to decisions based solely on automated processing. However, it is unclear what the phrase means in practice. The potential loophole (similarly seen in the German SCHUFA judgements), by which nominal involvement of a human at any stage of the automated process means the process is not solely automated, should be closed. There is still uncertainty if the usage of automated processes for the preparation of a decision constitutes solely automated processes, if the human that takes the final decision does not wish to interfere or to adopt the decision. Clarification can be offered by returning to the phrasing ‘solely or predominantly based on’ proposed by the EP¹²² in Article 20(5), or by providing specific examples of decision-making based solely and predominantly on automated processing of data.

4) Clarify the language of Article 22(1) to indicate what counts as a legal or significant effect of automated decision, including profiling

Article 22 only applies for automated decision-making with ‘legal effects’ or ‘similarly significant effects’.¹²³ Recital 71 only names two examples of such effects: automatic refusal of an online credit application and e-recruiting practices. The scope of these phrases should be made explicit: do they, for instance, refer only to effects identified in the Articles of the GDPR, or to some broader definition? At a minimum, the perspective to be taken in defining "significant effects" should be identified. Do effects need to be significant from the subjective perspective of the data subject, or according to some external standard?

5) Clarify the language of Article 22(2)a, ‘necessary for entering, or performance of a contract’

Article 22(2)a names this case as an exception of either the prohibition of automated decision-making or the right to object to automated decision-making. Since it is likely that the necessity of such measures will be defined by the data controller and lit (a) does not require consent of the data subject (since this is a separate exception listed under lit (c), this exemption runs the risk of weakening the rights of data subjects.

6) Clarify the language of Article 22 to indicate a prohibition

Ideally, the language that allows for two plausible interpretations should be clarified prior to 2018 when the GDPR comes into force. Due to the number of loopholes and weakening of Article 22(3) safeguards introduced if Article 22 is interpreted as a right to object, as well as wide implementation of Article 15 of the 1995 Directive as a prohibition, we recommend that the language used in Article 22(2) (‘Paragraph 1 shall not apply if the decision:’) be revised to indicate clearly and explicitly that Article 22 is intended as a prohibition against automated decision-making.

7) As a counterweight to trade secrets, introduce an external auditing mechanism for automated decision-making, or set internal auditing requirements for data controllers

Both the right of access and any future right to explanation will face significant limitations due to the sensitivity of trade secrets and intellectual property rights. As our examination of the 1995 Directive shows, explanations granted under the right of access are normally limited to system functionality and significantly limited to protect data controller interests. An ideal solution would allow for examination of automated decision-making systems, including the rationale and circumstances of specific decisions, by a trusted third party. This approach limits the risk to data controllers of exposing trade secrets, while also providing an oversight mechanism for data subjects that can operate when explanations are infeasible or too complex for lay comprehension. The powers of Supervisory Authorities could be expanded in this regard. Alternatively, a European regulator could be created specifically for auditing algorithms, before (certifications) and/or after algorithms are being deployed.¹²⁴

8) Support further research into the feasibility of explanations alternative accountability mechanisms

Even if a right to explanation is legally granted in the future, the feasibility and practical requirements to offer explanations to data subjects remain unclear. In line with current work on interpretable automated decision-making and machine learning methods,¹²⁵ research needs to be conducted in parallel to determine whether and how explanations can and should be offered to data subjects (or proxies thereof) with differing levels of expertise and interests. What counts as a meaningful explanation for one individual or group may not be meaningful for another; requirements for ‘meaningful explanations’ must be set if a legal right to explanation is to be practically useful. The right to explanation is also not the only way to achieve accountability and transparency in automated decision-making.¹²⁶ Further attention should be given to the development and deployment of alternative legal safeguards that can supplement the protections offered by the GDPR. Data controllers working in highly sensitive or risky sectors could, for instance, be required to use human interpretable decision-making methods.¹²⁷ Methods and (ethical) requirements for auditing algorithms¹²⁸ should also be further developed, both as standalone accountability tools and as mechanisms to provide an evidence trail for providing explanations of automated decisions.

As the ambiguities highlighted in these recommendations indicate, the GDPR can be a toothless or powerful mechanism to protect data subjects depending on its eventual legal interpretation. The effectiveness of the new framework will largely be determined by Supervisory Authorities, the Article 29 Working Party, the European Data Protection Board, the European Data Protection Supervisor, its Ethics Advisory Group,¹²⁹ as well as national courts and their future judgments.¹³⁰ As it stands, transparent and accountable automated decision-making is not yet guaranteed by the GDPR; nor is a right to explanation of specific decisions forthcoming. At best, data subjects will be granted a ‘right to be informed’ about the existence of automated decision-making and system functionality. These shortcomings should be addressed before the GDPR comes into force in 2018.

Footnotes

Author notes