<\/span><\/h2>\nThis is not an appeal to authority. It is an appeal to the fact that someone has already spent years on the hard version of this question, with lawyers arguing both sides, and the output is public.<\/p>\n
GDPR Article 22<\/strong> establishes that a data subject has the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning them or similarly significantly affects them. Where such processing is permitted, the controller must implement safeguards including, at minimum, the right to obtain human intervention, to express a point of view, and to contest the decision. Regulatory guidance treats automatic refusal of an online credit application and e-recruiting practices without human intervention as examples of the effects in scope.<\/p>\nRead the mechanism rather than the legalese. The trigger is not that the system might be inaccurate. The trigger is the combination of two things: the decision is made solely<\/em> by the machine, and the consequence for the person is significant<\/em>. Accuracy is not mentioned. A highly accurate system that refuses your loan with no human in the loop is still the thing the article is about.<\/p>\nThe EU AI Act, Regulation (EU) 2024\/1689,<\/strong> takes the same logic and gets specific. Article 14 requires high-risk systems to be designed so they can be effectively overseen by natural persons. Paragraph 4 then lists what the overseer must actually be able to do. It is, unexpectedly, one of the better decision checklists ever written, and it translates directly to personal use.<\/p>\nTable 1. The five oversight capabilities of AI Act Article 14(4), translated into personal decision questions (CEOtudent editorial framework, built on the verified text of Regulation (EU) 2024\/1689)<\/strong><\/p>\n\n\n\n| AI Act Article 14(4) requires the overseer to be able to<\/th>\n | The personal question it becomes<\/th>\n<\/tr>\n<\/thead>\n |
\n\n| (a) properly understand the relevant capacities and limitations of the system and monitor its operation, including detecting anomalies<\/td>\n | Do I actually know what this tool is bad at, or only what it is good at?<\/td>\n<\/tr>\n |
\n| (b) remain aware of the possible tendency of automatically relying or over-relying on the output (automation bias)<\/td>\n | If I found myself agreeing every time, would I notice?<\/td>\n<\/tr>\n |
\n| (c) correctly interpret the system’s output, taking into account available interpretation tools<\/td>\n | Can I read the output well enough to tell a good one from a confident wrong one?<\/td>\n<\/tr>\n |
\n| (d) decide, in any particular situation, not to use the system or to disregard, override or reverse the output<\/td>\n | Do I have a live alternative if I reject it, or am I committed the moment I ask?<\/td>\n<\/tr>\n |
\n| (e) intervene in the operation or interrupt the system through a stop button or similar procedure<\/td>\n | Can I stop this before it takes effect on anyone but me?<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Paragraph 5 adds one more rule worth borrowing personally. For remote biometric identification, no action or decision may be taken on the basis of the system’s identification unless it has been separately verified and confirmed by at least two natural persons with the necessary competence, training and authority. The law’s answer to the highest-stakes category is not a better algorithm. It is a second human.<\/p>\n Annex III lists the domains the Act designates as high-risk in the first place: biometrics; critical infrastructure; education and vocational training; employment and worker management; access to essential private and public services including creditworthiness and insurance pricing; law enforcement; migration, asylum and border control; and the administration of justice and democratic processes. Systems in these categories placed on the market after 2 August 2026 fall under the regime.<\/p>\n Look at what those eight have in common. They are not the hardest problems. Some are computationally trivial. They are the decisions where the person affected cannot easily undo the outcome, cannot readily verify the reasoning, and did not choose to be in the process. That is the pattern, and it is the pattern you can use.<\/p>\n <\/span>The variable everyone gets wrong is not accuracy<\/span><\/h2>\nHere is the case for taking the reversibility framing seriously rather than the accuracy framing.<\/p>\n In 2025, METR ran a randomised controlled trial with 16 experienced open-source developers working 246 real tasks in their own repositories, with tasks randomly assigned to allow or disallow AI tools. Before the study, participants forecast that AI would speed them up by 24%. After doing the work, they estimated it had sped them up by 20%. Measured, they were 19% slower. METR now labels the result historical, noting it does not necessarily reflect current tools or workflows, and that caveat should be taken seriously.<\/p>\n The number that survives the caveat is not the 19%. It is the 39-point gap between what expert practitioners measured and what those same expert practitioners believed, in their own domain, about work they had just finished.<\/p>\n That gap is the whole problem with accuracy-based delegation rules. A rule that says “delegate when the AI is reliable enough” requires you to know how reliable it was. The people best positioned to know were wrong by 39 points about themselves. The AI Act names this failure mode directly in Article 14(4)(b), which requires overseers to remain aware of the tendency to over-rely on automated output. Regulators built the checklist around it because they did not expect self-assessment to work either.<\/p>\n So stop asking how likely it is to be wrong. Ask what happens when it is.<\/p>\n |