Delegation without a contract

You can remove your president from office. You cannot remove the algorithm that decides your credit score, your employability, or which version of the world you see every morning. This is not rhetoric: these are real decisions, about you, being made right now. When did we agree to this?

The scale is hard to absorb. 98% of Fortune 500 companies use automated applicant tracking systems (ATS) to filter candidates ¹. Up to 75% of résumés are filtered out by ATS before a human ever reads them ². Your credit score determines whether you can get a mortgage. Your insurance premium is calculated using variables you’ll never know. And your news feed every morning is composed by an algorithm optimizing for your attention, not your understanding of the world. None of these systems asked for your consent. None of them owe you an explanation.

Modern democracies built specific mechanisms for the delegation of power: representation, transparency, reasoned justification, the right to appeal. If a public official denies you a benefit, they must explain why, and you can challenge the decision. If a judge convicts you, they must justify the sentence. That is the implicit contract of legitimate power: if you wrong me, I can demand an answer. With the algorithms that now make equivalent decisions about your life, that contract does not exist. And the opacity is not a technical flaw waiting to be fixed: as Frank Pasquale documents, it is a structural condition of the business model ³. What you cannot audit, you cannot question. And what you cannot question governs you without accountability.

If a government minister made decisions about your career with opaque criteria, no obligation to explain themselves, and no avenue for appeal, we would call it authoritarianism. When an algorithm does it, we call it efficiency. The “perceived neutrality” of technology — that illusion of mathematical objectivity that Cathy O’Neil dismantles in Weapons of Math Destruction ⁴ — disarms resistance. And what generates no resistance needs no legitimacy. A RAND Corporation study confirms it: citizens perceive algorithmic decisions as significantly less transparent than human ones ⁵. Yet institutions continue to shield themselves behind those systems to dilute accountability.

Back in 1979, an IBM internal training manual warned: “A computer can never be held accountable; therefore, a computer must never make a management decision” ⁶. Forty-five years later, we have done exactly the opposite.

We spent centuries building institutions to prevent humans from exercising power over other humans without accountability. We have not built them for algorithms. And we have not demanded them either. The deficit is not only in the technology. It is in us.