VAR, bias, and the consensus that doesn't exist

VAR promised to eliminate human error from soccer. But today half the crowd is still shouting “penalty” while the rest are convinced he dived. Can technology solve a problem if humans can’t agree on what the solution looks like?

Technology didn’t solve the problem because the problem was never technical. It was, and still is, that humans can’t agree on what constitutes a foul and what doesn’t. VAR doesn’t eliminate subjectivity; it makes it more visible.

Something similar happens with artificial intelligence and bias. We demand impartial, neutral, objective algorithms. But impartiality presupposes prior consensus on what is correct. And that consensus, in many cases, simply doesn’t exist.

At LLYC we saw this firsthand while training an AI to predict whether a message would have a positive, neutral, or negative impact on a brand’s reputation. We asked 160 experts to evaluate more than 13,000 messages; each message was analyzed by three different professionals. Only in 55% of cases was there full agreement. In the remaining 45%, experts with years of experience and shared frameworks reached different conclusions about the same message.

The question becomes unavoidable: if there’s no human consensus, what is the correct answer the AI should learn? The majority view? What if the minority is right?

This dilemma intensifies when we talk about bias. We all agree that a fair AI should avoid it. But is there really universal agreement on where, say, a racist attitude begins and ends? The line separating humor from offense, stereotype from discrimination, appropriate correction from overreach, shifts depending on who draws it. And if humans haven’t resolved that boundary, how can an algorithm trained on our own contradictions resolve it?

Perhaps some of AI’s unexpected outputs are not failures of intelligence. They are the computational expression of a disagreement that technology inherits but isn’t meant to resolve. So before trying to build a bias-free AI, it might be worth sitting with an uncomfortable question: are we sure we ourselves know exactly what we’re asking for?