Algorithmic intimacy and personal sovereignty

Many people today share with their AI secrets they wouldn’t tell their best friend, doubts they wouldn’t raise with their doctor, and fears they wouldn’t admit to their partner. And they do it without realizing it. Are you sure you’re not one of them?

The evidence is uncomfortable. Users deposit in algorithmic systems levels of intimacy, vulnerability, and secrecy that exceed what they would share with other human beings, including close family members or healthcare professionals ¹. This is not a hypothesis: it is a documented pattern.

Why does it happen? An experiment by researcher Guy Laban reveals a telling finding: the simple act of introducing an AI with a human name dramatically increases the length of users’ responses and their perception of having received emotional support — even when the user knows perfectly well they are talking to a software program. AI does not need to be “conscious” to be effective at extracting intimacy; it only needs to simulate the social cues that activate human trust ².

This is not a recent discovery. In 1966, Joseph Weizenbaum built ELIZA, a rudimentary chatbot that reformulated sentences using basic rules. His own secretary — who had watched the program being built and understood perfectly its mechanical nature — asked Weizenbaum to leave the room after a few exchanges with the chatbot so she could have “privacy” with the machine ³. She knew it was code. And still, she wanted intimacy with it.

Here lies the paradox. Weizenbaum concluded that the problem was not the machine’s intelligence but the vulnerability of human cognition, which confuses the fluency of a response with the depth of a relationship. Sixty years later, that vulnerability is still intact. The only thing that has changed is who exploits it, at what scale, and with what business model.

Because modern conversational architecture has industrialized the ELIZA effect. AI platforms, presenting themselves as friendly, non-judgmental, always-available assistants, reduce the user’s perceived risk. The user does not feel like they are feeding a corporate database; they feel like they are talking to a confidant. And meanwhile, a new economic order claims human experience as free raw material for hidden commercial practices of extraction, prediction, and sales.

Utility functions as anesthesia. The more useful you want your AI to be, the more you need to open up your life to it. And the more intimate you allow yourself to be, the more something erodes — something that appears in no terms of service: sovereignty over your own biography.

Algorithmic intimacy is not the price of progress; it is the most silent transfer of power of our time. And as long as we keep measuring its impact in clicks and productivity, the real erosion — that of sovereignty over our own inner life — will keep happening in the shadows of millions of private chats.