Paternalism in the algorithm

There is not a day in which we are not surrounded by algorithms that try to guide our choices. We see it, for example, in social media with systems that feed on our interaction, sometimes almost imperceptible, and adjust what is shown to us to tip the balance towards certain content. In that context, our decisions stop being completely ours, because they are made within an environment designed to push us in a direction.

We often assume that this delegation is done "for our own good", to save us time and avoid the mental load of deciding what to watch on one of our few evenings of rest. And, on the surface, it works. Help in the face of indecision is usually perceived as something positive. But the question changes when the algorithm decides unilaterally, based on partial, biased, incomplete or conditioned information, and furthermore, does so without the user being fully aware of how that recommendation was reached.

Algorithmic paternalism can be understood as the intervention of computational or artificial intelligence systems in human decision-making, guiding or limiting freedom of choice towards outcomes that the system, or those who design it, consider "better". The critical trait is that this guidance can operate in an opaque manner, so that the person does not always perceive how much of their decision truly comes from their deliberation and how much from the design of the environment.

I was surprised that algorithms were presented and marketed as an objective fact. A much more accurate description of an algorithm is that it is an opinion embedded in mathematics.

And if that opinion furthermore cannot be seen from the inside, then the problem is no longer just what the system decides, but how and for whom it is deciding.

Currently, algorithmic paternalism is not usually considered among the most visible risks associated with computing advances. However, as black box type models become widespread and explainability decreases, it becomes more difficult to audit why the system recommends, prioritises or discards certain options. In that scenario, we cannot assume that the delegation of decisions will always operate for the benefit of the user. It can also reflect data biases, commercial incentives or design criteria that do not align with our values, tastes or preferences.

The justification that has been given to us for the necessity of this phenomenon relies, to a great extent, on a prior idea, that people are irrational beings and that, therefore, we need "help" to decide better. Along this line, Dan Ariely popularises the thesis that we are "systematically and predictably irrational beings" in his book Predictably Irrational: The Hidden Forces That Shape Our Decisions. Ariely suggests that this irrationality appears as a recurring distance between the way we would like to behave and the way we actually do, a persistent gap that he calls The Potential Human Gap.

But not everyone buys that starting premise. In contrast, Herbert Simon nuanced that human beings can indeed be rational, although under very specific limits such as time, available information, computing capacity, context... That is, it is not so much irrationality as bounded rationality, we decide as best we can with what we have, not with what would be ideal to have.

Despite this nuance, the idea that we are irrational beings has always won this battle, undervaluing us as beings capable of making decisions and thus justifying the idea that technology, algorithms and AI will be more apt to make decisions than ourselves. Furthermore, with the idea that this irrationality is incorrigible, it invites the exploitation of these deficiencies in our capacities to redirect our behaviour, no longer just correcting it, but directing it intentionally.

A clear example in Catalonia is RisCanvi, an institutional tool for assessing risk (including recidivism) in the prison system. The ethical problem appears when that technical assessment functions as algorithmic paternalism, the system labels the person as more or less dangerous and that label conditions sensitive decisions such as intervention, classification, permits... displacing human deliberation towards a probabilistic logic. If furthermore the model is opaque or difficult to challenge, autonomy, explainability and justice in the face of errors or biases are weakened, reducing the person to a risk profile that is difficult to contradict when there is no comprehensible explanation.

Ultimately, the problem is not that systems that help decide exist, but that the help transforms into direction without fully informed consent. When we accept that a model, however efficient it may be, can substitute our deliberation, we are giving up something more than time. We give up the judgement on what counts as a good decision. And there algorithmic paternalism stops being a technological convenience to become a moral and political question: who defines what is best, to what ends, and under what conditions of transparency and accountability. If we cannot see, understand and question the logic that pushes us, then the delegation is no longer assistance, it is governance over our choices.