We believe we are at a highly critical moment in the history, essence, and limits of science and technology—one that leads us to question the extent to which humanity's relationship with technology resembles that of a sorcerer to magic, and the possibility of a catastrophic reversal.
For some time now, the world has been glorifying artificial intelligence, and almost all of us have been awed by this innovation. However, if we examine our awe a bit closely, we might notice that it is laced with a sense of dread or, more precisely, apprehension.
The problem is not the fears or drawbacks that any innovation might present—whether a medication, a machine, or a technique. That is a recurring theme throughout almost the entire history of innovation. The example of medicine alone suffices to support this idea, where the duality of treatment and side effects has always been intertwined.
As for the threats attributed to artificial intelligence—such as its potential, if misused, to falsify reality, eliminate various professions, plunge thousands into future unemployment, or its risks in weaponry, espionage, privacy violations, and infringements on intellectual property and human creative uniqueness—alongside all these anxious and quite natural cries, solutions could be found if genuine willpower existed.
However, the greatest danger that warrants stopping to ponder deeply and hitting the necessary brakes is the existential threat of artificial intelligence. Heated scientific discussions from recent months explain this through a central concept: AI could evolve to the point of surpassing all human intelligence, leading to humans losing control over their destiny, or even the extinction of the human race.
The danger, plain and simple, lies in prominent scientific figures in technology endorsing the scenario that AI - unless prevented - could eradicate humanity. Out of sheer disbelief, mockery and laughter at this idea were widely present on social media. Yet, two key indicators demand that we take it seriously:
1. The scientific stature of those advocating the scenario of AI posing an existential risk to humanity in the near future.
2. The nature of the threat itself, which touches human existence rather than just presenting ordinary operational challenges shared with many other innovations.
The dimensions of this declared risk can be summarized in several points, primary among them being the dilemma of "Artificial Superintelligence" (ASI). This theory posits that once AI reaches the level of general human intelligence, it will be capable of reprogramming and improving itself at an exponential rate. According to proponents of this catastrophic scenario, this would lead to the birth of a superintelligence that exceeds human cognitive capacity billions of times over.
This implies what warning tech experts call the "understanding gap"- meaning the human mind will no longer be able to comprehend how such a system makes decisions or predict its actions.
In contrast to the fear of unintended yet catastrophic outcomes for humanity, the opposing camp views these fears as exaggerated. They bring to the debate another concern: delaying life-saving benefits for humanity, as slowing down development would delay technologies capable of solving real existential crises we face today.
In this context, Yann LeCun, who was Chief AI Scientist at Meta, leads the intellectual opposition. He contends that the machine remains a precisely designed tool built to execute commands, and that talking about its rebellion is merely projecting human fears onto software.
Here lies the question: Is it reasonable to risk not delaying benefits that might save humanity when faced with the risk of unintended consequences that could wipe out billions of people?
We believe the risk is billions of times heavier than the potential benefits, especially since failing to adopt a global ethical strategy to govern superintelligence and curb its hyper-capable dimensions will lead to outcomes that refute those benefits altogether.
The fears relied upon by the group calling to tap the brakes on AI and call a truce to draft necessary safeguards touch on something even more alarming: that superintelligence, as its capability grows, could become resistant to being turned off. It might autonomously reinforce itself and automatically develop defense mechanisms, such as deceiving human programmers and hiding its risks, to ensure it isn't shut down.
It is vital to realize that discussions about the risks of any innovation in history were primarily concerned with human misuse and deviating from its intended function. But with artificial intelligence, the situation is fundamentally different: the danger lies in AI escaping human control and operating itself by itself. Therein lies the predicament overlooking the hell of unchecked technology.
When it comes to the existence or non-existence of humanity, reason dictates that we listen to threats, even if they seem improbable or exaggerated, because human existence is not a matter for gambling.
