The Bobiverse connection is apt. I'm unsure about the conclusion and about the premises. This appears to be a thought experiment based entirely in fiction. Whereas, other thought experiments, say...
The Bobiverse connection is apt. I'm unsure about the conclusion and about the premises. This appears to be a thought experiment based entirely in fiction. Whereas, other thought experiments, say Schrödinger's cat or Zeno's Paradox, take their premises in reality, the paperclip "problem" takes it's premise of super-intelligence from fantasy and then claims to derive a "real-world" conclusion. If the premises are founded in make-believe, then so are the conclusions.
In His Master's Voice, Stanislaw Lem imagines a message containing such super-intelligent message being beamed to earth from the stars. Far from using such an event for the betterment of humanity, the Great Powers try to discover its martial capabilities. Ultimately, they fail at this as well. In the end, our inability to understand the message blinds us to the potential benefits of this message, but thankfully, also prevents us from destroying ourselves with it.
This interpretation of the paperclip problem is wrong, of course. But, in my opinion, so are all such approaches. The premise is fictive, so all conclusions from are it also equally fictive.
Relevant XKCD: https://xkcd.com/1968/ I've always been far more concerned about bad actors commanding the swarms of kill bots and propaganda relays we already have rather than a magic computer...
I've always been far more concerned about bad actors commanding the swarms of kill bots and propaganda relays we already have rather than a magic computer intelligence ex nihilo hijacking them out of self defense.
I was reading the Bobiverse books and it referred to this issue. The paperclip problem is an interesting read but I am not sure about the conclusions
The Bobiverse connection is apt. I'm unsure about the conclusion and about the premises. This appears to be a thought experiment based entirely in fiction. Whereas, other thought experiments, say Schrödinger's cat or Zeno's Paradox, take their premises in reality, the paperclip "problem" takes it's premise of super-intelligence from fantasy and then claims to derive a "real-world" conclusion. If the premises are founded in make-believe, then so are the conclusions.
In His Master's Voice, Stanislaw Lem imagines a message containing such super-intelligent message being beamed to earth from the stars. Far from using such an event for the betterment of humanity, the Great Powers try to discover its martial capabilities. Ultimately, they fail at this as well. In the end, our inability to understand the message blinds us to the potential benefits of this message, but thankfully, also prevents us from destroying ourselves with it.
This interpretation of the paperclip problem is wrong, of course. But, in my opinion, so are all such approaches. The premise is fictive, so all conclusions from are it also equally fictive.
Relevant XKCD:
https://xkcd.com/1968/
I've always been far more concerned about bad actors commanding the swarms of kill bots and propaganda relays we already have rather than a magic computer intelligence ex nihilo hijacking them out of self defense.