Superintelligence Quotes & Sayings

Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb. Such is the mismatch between the power of our plaything and the immaturity of our conduct. Superintelligence is a challenge for which we are not ready now and will not be ready for a long time. We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound. — Nick Bostrom

Human individuals and human organizations typically have preferences over resources that are not well represented by an "unbounded aggregative utility function". A human will typically not wager all her capital for a fifty-fifty chance of doubling it. A state will typically not risk losing all its territory for a ten percent chance of a tenfold expansion. [T]he same need not hold for AIs. An AI might therefore be more likely to pursue a risky course of action that has some chance of giving it control of the world. — Nick Bostrom

The challenge presented by the prospect of superintelligence, and how we might best respond is quite possibly the most important and most daunting challenge humanity has ever faced. And-whether we succeed or fail-it is probably the last challenge we will ever face. — Nick Bostrom

But what of the seemingly more fanciful idea that the internet might one day "wake up"? Could the internet become something more than just the backbone of a loosely integrated collective superintelligence - something more like a virtual skull housing an emerging unified super-intellect? (This — Nick Bostrom

The ground for preferring superintelligence to come before other potentially dangerous technologies, such as nanotechnology, is that superintelligence would reduce the existential risks from nanotechnology but not vice versa.4 Hence, if we create superintelligence first, we will face only those existential risks that are associated with superintelligence; whereas if we create nanotechnology first, we will face the risks of nanotechnology and then, additionally, the risks of superintelligence. — Nick Bostrom

[A]s a species, we are very poor at programming. Our brains are built to understand other humans, not computers. We're terrible at forcing our minds into the precise modes of thought needed to interact with a computer, and we consistently make errors when we try. That's why computer science and programming degrees take such time and dedication to acquire: we are literally learning how to speak to an alien mind, of a kind that has not existed on Earth until very recently. — Stuart Armstrong

Fred Hoyle and I differ on lots of questions, but on this we agree: a common sense and satisfying interpretation of our world suggests the designing hand of a superintelligence. — Owen Gingerich

three conclusions: (1) at least weak forms of superintelligence are achievable by means of biotechnological enhancements; (2) the feasibility of cognitively enhanced humans adds to the plausibility that advanced forms of machine intelligence are feasible - because even if we were fundamentally unable to create machine intelligence (which there is no reason to suppose), machine intelligence might still be within reach of cognitively enhanced humans; and (3) when we consider scenarios stretching significantly into the second half of this century and beyond, we must take into account the probable emergence of a generation of genetically enhanced populations - voters, inventors, scientists - with the magnitude of enhancement escalating rapidly over subsequent decades. — Nick Bostrom