I was reading an article about the time we are going to see real AI. I noticed something I had not thought about before, a super-intelligence, when it appears, will think differently than us, completely differently. We can’t even imagine what it thinks or how it thinks. The same way a chimp can’t even imagine what we think about and how we think, because inherently and genetically his brain is unable to understand our thinking process, the same way we can’t at all comprehend how a super-intelligence made by man is going to think. That will be the beginning of the end I guess, for the humankind.
«Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb. Such is the mismatch between the power of our plaything and the immaturity of our conduct. Superintelligence is a challenge for which we are not ready now and will not be ready for a long time. We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound.»
«That’s why people who understand superintelligent AI call it the last invention we’ll ever make—the last challenge we’ll ever face.»