23. The Singularity

How can we improve our anticipation and design of artificial superintelligence (ASI)? What are the likely consequences of the arrival of ASI?

Important note: text versions of many of the key ideas in this top-level area of the Vital Syllabus are being assembled here.

Resources providing an overall introduction to the Singularity:

“What happens when our computers get smarter than us” by Nick Bostrom
“Artificial General Intelligence: Humanity’s Last Invention” by Ben Goertzel
“The Real Reason to be Afraid of Artificial Intelligence” by Peter Haas

23.1 The singularitarian stance

23.2 The singularity shadow

23.3 Different routes to superintelligence

23.4 Hard and soft take-off

23.5 Possible timescales to reach ASI

23.6 The Control Problem

Introductions to the Control Problem

“The case for taking AI seriously as a threat to humanity” – Vox article by Kelsey Piper

“Can we build AI without losing control over it?” – TED talk by Sam Harris
“Why Would AI Want to do Bad Things? Instrumental Convergence” by Robert Miles
“Intro to AI Safety”, by Robert Miles
“Why Asimov’s Laws of Robotics Don’t Work” – Robert Miles for Computerphile
Ten flawed reasons why people ignore the question of AI safety – by Robert Miles

23.7 The Alignment Problem

“Aligning AI systems with human intent” by OpenAI

23.8 Human-ASI merger

23.9 No Planet B

23.10 The singularity principles

23.11 AGI or not AGI: fundamental choices


Navigation aides:


Re-using this material:

The content of Vital Syllabus is available under CC BY 4.0 unless otherwise noted.