Nick Bostrom

Biography

Nick Bostrom is Professor in the Faculty of Philosophy at Oxford University, with a background in physics, computational neuroscience, mathematical logic, and philosophy. He is the founding director of the Future of Humanity Institute, a multidisciplinary research center at the University of Oxford which enables leading researchers to use mathematics, philosophy, and science to explore big-picture questions about humanity. Recently, the focus of the institute has been exploring questions regarding existential risks and the future of machine intelligence. The Future of Humanity institute works closely with the Centre for Effective Altruism .

In the beginning of 1998, the World Transhumanist Association was founded by Nick Bostrom and David Pearce. Its objective is “to provide a general organizational basis for all transhumanist groups and interests, across the political spectrum. The aim was also to develop a more mature and academically respectable form of transhumanism, freed from the “cultishness” which, at least in the eyes of some critics, had afflicted some of its earlier convocations.” The association has since changed its name to Humanity+. There were two founding documents of the World Transhumanist Association: the Transhumanist Declaration and the Transhumanist FAQ. The first document was a concise statement of the basic principles of transhumanism. The FAQ was a consensus document, more philosophical in its scope .

Bostrom is the author of the books Anthropic Bias (2002), and Superintelligence: Paths, Dangers, Strategies (2014). He served as an editor for the books Human Enhancement (2009), and Global Catastrophic Risks (2014). He has also published several journal papers, as well as contributing to books chapters, conference proceedings and articles. He is best known for his work in the simulation argument, existential risk, anthropics, impacts of future technology, and implications of consequentialism for global strategy .

He has received a Eugene R. Gannon Award, in which one person is selected annually worldwide from the fields of philosophy, mathematics, the arts and other humanities, and the natural sciences. He was selected to be part of Foreign Policy’s Top 100 Global Thinkers list and Prospect Magazine’s World Thinkers list .

He was born in March 10, 1973, in Helsingborg (Sweden). He grew up as an only child and didn’t particularly enjoyed school. He does not cite his parents as influences in exploring large philosophical questions. His father worked for an investment bank, and his mother for a Swedish corporation . When he was a teenager he had what he describes as an “epiphany experience”. In 1989 he picked up at random an anthology book of 19th-century German philosophy, containing works by Nietzsche and Schopenhauer. After reading it, he experience a dramatic sense of the possibilities of learning and proceeded to educating himself quickly, reading feverishly, and painting and writing poetry in the spare time . He did not pursue these artistic endeavors, giving priority to his mathematical pursuits. He took degrees in philosophy and mathematical logic at Gothenburg University and completed his PhD at the London School of Economics .

Existential risk

In his studies, Nick Bostrom has delved into the analysis of existential risks. These are the ones that threaten the entire future of humanity, although it may seem unthinkable. It may not, necessarily, mean the premature extinction of Earth-originating intelligent life, but instead be the permanent and drastic destruction of the potential for desirable future development. Human extinction risks are not well understood and tend to be underestimated by society . Some of the risks are relatively well know, such as asteroids or supervolcanoes, but there are others more obscure, arising from the development of human technology, and that are expected to grow in number and potency in the near future .

Usually, it is difficult to determine the probability of existential risks, but experts suppose that there is a significant risk confronting humanity over the next few centuries. It is estimated to be between 10-20 per cent in this century. This value relies in subjective judgement and so a more reasonable estimate could be substantially higher or lower. Some theories of value imply that a relatively small reduction in net existential risk could have enormous expected value . Bostrom uses philosophy and mathematics (specifically probability theory) to try and determine how humanity might prevent and survive on of many potential catastrophic events .

Not all risks have the same probability of occurring. According to Bostrom (2002), the description of the magnitude of a risk can be made using three dimension: scope, intensity, and probability. Here, “scope” is the size of the group of people that are at risk; “intensity” is how much each individual in the group would be affected; and “probability” is the best current subjective estimate of the probability of the adverse outcome .

There is a distinction between existential risks and global endurable risks. In the first one, all of humankind is imperiled. Examples of the latter are threats to the Earth’s biodiversity, moderate global warming, global economic recessions, or stifling cultural or religious eras (e.g. “dark ages”), assuming they are transitory. These examples, although endurable, are not meant to be taken as acceptable or not serious .

The first man-made existential risk was the detonation of an atomic bomb. There were concerns at the time that the explosion could start a chain-reaction by igniting the atmosphere. This risks has, since then, been dismissed as physically impossible, but it would still be qualified as an existential risk that was present at the time. This means that if there is some subjective probability of an adverse outcome (even if later the risk is dismissed), it still needs to be considered as one at the time. Concluding, one as to take into account the best current subjective estimate of what the objective risk factors are. Bostrom says that the risk of nuclear Armageddon and comet or asteroid strikes may be preludes to the risks that will be faced during the 21st century .

Existential risks can be classified into four categories. The names of each one has changed since first introduced by Bostrom, but their descriptions remain the same. These were, initially, “Bangs”, “Crunches”, “Shrieks”, and “Whimpers”. The current nomenclature of the four categories is: “Human extinction”, “Permanent stagnation”, “Flawed realization”, and “Subsequent ruination” .

In the “Human extinction” category, humanity goes extinct before reaching technological maturity, due to a sudden accidental disaster or a deliberate act of destruction. Examples include the deliberate misuse of nanotechnology, nuclear holocaust, the shutting down of a simulation (if we are living in one), a badly programmed superintelligence, or a genetically engineered biological agent. “Permanent stagnation” is when humanity never reaches technology maturity, even though human life continues in some form. Some scenarios that can lead to this category are resource depletion or ecological destruction, misguided world government stopping technological progress, “dysgenic” pressures, or technological arrest. In the category of “Flawed realization”, humankind achieves technological maturity but in a flawed way, that is irremediably flawed, existing in an extremely narrow band of what would be possible and desirable. Examples include a take-over by a transcending upload, flawed superintelligence, or a repressive totalitarian global regime. Finally, in “Subsequent ruination” humanity reaches technological maturity but future developments lead to the permanent ruination of future good prospects. Example scenarios for this are to be killed by an extraterrestrial civilization, the potential or even core values of humanity are eroded by evolutionary development, or something unforeseen .

In Bostrom (2013) some policy implications of existential risks are described. For example, existential risk as a concept can help bring focus on long term global efforts and sustainability concerns; morally it could be said that existential risk reduction is strictly more important than any other global public good; and maybe the cost-effective strategy to reduce existential risks, in a long-term, is to fund analysis of a wide range of risks and mitigation policies. It is considered that, between all of the existential risk scenarios, the ones with the greatest concern level are the anthropogenic existential risks that arise from human activity .

Bostrom also suggested the concept of “Maxipok”, which he describes it as an effort to “maximize the probability of an “OK outcome”, where an OK outcome is any outcome that avoids existential catastrophe.” This concept should be taken as a rule of thumb, and not as a principle of absolute validity. Its usefulness is as an aid to prioritization .

Anthropic principle

Another topic that Nick Bostrom has studied extensively is anthropic bias, or observational selection effects. A selection effect is a bias introduced by constraints in the data collection process, from the limitations of some measuring device for example. An observational selection effect is one that arises from the precondition that there is some observer properly positioned to examine the evidence. Bostrom has studied how to reason when it is suspected that evidence is biased by this observation selection effect . Some questions that involve reasoning from conditioned observations are, for example, “is the fact that life evolved on Earth evidence that life is abundant in the universe? “, “why does the universe appear fine-tuned for life?”, or “are we entitled to conclude from our being among the first sixty billion humans ever to have lived that probably no more than several trillion humans will ever come into existence - that is, that human extinction lies in the relatively near future?”

According to Bostrom (2002), anthropic reasoning seeks to detect, diagnose, and cure biases. It is a very rich philosophical field in terms of empirical implications, touching on so many important scientific questions, posing intricate paradoxes, and containing generous quantities of conceptual and methodological confusion that need to be sorted out.

The term “anthropic principle” is less than three decades old, and it was coined by Brandon Carter, a cosmologist, in a series of papers. The term “anthropic” is a misnomer, since reasoning about observation selection effects have no limited relation with a specific species (in this case Homo sapiens), but with observers in general. There is some confusion in the field, with several anthropic principles being formulated and defined in different ways by various authors. In Bostrom (2002), the author writes that “some reject anthropic reasoning out of hand as representing an obsolete and irrational form of anthropocentrism. Some hold that anthropic inferences rest on elementary mistakes in probability calculus. Some maintain that at least some of the anthropic principles are tautological and therefore indisputable. Tautological principles have been dismissed by some as empty and thus of no interest or ability to do explanatory work. Others have insisted that like some results in mathematics, though analytically true, anthropic principles can nonetheless be interesting and illuminating. Others still purport to derive empirical predictions from these same principles and regard them as testable hypotheses.” .

More recently, Bostrom and colleagues introduced the concept of anthropic shadow. It is an observation selection effect that prevents the observation of certain extreme risks that are close in terms of geological and evolutionary time. The anthropic shadow is cumulative with the “normal” selection effects that are applied to any sort of event. Correcting for this type of bias can affect the probability estimates for catastrophic events, and recognizing it might also help avoiding errors in risk analysis .

Superintelligence

The development of artificial intelligence (AI) could advance rapidly, possibly becoming an existential threat to humankind. Bostrom, in his book Superintelligence (2014), compares the development of AI to humans being like small children playing with a bomb. He also considers it “the most important thing to happen… since the rise of the human species”. Indeed, there is no reason why human psychology should be projected onto artificial minds, and assume that they would have the same emotional responses that humans developed during the evolutionary process. Expecting human characteristics from an AI could impede our understanding of what it might be like . This area of study has received some attention, with Elon Musk investing $10 million dollars to fund research about keeping AI friendly .

Bibliography

Books

• Bostrom, N. (2002). Anthropic bias: Observation Selection Effects in Science and Philosophy. New York, NY, Routledge.

• Savulescu, J. and Bostrom, N. (Eds.) (2009). Human Enhancement. Oxford, NY, Oxford University Press.

• Bostrom, N. and Circovic, M. M. (Eds.) (2011). Global Catastrophic Risks. Oxford, NY, Oxford University Press.

• Bostrom, N (2014). Superintelligence. Oxford, NY, Oxford University Press.

Selected articles

• Bostrom, N. (1998). How long before superintelligence? International Journal of Future Studies, 2.

• Bostrom, N. (2002). Existential risks. Journal of Evolution and Technology, 9(1).

• Bostrom, N. (2003). Are we living in a computer simulation? The Philosophical Quarterly, 53(211): 243-155.

• Bostrom, N. (2003). Human genetic enhancements: a transhumanist perspective. The Journal of Value Inquiry, 37(4): 493-506.

• Bostrom, N. (2005). In defense of posthuman dignity. Bioethics, 19(3): 202-214.

• Bostrom, N. (2005). A history of transhumanist thought. Journal of Evolution and Technology, 14(1).

• Bostrom, N. (2005). Transhumanist values. Journal of Philosophical Research, 30: 3-14.

• Bostrom, N. and Ord, T. (2006). The reversal test: eliminating status quo bias in applied ethics. Ethics, 116(4): 656-679.

• Bostrom, N. and Sandberg, A. (2009). Cognitive enhancement: methods, ethics, regulatory challenges. Science and Engineering Ethics, 15(3): 311-341.

• Bostrom, N. (2011). A patch for the simulation argument. Analysis, 71(1): 54-61.

• Bostrom, N. (2012). The superintelligent will: motivation and instrumental rationality in advanced artificial agents, 22(2): 71-85.

• Bostrom, N. (2013). Existential risk prevention as global priority. Global Policy, 4(1): 15-31.