h+ Magazine

Two A.I. Scientists agree that a Technological Singularity is coming… and that the super-intelligent machines that they are building will lead us there. However, they disagree on whether humans will survive the coming revolution.

I’ve argued that posthumans would have to be, in some sense of the term, “autonomous entities” capable of operating outside the scope of the socio-technical network I refer to as the Wide Human (Roden 2013). A being is autonomous if it is self-governing. According to the modern practical philosophy that follows Rousseau and Kant, autonomous beings (paradigmatically human beings) are those that can freely determine the principles by which they act and live.

In 2011, Alexander Kruel (XiXiDu) started a Q&A style interview series on LessWrong asking various experts in artificial intelligence about their perception of AI risks. He convened what was in essence a council of expert advisors to discuss AI development and risk.

It is known that benign looking AI objectives may result in powerful AI drives that may pose a risk to the human society. We examine the alternative scenario of what happens when universal goals that are not human-centric are used for designing AI agents. We follow a design approach that tries to exclude malevolent motivations from AI’s, however, we see that even objectives that seem benevolent at first may pose significant risk to humanity.

A cherished idea of AI risk proponents is that an expected utility maximizer will completely ignore anything which it is not specifically tasked to maximize. One example here is that if you tell a superintelligent expected utility maximizer to prevent human suffering it might simply kill all humans, notwithstanding that it is obviously not what humans want an AI to do and what humans mean by “prevent human suffering”.

Ben Goertzel converses with Hugo de Garis on his transhumanist argument for the reality of a Creator.