Sign In

Remember Me

Wendell Wallach on Machine Morality

Wendell Wallach, a lecturer and consultant at Yale University’s Interdisciplinary Center for Bioethics, has emerged as one of the leading voices on technology and ethics. His 2009 book Moral Machines (co-authored with Colin Allen) provides a solid conceptual framework for understanding the ethical issues related to artificial intelligences and robots, and reviews key perspectives on these issues, always with an attitude of constructive criticism. He also designed the first university course anywhere focused on Machine Ethics, which he has taught several times at Yale.

A few years ago, Wendell invited me to speak in Yale’s technology & ethics seminar series (see the slides from the talk here) – which was a rewarding experience for me, due both to the interesting questions from the audience, and also to the face-to-face dialogues on AI, Singularity, ethics, and consciousness that we shared afterwards. Some of the key points from our discussions are raised in the following interview that I did with Wendell for H+ Magazine:

Ben:

Ray Kurzweil has predicted a technological Singularity around 2045. Max More has asserted that what’s more likely is a progressive, ongoing Surge of improving technologies, without any brief interval of incredibly sudden increase. Which view do you think is more accurate, and why? — and do you think the different really matters (and if so, why)?

Wendell:

I’ve characterized my perspective as that of a “friendly skeptic” – friendly to the can do engineering spirit that animates the development of AI, skeptical that we understand enough about intelligence to create human-like intelligence in the next few decades. We are certainly in the midst of a surge in technological development, and will witness machines that increasingly outstrip human performance in a number of dimensions of intelligence. However, many of our present hypotheses will turn out to be wrong, or the implementation of our better theories will prove extremely difficult. Furthermore, I am not a technological determinist. There are societal, legal, and ethical challenges that will arise to thwart easy progress in developing technologies that are perceived to threaten human rights, the semblance of human equality, and the centrality of humanity in determining its own destiny. Periodic crises in which technology is complicit will moderate the more optimistic belief that there is a technological fix for every challenge.

Ben:

You’ve written a lot about the relation between morality and AI, including a whole book, “Moral Machines.” To start to probe into that topic, I’ll first ask you: How would you define morality, broadly speaking? What about ethics?

Wendell:

Morality is the sensitivity to the needs and concerns of others, and the willingness to often place more importance on those concerns than upon self-interest. Ethics and morality are often used interchangeably, but ethics can also refer to theories about how one determines what is right, good, or just. My vision of a moral machine is a computational system that is sensitive to the moral considerations that impinge upon a challenge, and which factors those considerations into its choices and actions.

Ben:

You’ve written and spoken about the difference between bottom-up and top-down approaches to ethics. Could you briefly elaborate on this, and explain what it means in the context of AI software, including near-term narrow-AI software as well as possible future AI software with high degrees of general intelligence. What do these concepts tell us about the best ways to make moral machines?

Wendell:

The inability of engineers to accurately predict how increasingly autonomous (ro)bots (embodied robots and computer bots within networks) will act when confronted with new challenges and new inputs is necessitating the development of computers that make explicit moral decisions in order to minimize harmful consequences. Initially these computers will evaluate options within very limited contexts.

Top-down refers to an approach where a moral theory for evaluating options is implemented in the system. For example, the Ten Commandments, utilitarianism, or even Asimov’s laws for robots might be implemented as principles used by the (ro)bot to evaluate which course of action is most acceptable. A strength of top-down approaches is that ethical goals are defined broadly to cover countless situations. However, if the goals are defined too broadly or abstractly their application to specific cases is debatable. Also static definitions lead to situational inflexibility.

Bottom-up approaches are inspired by moral development and learning as well as evolutionary psychology. The basic idea is that a system might either evolve moral acumen or go through an educational process where it learns to reason about moral considerations. The strength of bottom-up AI approaches is their ability to dynamically integrate input from many discrete subsystems. The weakness lies in the difficulty in defining a goal for the system. If there are many discrete components in the system, it is also a challenge to get them to function together.

Eventually, we will need artificial moral agents which maintain the dynamic and flexible morality of bottom-up systems that accommodate diverse inputs, while also subjecting the choices and actions to the evaluation of top-down principles that represent ideals we strive to meet. In addition to the ability to reason about moral challenges, moral machines may also require emotions, social skills, a theory of mind, consciousness, empathy, and be embodied in the world with other agents. These supra-rational capabilities will facilitate responding appropriately to challenges within certain domains. Future AI systems that integrate top-down and bottom-up approaches together with supra-rational capabilities will only be possible if we perfect strategies for artificial general intelligence.

Ben:

At what point do you think we have a moral responsibility to the AIs we create? How can we tell when an AI has the properties that mean we have a moral imperative to treat it like a conscious feeling agent rather than a tool?

Wendell:

An agent must have sentience and feel pleasure and pain for us to have obligations to it. Given society’s lack of concern for great apes and other creatures with consciousness and emotions, the bar for being morally responsible to (ro)bots is likely to be set very high.

How one determines that a machine truly has consciousness or somatic emotions will be the tricky part. We are becoming more knowledgeable about the cognitive sensibilities of non-human animals. Future scientists should be able to develop a Turing-like test for sentience for (ro)bots. We should be sensitive about falling into a “slave” mentality in relationship to future intelligent systems. While claims of feeling pain, or demands for freedom, can easily be programmed into a system, it will be important to take those demands and claims seriously if they are accompanied by the capabilities and sensitivities that one might expect from a sentient being.

Ben:

Humans are not really all that moral, when you come down to it. We often can be downright cruel to each other. So I wonder if the best way to make a highly moral AGI system might be not to emulate human intelligence too closely, but rather to make a system with morality more thoroughly at the core. Or do you think this is a flawed notion, and human morality is simply part and parcel of being human — so that to really understand and act in accordance with human morality, an AI would have to essentially be a human or something extremely similar?

Wendell:

While human behavior can be less than admirable, humans do have the capability to be sensitive to a wide array of moral considerations in responding to complicated situations. Our evolutionary and cultural ancestry has bequeathed us with an adaptive toolbox of propensities that compensate for each other. These compensating propensities may fail us at times. But I personally lament the pathologizing of human nature and the aggrandizing of what future (ro)bots might achieve. Our flaws and our strengths are one and the same. We humans are indeed remarkable creatures.

That said, a (ro)bot does not need to emulate a human to be a moral machine. But it does need to demonstrate sensitive to moral considerations if it is going to interact satisfactorily with humans in social contexts or make decisions that affect humanity’s well being. Humans don’t have sonar, but sonar together with light sensors may lead to excellent navigation skills. The speed of computer processing could contribute to moral reasoning abilities that exceeds the bounded morality of human decision making.

The test is whether the moral sensitive of the (ro)bot is satisfactory. (Ro)bots will need to pass some form of a Moral Turing Test, and demonstrate a willingness to compromise, or even suspend, their goals for the good of others.

One question is whether a (ro)bot without somatic emotions, without conscious awareness, or without being embodied in the world with other agents can actually demonstrate such sensitivity. There may be reasons, which are not fully understood today, why we humans have evolved into the kinds of creatures we are. To function successfully in social contexts (ro)bots may need to emulate many more human-like attributes than we presently appreciate.

Ben:

What are your thoughts about consciousness? What is it? Let’s say we build an intelligent computer program that is as smart as a human, or smarter. Would it necessarily be conscious? Could it possibly be conscious? Would its degree and/or type of consciousness depend on its internal structures and dynamics, as well as its behaviors?

Wendell:

There is still a touch of the mystic in my take on consciousness. I have been meditating for 43 years, and I perceive consciousness as having attributes that are ignored in some of the existing theories for building conscious machines. While I dismiss supernatural theories of consciousness and applaud the development of a science of consciousness, that science is still rather young. The human mind/body is more entangled in our world than models of the self-contained machine would suggest. Consciousness is an expression of relationship. In the attempt to capture some of that relational dynamic, philosophers have created concepts such as embodied cognition, intersubjectivity, and enkinaesthetia. There may even be aspects of consciousness that are peculiar to being carbon-based organic creatures.

We already have computers that are smarter than humans in some respects (e.g., mathematics and data-mining), but are certainly not conscious. Future (ro)bots that are smarter than humans may demonstrate functional abilities associated with consciousness. After all, even an amoeba is aware of its environment in a minimal way. But other higher-order capabilities such as being self-aware, feeling empathy, or experiencing transcendent states of mind depend upon being more fully conscious.

I suspect that without somatic emotions or without conscious awareness (ro)bots will fail to interact satisfactorily with humans in complex situations. In other words, without emotional and moral intelligence they will be dumber in some respects. However, if certain abilities can be said to require consciousness, than having the abilities is a demonstration that the agent has a form of consciousness. The degree and/or type of consciousness would depend on its internal structure and dynamics, not merely upon the (ro)bots demonstrating behavior equivalent to that of a human.

These reservations might leave some readers of this interview with the impression that I am dismissive of research on machine conscious. But that is not true. I have been working together with Stan Franklin on his LIDA model for artificial general intelligence. LIDA is a computation model that attempts to capture Bernard Baars’ global workspace theory of consciousness. Together with Franklin and Colin Allen, I have researched how LIDA, or a similar AGI, might make moral decisions and what role consciousness plays in the making of those decisions. That research is published in academic journals and is summarized in chapter 11 of Moral Machines: Teaching Robots Right From Wrong. The latest installment of this line of research will form the lead article for the coming issue of the International Journal of Machine Consciousness.

Ben:

Some theorists like to distinguish two types of consciousness – on the one hand, the “raw consciousness” that panpsychists see everywhere; and on the other hand, the “reflective, deliberative consciousness” that humans have a lot of, mice may have a little of, and rocks seem to have basically none of. What’s your view of this distinction? Meaningful and useful, or not?

Wendell:

The distinction is very useful. My question is how functional will an agent with reflective, deliberative consciousness be if it does not also explicitly tap into “raw consciousness?” If the agent can do everything, it perhaps demonstrates that the notion of a “raw consciousness” is false or an illusion. However, we won’t know without actually building the systems.

Ben:

It seems to me that the existence of such an agent (i.e. an agent with the same functionality as a conscious human, but no apparent role for special “raw consciousness” separate from its mechanisms) is ALSO consistent with the hypothesis of panpsychism — that “raw consciousness” is everywhere in everything (including mechanisms), and just manifested differently in different things…. Agree, or not? Or do you think the discussion becomes a kind of pointless language-game, by this point? ..

Wendell:

Panpsychism comes in many forms. Presuming that consciousness is all pervasive or is an attribute of the fundamental stuff of the universe (a fundamental attribute of matter?) doesn’t necessarily help us in discerning the quality of consciousness in another entity. While certain functional attributes might indicate the likelihood of conscious qualities, the functional attributes do not prove their existence. To my mind consciousness implies a capacity for intersubjectivity not only in the relationship between entities but also in the integration of self. There is no evidence that a rock taps into this. We humans have this capacity, though at times (perhaps much of the time) we also act like unconscious machines. But illuminating why we are conscious or what being conscious tells us about our universe has bedeviled humanity for a few thousand years, and I am skeptical that we will crack this nut in the near future. If an AI was able to perform tasks that were only possible for an entity with the capacity for intersubjectivity, perhaps this would prove it is conscious, or perhaps it would prove that intersubjectivity is an illusion. How would we know the difference?

Ben:

Do you think it’s possible to make a scientific test for whether a given system has “humanlike reflective consciousness” or not (say, an AI program, or for that matter an alien being)? How about an unscientific test — do you think you could somehow tell qualitatively if you were in the presence of a being with humanlike reflective consciousness, versus a “mere machine” without this kind of consciousness?

Wendell:

There are already some intriguing experiments that attempt to establish whether great apes or dolphins have metacognition. Those species do not have the ability to communicate verbally with us as future (ro)bots will. So yes to both questions. As to the accuracy of my own subjective judgment as a tester, that is another matter.

Ben:

I’d love to hear a little more about how you think people could tell qualitatively if they were in the presence of a machine with consciousness versus one without it. Do you think some people would be better at this than others? Do you think certain states of consciousness would help people make this identification better than others?

I’m thinking of Buber’s distinction between I-It and I-You interactions – one could argue that the feeling of an “I-You” interaction can’t be there with a non-conscious entity. But yet it gets subtle, because a young child can definitely have an I-You relationship with their favorite doll. Then you have to argue that the young child is not in the right sort of state of consciousness to accurately discern consciousness from its absence, and one needs to introduce some ontology or theory of states of consciousness….

Wendell:

There are physiological and neurological correlates to conscious states that in and of themselves do not tell us that a person is conscious, but are nevertheless fairly good indications of certain kinds of conscious states. Furthermore, expertise can be developed to discern these correlates. For examples, Paul Ekman and his colleagues have had some success in training people to read microexpressions on faces and thus deduce whether a person is truly happy, repressing emotions, or masking subterfuge and deception. Consciousness is similar. Engagement in the form of eye to eye contact and other perceived and felt cues tells us when a person is interacting with us and when they are distracted. Furthermore, I believe that conscious states are represented by certain energy patterns, that is, conscious attention directs energy as do other states of mind. In addition, there is recent evidence using fMRI for discerning positive and negative responses to questions from people in certain vegetative states. All this suggests that we are just beginning to develop methods for discerning conscious states in other humans. However, translating that understanding to entities built out of plastic and steel, and with entirely different morphologies, will be another matter. Nevertheless, I do believe that some humans are or can learn to be attuned to the conscious states of others, and they will also have insight into when an artificial entity is truly self-reflective or merely simulating a conscious state.

The I-Thou discernment raises a secondary question about mystical sensibilities that may be more common for some people than for others. Perhaps most people can distinguish an I-It from an I-Thou relationship, if only through the quality of response from the other. Most telling is when there is a quality of mutual self-recognition. For some people such a dynamic turns into a falling away of the distinction between self and other. I personally feel that such states are accompanied by both physiological and energetic correlates. That said, perhaps an artificial entity might also be sensitized to such correlates and respond appropriately. However, to date, we do not have sensors to quantitatively prove that these supposedly more mystical or spiritual states exists, so how would we build such sensitivities into an AGI? Even if we could build such sensors into an artificial entity, appropriate responses would not be the same as a full dynamic engagement. An individual sensitive to such states can usually tell when the other is fully engaged or only trying to be consciously engaged. Full engagement is like a subtle dance whose nuances from moment to moment would be extremely difficult, if not impossible, to simulate.

Ben:

As this interesting discussion highlights, there are a lot of unresolved issues around AI, intelligence, consciousness, morality and so forth. What are your thoughts about the morality of going ahead and creating very advanced AGI systems (say, Singularity-capable systems) before these various conceptual issues are resolved? There’s a lot of risk here; but also a lot of promise because these advanced AGI systems could help eliminate a lot of human suffering…

Wendell:

I am less concerned with building AGI systems than with attributing capabilities to these machines that they do not have. Worse yet would be designing a machine that was programmed to think that it is a superior being. We have enough problems with human psychopaths without also having to deal with machine psychopaths. I would not have wanted to give up the past 50 odd years of research on computers and genetics based on 1950’s fears of robot take-overs and giant mutant locust. Given that I perceive the development of AI as somewhat slower than you or Ray Kurzweil, and that I see nothing as being inevitable, at this juncture I would say, “let the research progress.” Kill switches and methods for interrupting the supply chain of potentially pathological systems will suffice for the next decades. Intelligent machines cause me less concern than the question of whether we humans have the intelligence to navigate the future and to monitor and manage complex technologies.

Ben:

Yes, I understand. So overall, what do you think are the greatest areas of concern in regard to intelligent machines and their future development?

Wendell:

There is a little too much attention being given to speculative future possibilities and not enough attention being given to nearer-term ethical, societal, and policy challenges. How we deal with the nearer-term challenges will help determine whether we have put in place foundations for maximizing the promise of AI and minimizing the risks.