Editor's Blog
Our knowledge of how the human brain works is rapidly increasing and some people will use that knowledge to enhance their minds – to make themselves more intelligent, to feel permanently happy or to choose different aspirations. This will challenge the existing political order in many ways: Hyper-intelligent people will find it easy to dominate their relatively dim fellows. Augmented reality and brain-computer links can provide governments with more than enough data to build an authoritarian state. Smarter terrorists will be able to build weapons of mass destruction such as deadly viruses and once they do, citizens will call for better protection and gladly give their governments more powers to provide it. The most likely outcome seems a phase of bloody chaos followed by an authoritarian regime that shapes the minds of its citizens according to its needs.
Most people will prefer a political system that gives more weight to their own preferences, a system that never forces anybody to have mind enhancement and that allows everybody to have it as far as this is reasonably possible. Normal people who prefer to stay normal could then continue life with dignity, while the majority of those who want to explore new frontiers could rush ahead. However, treatments that provide people with additional mental powers cannot be given to everybody. Some people are mentally disturbed, bear a grudge against the world or are committed to violent political action. Those would have to be barred to prevent disaster.
It seems pretty obvious that people barred from mind enhancement and therefore practically from any higher position in society, will not accept that decision. They will immediately blame their fate on racial, tribal or gender bias, corruption or faulty procedure. In many cases, they will be right, so they will easily find others who agree with them and try to dismantle the whole approval process. When they manage, terrorists and psychopaths will do more damage, scared citizens will give more power to their governments and the route to an authoritarian system is again the most likely one. Bared people will always be disappointed or angry, but the echo they find in the general population could be significantly reduced if the approval process were fully automated and completely open to public scrutiny.
The principle might best be described in the form of a conceptual prototype. An independent testing organisation (not the government) obtains a full genetic scan, medical history and biographical information of a person. Thereafter, the applicant plays a series of computer games in which he has to make decisions while his brain activity is measured by appropriate devices (fNMR, EEG, etc.). All information is integrated in a computer model that predicts his personal decisions in given situations. Appropriate research data are used to extend the model, so that it predicts likely decisions after the planned mind enhancement treatment. The model is then applied to a series of hypothetical situations to see whether the applicant would be likely to use his newfound abilitiesin a harmful way (e.g. help terrorists or take part in a military coup). The results determine the applicant’s “trustworthiness” and only those who get the required minimum score may proceed with the mind enhancement treatment of their choice.
Automation of this kind would make it possible to cope with the millions of applications that have to be handled every time a new enhancement treatment comes onto the market. The focus on harmful behaviour would deliver practical results without fruitless discussions about the ideal mix of personal characteristics. A slight modification of the behaviour prediction model could be used to determine whether somebody involved in trustworthiness testing is likely to abuse his position – and prevent corruption. After a short time, society would be dominated by people who were found to be trustworthy and who will therefore tend to be comfortable with the process. This gives the process a self-sustaining quality.
The promise of long-term stability is also a potential problem. Any regime or ideology that manages to establish a trustworthiness testing process of this kind will be able to hold on to power with no end in sight – even a regime that looks unspeakably horrible from our current point of view. It obviously matters who gets there first.
The most likely outcome seems a stable oligarchy that may come about as follows: Most people ignore these considerations, because they seem too far removed from real life. The technology required for a workable trustworthiness tester matures and some people in the current elite discover it as an ideal tool to grab power. They only apply it to each other to ensure that they can trust their fellows. Once they have a few seriously wealthy and influential members, they can offer new members a safe place in a future aristocracy – safe because trustworthiness testing among the conspirators has been arranged to prevent internal struggles, personality cults, betrayals and all the other things that make power grabs risky – and safe because most of those invited will not be able to resist the temptation and the conspiracy will win; and if not this one, then the next or the one thereafter.
Those who prefer to stay in charge of their own fate will have to find a way to pre-empt this development and use trustworthiness testing technology to establish a free political system that involves all citizens before others can use it to build an oligarchy. To do this, they will need majority support, which requires that most people know the technology well and trust it. There will be no time to do this familiarisation once the technology is ready to use and the conspirators are ready to strike. Word has to get around before, while the technology is still in its infancy and therefore harmless.
This might best be done in an open project to build a trustworthiness tester and to promote awareness regarding the associated opportunities and threats. In the early stages of such a project, it would be irresponsible to use the technology for anything “serious” such as politics. It would be more appropriate to focus on applications that are not serious at all but do attract wide public attention – things that are fun to watch or do such as TV shows based on trustworthiness testing, public competitions or computer games. It might therefore be worth the effort to build a trustworthiness tester just for “fun”.
Roland Schiefer holds an MSc in Biophysics and a PhD in Medicine and has applied himself to various projects in contract research and development. His book “All In The Mind” at amazon.com is also available in Kindle format.
Sign up for the Humanity+ newsletter:
24 Comments
Very thought provoking, but the argument seems to assume that capabilities can be enhanced without affecting motivations. I am not sure that this is the case (one finds a lot more stupider-than-average terrorists and criminals than one finds more-intelligent-than-average ones), but at the very least it is something that needs to be investigated before being assumed.
Excuse my late response, but I did not want to jump onto the stage too early and distort all the valuable feedback.
I do not assume that mind enhancement will leave motivations unchanged. Just the opposite: When people can choose from a variety of changes to their personalities, they will be unlikely to limit themselves to increasing their IQs. Some will want to be happy all the time and once they are, they might care less about ideal partners and salary increases. Some will want more eagerness to strive. Some will want to make themselves care more about the natural environment because they think this is a noble thing to do. The list is endless.
We should allow everybody to choose whatever he likes – as long as this will not harm his fellows (by hurting them or wrecking the society they live in). Maybe, measured trustworthiness is a way to make this possible.
Roland
fMRI, EEG– all almost certainly lack the resolution necessary for this task. You’ll likely need a full “connectome”.
You are basically discussing simulating a person on a computer (and if you are not talking about simulating a person, there’s no way you’ll get the necessary precision from the answers such a system would give). When we have that capability, society will change drastically and the rest of your article will likely not apply. Disregarding that for the moment, there are some ethical issues with simulating a person to gain knowledge like this and then turning off the simulation: when you turn off the simulation, is that equivalent to murder?
Also, how do you know the new smarter person won’t be able to tell when it is being simulated and when it is in the real world, behaving differently in either case?
This is similar to AI-boxing, i.e., running a machine super intelligence of unknown friendliness in a box to see if it will be friendly or not. The short answer is, that’s extremely unsafe. For more, see: http://yudkowsky.net/singularity/aibox
hrm, the blockquote at the beginning of that doesn’t look distinct. Oops, formatting fail.
The measured trustworthiness concept is based on a simulation of likely decision behaviour. This does not require a one-to-one mapping of the mind, which is currently far out of reach and might indeed cause the ethical problems you mention. Please allow me a trivial example. When we want to predict the behaviour of a car suspension as far as it is relevant for driving behaviour, we do not have to model the suspension molecule by molecule. We just need a few equations involving the mass distribution in the car, the spring constants, friction values and so on. When we want to predict the likely behaviour of a person as far as harmful behaviour is concerned, the same applies. A look at fMRI or EEG does not reveal what “really happens”; it just provides spatial and time-series patterns that can be correlated with decision events and I think that would be good enough.
Imagine a society that has just developed a few powerful mind enhancement methods. Keeping them all outlawed means that those who cheat first will rule over their fellows. Spreading them indiscriminately will enable a few psychopaths or terrorists to create havoc. A method that is only 95% effective (i.e. misses 5% potential psychopaths or terrorists) will still reduce harm done by a factor 20 – and may buy enough time to let the enhanced people come up with a better plan.
Roland
Reply to Eliezer Part II
Your AI-boxing example is indeed intriguing. (For the casual reader: It involves an advanced Artificial Intelligence locked up in a box that has to persuade a gate-keeper to let it out). When I imagine myself in the role of a responsible gate keeper, I would dearly like to measure the trustworthiness of the AI – even when the measurement only provides a rough indication, for example: Now it uses a processing unit that is usually active when an AI of that kind inserts an intermediate layer between what it thinks and what it says – i.e. when it tries to lie. Now it is very excited – maybe because of what it says or what it does not say. No, the unit that indicates strong focus on something came on earlier. It tries to make me think that it finds the things it is now talking about very exciting – but that seems to be a ruse. This would give me an edge that might compensate for the fact that the AI is smarter than I am (at lest I hope so). It might then take a super-advanced AI to fool me – but by then I might be smarter as well and have a better trustworthiness tester at hand.
Roland
Very thought provoking and well written.
But i have a quick counter idea: How about the legislative powers pass a law, requiring a kinda “brain spring cleaning” before any enhancements can be preformed. This “brain spring cleaning” could include removal of current, or upcoming, mental illness (psychosis, psychopathy, depression, bi-polar schizophrenia etc.), as well as a mandatory improvement to ones empathy/sympathy. That way potentially dangerous people would become non-violent and sane, making it harder for them to commit any atrocities against mankind.
The ethical implications of a legislative power deciding what to add/remove might be grand, but if done in a transparent way, the average citizen might come to support it. One might make the argument, to the average citizen, that brain enhancements are like a gun and the “brain spring cleaning” makes sure that the gun will only be aimed at problems and not people.
People are likely to differ when it comes to identifying the parts of their minds that should go in the “spring clean” and the parts that they would like to keep. If a government would prescribe a standard solution against the will of the individual citizens, quite a few people might take up arms. That would surely not promote harmony and rational decision making.
Even a group of people that can agree on moral standards might choose badly. There are so many apparently good things that nobody would dare to reject, like more tolerance, more empathy, more openness towards anything and everything – I think the ideal being designed by a well-meaning committee would be unfit for life.
I prefer to keep the messy variety of personalities we currently have and gladly chose a system that deprives me of messing with the mind of others when it deprives others to mess with mine. Measured trustworthiness is meant to do just that.
Roland
I know people might differ, which is why i suggested going for things that are universally good or bad, like mental illness and an increase in sympathy/empathy.
http://www.scn.org/~mentifex/Dushka.html is an AI Mind that thinks and communicates in Russian. She is not yet advanced enough to answer questions about the trustworthiness of human beings. Her http://www.scn.org/~mentifex/RuAiUser.html manual does not even address this topic — but it is worthy of consideration.
Unfortunately, I do not speak Russian, so I cannot ask her.
Roland
Some psychologists believe that once we achieve biological immortality we will completely loose motives for struggle for power or for gathering in big collectives like states. If that is not the case then your tests will not be enough. First of all we don’t have global government.. And besides people don’t usually get violent without reason. Trustworthiness tests might steer governments from solving problems towards more repressive solutions.
My personal experience does not agree with these psychologists. Healthy young people are hardly ever concerned about their own death – and prove that you can keep your motivation when death is no issue. And I remember a friend saying about her marriage: “It’s not worth splitting up anymore” – which shows that the finality of life can also cause inaction.
However, I do agree that trustworthiness testing can be used to keep repressive systems stable – even those too horrible to imagine. It will not be a benevolent technology that will inevitably make the world a better place just by being there. But it will give an almost unbeatable advantage to those who use it first. Once that becomes more widely accepted, the race is on.
Roland
Moral Enhancement if not given to everyone will lead individuals in power simply to legislate special privileges national security spots as to the military DARPA’s enhancement program. Factions will be created unless a standard for moral enhancement is dealt with. If this happens in more than a single country debates on what considerations moral enhancement will take place.
Since thousands of years, people in power have used every trick in the book to stay in power – which includes using the latest technology. But any constitutional monarchy proves that people in power can sometimes see that things are changing and that they can be persuaded to agree to a deal – to give up power step by step as long as the resulting society seems acceptable and the deal leaves them better off than the likely, alternative outcome would. If that lesson applies, one should not gear up to fight the powerful, but rather try to offer them a deal they cannot refuse.
Roland
So… You do realize that all your possible outcomes are the same, right? They’re all just outcomes where whatever whoever holds power considers desirable is sustained, and all else is crushed. You talk about only allowing individuals with high morality to be enhanced, thus leading to an indefenite perpetuation of modern moral standards as defined by the majority. What is “moral” today WILL NOT BE MORAL TOMORROW. If we had had brain enhancement in the 1950′s, no homosexual would ever be allowed enhancement.
What you’re describing is an evolutionary arms race, and it’s nothing new- though the way it’s framed here is certainly unique to the times.
Like all other arms races it may slow, but it will not end and the first to start running will usually stay in the lead.
Go.
China must already be using some kind of “trustworthiness-checker” for their periodic national meetings in which they choose their top leaders. Their leadership is both patriotic and self-serving. They want to hold onto power at any cost, but they also do not want to make a complete train-wreck out of China as the largest country. Perhaps the Chinese should look into a computerized “Trustworthiness-Tester” as described above by Roland Schiefer, who by the way has done a good job of posting follow-ups to the comments.
Now how could “help terrorists or take part in a military coup” be held against you? (Ever heard of Stauffenberg?) Who runs the approval process? Big brother? I would already be barred for a cautioning comment. Not very bright, folks.
And what’s wrong with enhancing everybody’s mind? A brighter mind might grasp the idea that we are all the same and therefor seize to hurt his bros & sis’. If you intend to leave decisional power over this knowledge and science in the hands of the usual suspects* (not very intelligent, driven by greed and anger – e.g. your everyday politician) it is going to be a brutal weapon against mankind and will lead to even more slavery, more war, more famine and more hitlers. For those unfamiliar with reasoning there’s a simple formula: What cannot be given to everybody must not be given to anybody.
My first reply, above, was a bit hasty and devoid of constructive feedback. Let me try again.
This could fall in line with evidence-based politics- that is, a political system based on what we know about human behavior and the behavior of groups of humans, namely psychology and sociology.
An implication of such a trustworthiness detector could be the implementation of citizenship metrics. For example, measurements could be taken of the reward and pleasure pathways of the brain and, like you’ve said, associated with their corresponding behaviors. If a person is and remains sane- that is, they are not motivated by self-destructive or socially destructive behaviors- they are given the benefits of citizenship, among them the opportunity to direct government and legislation.
However, should that person fall outside of certain metrics they would not be allowed the benefits of citizenship and it would be up to them to seek treatment, or if society had an interest in it, they could provide treatment free of charge.
Thoughts?
>Those would have to be barred to prevent disaster.
Who gets to decide who falls in to this category? You? “Society”? (whatever that means)
Under your own definition you fall squarely in to this “committed to violent political action”, if you wish to prevent using force if necessary a category of people from modifying their own minds/bodies, so it follows that you should also be restricted.
I would questiom whether a small group of persons (with augmented intelligences) would be a bad thing for society as a greater whole. Given their higher intelligence would they not deduce that holding their position would become a lot easier if the general public actually [i]wanted[/i] them there. I.e. That by performing actions that benefited John & Jane Doe, said citizens would cease to care about said leaders.
In my workplace experience, so long as employees believe that they are better off under Manager A than B, they will remain loyal to Manager A and will support them. If Manager A abuses their power they find a much less willing employee. Not the best example, but the only I could think of that involves unelected individuals with power over a community.
Im also not suggesting this would be the only outcome, but it is one that bears consideration.
I think the entire situation is entirely to simple and much simpler than it should be and this harms the idea, the process and the final outcome.
If you want to invent more terrorists for even more reasons that the insipid ones that exist today, keep barring people from the game. I find horrifying that people who are “terrorists” or “insurgents” are people who just want to feed, clothe, educate and govern themselves. Please read James Martin’s “The Meaning of the 21st Century” and when you do you realize that to stop terrorism, you do not march off your army to parts unknown to bring the fight there. You remove reasons for people to be terrorists. You make sure people have food, shelter, medicine and education and contribute to society as best as possible.
Denying anyone what becomes, in essence, life-sustaining technology is to condemn them to death. In the end, what does that serve? Who decides the standard by which to administer your ridiculous test and how can anyone possibly sanction the unethical result it produces? For, in the end, the result must be unethical if it deprives someone of their life, of a way to earn a living or of leading the life of their own choice.
Socially “unacceptable” people today were the great hunters and leaders of yesteryear. They were people that, while certainly dangerous, people followed because they knew they were eating that day. Their existence is just another expression of humanity, one which you seem determined to eradicate – and yet, this is a result of evolution of a predatory species. You can try to drum it out with your insipid test all you like but to get rid of it all flies in the face of evolution, a process you clearly do not understand regardless of your advanced degree.
Mind you, this is not a defense of the psychopath or the terrorist. Certainly, I would not want to be caught alone in a room with either. But it is an acknowledgement that these people exist whether we want them to or not and that evolution through natural selection perpetuated situations for these people for reasons that, while they may no longer apply in society, did apply at some point in the past.
Clearly, you need to give this area much more thought.
One obvious assumption you seem to make is that people are people regardless of their situation; hence, “good” people will remain good even after massive enhancement and “bad” people will be bad. What is your basis for this? How many times have we seen a lottery winner “changed by the money”? How many times have we heard from interviews with his friends how this good person became cold, distant and even rude or violent when before he was always of service to his community? And then they continue by blaming the money and not the individual or other variables such as education (both of that individual and of those friends).
To think that people will not change after such an event is even more ridiculous than your test. Reality is much more complex than this. Let me suggest that instead of worrying about control over this process that we arm everyone with this ability regardless of his or her previous activities. The universe is a very complex place and we may very well need every mind operating to their fullest capacity, bringing their unique viewpoint and contribution to bear on the problem of what comes next.
In the end, where we may create a vicious super-criminal, equally clever people will exist to create balance. But to think that this technology will simply materialize and that we must dutifully select out people like the sign at the carnival that insists “You must be this tall…” is exactly, precisely the most backward and wrong answer possible. It may be that your test creates more monsters and leaves those able to stop them hampered as a sub-class of humanity.
Instead, this technology will emerge like everything else, in fits and starts. Early adoption along with a process of education is the single best way to deal with it.
A better, more productive conversation might have been around the discussion about what kind of education may be required leading up to the implant and then continuing education afterwards. Mind you, this is unadulterated education, not indoctrination. Not nationalization or preprogrammed political answers but education on a global basis that leaves out nothing, that takes into consideration the full dynamic of historical events. In the end, such a person will eventually discover these items anyway and I for one would sleep better knowing we had taught them the truth rather than the kind of polished creature that passes for, say, American history.
See, the issue here is not what happens leading up to the implant event. The issue is about what happens next to nation-states, hemispheric alliances and other situations when these people begin to reach critical mass. Such a mass may be more than a value of one because even after this conversion the value will not be a shared knowledge base from which to draw facts but rather the kinds of solutions and directions that suggest themselves to these people. The value here is not passing a certification for the next level by conquering your ID but by understanding that our history is just that – ours – and by moving on to what comes next.