AGI and the Emerging Peer-to-Peer Economy: Ben Goertzel Interviews AI Researcher Mohamad Tarifi

I first encountered Mohamad Tarifi on an AI email list, where he was discussing his deep and fascinating work on hierarchical temporal memory architectures.  Using a combination of rigorous mathematics and computational experiments, he treats these constructs very broadly as an approach to both vision processing and artificial general intelligence.  This is an area I’ve been delving into a fair bit lately due to my work hybridizing Itamar Arel’s DeSTIN vision system with the  OpenCog proto-AGI architecture, and I found Mohamad’s work helpful in understanding this area of AI better.

But through our online conversations, it quickly became apparent that Mohamad’s depth of thinking about technology and the future extends far beyond technical AI topics – for example he has developed a rich knowledge and insight regarding some themes raised in my recent interview with Linas Vepstas, e.g. the relation between open source software, AI and the rapidly transforming economy.   And when Mohamad and I decided to do an interview, we chose to focus on these themes – what is happening to the world economy now, what’s going to happen in the next few decades, and how do AGI and open source fit into the picture?

Ben: Before we get started with the meat of the interview, could you fill me (and the readership) in on some of the details of your background?

Mohamad: Sure.  I received a bachelor of engineering in computer and communications engineering, with minors in mathematics and physics, from the American University of Beirut, and spent 2 semesters in exchange at the University of Waterloo. After that, I did my masters on computer science at the University of Florida (UF) and helped start a research group on quantum computing with researchers from the UF and Caltech. During graduate school, I worked full-time for over 3 years doing analysis, software development, and management for several companies in the financial, video games and internet industries. I technically lead, mentored, and consulted for several young startups, industry, and research projects. Through my industry and graduate experience, I have been in involved in projects spanning many diverse areas such as bioinformatics, psychology, quantum computing, nanotechnology, economics, computational neuroscience, biomedical devices, finance, computer graphics, marketing, video game development, machine learning, complexity, and various mathematical disciplines such as combinatorics, linear algebra, statistics, and theoretical CS. I’m currently focusing full-time on the final stages of my PhD work, with the thesis titled  “Foundations Towards a Computational Theory of Intelligence”.

Ben:
Wow, certainly a rich and diverse set of interests and experiences!   But it seems that, at least in certain sectors of society, one hears that sort of story more and more.  As the economy and the technology world get more complex, conventionally single-focused careers become decreasingly common.

To get the ball rolling, why don’t you tell me a bit about your thinking on open source and the future of the economy … and how you see AI fitting into the picture going forward?

Mohamad: As we all know, our society is being transformed by technology. Traditional economic forces push towards optimizing labor efficiency. The automation of simple labor by machines was one such leap in efficiency that started the industrial age. Further push towards labor efficiency dictated the next technological leap to the information age. The next step in the evolution of technology is Artificial General Intelligence (AGI). The realization of the significant impact of Artificial General Intelligence (AGI) on the future of humanity motivated me to investigate the theoretical foundations for AGI as my PhD topic.

Ben:
So, just to be sure it’s clear to everyone – when you say AGI, you mean the whole shebang, right?  Human-level AGI, and then eventually general intelligence massively greater than the human level?

Mohamad:
If by human level of intelligence, you mean current human level of intelligence, then I certainly think that AGI will easily surpass us. The more interesting question is what parts of our substrate is limiting us and how we will augment ourselves with AGI.

Ben:
OK, great.  So let’s probe into the potential transformative impact of AGI a little.  You isolate AGI as the next big step in human and technological evolution, and I tend to agree with you.  But others might point to other technologies as critical, e.g. human brain enhancement or strong nanotech.  Why do you isolate AGI in particular? Do you think human-level AGI is going to come before radical advancements in these other areas?  Or do you think these other areas on their own wouldn’t have such dramatic transformative potential, without AGI to accelerate them?

Mohamad:
I agree with both lines of reasoning you offered. Those technologies will be synergetic, and AGI seems to be the closest feasible target.

Ben:

I suppose most of our readers won’t be familiar with your specific take on AI research … so maybe it would be worthwhile for you to summarize briefly the key ideas of your investigations…. 

Mohamad:
With the guidance of my advisers Dr Meera Sitharam and Dr Jeffery Ho, I’m pursuing a mathematically rigorous integrated theory intelligence from the perspective of computer science that builds upon ideas from neuroscience, machine learning, and many areas of mathematics and theoretical computer science such as approximation theory and computational learning theory. We put a preprint of the very first steps towards our integrated approach online at arxiv.org. The paper introduces a fundamental circuit element for bottom-up processing that generalizes and formalizes existing hierarchical/deep models in the literature. Of course, the model will be extended and formally analyzed, along with new algorithms and experiments, in future publications.

Ben:
Great, thanks.  Of course that interests me a lot, and maybe we’ll do another interview digging into that sometime.  But for now let’s focus on the economic side of things.  What do you think is happening to the world economy, and how do you see AGI fitting into that?

Mohamad:
The limitations of the assumptions of traditional economic theory are increasingly manifest in our society through economic inequalities, over-competitiveness, selfishness, short term thinking, and instability. Traditional currencies suffer from inherent structural instabilities, with cycles of inflation followed by crises, a pattern of samsara. To cope with the divergence between theory and reality, economists are attempting to isolate and formalize economically significant human behavioral patterns into their models. For instance, Ernst Fehr’s research incorporates ideas from neuroscience to model agent behavior in various economic games. What is needed, however, is an alternative infrastructure.

Ben:
A “pattern of samsara” – that’s a choice phrase!

Mohamad:
As you know It is as a metaphor for cycles of death and rebirth. I can go on about how this relates to the dynamics of Jungian archetypes in the collective subconscious of society, but this interview will probably never end. Plus these types of metaphors are best left a bit ambiguous since that generalizes their scope of applicability by allowing multiple interpretations. This is why for instance the ancient Vedic scripts are intentionally cryptic. You see how this discussion can quickly become too interesting to end? So I’ll stop here.

Ben:
Heh – but I find that, unlike you, I can’t resist the urge to explicate the obvious at this point! Presumably you mean to imply that AGI has the potential get us beyond this sort of cycle into a realm dominated by the pattern of nirvana? 😉

Mohamad:
Ahahaha, yes!

Ben:
Well, it’s tempting to riff more about AGI and nirvana, but I suppose I’ll take your cue and steer back toward our agreed thematic of the future of economy!  So could you say a little more about the practical specifics of his work.

Mohamad:
This is a link to his research summary and some of his articles. According to his webpage, his research “focuses on the proximate pattern and the evolutionary origins of human altruism, and the interplay between social preferences, social norm and strategic interactions”.

For instance, he analyzes the impact of inequity aversion in bilateral ultimatum games. In that setting, a proposer has to split a predetermined amount of money with a responder. If the responder accepts the percentage share offered by the proposer, both parties retain their share, otherwise both loose. It turns out that human responders will generally not accept the proposer’s offer if it deviates far from equality, and correspondingly proposers generally offer reasonable cuts. The traditional utility function of material self-interest does not explain this behavior. By adding the assumption of inequity aversion to the utility function, he shows that the Nash equilibrium can be shifted to account for the observed human trials.

Ben:
That’s certainly interesting, I look forward to digging into it further.

Actually, I find the very concept of money gets a bit slippery these days, what with the complexity of the international banking system, and the emergence of attention and reputation as often more economically important than material goods, and so forth.

Mohamad:
Yes, money is changing, and will likely change a lot more as AGI advances.  Money is traditionally perceived as a medium for the exchange of human effort.   But with the increased automation of labor, as we approach Artificial General Intelligence (AGI), the cost of effort is decreasing for the majority of the population. As the automation process continues it’s exponential increase, the rationale behind the concept of money is breaking down. Unless we do something about it, and I’m confident that we will, instability concerns will become even more pressing as we move towards AGI. With the breakdown of our traditional monetary system, we need a major shift in our collective consciousness as a species, to redefine the concept and meaning of work, and center it around human values.

Ben:
Hmmm … how would you specifically tie automation and its impact on the nature of money, to the recent financial crisis at the end of 2008?

Mohamad:
Automation creates a more efficient and connected system. Results in complexity theory show that in network follow systems (such as ecosystems, electric circuits, or monetary transactions), there is a single measure based on connectivity that trades off efficiency with sustainability/stability. There is beautiful analogy with a concept in statistics/machine learning called the Bias-Variance trade-off: if we build a statistical model with high complexity, we are at risk of over-fitting the training data. This means that we can explain the current training data ever more precisely (efficiency), at the price of incorrectly generalizing to future test data (stability).

Ben:
I guess this gets a bit technical for most of our readers, but still I’m curious: What’s the measure of complexity you refer to?

Mohamad:
An overview paper is “Quantifying Complexity: Resilience, efficiency, and the return of information theory”.

Ben:
I’m reminded of how the great social philosopher Hannah Arendt distinguished work (purposeful activity which need not be tied to material compensation, and may be done for its intrinsic or social reward) from labor (effort expended with the primary goal of gaining resources).  How does her distinction resonate with your thinking?

My own thought is that AGI and allied technologies may eventually eliminate labor in her sense, but work in her sense will remain — and  may shift largely toward “artistic” work, since practical work will be done by robots, and science, engineering and math will lose part of their appeal when AGI scientists can do them better (they  may still be fun to do, but will lose the  practicality that is now part of their appeal).

Mohamad:
That resonates well with my thinking. Personally, I enjoy science and engineering. In my little spare time, I read current research papers and work on conceptual problems in theoretical physics (cosmological models, quantum field theories, quantum gravity, etc) and many areas of mathematics (pure and applied to the various sciences). I also enjoy solving problems, programming, and hacking art such as building video games, experimenting with collaborative narrative, and real-time art. With a bit more free time on my hands, I would continue pursuing these intellectual ventures because they appeal to me aesthetically.

I think in a way this has already happened. Most people in the western world do not need to exert much physical labor. We no longer need to walk miles to a clean water source or hunt. Yet, we enjoy regular exercise, working out at the gym, and running marathons!

Ben:
So money is changing meaning, and at the same time our relationship to money is changing, as we construct and enter into more complex networks of value.

Mohamad:
This transition is occurring through P2P network economies afforded by the internet infrastructure, with the emergence of alternative currency systems, such as Bernard Lietaer’s research showing how complimentary currencies can address instability. By accounting and scaling previously informal economies, these currency systems promote a more cooperative and ethical society that tailors to our humanity. Many of these ideas have appeared in different forms throughout human history. The difference right now is that we can implement and scale them with computers and the internet. 

Recently one of my friends wrote a really nice paper on P2P economies along these lines, along with proofs of liquidity.

Ben:
Yes, I see.  This network of ideas is rapidly advancing and we’re understanding more all the time, both practically and theoretically.

Could you say a little about how you see the mid-term future of alternative currency systems?  Do you really think they’ll become a practical alternative for a large variety of economic  transactions soon?  What segment of the economy do you think they’ll seriously invade first?

Mohamad:
Alternative currency systems are already being used on small and medium scales in niche markets, local communities and businesses (please see bellow, and Lietaer’s site for some references)

What is needed is to raise global awareness in order to mainstream them and better integrate them to our legal and economic infrastructure. For instance, in some cases, voting to allow taxes to be collected in alternative currencies.

By adding a new class of abundant connections to the economic network flow, specifically those derived from P2P and social interactions, these complimentary currencies increase the stability of the entire system. For example, the relative economic stability of Switzerland can be partially attributed to WIR independent complimentary currency system employed by medium and small sized businesses. The volume of transactions in the WIR increases in periods of global periods of economic stress, providing the liquidity required to keep the businesses running.

Ben:
That’s an intriguing claim.  How exactly would these alternative currencies promote more ethical interactions?

Mohamad:
I prefer to keep value judgments on the moral issues to a minimum, in order to avoid people’s predetermined views on morality from interfering with the natural process of assimilating this research. On the other hand, the research results in themselves shed clear light into morally significant conundrums.

For instance, alternative P2P currencies promote long term thinking through demurrage. Positive interest rate amounts to preferring short term gains since returns on long term projects are exponentially discounted to present value. Demurrage currencies have the opposite effect, it’s much more profitable to spend them on fruitful long term projects.
The Fureai Kippu is a complimentary currency employed in Japan to account for hours of service to elderly people. These credits accumulate and the user may then later use them for oneself or transfer them to someone else. This nurtures the community ethic that a trained professional can hardly match. Surveys show that elderly people consistently prefer services provided by people paid in Fureai Kippu over professionals paid in Yen.
Ben:
You’ve also talked before about automated barter systems as part of the mix in a future economy…
Mohamad:
Yes.  For instance, if we choose to follow a mixed economical system: scarcity for resources which are continuously distributed in an egalitarian way, and abundance for everything else, then Barter systems for exchanging resources can be algorithmically automated.
Ben:
Hmmm, but what are the advantages of barter over currency-based systems?  I don’t really get that.  Barter just seems intrinsically inefficient compared to money.  It may require multiple trades to do the same thing as can be done with a single transaction using money.  Granted, using computers and the Net, multiple trades may be done rapidly.  But, what’s the point of using barter instead of some kind of money?
Mohamad:

This was just one example of a candidate model for a P2P economy. The government can continually distribute rights to newly discovered scarce natural resources equally to it’s people. These resources can then be traded among the community with a Barter system. This does not mean that we trade the actual resources themselves, but the rights to them, in a P2P manner. Algorithms can then be implemented to find the most efficient series of exchanges than can maximize a collective measure of utility.

The P2P Foundation explores several of interesting alternative models that either compliments or replaces traditional social, economic, and political infrastructure. These include Ripple pay for P2P money, P2P governance, P2P manufacturing, and so on. While these models are not yet fully mainstream, they are already being implemented on large scales. The open source community is a great example of P2P production that significantly affects our daily lives.

Ben:
Ah yes, I see. And I suppose open governance and open source government tie into all this as well.

Mohamad:
Yes indeed! But I’ve already talked too much today, so perhaps we can leave this for another discussion some time in the future?

Ben:
That’s fine. But there’s one more thing I have to bring up before I let you go.

All this discussion about the future of the economy seems to assume a fairly optimistic future scenario where Hugo de Garis is wrong and the artilects don’t simply squash us all and co-opt our molecules for their own purposes! What is your reaction to that sort of darker, more violent future vision?

Mohamad:
I think Hugo de Garis’s thinking on the topic is a bit immature. While I agree there is a threat of unfriendly AGI, I think we will respond to this threat with corresponding security precautions. These concerns affect the lives of real humans, and no less than the fate of our species. The gravity of these issues will be addressed with appropriate scrutiny and depth, rather than merely wild fantasies.

I personally like the word “human” better than “transhuman”. The technological revolution is interesting precisely because it will allow us to tap into our vast potential and fully explore our humanity.

Ben:
Well, I agree that Hugo’s proclamations on these topics can be a bit oversimplistic sometimes (though knowing him personally, I’m aware his thinking is subtler than his proclamations sometimes make it seem). However, nevertheless, there seems some validity to the fear that an AGI with general intelligence 10x or 100x that of a human [and yeah, I know there’s no accepted practical measure of general intelligence, but I’m speaking qualitatively here], might find little use for humans, and treat us in a manner similarly to how we treat, say, gnats.

Mohamad:
I do agree unfriendly AGI is a concern, especially if the AGI is designed to be actively hostile.

There will be various protection mechanisms designed along side with the system. For instance, we can directly monitor the progress and thinking patterns of an AGI in real time, or preemptively by performing simulations in controlled environments. In my work, I favor embedding AGI in a virtual reality world. This has many advantages including better control, duplicability, and better security.

The core premise of my view here is a bit more subtle, though, than previous distinctions made by AGI enthusiasts. To begin with, I think that reinforcement learning is not a good model for action. Action is better thought of as Active Inference. With active inference, an AGI is motivated do nothing besides minimize a bound on the entropy of its sensorium. That is, without a “body”, goal oriented behavior quickly breaks down. The key to sustainable behavior is to infuse some of the “soul” of humanity with AGI. I’m speaking in metaphorical terms, of course. There are many interesting concepts that we can explore here, but I think the details of this argument deserves an independent treatment, perhaps in a future interview.

Ben:
Hmmm…. I don’t see any guarantee that massively superhuman AGIs would treat
us badly or dismissively, but I don’t see how one can be confident of “security measures” against such a possibility. Unless you mean security measures to stop people or human-level AIs from developing to a significantly super-human level of intelligence — some kind of panopticon-powered nanny mechanism…

Mohamad:
Ahahaha, I think people hopefully know by now that such protectionism does not work. It is simply too unstable a scenario.

Ben:
I’m not so sure of that myself – but let’s move on. I want to go back to your statement that you like the word “human” better than “transhuman”. What exactly do you mean by that?

I mean, AGI technology may allow us to create “virtual humans”, but it may also allow us to create powerful intelligences that are quite different from humans — don’t you think? And if these are also dramatically more generally intelligent than humans, perhaps they would deserve the label “transhuman”. And if humans somehow hybridize their brains with such non-human AGIs, doesn’t this make them
“transhuman” rather than merely “humans exploring their humanity”? At what point is a cyborg no longer a human and something else instead?

Mohamad:
I agree with you on all the points above.

I think the sort of advanced beyond the current human level that you describe is possible, and in many different ways. Simple arguments can show that this can be done, trivially, as in one example: just build a human level AGI but with the capacity to replicate itself, you can then do many tasks in parallel, that the attention bandwidth of a single human can not do. As someone who has managed working up to 70 hour weeks in the industry, while attending graduate school, and keeping a relatively healthy and social lifestyle, I know how much easier things would be if I could simply fork copies of myself!

I think we’re just comfortable with different words to describe the same process. To me, “Transhuman” feels detached from human concerns. I think of the future changes as a self-actualization of our human destiny. This process is continuous, I can imagine if the two of us were to have this conversation back in prehistoric times, you might have preferred to call humanity now as “Transhuman”. This touches on the ancient and ever-so-interesting debate between Monism, Dualism, and Dualistic-Monism outlooks in spiritual philosophies but this is another topic that will stretch this interview.

Ben:
Yes, we’ve covered a lot of ground already, and I thank you for taking the time to share your views with me and the readers of H+ Magazine. I look forward to following your work and your thinking as they develop!

4 Responses

  1. filou says:

    About mind-files, by the time this technology will work, we’ll have become bio-survivors ? ..

  2. CygnusX1 says:

    Thanks for this “mind-blowing” interview! The economic complexities left my mind at 1st base, but it’s exhilarating to know there is this much serious work engaged in transforming global economics – awesome!

    I myself, have faith and trust that Supercomputer nodes can and will manage future global trade and economics to provide for a more egalitarian future and provision of basic needs for all – which leaves us with the question regarding human “Free” time and idle minds and their rewards?

    We really need to crowd source and accumulate more data trending for these economic models, and by way of guiding the global collective online consciousness towards these ideas for economic change – and I think most everyone would embrace these new ideas and change?

    Time = Money, now there’s a clue? Yet “idle” time is a useless, unproductive and an un-creative commodity!

    Here’s looking forward to a future of extended Google++ interactive learning and participation, (and trending)!

    _/|\_

  3. Nick Land says:

    One minor quibble with this fascinating interview concerns the comments on Hugo de Garis. The Artilect War scenario is focused upon ‘Terran’ EXPECTATIONS of AGI, so to counter that the coming artilects won’t in fact “simply squash us all and co-opt our molecules for their own purposes” is to entirely miss the point.

    Will the prospect of AGI scare enough people to prompt a militant species-conservative counter-politics? — that’s the question.

Leave a Reply