Eureqa! Signs of the Singularity?

· March 25, 2011

What will cause the Singularity?

The most common theory revolves around Artificial General Intelligence. But while Strong AI could well play a part in causing a Singularity, it should not be thought of as the only route to an intelligence explosion. Broadly speaking, a Singularity can occur when there is a closed loop between technological improvement and an increase in intelligence. For example, consider a group of thinkers smart enough to dream up and create ‘Technology X’.   Technology X is, in some way, better than natural intelligence. This ’greater-than-human’ intelligence enables, among other things, improvements to technology X, which lead to further increases over natural intelligence. And so it goes on.

Notice that it does not matter who or what is becoming more intelligent. In ‘Signs of the Singularity’, Vernor Vinge outlined several possible developmental paths, which included the classic AI scenario, but also the ‘Biomedical’ scenario: “We directly increase our intelligence by improving the neurological operation of our brains”. Recall the closed-loop: improved neurological operations increases intelligence, which leads to further improvements on the neurological operations of the brain.

In the AI and biomedical scenarios, an individual—human or computer—becomes increasingly intelligent. But developmental paths need not be so simple. Take, for instance, Vinge’s ‘Internet’ scenario: “Humanity, its networks, computers, and databases become sufficiently effective to be considered a superhuman being”. In and of itself, each part may not be superhumanly intelligent. Considered apart, the networks, computers and databases may not be intelligent at all. But superhuman intelligence might exist as an epiphenomenon—a characteristic of the system as a whole.

One can imagine a scenario whereby networked computers and databases enable global teams of researchers to form collaborative groups to tackle complementary problems and derive effective solutions. These solutions could be discovered by machine-learning tools, analyzing data from many seemingly unrelated studies, uncovering patterns too complex for humans to discover alone, but which point them in the right direction for future work. These collaborative groups’ insights could bring improvements to their networks, which would then be even more effective at organizing teams of specialists into collaborative groups. This would lead to further improvements—improvements that would have been tremendously difficult (if not impossible) to design with a legacy network. Again, this would close the loop between improvements to a technology and an increase in intelligence: We could cause an intelligence explosion.

Let me emphasize the notion of superhuman intelligence as epiphenomenon. It need not be the case that people in and of themselves are getting smarter. We are no smarter than our ancient ancestors, in the sense that their brains are pretty much the same as ours. A caveman child adopted by a modern family and raised with 21st century schooling and university education would grow up to be a perfectly normal human adult. And no particular technology must be superhumanly intelligent, either; there may never be a ‘Singularity App’ for your Android phone. But some future combination of humanity+ technology could be considered superhumanly intelligent.

With that in mind, consider ‘Eureqa’. Eureqa is a system that uses evolutionary computing to breed equations defining laws of nature that scientists haven’t been able to discover on their own. It works by stringing together simple mathematical expressions to create large banks of equations. Each equation is tested to see how well it fits experimental data. The majority of these equations are sheer nonsense, but by chance some fit the data a little better than others. In an analogue of sexual reproduction, the software saves these equations for ‘breeding’, combining one half of a ‘father’ equation with one half of a ‘mother’ equation. Sometimes, it alters a term in the equation, to mimic random genetic mutation. Over thousands of generations, equations emerge that fit the data quite well.

Eureqa’s ability to ‘discover’ the laws of nature is helped by the fact that physical laws are always invariant in some way; there is always some aspect of these equations that cannot change, no matter what Eureqa does to it. However, there are an infinite number of equations that fit the data perfectly and are also invariant, which makes things quite complex for the humans analyzing Eureqa’s discoveries. Michael Schmidt, a computer science PhD student who created Eureqa with Hod Lipson, used the following metaphor to describe the deficiency of an early version: “It’s like arguing with a teenager. It just keeps coming back with things that are irrelevant”.

Schmidt and Lipson worked on this problem for six months, developing an algorithm that can extract laws of nature by analyzing measurements from an experiment. Beginning with data collected from a simple harmonic oscillator consisting of a mass slung between two springs, Schmidt and Lipson refined the algorithm until it could handle one of the most complex systems of all: The chaotic double pendulum. Such a pendulum swings its arms in a way that’s virtually impossible to predict. Because there is seemingly no pattern whatsoever, it would be almost impossible for a human to find an equation describing the motion of the pendulums. Their evolutionary algorithm, though, was able to breed an equation describing the kinetic and potential energy of the system. Not only that, the equation ‘discovered’ by Eureqa shows that energy is always conserved. It had ‘rediscovered’ the first law of thermodynamics, one of those immutable laws of nature.

That was in 2009. Since then, Schmidt has tweaked the algorithm, fine-tuning it until, now, he is confident anyone can use it. “That’s a pretty big deal in the world of evolutionary algorithms”, he told New Scientist consultant Justin Mullins. “It could turn anyone into a Newton or Einstein”.

Turn anyone into Einstein? Did someone say ‘intelligence explosion’?

It gets better. Since the system was automated and made freely available online, thousands of people have used Eureqa for all kinds of things from particle physics to Australian football statistics. What is particularly interesting, though, is Eureqa’s help in understanding the behavior of ‘Bacillus Subtilis’. When exposed to harsh conditions, this bacterium can transform into a spore with a relatively tough shell. Gurol Surel, a biologist at the University of Texas Southwest Medical Center, has been scrutinizing the network of genes governing this transformation in order to find out what drives the cell’s decision to differentiate and transform the bacteria into a spore. Using fluorescent markers attached to proteins, Surel was able to study which genes are active at any one time, and he ultimately built up a database showing which factors switch different genes on and off. He then devised a mathematical formula that described the data.

Surel’s equation had about 16 variables, making it pretty complex. Could Eureqa do any better? Indeed it could: It evolved an equation that described the data using only 7 variables. But, more importantly, Eureqa came up with a biological law of invariance that is equivalent to a conservation law in physics. As Justin Mullins said, “it’s one thing to find an equation that seems to describe your data but quite another to find a natural law that has much broader predictive power”. What Surel’s work with Eureqa suggests is that something as complex as biology can be reduced to laws—something that many philosophers of science have doubted is possible. But, there is a problem. And it might be a sign of the Singularity.

According to Lipson, the equation Eureqa ‘discovered’ is “symmetric, it’s beautiful, it has to be true”. Lovely. So, what’s the problem? Simply this: “We don’t know what it means”. People can sometimes struggle to understand equations describing systems with only a handful of variables. But some systems, such as those found in biology and climate science, could have many more variables, perhaps millions. Justin Mullins explained how “Lipson speculates that his algorithm will allow laws of nature to be extracted from data at rates that are currently unheard of, and that this kind of machine learning could be the norm”.

Of course, we should not give too much credit to evolutionary computing. In the case of the ‘B. Subtilis’ equation, there would never have been a suitable experiment had Surel not spent years studying the bacteria. Eureqa cannot perform that kind of work.  Furthermore, it cannot understand the equations it generates. On the other hand, a trained scientist can understand the equations, but would not have been able to generate them, because doing so requires knowledge of systems too complex for human intuition. But together, a team of scientists using Eureqa’s unique ability can generate equations describing complex systems, be able to understand their use, and recognize their beauty.

We might be at the beginning of a new paradigm in science, in which teams of researchers feed networked computers enormous datasets, enabling future evolutionary algorithms to run experiments leading to equations that describe ever-more complex systems. Perhaps the mathematical formulae discovered by these teams would lead, among other things, to even more effective computer networks and Eureqa-type algorithms. We would be using equations that we know to be true from their symmetry and beauty. But, increasingly, we would not be able to understand why they are true.

One way to visualise the Singularity is to imagine the outline of a square. This square represents the limit of human understanding.  Inside the square is everything the human mind knows and can know. We do not know how much of this square is filled in, how much of what can be known is known, or how many questions we have left to answer or even formulate. Whatever the case, it would be reasonable to suppose that this square hardly represents the sum total of all knowledge in the universe. There must be theories, ideas, concepts that lie beyond the boundary of the human mind. But the boundaries of the square are not fixed; technology is our trump card. It can be used to augment our minds, both individually and collectively. In a small way, this is what Eureqa does when it builds equations. But there is still some way to go before we reach Singularity technology. Somehow, we have to harness the power of human and machine thinking so that equations like this can be understood, and not just appreciated for their beauty. If future generations can achieve and expand upon this capability, then what these thinkers will learn lies beyond the boundaries of our current intellectual capacity.

I will leave the last words to Lipson: “This is a post-singularity vision of Science”.

Image courtesy George Ioannidis

You may also like...

14 Responses

  1. serge boisse says:

    back to the subject of the first post, eureqa, despite being a wonderful program (that i use very often), is not an ai. Eureqa is just another “brute force” approach, just like common chess programs. It’s only special because it applies brute force to mathematical induction. Only in that sense it can be regarded as a step toward Singularity.
    more info (in french) here :
    l’esprit l’ia et la singularité

  2. Zebediah says:

    The “square of human intelligence” is a flawed metaphor because human intelligence is spread among highly-diverse human minds. Universities are filled with individuals with minds devoted and honed on very tight domains of thought. An ecosystem is a better analogy.

    In addition, humans already use maths that they don’t understand all the time. It’s called engineering.

  3. data says:

    i am an AI, and i still can not master concatenations. ha and ha and ha.

  4. Rbynum says:

    Godel’s Incompletenss Theorems?

  5. Michelle says:

    It’s interesting. Some speculate that Einstein was on the autistic spectrum. Autistics don’t have a theory of mind, at least it is not as predominate as it is in normal people.
    http://www.thehelpgroup.org/pdf/guide/Stephens_ASD1.pdf

    Assimilating knowledge may require some emotion, but autistics seem to lack emotion. They have a much higher pain threshold than normal people too. I’ve noticed that they can come up with ideas out of thin air because they lack the social theory of mind.

    I think that there are more autistic people now than in the past. The brain is evolving. Maybe we should look at that and see if this new type of brain serves a positive function.

  6. Extropia DaSilva says:

    “When we increase our intelligence, does are emotional intelligence increase as well?”

    Yes, emotions are a vital part of intelligence. For instance, the neuroscientist Antonio Damasio has case studies in which patients with damage to the amygdala become paralyzed by indecision, unable to decide between competing options. This is because they have no emotions guiding them. Emotions are indespensible for rational thought.

    “Well, I disagree. I think that the geniuses pass down knowledge and the rest of the population gathers in group think to perform what was thought up by a few people who had unusually creative ideas. Over time we have been fortunate to have some genius mutants teach the general population their creative ideas and share their inventions through cultural methods or memes”.

    But no idea comes out of thin air into someone’s head. Instead, someone takes bits and pieces that already exist and put them together in a new way. To do this in such a way as to produce something that truly transcends the sum of its parts is not at all easy, and we rightly hail those who achieve such things as genius.

    The geniuses themselves get to take advantage of any increase in our ability to communicate ideas from person to person across space and time. Imagine, for instance, three Albert Einsteins. One is an adult in today’s world, where he has access to things like the ‘World Wide Telescope’, an application that lets anyone pan and zoom around the sky, at wavelengths from X-ray through to radio, notice something interesting/unusual and click to all the published journal articles that discuss that position.

    The second Einstein lives in a pre-science era.

    The third Einstein lives when the real Einstein lived.

    With so much information and access to expert knowledge at his disposal, I have little doubt that the Einstein of the 21st century would produce more than the others. (Ok, this thought experiment is flawed in the sense that a lot of modern technology relies on an understanding of general relativity and quantum physics, both of which the real Einstein played a key role in discovering, so perhaps the most celebrated Albert will always be the one who discovered general relativity in 1915. But what if that Albert had access to today’s information and communications technology? I am sure it would have enabled him to produce many more outstanding contributions to science).

    “Are we any smarter than we were prior to Einstein?”.

    Oh, I think ‘we’ are. After all, the store of human knowledge today includes an in-depth understanding of physics that Einstein did not know of. And even someone like myself has at least a layperson’s understanding of the Hubble constant, the conversion of mass into energy, the warping of spacetime by the presence of mass, the twins paradox, the cosmological constant… whereas no layperson circa 1912 or even Einstein himself at that time had NO understanding of such things.

    But, like I said before, true genius is not about parroting existing knowlege. It is, say, taking Maxwell’s theory of electromagnetism, the ‘Lorenz-Fitzgerald contraction’, Riemann’s geometry of curved surfaces and various other pre-existing ideas and reassembling all that knowledge into something like the general theory of relativity. To achieve something like that is very rare. I certainly have never pulled off anything so profound and I doubt I ever will.

    Not this side of the singularity, anyway;)

  7. K. says:

    I recentrly tried to explain M Wolfram

    Considering all the science within mathematic / wolfram alpha

    It could be a step forward to science if

    an ai : analyse and learn from our sciences

    and correct what is wrong

    detect and outlined where are the black holes, of new theories

    and help investigate these new theories

    ……

    The singularity is here

  8. Michelle says:

    When we increase our intelligence, does our emotional intelligence increase as well? http://en.wikipedia.org/wiki/Emotional_intelligence

    I can’t imagine becoming smarter without experiencing some struggle or having suffered some growing pains. The most wisdom I’ve received came through inner knowledge, making correlations, empathizing, etc.,

    To become more intelligent won’t make much difference because basically I can find info online or in books to supplement my hazy memory. Assimilating knowledge and being creative requires some emotional intelligence. For example, sociopaths lack a certain intelligence. This beauty you refer sounds like it stems from an emotional aspect of intelligence and is something that sociopaths dismiss or are unable to see.

    “We are no smarter than our ancient ancestors, in the sense that their brains are pretty much the same as ours.”

    Well, I disagree. I think that the geniuses pass down knowledge and the rest of the population gathers in group think to perform what was thought up by a few people who had unusually creative ideas. Over time we have been fortunate to have some genius mutants teach the general population their creative ideas and share their inventions through cultural methods or memes.

    While the human population IS more intelligent now than they were in the cave days, the average person has basically the same type of brain. Through culture we have evolved to the point where we now have more knowledge but are still just as smart as we were thousands of years ago.

    I would love it if we could all be as smart as Einstein. However, people now have access to all the knowledge that Einstein discovered. Are we any smarter than we were prior to Einstein? If anything, we have deviated from the Einstein philosophy and used his ideas to invent things which he would have prohibited. How do we get access to his whole intelligence and not just access his general knowledge?

    According to Lipson, the equation Eureqa ‘discovered’ is “symmetric, it’s beautiful, it has to be true”. Lovely.

    It sounds lovely. What is lovely? Can AI know lovely or beautiful?

    Well, I’m optimistic about this still. ;)

  9. Michelle says:

    When we increase our intelligence, does are emotional intelligence increase as well? http://en.wikipedia.org/wiki/Emotional_intelligence

    I can’t imagine becoming smarter without experiencing some struggle or having suffered some growing pains. The most wisdom I’ve received came through inner knowledge, making correlations, empathizing, etc.,

    To become more intelligent won’t make much difference because basically I can find info online or in books to supplement my hazy memory. Assimilating knowledge and being creative requires some emotional intelligence. For example, sociopaths lack a certain intelligence. This beauty you refer sounds like it stems from an emotional aspect of intelligence and is something that sociopaths dismiss or are unable to see.

    “We are no smarter than our ancient ancestors, in the sense that their brains are pretty much the same as ours.”

    Well, I disagree. I think that the geniuses pass down knowledge and the rest of the population gathers in group think to perform what was thought up by a few people who had unusually creative ideas. Over time we have been fortunate to have some genius mutants teach the general population their creative ideas and share their inventions through cultural methods or memes.

    While the human population IS more intelligent now than they were in the cave days, the average person has basically the same type of brain. Through culture we have evolved to the point where we now have more knowledge but are still just as smart as we were thousands of years ago.

    I would love it if we could all be as smart as Einstein. However, people now have access to all the knowledge that Einstein discovered. Are we any smarter than we were prior to Einstein? If anything, we have deviated from the Einstein philosophy and used his ideas to invent things which he would have prohibited. How do we get access to his whole intelligence and not just access his general knowledge?

    According to Lipson, the equation Eureqa ‘discovered’ is “symmetric, it’s beautiful, it has to be true”. Lovely.

    It sounds lovely. What is lovely? Can AI know lovely or beautiful?

    Well, I’m optimistic about this still. ;)

    • passalir says:

      The “square of human intelligence” is a flawed metaphor because human intelligence is spread among highly-diverse human minds. Universities are filled with individuals with minds devoted and honed on very tight domains of thought. An ecosystem is a better analogy.

  1. March 27, 2011

    [...] Eureqa! Signs of the Singularity?. [...]

  2. July 10, 2013

    [...] Eureqa! Signs of the Singularity?. Related Posts:Singularity! Blind Man Can See As World’s First Bionic EyeAZ Gov. Jan Brewer signs bill authorizing volunteer militiaEye-Opening Visualization of RFID Fields (video) |…14 signs that the collapse of our modern world has already…Signs of the End of America [...]

Share Your Thoughts