Decelerating Change in Supercomputer Power: Is the Singularity Stalling?

 

IEEE Spectrum ran an interesting article the other day, titled: Next-Generation Supercomputers: Supercomputers are now running our search engines and social networks. But the heady days of stunning performance increases are over.

The article is written by one of the authors of a 278-page DARPA report on the future of supercomputing, one of the conclusions of which was that the accelerating increase in supercomputer power we’ve seen in the past is not likely to continue. To quote:

to construct an exaflops-class supercomputer that occupies a large building or a petaflops-class one that fits in a couple of refrigerator-size racks … would require engineers to rethink entirely how they construct number crunchers in the future.

The problem identified isn’t computer architecture per se but rather power consumption. They estimate that making an exaflops-class supercomputer by simply scaling up current technology would result in a machine taking up more than 0.1 percent of the total U.S. power grid, or about 1.5 gigawatts of power. Oops.

Singularity skeptics are often heard claiming that technology development is going to be an S-curve rather than an ongoing exponential explosion. Are we finally hitting the “slowdown” part of the S?

Of course, that’s not what DARPA’s report told them. Rather, the conclusion was basically to get with the times, and focus on dividing up problems among a larger set of smaller computers, rather than focusing on building individual super-fast computers. Even if the speed of the fastest supercomputer does decelerate, what about, for instance, the total amount of computer power on Earth? I haven’t seen any good estimates of this (let me know if you find one!) but I’ll bet you it’s accelerating steadily and dramatically.

Another lesson from this report is the synergy between various advanced technologies. The bottleneck in supercomputer speed isn’t human ingenuity at designing new computers nor the laws of physics, it’s the current state of electrical power generation technology. Or to put it differently, a breakthrough in power generation (plus some “business as usual”) will yield a breakthrough in supercomputer speed.

And then of course, there is the emerging area of low power consumption supercomputing, using technology similar to that used in mobile devices, aimed specifically at bypassing these problems.

Still, this sort of data is a caution against approaching the future via mindless curve-plotting and trend extrapolation. There are many trends one can look at, and ultimately one needs to interpret them in the context of one’s holistic understanding of the future. Numeric extrapolation and qualitative extrapolation need to proceed in synergy, a methodology that strips away the illusion that we can do any kind of fully objective futurology — but leaves plenty of room for real understanding.

15 Comments

  1. Ben,

    Regarding good estimates of the total amount of computer power on Earth, have you seen the paper:

    The World’s Technological Capacity to Store, Communicate, and Compute Information
    by Martin Hilbert, and Priscila López ?

    http://www.sciencemag.org/content/early/2011/02/09/science.1200970

  2. I also wrote my messages using mostly a male’s perspective, so a woman’s criticism might be needed if did other mistakes that I didn’t notice yet.

  3. I’m sorry for a mistake of grammar that I’ve just committed in my previous message because I am not a native english speaker. When I wrote that the average Joe is happy to watch on his screen some pretty chick in “its” underwear, I didn’t intend to sound mysoginistic. Although I think that through this error I have involuntarily described quite well that particular type of male behaviour, behaviour that actually involves the treatment of the woman as an object rather than a person.
    Mea culpa.

  4. There actually might exist another potential hindrance for the advancement of technology, other than that of the physical limits of silicon, those of the graphene or of other conceivable material that can be used for building processors. I am thinking of the limits that might be displayed by the user of this technology himself/herself. Every product of technology is actually created in accordance with the consumer’s needs and demands. And I am thinking now about the mass consumer, the average one, not the specialized one(like, for example, the qualified researcher in a scientific laboratory).
    Imagine, if you like, that today you will be able to sell to an average household consumer a desktop computer or a laptop with the processing power of an actual supercomputer. What will do that person with all that processing power, what uses will he/she find for it? Wouldn’t this situation resemble very much the act of selling a Ferrari sports car to a subsaharan tribesman?
    In order to get to such a degree of technological advancement as many dream of, first you need new consumer habits that would require that kind of progress and will sustain it financially. But the typical household consumer of today is still pretty much under the rule of some primal urges, like the need to socialize with others(see for example the huge success of Facebook), the need of sex(see the huge amount of porn on the net) and even the need of violence(it might be a controversial idea, but i think that it is a real unconscious need that was developed in our distant prehistoric past when violence was commonly used either as a mean to get to the top of the social hierarchy or as an essential way of protection against other individuals or groups – nowadays this need manifests itself in a sublimated form in the proliferation of violent videogames or of movies with violent content) etc. What if some level of technological advancement will prove to be quite sufficient in order to fulfill those human urges and this situation will create another bottleneck? What if a certain amount of processing power will prove to be sufficient to create a virtual reality almost indistinguishable from the real life, a virtual reality able to provide tot the average Joe what he would desire in this world but presently can’t have it? Then the average Joe will have more additional motives to desire a continued technological advancement or will he settle for that technological level and will be content with it?
    I am thinking now also to the current behaviours that I see manifested by some people that I know. In their case, the main motivation for changing periodically their computers for better ones is actually that of being able to play videogames with improved graphic caracteristics. The other activities they currently do with their computers actually don’t require very much processing power(they would eventually desire more storing capacity for family photos, movies etc.).
    The people really interested in knowledge and its possible nontrivial applications via technology(the kind of people that form the public of this website, I presume) do not, unfortunately, form the majority of the general population. The average Joe is actually pretty much content to be able to play a thrilling shooter, to chat with some friends and occasionally to watch on the screen some pretty chick in its underwear(or rather without). But he is also the one who influences greatly what the computer companies will produce because the likes of him form the biggest part of the household consumer market.
    And even the other type of consumer, the one that purchases a computer for a lucrative purpose, might find in some cases that a certain level of computer power could be enough for what he intends to do.
    So one interesting problem will be in my opinion how to make sure that the mass demand will continue to increase constantly and in the measure required to achieve the level of technological progress that will unleash the singularity. What if this demand could also, unfortunately, tend to exhibit some kind of S shape?

    • Necessity breeds creation is a popular saying. Some of the time, the inverse of this statement is true; creation breeds necessity. I can remember a time when a 500 megabyte hard drive seemed like a gross excess of memory to me. I believe that as the ability to process more and more information becomes available we will see larger and more complex programs that utilize the processing power.

      The point at which “the average Joe” will be happy with current technology is still very far away as you’ve defined it. This is simply due to the fact that displays are still in the stone age compared to computer processors. I’m fairly certain that economic demand for novel technology will continue but you do raise a very good point with that question. For instance, if 3-D printing really takes off and changes the infrastructure economy (as mentioned in a previous H+ article,) how will that effect consumer reaction to novel technology?

      p.s. I’ve seen a lot worse writing from many native English speakers. Kudos. 🙂

  5. Well it seems we’re well on our way with graphene if the latest breakthroughs are any indication.

    http://www.physorg.com/news/2011-03-single-atom-lithography-graphene.html

    I remember also reading about the U-bend breaktrhough.

    http://nextbigfuture.com/2011/02/u-bend-graphene-transistor-breakthrough.html

    I’m sure there’s lots left to overcome with graphene but it’s pretty cool that these major breakthroughs came out within weeks of each other.

  6. It seems it is time to build solar space power stations.

  7. I don’t believe infinite computing power is a necessary precursor to the singularity event horizon or in the creation of Strong A.I, therefore poor relative levels of investment in research and development of A.I. seems to me the prevailing parameter in the stalling of the singularity

  8. Do the companies dare to risk not going for graphene (or something other than silicon) as soon as possible? If one of the big companies would start to sell better CPU’s the other ones would probably go bankrupt very fast 🙂

    • Dare they do. If you invest $1,000 into something, would you abandon that investment after getting a $0.01 return? Virtually every major chip manufacturer is invested in the raw resource of Silicon. Since everyone is heavily invested, no one’s going to make big changes.

      • Actually, considering that the “Big Chip Companies” are NOT making their own chips these days, and are mounting enormous pressure on the real manufacturers in Taiwan, it’s a factor in shift to 3d printers. Roll to Roll graphene printing is becoming more and more likely with each new development that allows ALD of graphene sheets to cover a larger area. They are also discovering that “silicon” is not terribly compatible with silicon, and works better on Boron Nitride

        http://nextbigfuture.com/2010/03/atomic-layers-of-hybridized-boron.html

        The “Big Chip Companies” are little more than design studios. If they design a “graphene chip” based on emerging technology, it’s not them who will be “eating the cost” of that silicon tech. And the Taiwanese companies will cut each others throats to be the first to develop graphene tech, because who ever does will get almost ALL the manufacturing orders.

        • So, who’s going to design the graphene chips for the Taiwanese companies? Who will “eat the cost?” I agree that roll to roll graphene printing could speed up the transition from silicon to graphene chips, but I still maintain that we’re stuck with silicon until it cannot be scaled down any farther. I’m interviewing a silicon crystal manufacturer next week to track down some solid sources to my somewhat vague statements.

  9. Hey Ben, what’s your take on non-silicon based innovations like graphene and molybdenite? Both of which look like they could offer orders of magnitude increases in speed without such massive power requirements? I know it is still in the laboratory research stage, but if silicon is reaching the plateau of the s-curve, then it would seem graphene and moly are strong contenders for the “new paradigm” that Kurzwiel predicts will occur as we reach the “end of stage” for silicon tech.

    • My gut feel is that somebody will kick the world’s ass with graphene processors, but I’m not sure who or when…

      If the big Western chip companies are too addicted to silicon to make it happen, maybe it will be the Chinese, note this work

      http://english.cas.cn/ST/BR/br_progress/201012/t20101209_62847.shtml

      on “Controllable N-Doping in Graphene” from China’s “National Center for Nanoscience and Technology”.

      I’m not saying the Chinese are ahead of us in this area (I don’t know the area well enough to say) … but just pointing out that Western companies are not the only ones with the capability to take the next big step…

  10. I recently learned that the computing industry spent billions of dollars to continue the manufacturing of silicon for computer chips. Since many major chip manufacturers are now deeply invested in silicon, it could stall the production of faster, more energy efficient, and novel chip technology.

    It seems that graphene would be the next logical step in material upgrades since it works well with the current infrastructure of chip making but it will be held off for quite some time simply due to the fact that all of these companies now want their investment back.

    I think we’re going to have to bank on less of a steady smooth curve and more on a anomalous jump in computing power, possibly made through a breakthrough in QC. Silicon is reaching the end of its run (~2019) due to the fact that it won’t be able to be scaled down any farther. Most other options in research right now to continue the curve will not work with current infrastructure and will not be ideal for business.

    My projections of what will happen based on these thoughts:

    1. (Delay) The curve will trend more like an S curve up until Silicon’s potential has been maxed out and then businesses will begin building novel infrastructures for newer/better technology. Once the new infrastructure is in place, the curve will continue to trend exponentially.

    2. (Blip) Companies will appropriately anticipate for the need for restructuring. We’ll see some low numbers as far as processing increases toward the end of Silicon’s scale down but will smoothly transition back into the curve trend after they’ve soaked all the money out of Silicon that they can.

    3. (Jump) A new company not limited by having an aging infrastructure will come out with novel technology that is vastly superior to current processors. The curve begins increasing faster. (Note: this could fit exponentially modeled growth growing exponentially, for all you Kurzweil fans.)

    Thoughts?

Leave a Reply