Does This Headline Know You’re Reading It?

Not yet, but it could.

Ralf Biedert and colleagues at the German Research Center for Artificial Intelligence (DFKI) are using eye-trackers from Tobii Technology of Sweden along with HTML, CSS, and JavaScript to create a reading enhancement technology called Text 2.0.

This is not simply a case of using infrared light, a camera, and eye movement to move a cursor and click buttons: Text 2.0 infers user intentions and enhances the reading experience in far more complex ways. Reading certain words, phrases, or names can trigger the appearance of footnotes, translations, definitions, biographies, even sound effects or animations. Ask how a word is pronounced and you get a verbal answer. If you begin skimming the text, it fades out the less important words. If you glance away, a bookmark automatically appears, pointing to where you stopped reading.

If you think this sounds like a big step toward the Young Lady’s Illustrated Primer from Neal Stephenson’s The Diamond Age, you’re right, as this introductory video shows:

DFKI has also put their Processing Easy Eye Tracker Plugin (PEEP) to somewhat less futuristic uses. Here they’ve used it with Webkit’s 3D capability to create (in four hours!) a window-manipulation system they call “gaze controlled tab exposé”:

PEEP is free to download and use in your own eye-tracking projects. Here’s a video tutorial:

Biedert foresees this technology giving authors and artists new tools to create things like multimedia-enhanced “Hollywood books.” That’s certainly one possibility, once the hardware costs come down and the bugs are worked out. (Tobii eye-trackers currently cost roughly $7,000–$35,000, and might not work well with some eyeglasses, contact lenses, or lighting conditions.)

If you think this sounds like a big step toward the Young Lady’s Illustrated Primer from Neal Stephenson’s The Diamond Age, you’re right.

Despite the obvious potential for the enhancement of learning and other things good and cool, your inner skeptic may worry about using technology to enhance something as traditional and pervasive as reading. Certainly Text 2.0’s automatic bookmark seems helpful, non-intrusive, and easy to automate. On the other hand, automatic visual or special effects, while potentially wonderful for language learning or artistic purposes, could be like those distracting animated ads on a web page that you’re trying to read.

Like any new technology, you can bet this will be used for advertising. Google and others have built businesses around contextual ads, often generated automatically. Sometimes these are helpful (sponsored links on Google results pages) and sometimes not (those double-underline links in articles that pop up ads when you accidentally hover over them). Imagine how annoying pop up ads could be if they were triggered when you looked at them. And while you may appreciate the fact that a publication’s ads are targeted to you, do you want them changing based on where your eyes linger?

Automatically generating anything involving language is risky. When you are skimming, does the computer really know which words are “less important” to you? If you’re reading about feral horses of the West, will you see ads for new Ford Mustangs? My favorite example of clueless ad automation: a political piece positing a coming dark age, accompanied by a large ad selling “Dark Ages ringtones.”

Tobii x120 Eye Tracker. Photo: text20.netSome will find the whole idea Orwellian. I know people so concerned about their privacy that they won’t sign up for a supermarket discount card. If you forego cheaper groceries because you don’t want Safeway to track what you buy, will you want your ebook reader to track which words you found interesting or hard to understand?

If Text 2.0 is to reach the mainstream, designers will have to work hard to make it affordable, useful, easy to use, and unintrusive. Sounds like a job for Apple and their top-notch design and UI skills. Indeed, Apple is known to have bought Tobii eye-trackers (see Resources). Whether these are for internal research only or for a future product, Apple is (characteristically) not saying. In early speculations about the iPad, people wondered if it would include eye-tracking tech, but the initial versions won’t even include a camera. In the coming years, though, who knows? Your iPad may know exactly what you’re reading, and you may be happy that it does.

32 Responses

  1. For me Text 2.0 will be the next step into a total controlled world. It will be possible for governments and big companies to know everything about. Do we really need this?

  2. sağlık says:

    I am happy to find this very useful for me, as it contains lot of information. I always prefer to read the quality content
    aj.

  3. Pop Up ads while hovering them? Won’t it irritate users? Almost all of genuine internet users enable their pop up blocker to disable pop ups. Even browsers do have Pop Up blocker by default.

  4. There’s no privacy issue if the e-book reader is free-software so it can be set not to sent information to any corporations. Of course, most people prefer proprietary software..

  5. anime says:

    Text 2.0 is a technology for improving reading that is being developed through the collaboration of the German Research Center for Artificial Intelligence and a Swedish company Tobii Technology. With the help of this technology, the words can be found on the page you are viewing. Based on this information, the electronic reader will react accordingly.

  6. However if you are reading some article you can just imagine adverts popping up continuously with no way to avoid them…………thats all………..

  7. Hot Sauce says:

    Love it. I equate this to the same thing as cookie based advertising.

  8. Pop Up ads while hovering them? Won’t it irritate users? Almost all of genuine internet users enable their pop up blocker to disable pop ups. Even browsers do have Pop Up blocker by default. It shows how much users are irritated with these pop ups.

  9. Anonymous says:

    There is a way to use AI to intelligently manipulate and deliver to users the exact content they were viewing the last time. Thats if they come back though?

  10. anime says:

    I remember when Minority Report is a futuristic film. Now it seems so old! Why is John Anderton shaking hands so much? He must only move their eyes to navigate the screen, or focus on a window (perhaps with a frown forehead) to maximize.

  11. Almost all of genuine internet users enable their pop up blocker to disable pop ups. Even browsers do have Pop Up blocker by default. It shows how much users are irritated with these pop ups.

  12. Firma says:

    I know people so concerned about their privacy that they won’t sign up for a supermarket discount card. If you forego cheaper a groceries because you don’t want Safeway to track what you buy, will you want your ebook reader to track which words you found interesting or hard to understand

  13. irena says:

    I am very impressed by Text 2.0. The way it infers user intentions and enhances the reading experience is very interesting. Irena from cheap hosting

  14. ksi united says:

    It’s very impressive how the technology advances evary day, but I.m sure this will have a lot of negative aspects. It will be mainly used for advertising, and that can be very annoying, but it can also help disabled people and make using a computer much easier for them.

  15. Won’t it irritate users? Almost all of genuine internet users enable their pop up blocker to disable pop ups. Even browsers do have Pop Up blocker by default. It shows how much users are irritated with these pop ups.

  16. If you’re reading about feral horses of the West, will you see ads for new Ford Mustangs? My favorite example of clueless ad automation

Leave a Reply