h+ Magazine

Not yet, but it could.

Ralf Biedert and colleagues at the German Research Center for Artificial Intelligence (DFKI) are using eye-trackers from Tobii Technology of Sweden along with HTML, CSS, and JavaScript to create a reading enhancement technology called Text 2.0.

This is not simply a case of using infrared light, a camera, and eye movement to move a cursor and click buttons: Text 2.0 infers user intentions and enhances the reading experience in far more complex ways. Reading certain words, phrases, or names can trigger the appearance of footnotes, translations, definitions, biographies, even sound effects or animations. Ask how a word is pronounced and you get a verbal answer. If you begin skimming the text, it fades out the less important words. If you glance away, a bookmark automatically appears, pointing to where you stopped reading.

If you think this sounds like a big step toward the Young Lady’s Illustrated Primer from Neal Stephenson’s The Diamond Age, you’re right, as this introductory video shows:

DFKI has also put their Processing Easy Eye Tracker Plugin (PEEP) to somewhat less futuristic uses. Here they’ve used it with Webkit’s 3D capability to create (in four hours!) a window-manipulation system they call “gaze controlled tab exposé”:

PEEP is free to download and use in your own eye-tracking projects. Here’s a video tutorial:

Biedert foresees this technology giving authors and artists new tools to create things like multimedia-enhanced “Hollywood books.” That’s certainly one possibility, once the hardware costs come down and the bugs are worked out. (Tobii eye-trackers currently cost roughly $7,000–$35,000, and might not work well with some eyeglasses, contact lenses, or lighting conditions.)

Despite the obvious potential for the enhancement of learning and other things good and cool, your inner skeptic may worry about using technology to enhance something as traditional and pervasive as reading. Certainly Text 2.0’s automatic bookmark seems helpful, non-intrusive, and easy to automate. On the other hand, automatic visual or special effects, while potentially wonderful for language learning or artistic purposes, could be like those distracting animated ads on a web page that you’re trying to read.

Like any new technology, you can bet this will be used for advertising. Google and others have built businesses around contextual ads, often generated automatically. Sometimes these are helpful (sponsored links on Google results pages) and sometimes not (those double-underline links in articles that pop up ads when you accidentally hover over them). Imagine how annoying pop up ads could be if they were triggered when you looked at them. And while you may appreciate the fact that a publication’s ads are targeted to you, do you want them changing based on where your eyes linger?

Automatically generating anything involving language is risky. When you are skimming, does the computer really know which words are “less important” to you? If you’re reading about feral horses of the West, will you see ads for new Ford Mustangs? My favorite example of clueless ad automation: a political piece positing a coming dark age, accompanied by a large ad selling “Dark Ages ringtones.”

Some will find the whole idea Orwellian. I know people so concerned about their privacy that they won’t sign up for a supermarket discount card. If you forego cheaper groceries because you don’t want Safeway to track what you buy, will you want your ebook reader to track which words you found interesting or hard to understand?

If Text 2.0 is to reach the mainstream, designers will have to work hard to make it affordable, useful, easy to use, and unintrusive. Sounds like a job for Apple and their top-notch design and UI skills. Indeed, Apple is known to have bought Tobii eye-trackers (see Resources). Whether these are for internal research only or for a future product, Apple is (characteristically) not saying. In early speculations about the iPad, people wondered if it would include eye-tracking tech, but the initial versions won’t even include a camera. In the coming years, though, who knows? Your iPad may know exactly what you’re reading, and you may be happy that it does.