Sign In

Remember Me

By Thought Alone: Mind Over Keyboard

Check out our featured article: Does This Headline Know You’re Reading It?

——————————–

Why bother to type a document using a keyboard when you can write it by simply thinking about the letters you need to type?

A brain wave study presented at the 2009 annual meeting of the American Epilepsy Society shows that people with electrodes in their brains can “type” (input data into a computer) using just their minds. Neurologist Jerry Shih, M.D. Shih and other Mayo Clinic researchers worked with Dean Krusienski, Ph.D., from the University of North Florida in an experiment involving two patients with epilepsy. Both patients were already being monitored for seizure activity using electrocorticography (ECoG), in which a sheet of electrodes is laid directly on the surface of the brain. This procedure requires a craniotomy, a surgical incision into the skull. Dr. Shih and colleagues hypothesized that feedback from electrodes placed directly on the brain would be much more specific than data collected with EEG (electroencephalography) alone, in which electrodes are placed on the scalp. Most studies of mind-machine interaction have occurred with EEG. "There is a big difference in the quality of information you get from ECoG compared to EEG. The scalp and bony skull diffuses and distorts the signal, rather like how the Earth’s atmosphere blurs the light from stars," says Dr. Shih. "That’s why progress to date on developing these kinds of mind interfaces has been slow."

Electrodes placed directly on the surface of peoples' brains allow them to type just by thinking of letters. Photo: livescience.com / stockexpertDr. Shih’s patients at the Mayo Clinic were asked to look at a computer screen containing a 6-by-6 matrix with a single alphanumeric character inside each square. Every time the square with a certain letter flashed, the patient focused on it and a computer application recorded the brain’s response to the flashing letter. The computer software calibrated the system with the individual patient’s specific brain wave patterns. When the patient then focused on a letter, the letter appeared on the screen. "We were able to consistently predict the desired letters for our patients at or near 100 percent accuracy," Shih explains. "While this is comparable to other researchers’ results with EEGs, this approach is more localized and can potentially provide a faster communication rate.”

A recent h+ article, “Mind Reading (Neuro Decoding) Goes Mainstream” (see Resources) describes a similar study by Dr. Gerwin Schalk, who worked with patients using ECoG at the Wadsworth Center, in Albany, NY. The patients were asked to say or imagine words flashed on a screen while their brain activity was recorded. Schalk’s team then used specially designed decoder algorithms to predict the vowels and consonants of the word, using only the pattern of brain activity. They found that both speaking and imagining the word gave roughly the same level of accuracy.

 

Brain Wave applications include the ability to “mind read” vowels, consonants, and individual letters; algorithms to turn brain waves into musical scores; even twittering by thought alone.

In addition to the ability to “mind read” vowels, consonants, and individual letters, brain wave applications also include algorithms to turn brain waves into music and even “tweeting” (using the popular Twitter Internet application) by thought alone. Brain music therapy is a form of neurofeedback using EEG based on a variable ratio of fast and slow rhythms –- it can be used to turn a person’s brain waves into music notes using a computerized mathematical formula. Dr. Galina Mindlin, a neuropsychiatrist with the Brain Music Therapy Center in New York City brought this to the U.S. from Moscow in 2006 as a form of entrainment therapy. Interviewed on NBC’s Today Show, she said, “Brain waves are translated into music digitally with a special algorithm. Once the brain waves are converted into musical sounds, they are placed on a CD with a relaxing file and activating file and instructions on how to use them.” What does this mind-machine interface sound like? “It sounds like classical piano music,” says Dr. Mindlin. Here’s a video showing the use of an EEG mind-machine interface to control sampled sound clips on a piano:

In addition to his ECoG research at the Wadsworth Center, Dr. Gerwin Schalk and his colleagues also worked with University of Wisconsin-Madison biomedical engineering doctoral student Adam Wilson to develop an interface that involves a keyboard displayed on a computer screen that interprets brain waves to send Twitter messages (tweets). "We started thinking that moving a cursor on a screen is a good scientific exercise," said Justin Williams, a University of Wisconsin-Madison assistant professor of biomedical engineering and Wilson’s adviser. "But when we talk to people who have locked-in syndrome or a spinal-cord injury, their number one concern is communication."

Using the EEG-based interface, "All the letters come up, and each one of them flashes individually," explains Williams. "And what your brain does is –- if you’re looking at the ‘R’ on the screen and all the other letters are flashing – nothing happens. But when the ‘R’ flashes, your brain says, ‘Hey, wait a minute. Something’s different about what I was just paying attention to.’ And you see a momentary change in brain activity."

Wilson was able to tweet by thought alone using EEG. Here’s a video showing the brain-Twitter interface:

Tweeting by thought alone is a somewhat slow process using this prototype technology –- we speak at approximately 120 words per minute. But, as with texting, users can improve as they practice using the interface. "I’ve seen people do up to eight characters per minute," Wilson says.

Brain wave applications in the laboratory –- whether using EEG or the more invasive ECoG –- now include the ability to “mind read” vowels, consonants, and individual letters; algorithms to turn brain waves into musical scores; and even twittering by thought alone. Who needs a keyboard when you can simply think about what you want to say (or play musically) and have it recorded and/or communicated?

15 Comments

  1. Tendo algunas dudas sobre estos chips, es posible saber si una persona a sido operada con un chip (lector de mente)?
    Que durabilidad tienen este tipo de chips en el cerebro?
    Es de suponer que estos chips tendran sus contras.

  2. this research is hardly about entering by believed. the actual idea is making a man/machine program and learn more about how to program so that more innovative ideas can be converted into activity.

  3. Where can I sign up?

  4. Wow, are people so lazy these days that they’ll actually submit to having their skull opened just to avoid moving their fingers??

  5. this research is hardly about typing by thought. the underlying concept is making a man/machine interface and learn more about how to interface so that more advanced thoughts can be translated into action.

    besides, some people are unable to type due to physical disability, so being able to type by thought would be a godsend.

  6. You tool it has nothing to do with being lazy at all it has to do with the future of technology. Do you honestly think that in 100 years we will be using something at primative as a keyboard? Why do you think devices are touch screen? Because the mouse is a primitave tool just like they keyboard. Sure it is the standard and easy to use but eventually it will be in a museum.

  7. could you please post the reference to the “brain wave study” you mention in your blog post (preferably in your resources list). thank you.

  8. ever had the misfortune/occassion to attend an epileptic ;than you would have known !
    this is all about finding a cure for them.
    in addition it also helps the department of dirty tricks of CIA/ KGB/MI5 etc etc.

  9. This would be great for anyone who has lost the use of their hands for typing, and voice for speaking… I think it’d be a little intrusive to have to open their brains, but it could really help the handicapped community out. I think it would enable some to have jobs and make money. Thumbs up!
    -Sylvia
    Citizen Campanola

  10. @laughing man nice GITS reference.

  11. Brain Wave study is amazing. I am quite fascinated by it.

  12. please what is the name of this machine again i just had a friend who was in a car accident and he cant speak his parents are praying for a miracle but the problem is will this machine work even if the part of his brain that produces speech is damaged ???
    please help !!!!

  13. think it’d be a little intrusive to have to open their brains, but it could really help the handicapped community out. I think it would enable some to have jobs and make money ilan ver

  14. I find this absolutely terrifying … imagine typing to your boss this way and accidentally spilling private thoughts? Also, how will this technology interface with existing computers and operating systems? Sounds like there might be some serious development and possible pc errors along the way. Can you imagine debugging the error codes for something like this?

  15. Then you are misunderstanding how this works.

    This is using a “BCI” to move a mouse cursor on a screen to type on the on screen keyboard, this is not “Thought to text” the way “speech to text” enables you to talk and have it recorded.

    Over the next decade, this BCI will improve, but it will still OPERATE the exact same way. It’s just that instead of using your hand to control a mouse, it will use your brainwaves. As we switch to VR and HUD UIs over the next decade, we will be using such devices to control the UI but it will still require conscious volition to type a text, or open a voice line, or connect to another persons VR space.

    Integration is not a problem, because this is just a glorified hands free mouse and keyboard.