Sign In

Remember Me

DARPA-funded Efforts in the Development of Novel Brain–Computer Interface Technologies


The Defense Advanced Research Projects Agency (DARPA) has funded innovative scientific research and technology developments in the field of brain–computer interfaces (BCI) since the 1970s. This review highlights some of DARPA’s major advances in the field of BCI, particularly those made in recent years. Two broad categories of DARPA programs are presented with respect to the ultimate goals of supporting the nation’s warfighters: (1) BCI efforts aimed at restoring neural and/or behavioral function, and (2) BCI efforts aimed at improving human training and performance. The programs discussed are synergistic and complementary to one another, and, moreover, promote interdisciplinary collaborations among researchers, engineers, and clinicians. Finally, this review includes a summary of some of the remaining challenges for the field of BCI, as well as the goals of new DARPA efforts in this domain.

1. Introduction

Brain–computer interfaces (BCI) are systems that mediate signaling between the brain and various technological devices. The first demonstrations of BCI in humans and animals took place in the 1960s. In 1964, Grey Walter demonstrated use of non-invasively recorded encephalogram (EEG) signals from a human subject to control a slide projector (Graimann et al., 2010). Shortly thereafter, Fetz demonstrated that, by providing food reward to awake, non-human primates along with auditory or visual feedback on the firing rates of neurons in the motor cortex, these neurons could be operantly conditioned to increase their firing rates by 50–500% (Fetz, 1969). In 1971 the term brain-computer interface (BCI) was coined by Jaceques J. Vidal, who laid out a comprehensive experimental research plan to interface the human brain with computers (Vidal, 1973), including the XDS Sigma 7 at the University of California at Los Angeles that coincidently also served as the first node of the Advanced Research Projects Agency Network (ARPANET). Following these initial demonstrations, the field of BCI has expanded significantly, encompassing both invasive and non-invasive neural recordings in humans and animals, spanning a range of sensorimotor and cognitive functions, and incorporating novel feedback mechanisms in closed-loop systems.

While most closed-loop BCI systems provide feedback to the user on system performance through the presentation of sensory (primarily visual) information, approaches have also been developed to provide sensory feedback through direct stimulation of the nervous system. For instance, unidirectional systems such as cochlear and retinal implants can provide partial restoration of sight and hearing through the direct stimulation of neurons within the cochlea and retina, respectively (see Géléoc and Holt, 2014 and Chuang et al., 2014 for review). Recent explorations of targeted reinnervation in amputees suggest that such approaches may not only enable prosthetic limb control through peripheral nerve signals but may also provide a means of conveying somatosensory sensation of touch, temperature, pain, and vibration to these patients (Hebert et al., 2013).

Sensory percepts can also be elicited through direct brain stimulation (Schiller et al., 2011, Kar and Krekelberg, 2012, Larson and Cheung, 2012, Tabot et al., 2013, Zaaimi et al., 2013, May et al., 2013 and Johnson et al., 2013). Such findings provide a proof of concept for the integration of stimulation-induced sensory feedback into BCI systems as a novel mechanism of closing the loop (e.g., see O’Doherty et al., 2011). In addition to providing an alternative means of sensory perception, direct brain stimulation can also be used to bridge the gap across perturbed neural connections (Berger et al., 2011). Studies suggest that neural stimulation may even have the potential to restore functional connectivity and associated behaviors through modulation of molecular mechanisms of synaptic efficacy (Jacobs et al., 2012, Rahman et al., 2013 and Song et al., 2013). In this regard, BCI technologies may not only be useful for enabling function, but also have the potential for implementation as a therapeutic device for restoringfunction.

A primary application of BCI is to provide a mechanism for movement or communication by patients who are unable to move or communicate through normal pathways. Such approaches have included the translation of recorded neural signals associated with sensory and goal-directed mechanisms into navigation or selection commands, enabling the user to move through a virtual or real environment or to select letters to type for purposes of communication (Thurlings et al., 2010). Other approaches have included decoding of neural signals directly associated with the intent to move (Collinger et al., 2013 and Doud et al., 2011) or speak (Pei et al., 2011). In addition to approaches that leverage correlates of user intent, BCI has been utilized to provide neurofeedback to users, enabling them to regulate neural and behavioral functions normally not under volitional control. Such functions include attention, pain, emotion, and memory (Birbaumer et al., 2009).

While people living with injury remain a primary end-user target population for the field of BCI, the increasing availability of portable hardware for real-time non-invasive sensing of neural activity has also led to the development of commercial BCI applications for healthy individuals, as seen by recent incorporation of BCI within the gaming industry. BCI games have used neural signals to control or influence functions such as steering through virtual environments, changing the form and function of avatars, or controlling the movement of a virtual ball through collaboration or competition among multiple users (for review, see Coyle et al., 2013 and Marshall et al., 2013).

Recent advances in the field of BCI have been achieved via a broad spectrum of funding sources across academic, industry, clinical, and various international government organizations. The current review, however, is focused on BCI research funded by the Defense Advanced Research Projects Agency (DARPA). Established in 1958 in response to the Soviet launch of the world’s first satellite, Sputnik, DARPA’s mission is to maintain technological superiority of the United States military and prevent technological surprise by U.S. adversaries (Defense Advanced Research Projects and Agency, 2013). To achieve this mission, DARPA invests in revolutionary, high-risk/high-reward research efforts ranging from fundamental scientific discoveries to the application of these discoveries for military use. DARPA’s primary constituents are the military services and American warfighters. The agency’s goal is to provide these constituents with the capabilities to perform their complex duties and to quickly and effectively recover from adverse events.

While DARPA itself does not conduct scientific research, the agency’s program managers and directors, whose expertise spans diverse scientific and military fields, are highly immersed in the scientific research community as well as the U.S. military community. Through interactions with these communities, DARPA assesses current needs and state-of-the art scientific and technological achievements and identifies areas in which groundbreaking advances could revolutionize national security capabilities. Through its programs, DARPA funds research and development efforts conducted by a broad spectrum of industry, academic, and other government organizations. These efforts range from fundamental scientific exploration to development of prototype technological devices with specific end-user applications. Additionally, DARPA facilitates transition and operationalization of successful results from its programs for military and commercial use.

In recent years, DARPA has supported highly innovative research in the field of neuroscience, fostering multi-disciplinary collaborations among neurobiologists, neuropsychologists, mathematicians, and engineers. The goals of these efforts span four major challenges:

• Detect – Develop diagnostics, models, and devices to characterize and mitigate threats to the human brain.
• Emulate – Leverage inspiration from functional brain networks to efficiently synthesize information.
• Restore – Reestablish behavioral and cognitive function lost as a result of injury to the brain or body.
• Improve – Develop brain-in-the loop systems to accelerate training and improve functional behaviors.

Of relevance to this special issue, many of DARPA’s investments in neuroscience have encompassed the development of novel BCI technologies. These DARPA-funded efforts have enabled new neural interface technologies for detecting multi-scale and multi-region brain function in real time, as well as complex mathematical algorithms that emulate the translation of neural activity into activity in downstream brain areas and resulting behavioral functions. Together, these neural interfaces and mathematical models are integrated into BCI systems that can restore and/or facilitate near – natural neural and behavioral function.

DARPA’s initial investments in BCI began in 1974 under the Close-Coupled Man/Machine Systems (later renamed Biocybernetics) program. This program investigated the application of human physiological signals, including brain signals as measured non-invasively using either EEG or magnetoencephalography (MEG), to enable direct communication between humans and machines and to monitor neural states associated with vigilance, fatigue, emotions, decision-making, perception, and general cognitive ability. The program yielded notable advancements, such as detailed understanding of single-trial, sensory-evoked responses in the EEG of human participants. These efforts demonstrated that neural activity in response to visual checkerboard stimuli, alternating at different frequencies at each of four fixation points, could be decoded in real time and used to navigate a cursor through a simple maze ( Vidal, 1977). In 2002 DARPA took a deeper dive into the field of BCI by launching its Brain Machine Interface (BMI) program, shortly followed by the Human Assisted Neural Devices (HAND) program. These early programs tackled a wide array of BCI challenges including sensorimotor control of prosthetic devices ( Carmena et al., 2003), facilitation of memory encoding ( Song et al., 2007), decoding of visual inputs ( Hung et al., 2005), development of dynamic neural decoding algorithms ( Gage et al., 2005), as well as the development of new devices for high-resolution neural imaging ( Vetter et al., 2004). These DARPA-funded efforts provided many of the foundational discoveries and technologies that have enabled more recent developments in this field.

This review highlights several recent and ongoing DARPA-funded programs that are aimed at utilizing BCI to either restore neural and behavioral function following injury to the brain, or to improve human performance through intervention during training or operational tasks. Notably, under President Obama’s Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative, announced in April 2013, DARPA is currently supporting new research efforts aimed at the development of novel BCI technologies for restoring function in human clinical populations with either neuropsychiatric or memory dysfunction. The goals of these new programs will be described further in the conclusion of this review.

2. DARPA BCI efforts to restore neural and behavioral function

Recent and ongoing DARPA programs supporting the development of BCI technologies to restore neural and behavioral function include Revolutionizing Prosthetics, Reorganization and Plasticity to Accelerate Injury Recovery (REPAIR), Restorative Encoding Memory Integration Neural Device (REMIND), andReliable Neural Interface Technology (RE-NET). These programs are complementary and synergistic, leveraging novel techniques to interface with the nervous system, providing new fundamental approaches to modeling the nervous system, and enabling direct communication with the brain, body, and environment. For the application of actuation, Revolutionizing Prosthetics is translating state of the art BCI systems to restore sensorimotor function in humans, and REPAIR is utilizing animal models to advance neural decoder capabilities through the incorporation of multi-scale, dynamic models that account for the brain’s plastic changes underling sensorimotor function during learning or following injury. The BCI system developed by the REMIND program targets a different neurobehavioral system – memory – and has demonstrated the improvement and restoration of performance on memory tasks in animal models. Finally, the RE-NET program is addressing challenges involved in developing safe, robust BCI systems for chronic use and is applicable to a broad spectrum of BCI applications.

2.1. Revolutionizing Prosthetics

The Revolutionizing Prosthetics program began in 2006 with the vision of restoring near-natural dexterity for people with loss of upper-limb control. The objective was to allow Wounded Warrior amputees to improve quality of life, maximize function and independence, enable activities of daily living, and return to service (if desired). DARPA embarked on this challenge in response to the increased incidence of amputations and injuries to the nervous system suffered by service members. Major upper extremity disabilities are a significant problem for the Department of Defense (DoD). Between 2000 and 2011, there were nearly 6000 amputations of service members within the US Armed Forces, with over two-thirds of these instances involving upper extremity amputations ( O’Donnell, 2012). Approximately 16.5% of amputees returned to active duty, with return-to-duty rates of single amputees reaching 20% ( Stinner et al., 2010). Therefore, there is a need for functional solutions that enable service members to deliver high performance. In addition to amputees, the Revolutionizing Prosthetics program also serves individuals with loss of upper extremity function as a result of spinal cord injury (SCI). It is estimated that there are nearly 300,000 individuals in the US living with SCI with approximately 12,000 new cases every year ( National Spinal Cord Injury Statistical Center, 2013).

Prior to DARPA’s investments in this area, there were few options for military personnel suffering with these disorders. Remarkably, one of the most commonly used solutions to upper extremity loss was the split hook prosthetic developed in 1912 (Dorrance, 1912). Evaluation of the state-of-the-art revealed that very little progress had been made in prosthetic innovation since this time with meager advances with the 1938 Becker design (Becker, 1942), National Academy of Science Artificial Limb Program of 1945 (Furman, 1962), and the available one-degree-of-freedom (DOF) hands that were on the market in the early 2000s. DARPA responded to the lack of advanced prosthetic limb options for upper-extremity amputees with a two-pronged development strategy. Both efforts focused on developing modular arm systems that could provide support to a variety of amputees including transradial, transhumeral, as well as full shoulder disarticulation. Development of these prosthetic limbs involved highly demanding specifications that mimicked attributes and capabilities of real human arms including weight, shape, and grip strength. Two teams of investigators, DEKA and The Johns Hopkins University Applied Physics Laboratory (JHU/APL), took on the task of designing and assembling these next generation arms to meet the program demands (Johannes et al., 2011 and Resnik et al., 2013). The advanced DARPA arm systems, one with ten and the other with seventeen miniature motors, enabled replication of near-natural hand and arm movements and are available to the research and clinical communities. Additionally, a virtual arm system is available for prototyping (Armiger et al., 2011 and Collinger et al., 2014).

In addition to the arms themselves, the Revolutionizing Prosthetics program also produced innovations in the user control interface. Since the diversity in loss of upper extremity function was high among military personnel, the control interfaces had to provide multiple options such that the user could choose the solution most appropriate to their own needs, thus leading to personalized medicine. The first control interface developed under the Revolutionizing Prosthetics program was a non-invasive control modality that does not require surgical procedures. It consisted of inertial measurement units that could be placed on the shoes as well as pressure/bump switches that could be attached to the torso. Movement of the inertial units enabled actuation of all of the degrees of freedom while the bump switches enabled changes in the arm modes (hand grips, for example). The use of these interfaces was well received by prosthetic limb users, and over 7000 h among 77 amputees and quadriplegics have been logged during pilot testing. Other viable approaches for peripheral control of these arm systems include the electromyogram (EMG), targeted muscle reinnervation (TMR), as well as nerve interfaces such as implanted myoelectric sensors (IMES) (for a review, see Ortiz-Catalan et al., 2012). In May 2014, the DEKA arm system received U.S. Food and Drug Administration (FDA) approval. Initial input control modalities in this approval include inertial and EMG control. Efforts are ongoing to make these systems available to military and civilian personnel.

Beyond the use of inertial or EMG control, Revolutionizing Prosthetics has provided transformative innovations in direct brain control of prosthetic limbs that enabled human users to think about moving in much the same way they would control their own arm to actuate the prosthetic arm systems (Collinger et al., 2013). This line of research and development involved real-time recording and decoding of motor cortical signals to provide research participants with tetraplegia the ability to control up to ten DOF with the prosthetic arm systems. The main enablers for near-natural control include micro-electrode arrays for recording brain signals and complex algorithms to translate neural activity into commands for the motors throughout the prosthetic arm system. In the Revolutionizing Prosthetics program, single unit neuronal recordings have been achieved in a human clinical patient via the implantation of two intracortical microelectrode arrays (Blackrock Microsystems, Salt Lake City, UT), each with 96 electrode shanks. The 4 mm × 4 mm assembly was implanted in the participant’s motor cortex (M1) and connected percutaneously through the use of two head-mounted pedestals. Using preoperative structural and functional MRI and magnetoencephalography (MEG) to identify hand and finger activation areas in M1 for the purpose of decoding grasp behaviors, the arrays were implanted 14 mm apart through the use of a stereotaxic surgical navigation system (Collinger et al., 2014).

The combined signals from these arrays allowed for the simultaneous recording of over 250 unique single units, which were then processed in real-time to ultimately send commands to move the JHU/APL prosthetic limb. The recorded signals passed through a Blackrock Microsystems NeuroPort data acquisition system, which converted neuronal firing rate (30-ms bins) into a functional mapping for prosthetic limb commands in endpoint velocity space. Real-time visual feedback from the prosthetic limb to the participant enabled closed-loop control. Using this system, the participant achieved control of the arm in three DOF (endpoint of the wrist) within two weeks of implantation, and began operating on seven DOF within five weeks (Collinger et al., 2013). Depending on the types of signals acquired from the brain (single neuron vs. electrocorticogram (ECoG)), new population vector decoding methodologies and shared control architectures needed to be developed to allow users to initialize their control of the system and then adaptively learn to obtain increasing control of a greater number of degrees of freedom (e.g., see Collinger et al., 2013). Applying such methodologies to novel behavioral paradigms, Revolutionizing Prosthetics efforts have further elucidated the neural mechanisms underlying human-tool interaction. These discoveries have enabled a deeper understanding of how the brain represents motor control, environmental cues, object interaction, and perception of neuroprosthetic control (Hauschild et al., 2012 and Collinger et al., 2013). Performance was established both in terms of completion of various reach and grasp tasks, and also by way of evaluation against functional metrics such as the Action Research Arm Test (ARAT), a clinical outcome measure derived from the stroke rehabilitation community (Lyle, 1981).

To continue to push the frontier of the intersection between brain science and technology and deliver the most natural arm systems, DARPA researchers have laid the foundation for adding the next generation of neuroprosthetic control by exploring the restoration of the sense of touch. Constructing closed-loop complete sensorimotor systems is essential for identifying objects, manipulating objects, and even grasping objects in the absence of vision. Some of the first steps in developing these next generation interfaces have involved investigating how the non-human primate brain encodes sensory information provided via natural means (tactile stimulation of the subject’s own fingers), and comparing that to the psychometric evaluation of the encoding of sensory information delivered through cortical stimulation (Tabot et al., 2013 and Zaaimi et al., 2013). These efforts explored simple percepts of touch, as well as complex encoding of slip and texture. For the Revolutionizing Prosthetics program, a series of experiments were performed at the University of Chicago to demonstrate safety of chronically implanted stimulating electrode arrays in the somatosensory cortex of non-human primates. The implant configuration consisted of two 100-channel, sputtered iridium oxide film (SIROF)-tipped Utah electrode arrays (UEA) connected via Cereport connectors to a CereStim R96 stimulator, all manufactured by Blackrock Microsystems. Stimulation was delivered to three non-human primates at 300 Hz across a range of charge amplitudes, duty cycles, and interval durations. Sensory stimulation was performed for 4 h per day over a period of six months. The results revealed no deficits in fine motor control and demonstrated safety of the electrode-tissue interface (Chen et al., 2014). In tandem with the safety study, an efficacy study was also performed at University of Chicago to characterize the relationship between mechanical and electrical stimulation on tactile tasks (Berg et al., 2013).

Using both a 96-electrode SIROF-tipped UEA implanted in the hand representation of Brodmann’s area 1 and two 16-electrode Floating Microelectrode Arrays (MicroProbes for Life Sciences, Gaithersburg, MD) targeting the hand region in Brodmann’s area 3b, the implants and corresponding stimulation via a CereStim stimulator were used in a series of electrical detection tasks and compared against mechanical detection tasks. It was demonstrated that electrical stimulation sent in response to tactile stimulation of the prosthetic finger showed equivalent detection performance to mechanical stimulation of the native finger, with a psychometric curve function defining the relationship between mechanical and electrical sensation for use in subsequent stimulation experiments. This combined suite of safety and efficacy data has been critical in the support of FDA Investigational Device Exemption (IDE) approval for testing in human clinical populations. The ultimate vision for transitioning these efforts for clinical use is to enable signals from sensors on prosthetic fingers to be translated into stimulation signals delivered directly to the sensory cortex, enabling patients to ‘feel’ when their prosthetic hand touches objects. This transition from visually driven closed-loop control to full sensorimotor closed-loop control (see Fig. 1) is anticipated to enable increased user control of prosthetic limbs, with faster response times and near-natural sensation during performance of tasks with occluded views or those that require tactile feedback. It is hoped that these advances will continue to improve independence and quality of life after injury in users of prosthetic limbs.

Full-size image (58 K)
Fig. 1. Idealized bidirectional brain–computer interface for closed-loop prosthetic control. Neural correlates of motor intent are recorded from electrode arrays implanted in motor areas of the brain such as the primary motor cortex (M1). The signals are decoded and used to control the movement of a prosthetic arm. Sensors on the robotic arm detect information on touch (via contacts with external objects) and/or proprioception (via movement and position of the prosthetic limb). Outputs from these sensors are then converted to patterns of stimulus pulses that are delivered via implanted electrode arrays to sensory regions of the brain, such as primary somatosensory cortex (S1). Reprinted from “Restoring sensorimotor function through intracortical interfaces: progress and looming challenges,” by S.J. Bensmaia and L.E. Miller, 2014, Nature Reviews Neuroscience, 15, p. 315. Copyright 2014 by Nature Publishing Group. Reprinted with permission.

2.2. Reorganization and Plasticity to Accelerate Injury Recovery (REPAIR)

Although the BCI technologies developed under the Revolutionizing Prosthetics program have proved quite remarkable in enabling direct neural control of robotic limbs, the neural decoding algorithms do not capitalize on one of the brain’s fundamental characteristics – plasticity. Importantly, brain function is not fixed or static, but rather it adapts in response to learning new information (such as meeting a new person) or new behavioral skills (such as riding a bicycle). Moreover, these learning processes are subserved by brain activity at multiple spatial scales, ranging from the activity of single neurons to coordinated patterns of activity across small and large networks of neurons, and ultimately, to behavior. Such changes also occur across multiple temporal scales, encompassing millisecond level functional changes as well as structural changes that can result in alterations in neural activity over days, weeks, months, or longer. In addition to adaptation of the brain in response to learning, the brain’s function can also be dynamically altered by injury, either of the brain itself, such as in the case of traumatic brain injury (TBI), or of the body – for instance, amputees no longer have normal sensations of touch that are sent to the brain’s sensory regions, and they can no longer use the brain’s motor control systems to directly move their affected limb (e.g., see Pohlmeyer et al., 2014).

The goal of DARPA’s REPAIR program, initiated in 2010 and projected to continue through 2015, is to develop a multi-scale, biologically accurate model of neural function that accounts for the brain’s adaptation over time. The computational models developed under REPAIR have focused on sensorimotor function, that is, how the brain learns to use sensory information such as touch or vision to generate appropriate reaching and grasping behaviors in order to perform complex tasks (e.g., see Sanchez et al., 2012, Shenoy and Nurmikko, 2012 and Andersen et al., 2012). Under this program, new neural interface tools have been designed to detect and alter the activity of large populations of neurons in awake, behaving non-human primates across multiple spatial and temporal scales. Neural recordings from these interfaces have been used to develop and validate computational models that emulate dynamic functions of the brain and resulting behaviors. Through highly collaborative efforts across eight universities, the computational models developed under REPAIR are being integrated with one another and interfaced directly with the brain for the purpose of restoring neural and behavioral function following neural injury or sensory deprivation. The final dynamic model developed under the REPAIR program is anticipated to be incorporated into a co-adaptive BCI system that optimally facilitates sensorimotor function in a non-human primate by enabling adaptation of both the in silico model and biological brain. While research efforts conducted under REPAIR have utilized only animal models, the new technologies and scientific insights will undoubtedly provide a foundation for the development of novel therapeutic devices for human clinical populations.

Computational models developed under REPAIR include biomimetic models that mimic the properties of neuronal firing and dynamic connectivity across networks of neurons ( Kerr et al., 2012). Such models have been integrated with higher-level biomimetic models of reinforcement learning that the brain undergoes while learning to perform new tasks ( Mahmoudi and Sanchez, 2011 and Neymotin et al., 2013). REPAIR researchers have also developed low-dimensional models that predict state space trajectories of population level neural activity involved in planning and execution of sensorimotor behaviors ( Shenoy et al., 2013 and Ames et al., 2014). Additionally, new sophisticated algorithms have been developed that decode neural representations of force related variables, such as torque, in addition to traditional kinematic variables (e.g., position), resulting in more natural sensorimotor prosthetic control and the ability to compensate for novel dynamic environments, compared to traditional decoders based on kinematic variables alone ( Chhatbar and Francis, 2013). REPAIR investigators are developing and testing their models by recording neural activity in animals performing complex behavioral tasks, and, importantly, determining whether the models correctly predict multi-scale changes in brain activity and behavior when certain aspects of the brain’s activity are temporarily perturbed. Many of these studies are investigating neural correlates of complex behaviors in freely moving non-human primates and have been made possible by recent developments in wireless neural interfaces ( Foster et al., 2012 and Borton et al., 2013).

 To perform precise, reversible perturbations of brain activity, REPAIR researchers have leveraged and expanded upon recent developments in optogenetics, enabling expression of bioengineered opsins (light-sensitive channels) within the cell membranes of specific types of neurons ( Diester et al., 2011 and Zalocusky and Deisseroth, 2013). These opsins undergo modified configurations in response to specific wavelengths of light, altering the thresholds for neuronal action potentials, and ultimately resulting in an increase or decrease of neuronal firing activity in the presence of specific colors of light. Moreover, new neural interface hardware developed under the REPAIR program has enabled optical neuromodulation simultaneous with electrical recording of single and multi-unit activity in awake, behaving non-human primates (e.g., see Fig. 2) ( Shenoy and Nurmikko, 2012 and Ozden et al., 2013). These new developments have been of utmost importance in the REPAIR program to facilitate modeling of the brain’s dynamic responses to perturbations of neural activity.
Full-size image (97 K)
Fig. 2. Coaxial optrode for simultaneous electrical recording and optogenetic neuromodulation. (a) Coaxial optrode cross-sectional schematic (top) and photograph (bottom). (b) Scanning electron microscope images of coaxial optrode tip. (c) Full-length photograph of coaxial optrode. (d) Photograph of coaxial optrode depicting reinforcing thin stainless steel tube (310 μm diameter) and tissue penetrating portion of shaft (165 μm diameter). (e) Performance of the coaxial optrode is shown by optically modulated, in vivo electrophysiological recordings from somatosensory cortices in an anesthetized mouse (transgenic Thy1-ChR2/YFP, left), anesthetized rat (transduced with viral construct AAV5-CAMKIIα-C1V1-eYFP, middle), and awake behaving non-human primate (transduced with AAV5-CAMKIIα-C1V1-eYFP, right). Blue and green bars indicate timeframes of light delivery (at 473 nm for the mouse and 561 nm for the rat and non-human primate, respectively), highlighting light-induced increases in neuronal spiking activity. (For interpretation of the references to color in figure legend, the reader is referred to the web version of the article.) Adapted from “A coaxial optrode as multifunction write-read probe for optogenetic studies in non-human primates,” by I. Ozden, J. Wang, Y. Lu, T. May, J. Lee, W. Goo, D. J. O’Shea, P. Kalanithi, I. Diester, M. Diagne, K. Deisseroth, K. V. Shenoy, and A. V. Nurmikko, 2013, Journal of Neuroscience Methods, 219(1), p. 144 & 148. Copyright 2014 by Elsevier. Adapted with permission.

While traditional closed-loop BCI systems provide feedback to the user, REPAIR researchers have incorporated the ability to provide feedback to both the user and the BCI system’s neural decoder using adaptive models of brain function based on reinforcement learning. Some of these models have been interfaced directly with the brain, paving the way for enabling both the brain and the model to adapt to one another and to learn new behavioral tasks through closed-loop BCI performance ( Marsh et al., 2013 and Pohlmeyer et al., 2014). Recent REPAIR efforts have developed the foundation for a fully autonomous, closed-loop BCI that uses real-time recorded neural activity from the non-human primate brain, both to directly control virtual reaching movements, and also to provide feedback that enables the system to automatically “learn” how to decode the brain’s motor output signals ( Marsh et al., 2013). In typical BCI systems, the neural decoder that controls virtual or robotic motor outputs must be trained on how to effectively translate signals from the brain’s motor cortical regions into appropriate motor commands. However, REPAIR researchers discovered that neurons in the motor cortex also respond differentially depending on whether or not the virtual reaching movements accurately reflect the animal’s intent. The researchers demonstrated that these biological “critic” signals can be used as feedback to the BCI in order to iteratively update the motor output decoder so that its output more accurately reflects the animal’s intent. Such an approach is a radical departure from traditional BCIs that use kinematic signals for feedback. This new functionality has the potential to dramatically increase the flexibility of BCIs to quickly and automatically adapt to performance in dynamic environments, including performing tasks in changing environmental conditions, learning completely new tasks, and adapting to the brain’s structural and functional changes that occur during injury and recovery.

2.3. Co-adaptive BCI for restoration of sensorimotor function

While the efforts described above are focused largely on invasive BCI technologies, an additional DARPA Small Business Innovation Research (SBIR) effort is supporting a collaborative effort between researchers at Advanced Brain Monitoring, Inc. and the University of Miami to develop a BCI system that utilizes non-invasively recorded EEG signals in patients with spinal cord injury to enable, and potentially even restore, movement of their paralyzed arms (Roset et al., 2013). To achieve this goal, the patient imagines opening or closing his or her hand while EEG signals are recorded. These EEG signals, particularly those generated in motor cortical areas associated with motor imagery, are detected, decoded, and used to trigger the activation of a functional electrical stimulation (FES) device worn on the patient’s arm and hand. The FES device non-invasively delivers pulses of electrical stimulation to nerves innervating the muscles of the arm and hand. Depending on the locations of these pulses, the device can stimulate the patient’s hand to either open or close. Importantly, if the patient perceives that the output of the FES device is not consistent with the patient’s intent, the patient’s brain generates “error potentials” in the ongoing EEG signal. Similar to the autonomous BCI concept investigated via invasive measures under the REPAIRprogram (described above), these non-invasively recorded error potentials are fed back to the closed-loop BCI system, enabling automatic adjustment of the algorithms that decode the brain’s motor EEG signals. In this way, the system effectively “learns” the neural correlates underlying the user’s intentions to move, thus restoring behavioral function. It remains to be determined whether long-term use of this BCI system can also restore disrupted connectivity between the central and peripheral nervous systems, ultimately improving the ability of patients with spinal cord injury to make limb movements without reliance on the BCI.

2.4. Restorative Encoding Memory Integration Neural Device (REMIND)

While a majority of BCI efforts have focused on restoring sensory information (e.g., restoration of sight or hearing via retinal or cochlear implants) or motor behaviors (e.g., through peripheral or central nervous system control of computer cursor movements or prosthetic limbs), very few have attempted to develop a cognitive prosthesis. One such unique effort, initiated in 2002 under DARPA’s Brain–Machine Interface (BMI) and Human Assisted Neural Devices (HAND) programs, and later supported by DARPA’s REMINDprogram (2009–2014), focused on the development of a BCI system for memory restoration. Under this program, collaborative efforts between teams led by researchers at Wake Forest University and University of Southern California demonstrated the ability to detect patterns of functional brain connectivity in the hippocampus and prefrontal cortex associated with successful memory encoding and retrieval, respectively. These naturally occurring spatiotemporal firing patterns across brain regions are emulated by a non-linear, multi-input, multi-output (“MIMO”) model, which inputs neural firing patterns from one brain region and outputs predicted firing patterns in a downstream brain area ( Song et al., 2007 and Song et al., 2009). Moreover, in their investigation of neural correlates of behavioral performance on the memory task, the researchers discovered that the spatiotemporal patterns of hippocampal activity during memory encoding vary with respect to (1) whether the encoded item or event will subsequently be remembered or forgotten after a delay period, and (2) the nature of the item or event being encoded (in the rodent studies, encoded events are either a right or left lever press).

Taking their results a step (or rather, a leap) further, the researchers translated outputs of their computational model into electrical stimulation patterns that mimic the hippocampal firing patterns associated with correct encoding of a specific event. When applied to the hippocampus during memory encoding, the stimulation significantly improved, on a greater number of trials, the ability of rodents to subsequently remember an event (right vs. left lever press) following a long delay period (Berger et al., 2011 and Hampson et al., 2012b). Importantly, when the investigators examined the effects of reversed stimulation (i.e., stimulating with a biomimetic pattern associated with successfully encoding a given lever press, but while the animal was actually pressing the opposite lever), they found that the animals’ performance accuracy dropped below normal levels, further suggesting specificity of these stimulation patterns.

More recently, REMIND investigators demonstrated improvement of memory encoding and retrieval of items and their locations by non-human primates through stimulation of the hippocampus and prefrontal cortex, respectively ( Hampson et al., 2012c and Hampson et al., 2013). Achieving this challenge was largely enabled by newly developed techniques for conformal recording in the non-human primate brain described in this special issue ( Opris et al., 2014 and Fetterhoff et al., 2014). The biomorphic microelectrode arrays described by Opris et al. in this special issue have enabled conformal electrophysiological recordings as well as neurochemical measurements of glutamate concentration within cortical microcircuits, enabling the researchers to assess effects of electrical stimulation as well as molecular correlates of task performance in non-human primates ( Opris et al., 2012 and Opris et al., 2014).

Using the rodent model, REMIND researchers investigated the effect of hippocampal stimulation on restoration of memory performance in rodents following infusion of the glutamate receptor antagonist MK801 into the hippocampus ( Berger et al., 2011). The MK801 infusion reversibly disrupted the signaling between two regions of the hippocampus that are crucial for memory encoding and drastically decreased behavioral performance on the memory task. However, when the animals received model-derived hippocampal stimulation, their performance on the memory task was in large part recovered. The researchers demonstrated similar restoration of performance by applying model-derived stimulation patterns to the prefrontal cortex of non-human primates following reversible pharmacological disruption of activity in this brain region ( Hampson et al., 2012c).

During their initial studies, the REMIND investigators derived memory prosthetic stimulation patterns for a given animal based on a model of that animal’s own brain activity. However, they later discovered that aspects of these spatiotemporal patterns of brain activity exhibited remarkable consistency across animals ( Deadwyler et al., 2013). To demonstrate the applicability of the model outputs across animals, the researchers trained one subset of rodents to be “experts” on the memory task – that is, these animals performed very well on the task, even when they had to remember the event for delays up to 60 s (which is quite difficult for a rat). Another subset of “delay-naïve” animals was trained on the task, but these animals were not trained to remember events across time delays of any length. When presented with trials containing a delay period between the initial lever press and the non-match response, the delay-naïve animals were not able to respond correctly. However, in a novel “donor-recipient” paradigm, in which an expert rat and a delay-naïve rat performed the memory task concurrently in separate chambers, hippocampal activity was recorded from the expert rat during memory encoding and translated in real-time to stimulation patterns that were immediately applied to the hippocampus of the delay-naïve rat. The performance of delay-naïve rates on long-delay trials was significantly improved, as compared to long-delay trials in which no stimulation was delivered. The researchers also leveraged the similarity of hippocampal firing patterns across over 40 rodents to develop a “generic” stimulation pattern, that, when applied to the hippocampus of a different animal during memory encoding, significantly boosted task performance ( Hampson et al., 2012a). These studies, combined with the pharmacological disruption studies described above, raise the question of whether “generic” stimulation patterns may also be leveraged to recover memory performance in individuals with memory impairments resulting from neural injury or dysfunction.

Interestingly, Hampson et al. observed that applying model-derived stimulation patterns to the rodent hippocampus during memory encoding resulted in behavioral improvement of task performance not only on stimulated trials, but also to a lesser extent on intervening trials in which stimulation was not applied (Hampson et al., 2012a and Hampson et al., 2012b). Moreover, performance on non-stimulated trials remained elevated for at least a week after stimulation was terminated. Such improvements in behavioral performance were also associated with increased prevalence of neural activity patterns that had been found to underlie the successful encoding of right or left lever presses on difficult trials (i.e., those with long delays). While neuronal mechanisms of stimulation-induced plasticity have not been directly investigated in the context of memory restoration, a new computational framework described in this special issue may provide the capability to model long-term changes in functional neuronal connectivity based on nonlinear, dynamic associations between spiking patterns of multiple neurons (Song et al., 2014). This computational modeling capability could provide key insights for future investigations of the effects of BCI usage on neural plasticity.

2.5. Reliable Neural Interface Technology (RE-NET)

With the remarkable success demonstrated under DARPA’s BMI, HAND, and Revolutionizing Prostheticsprograms, the formation of multi-disciplinary research teams such as BrainGate, and the advancement of BCI study at universities across the globe, researchers began to notice that the longevity of their BCI study capability varied dramatically. Initially, the vast majority of cortical implants were used in acute animal studies aimed at understanding and decoding brain activity. Some researchers transitioned from acute to chronic studies and were able to maintain partial cortical implant recording capability for years, while other researchers experienced interface failures much earlier.

The neural prosthesis research community began to form initial hypotheses as to the cause of these failures through histological analysis of explanted devices and surrounding tissue (Biran et al., 2007 and He and Bellamkonda, 2008). However, hypothesis testing proved challenging given the large number of variables to control, including surgical protocols, animal models, decoding approaches, and electrode technologies. Neural interfaces that are unreliable present a significant challenge to the clinical translation of these devices and can also delay progress in basic neuroscience research. Therefore, understanding the mechanisms of these failures became critically important. Today, overcoming interface failure is arguably the largest remaining hurdle that researchers striving to understand and decode brain function must achieve. Without overcoming these issues, the vision of life-long BCI systems will remain just outside of reach.

The Reliable Neural Interface Technology (RE-NET) program was established to identify and solve problems of extracting neural information from the nervous system at the scale and rate necessary to control high-performance prosthetic-limbs for chronic periods of time ( Judy, 2012). This DARPA effort contained three separate components designed to identify and eliminate interface failure. The first effort,Histology for Interface Stability over Time (HIST), initiated in 2010, was developed to characterize and quantify neural signal detection failure associated with chronically implanted electrode arrays. The Reliable Central nervous system Interfaces (RCI) and Reliable Peripheral Interfaces (RPI) programs, both of which began in 2011, were aimed at preventing interface failures, advancing motor decoding performance of neural interface technologies placed in the brain, peripheral nerves, and muscles, and restoring somatosensory function.

The ability to detect neural activity is dependent on the quality of the device. The HIST program evaluated both biotic and abiotic device failure modes, that is, the biological or tissue responses that may be related to the disrupted or reduced ability to detect neural activity ( Karumbaiah et al., 2013 and Saxena et al., 2013), as well as the device-related or system-related failures such as material, manufacturing, or system interconnects ( Prasad and Sanchez, 2012 and Prasad et al., 2014).

RCI efforts developed and demonstrated novel materials, electrodes, and interface systems to increase the reliability and functional duration of cortical interface systems (Tien et al., 2013). Additional efforts under this program have focused on emulating and subsequently decoding motor control signals from functional brain networks in order to control advanced prosthetic devices. One such effort, presented in this special issue, involved the development of a novel virtual reality system for non-human primates capable of emulating all the natural movements of a healthy limb (Putrino et al., 2014). This RCI outcome will enable researchers to demonstrate reliable high-bandwidth control of unconstrained, many-degree-of-freedom movements.

Restoration of function has long-been the leading goal of researchers developing novel medical devices. The RPI effort improved peripheral nervous system recording capability through the use of penetrating and non-penetrating electrodes placed in spinal and peripheral nerves. Additional developments by the RPIprogram included novel peripheral interface devices, advanced targeted muscle re-innervation techniques ( Abidian et al., 2012 and Baghmanli et al., 2013), new algorithms for on-line decoding of motor control signals, and new methods for naturalistic sensory percept generation through modulation of peripheral nerve electrical stimulation parameters. In this special issue, RE-NET RPI researchers describe the development of a peripheral interface system to record and stimulate at various locations across multiple nerves ( Thota et al., 2014). In addition, one service member at Walter Reed National Military Medical Center has become the first amputee to receive implantable technology in effort to improve control of his prosthetic device. This work is planned for expansion and holds great promise for the future of prosthetic interface technology and the restoration of upper-limb function in amputees, as described by Pasquina et al. in this special issue ( Pasquina et al., 2014).

RE-NET efforts are projected to continue through 2015 and will build on the momentum of the program with a heavy emphasis on demonstration of life-long neural interface systems. The renewed focus on the peripheral nervous system has produced a level of prosthetic control above projected capability. Peripheral nerves and muscles are ideal targets for amputees since peripheral nerve surgeries are routinely performed in this population; therefore, inclusion of nerve implants poses little added risk. Novel approaches to record and stimulate sensory nerves are needed to restore touch and proprioception. RE-NET will continue to capitalize on this momentum by developing advanced peripheral interface devices with chronic, stable signal capture and decoding capabilities. Moreover, these systems will provide amputees with prosthetic sensory feedback to further improve sensorimotor control. Ultimately, these advances are intended to increase the extent of embodiment of neurally controlled prosthetic devices so that such devices will be widely accepted and utilized regularly by clinical populations. Achieving this vision is the main thrust of DARPA’s new Hand, Proprioception, & Touch Interfaces (HAPTIX) program which is projected to begin in late 2014 and run until 2020.

3. DARPA BCI efforts to improve human training and performance

While the DARPA-funded efforts featured above are focused on development of BCI interfaces ultimately aimed at restoring neural and behavioral function, DARPA has also made notable investments in the development of BCI systems intended to improve training and performance of healthy individuals. TheAccelerated Learning program has developed novel training paradigms, including those leveraging BCI, to accelerate improvements in human performance. The Narrative Networks (N2) program is developing new techniques to quantify the effect of narratives on human cognition and behavior, including initial development of a closed-loop BCI system that adapts a narrative in response to a listener’s EEG signals. Such a system would have numerous applications to training and human performance domains. TheNeurotechnology for Intelligence Analysts (NIA) and Cognitive Technology Threat Warning System(CT2WS) programs both have utilized non-invasively recorded “target detection” brain signals to improve the efficiency of imagery analysis and real-time threat detection, respectively. The Low-cost EEG Technologies effort aims to develop more affordable EEG recording systems, thus expanding the reach of BCI development opportunities to a broader community of both professional and amateur neuroscientists. These DARPA programs have funded scientific advances that range from fundamental new discoveries about the functions of the human brain to the development of new human-in-the-loop systems that leverage non-invasively recorded neural responses to optimize human performance.

3.1. Accelerated Learning

The Accelerated Learning program (2007–2012) sought to revolutionize learning in the military environment through the development of reliable and quantitative methods for measuring, tracking, and accelerating skills acquisition. The program primarily focused on detecting non-invasively measured neural and other physiological correlates of task learning, with an end goal of producing a two-fold increase in an individual’s learning rate. In Phase I of the program, researchers conducted fundamental research to identify neural correlates of task learning and to develop a proof of principle that these findings could be leveraged to accelerate learning. The methods used included neurophysiologically driven training regimes, neurally optimized stimuli, and development of feedback interventions delivered through a closed-loop BCI. Phase II focused on demonstrating a two-fold increase in the rate of learning in a military population on an operationally relevant task. Complementary components that further aided in attaining the goals of the program included development of neurally based techniques for maintenance of acquired skills, prediction of skill acquisition based on real-time neural activity, and the ability to present neural state status in the form of sensory feedback to accelerate an individual’s learning progression and thus improve human performance.

Efforts under Accelerated Learning demonstrated that it was possible to leverage brain imaging, EEG, and other neurophysiological measures to quantitatively characterize physiological states reflective of novice, intermediate, and expert levels of performance. Researchers also succeeded in identifying specific brain regions, activity patterns, and networks associated with the acquisition of complex tasks, utilizing these findings to accelerate learning of those tasks by a factor of two. Relevant to this review, Advanced Brain Monitoring, Inc. developed a suite of adaptive and interactive neuro-educational technologies (Interactive Neuro-Educational Technology, or I-NET®) to accelerate skill learning and incorporated these tools into a closed-loop system known as the Adaptive Peak Performance Trainer (APPT®) ( Raphael et al., 2009). This system includes four main components: integration of real-time EEG into closed-loop tutorials, identification of psychophysiological characteristics of expertise using Advanced Brain Monitoring’s wireless EEG acquisition system ( Berka et al., 2004), development of sensor-based feedback to deliver real-time physiological state feedback, and identification of neurocognitive factors that are predictive of skill acquisition to enable early interventions. Rifle marksmanship training was used as the militarily relevant task, as it is a core military skill that involves both classroom learning and field practice requiring sensory, motor, and cognitive skills. The APPT® system incorporates knowledge of EEG, electrocardiography (ECG), respiration rate, and eye tracking signatures of learning stages. The system can provide continuous physiological monitoring and feedback (visual, auditory, or haptic) to the trainee in real-time through integration of algorithms that derive physiological state changes based on sensor inputs. A preliminary study suggests that use of the I-NET/APPT increased the learning trajectory of novice participants by a factor of 2.3, compared to novice participants who trained with an identical protocol without the APPT® (Behneman et al., 2012). Importantly, the system provides a closed-loop BCI platform for assessing combinations of pre-training interventions designed to accelerate psychophysiological control and combat-relevant skills acquisition.

3.2. Narrative Networks (N2)

Initiated in 2011 and anticipated to continue through 2015, the N2 program was created to develop a quantitative approach to the analysis of narratives and their influence on human cognition and behavior. Narratives exert a powerful influence on human thoughts, emotions, memories, and behavior, and can be particularly important in security contexts ( Casebeer and Russell, 2005). Through an improved basic understanding of narrative effects, tools are being developed to detect brain activity associated with narrative influence and to emulate this activity in the context of larger environmental factors with models of narrative influence on individual and group behavior. These tools will facilitate faster and better communication of information in foreign information operations. To this end, one goal of the program is to create BCI technologies that close the loop between the storywriter and consumer, allowing neural responses to a narrative stimulus to dictate the story’s trajectory. In this way, moment-by-moment neural activity would drive the subsequent story outcome, resulting in an individualized narrative tailored by neural signatures associated with cognitive processes such as attention and empathy.

Since the brain is the proximate cause of behavior, N2 has focused much of its early research on understanding how stories impact the brain. Using non-invasive brain imaging techniques such as EEG and functional magnetic resonance imaging (fMRI), researchers have detected various brain responses to particular story elements and correlated these responses with emotional, attitudinal, and behavioral effects in the story consumer. When an individual listens to a compelling story, particular patterns of EEG activity have been shown across individuals to correlate with levels of sensory engagement, empathy, and narrative cohesion in the listener. Similarly, while watching suspenseful video clips, individuals display characteristic fMRI patterns that correlate with moments of high narrative transportation, when the listener feels more transported into the narrative world. These decoded patterns of neural activity are correlated with behavioral changes induced by the narrative. For example, investigators have predicted with high accuracy whether or not an individual will donate to a given charity after watching a related narrative, based on their neural responses during the video.

In conflict resolution and counterterrorism scenarios, detecting the neural response underlying empathy induced by stories is of critical importance. N2 researchers have explored how narratives can reinforce in-group and out-group memberships and induce profound empathy gaps between members of these groups. In exploration of the neural code representing physical pain and emotional suffering, investigators used a series of 96 text-based stories, which varied in the levels of physical pain and emotional suffering experienced by the protagonist. While subjective ratings of pain and suffering were highly correlated across the various stories, the neural responses to these measures were distinct, even within the pain network of the brain ( Bruneau et al., 2013). Therefore, measures of neural activity may give us a more precise window into the feelings and emotions induced by narrative consumption.

Having detected a number of neural states associated with narrative influence, investigators are using this information to develop novel brain-in-the-loop systems to improve narrative creation and delivery. Efforts by Advanced Brain Monitoring initiated under N2 include the development of a closed-loop BCI system that integrates the human audience member into the storytelling process by recording the story recipient’s EEG during presentation of a narrative video. Brain activity recordings are outputted to the system, and based on algorithms designed to detect the neural states previously mentioned, the story is intended to branch into alternative scenarios when certain states reach threshold. For example, if engagement levels drop below an optimal level, the narrative would take an alternate branch designed to increase aspects of story engagement. In this way, this narrative testbed could be used to create optimal narratives tailored to a specific individual or group of people. Notably, Advanced Brain Monitoring’s system is designed to synchronize multiple users’ neural inputs into the closed-loop system ( Stevens et al., 2012), allowing for the potential to enable group-level activity to influence the narrative outcomes. Clear applications for this BCI technology exist not only for N2-related goals but also in the education and entertainment industries.

3.3. Neurotechnology for Intelligence Analysts (NIA)

The goal of the NIA program (2005–2014) was to develop new BCI systems utilizing non-invasively recorded brain signals to significantly increase the efficiency and throughput of imagery analysis. In recent years, operational technologies have been developed that have exponentially increased collection and storage capabilities of intelligence data, including overhead imagery collected via satellites and remotely piloted aircraft. The increase in the availability of enormous imagery datasets has led to the challenge of effectively searching through and analyzing the imagery in a timely manner with limited analytical resources. The current operational procedure for broad area search for targets in overhead imagery is an extremely time- and labor-intensive brute force method, and current computer vision approaches to imagery analysis are nowhere close to matching the target detection capabilities of the human visual system.

The neuroscientific basis of the BCI systems developed under NIA is that, upon seeing a target of interest, a unique set of neural responses, including an event-related potential known as a “P300,” is elicited in a person’s non-invasively recorded EEG. This P300 response is typically observed approximately 300 ms following presentation of a target image and appears as a positive-going deflection in the EEG signal, compared to the signal elicited by non-target images. Neural signatures of target detection have been observed in response to targets in images presented sequentially at rapid rates of up to 20 images per second ( Sajda et al., 2003). At such fast presentation rates it is not feasible to match target images with behavioral responses such as button presses, which are generally slower than 300 ms and quite variable in response time.

The NIA systems encompassed a “human-in-the-loop” approach that leveraged the use of new decoding algorithms to detect neural signatures of target detection on a single-trial basis (see Fig. 3). Satellite images were divided into small segments and then presented to imagery analysts at rates between 0.5 and 10 images per second while the analysts’ brain responses were measured with EEG ( Qian et al., 2009,Sajda et al., 2010 and Macdonald et al., 2011). The NIA systems’ decoding classifiers were designed to automatically detect spatiotemporal features within the EEG signal associated with viewing targets as compared to non-target images. Additionally, the NIA target detection classifiers integrated computer vision, eye-tracking data, and/or EEG correlates of attention. Based on output from the systems’ classification algorithms, image segments were either given a prioritization score based on their calculated probability of containing a target, or marked as potential target images based on the classifier’s threshold levels of brain responses. Satellite imagery locations most likely to contain targets were then presented back to the user for final verification and assessment. Importantly, given that target detection brain responses are largely independent of target type or modality, the BCI systems developed under the NIAprogram are able to immediately adapt to changes in operational requirements, such as searching for new and/or multiple classes of targets in different imagery modalities. Thus, the NIA approach eliminates the need for cumbersome system parameter adjustments required for detection of new types of targets by systems relying solely on computer vision approaches. In a series of formal evaluations conducted on three of the NIA systems with over 40 professional imagery analysts in 2008–2009, use of the NIA systems resulted in up to a 10-fold increase in analysis throughput (area of imagery analyzed per unit time) with no loss of target detection sensitivity, as compared to the analysts’ performance using their standard imagery analysis approaches.

Full-size image (70 K)
Fig. 3. General workflow procedures implemented by the Neurotechology for Intelligence Analysts BCI systems to detect targets of interest in satellite imagery. (1) The analyst user is fitted with an EEG cap containing electrodes that non-invasively record the analyst’s neural signals. (Photo courtesy of Neuromatters, LLC). (2) Overhead imagery is divided into segments or “chips,” which are then (3) presented to the user in rapid succession (0.5–10 images per second). (4) Each image chip containing one or more targets of interest elicits a distinctive neural response in the analyst, which is measured by the electrodes in the analyst’s EEG cap. (5) The NIA system decodes the analyst’s neural signals and automatically prioritizes locations within the image that are most likely to contain targets. Using standard imagery analysis software, these image locations are presented to the analyst for review, validation, and further annotation. 

3.4. Cognitive Technology Threat Warning System (CT2WS)

Like the NIA program, CT2WS, initiated in 2007, funded the development of a non-invasive BCI system that detects the human user’s “target detection” brain responses including the P300, as measured by EEG ( Khosla et al., 2011 and Weiden et al., 2012). Rather than detecting targets in overhead imagery, however, the goal of CT2WS is to detect potential threats during real-time surveillance operations. For the purposes of demonstration, the program focused on surveillance from a forward operating base, for which threats included vehicles (mounted forces) and individuals on foot (dismounted forces). The system, developed by a research team at HRL Laboratories, LLC, utilizes advanced flat-field, wide-angle optics and high pixel-count digital imagers to record real-time surveillance video ( Huber et al., 2013). Neuromorphic computer vision algorithms designed to emulate the brain’s visual system are used to detect potential threats within the video ( Khosla et al., 2013). The system then presents still frames containing these potential threats in rapid succession to the user (a method referred to as Rapid Serial Visual Presentation or RSVP). The user’s neural signals enable further classification of potential threats; when a threat is detected, a visual cue is provided to the user, alerting the user to focus on that image.

During a weeklong evaluation at Camp Roberts, CA in 2012, the threat detection performance of theCT2WS system (four cameras with a 120-degree field-of-view, see Fig. 4) was compared side by side to the performance of the Cerberus Scout, a state-of-the-art commercial surveillance system used in theater by Army and Marine Corps units. The CT2WS system demonstrated a probability of threat detection almost twice that of the Scout (91% and 53%, respectively), and an extremely low rate of five false alarms per hour out of more than 2300 events per hour (not measured for Scout, but substantially higher). Also in 2012, a pilot study with five subjects indicated that using the CT2WS system reduced the operator workload (measured by the mean for the NASA-TLX categories of mental demand, physical demand, temporal demand, performance, effort, and frustration) by a factor of two compared to the unaided eye.

Full-size image (49 K)
Fig. 4. Equipment used to demonstrate threat detection performance of the CT2WS BCI system at Camp Roberts, CA in 2012. (a) A four-camera system recorded real-time surveillance video with 120-degree field-of-view and 120-megapixel resolution. (b) EEG cap (Advanced Brain Monitoring B-Alert X24, 20 EEG channels) used to non-invasively record operators’ neural activity while viewing rapidly presented visual stimuli.

The CT2WS program is also funding the development of dry-electrode EEG headsets that would improve the usability of fieldable BCI systems for applications such as threat detection (described above). Traditional EEG headsets rely on conductive paste, gel, or saline solution to non-invasively record the brain’s electrical signals. As a consequence, these devices are often cumbersome to apply and unappealing to the user, given the wetness or residue that remains on the user’s scalp and hair following removal of the headset. Off-the shelf, dry-electrode headsets have been previously developed, primarily for the gaming industry; however, the signal-to-noise ratio of these systems is too low to reliably detect many EEG components of interest for neuroscience efforts aimed at improving human training and performance, particularly when single trial analysis is required. Additionally, currently available dry electrode headsets are often more susceptible to interference such as motion artifacts, compared to systems utilizing paste, gel, or saline. The CT2WS program developed three novel dry-electrode EEG headsets (see Fig. 5), all wireless, with sensor channels ranging in number from 20 to 32, and weight ranging from 40 to 635 g ( Chi et al., 2012, Chi et al., 2013, Davis et al., 2013 and Halford et al., 2013). Empirical tests of the dry-electrode headsets and a wet-electrode baseline system compared P300 response performance on image stimuli, “oddball” image stimuli, video stimuli, steady state visually evoked potential, auditory evoked potential, and artifacts including eyes open, eyes closed, blinks, and jaw clench. All three systems correlated reasonably well to the baseline. Further research and development is needed to optimize system robustness to motion artifacts.

Full-size image (43 K)
Fig. 5. Three dry-electrode EEG caps designed and tested under the CT2WS program. (Left) B-Alert X24 Wireless EEG Headset System designed by Advanced Brain Monitoring, Inc. (ABM), with 24 standard wet sensors replaced with semi-dry, hydrogel-based sensors. (Middle) Dry-electrode EEG system developed by Quantum Applied Science and Research, Inc. (QUASAR), containing 20 hybrid resistive-capacitive dry-electrode sensors. (Right) EEG system developed by researchers at the University of California at San Diego (UCSD), containing 32 dry-electrode sensors in a soft, adjustable headset.

3.5. Low-cost EEG Technologies

The CT2WS effort demonstrated that dry-electrode headsets enable collection of EEG data with high signal-to-noise ratios, suitable for research grade data collection. However, these cutting-edge systems are typically too expensive and difficult for non-neuroscientists to use. In the last few years, commercial companies have developed lower-cost EEG headsets, but these systems often have poor signal-to-noise ratios or encrypt the data stream, requiring substantial investment to gain full data access. In 2013, a DARPA SBIR program was launched to address this EEG cost, data quality, and usability gap. The goal of this SBIR effort is to facilitate brain-in-the-loop research by developing a low-cost ($30, price of parts), research grade EEG system and accompanying software to enable usage by non-neuroscientists. Such a system would allow for large-scale crowdsourced neuroimaging research, not currently feasible. The neuroscientist community has leveraged crowdsourcing for large volume imagery data analysis with projects such as EyeWire and KNOSSOS ( Marx, 2013). Likewise, the psychologist community has leveraged crowdsourcing for large volume survey and behavioral data collection with tools like Amazon’s Mechanical Turk and Qualtrics. A $30 EEG system would enable similar crowdsourcing data collection efforts for neuroimaging as well.

Development to date has focused on a number of different approaches. With regard to the sensors, teams have explored 3-D printing of dry electrodes directly to the headgear. For electronics development, others have adopted a completely open source approach to the design, iterating with the citizen scientist community for improved electronics’ designs. Still others have created augmented reality software and apps to aid in the donning and use of the EEG devices.

This SBIR effort is anticipated to continue through 2015 and aims to develop a truly enabling technology. With respect to the President’s BRAIN initiative, novel BCI technologies are needed that not only extend what information can be extracted from the brain, but also who is able to conduct and participate in those studies. Applications exist not only in the crowdsourced neuroscience domain, but also in education, entertainment, and a myriad of possible BCI-related projects.

4. Future efforts and challenges for brain–computer interface technologies

Given the latest advances in brain–computer interfaces, a number of challenges remain in optimizing the capabilities, robustness, and usability of such systems, including the development of BCI devices for human use. One needed capability advancement is the development of enhanced recording techniques that enable real-time measurements of neural activity and structure across a wide range of spatial scales (from one to millions of neurons) and temporal scales (from milliseconds to years). A new DARPA program entitled Neuro Function, Activity, Structure, and Technology (NeuroFAST) aims to develop novel optical methods to enable real-time functional recording of thousands of neurons, with single-neuron resolution, in awake, behaving animals. Likewise, to effectively implement BCI systems for clinical use by individuals with less severe clinical cases or who are unwilling to undergo the risks of neurosurgery, or for enabling efficient performance by healthy users, there exists a need for the development of subcutaneous and fully non-invasive neural interfaces that are both portable and capable of recording activity from cortical and deep brain structures at high spatial and temporal resolution.

As new technologies enable progressively more neural, physiological, and behavioral data to be collected, the need for improved computational processing and mathematical dimensionality reduction techniques will increase exponentially in order to facilitate effective analysis and use of the data. These needs are especially essential for closed-loop BCI systems that rely on automated, real-time analysis of recorded neural signals. Effective analysis and integration of data across research groups will also require revolutionary new approaches to data sharing, including standardization of protocols and data formats, ability to effectively merge data across multiple modalities and scales, and incentive structures that promote sharing of data and collaborative efforts across individual laboratories. Notably, for new efforts under DARPA’s brain function research portfolio, high priority is being placed on the sharing of newly developed protocols and resulting data. While DARPA has long placed an emphasis on collaboration across multi-disciplinary research teams, selection criteria for new efforts also include requirements for sharing data among DARPA-funded teams, as well as making data available to the broader scientific research community at an accelerated pace.

With respect to the ultimate goal of transitioning BCI systems for chronic use by humans, the need for robust interface hardware and computational models – beyond the advances made by the DARPA-funded programs described above – is becoming increasingly important. Implanted neural interface hardware must remain functional and biologically compatible over years and even decades to reduce the need for multiple surgical procedures. Additionally, these systems should enable the application of new software updates without requiring surgical removal of implanted hardware. While surgical risks are not applicable to non-invasive BCI hardware, non-invasive systems are associated with a distinct set of challenges, including the need to minimize system set-up time, as well as the need for robustness in changing environmental conditions and against variation in sensor placement across multiple uses. Computational model components of BCI systems should be compatible with neural plasticity over chronic use and should function across various contexts and associated brain states, without the requirement for extensive and/or frequent calibration procedures. The systems must also remain robust in the midst of external interference, such as electrical interference (e.g., while using a mobile phone), and must include sufficient safeguards to prevent deliberate interference by unauthorized individuals.

While a majority of state-of-the-art, closed-loop BCI systems translate recorded neural signals into functions performed by a machine – for instance, the movement of a computer cursor or robotic limb – recent BCI efforts have begun to utilize recorded neural activity to generate feedback signals that are delivered directly to the user through stimulation of muscles, nerves, or the brain itself. Advances in both invasive and non-invasive neural interfaces, as well as computational models, are needed to enable precisely targeted stimulation with high spatial and temporal resolution to the brain or periphery of humans to enable (1) substitution for lost sensory inputs (e.g., prosthetic sensation of touch for amputees), (2) immediate correction of dysfunctional networks (e.g., abnormal neural firing detection and mitigation), and (3) long-term restoration of healthy functional networks through leveraging of the brain’s natural plasticity mechanisms.

Two new programs under DARPA’s brain function research portfolio were established in 2014 to quantitatively characterize the complex neuronal networks underlying both healthy and aberrant cognitive and behavioral function, and to develop devices that deliver advanced therapeutic neural stimulation to human clinical populations. The Systems-Based Neurotechnology for Emerging Therapies (SUBNETS)program aims to address the problem that state-of-the-art methodologies for treating complex neuropsychiatric and neurologic disorders involve imprecise surgical, pharmacological, psychotherapy, or deep brain stimulation (DBS) approaches that are implemented through a slow, trial-and-error based process. This is a particular challenge for the U.S. military, as in recent years, veterans receiving mental health care from the Department of Veterans Affairs (VA) have constituted almost a third of the total number of veterans receiving health care from the VA ( U.S. Government Accountability Office, 2011). TheSUBNETS program will attempt to develop novel neural interfaces and therapeutic approaches for human patients with intractable illnesses by developing fully implantable medical devices for multi-site, systems-based neural recordings, deriving new computational models to characterize the distributed neural interactions that underlie a variety of neuropsychiatric and neurological conditions, and delivering safe, targeted therapeutic neural stimulation through closed-loop BCI.

The vision of DARPA’s Restoring Active Memory (RAM) program is to develop computational models that quantitatively characterize the biological underpinnings of the complex organization of memories in the human brain, and to integrate these models into a therapeutic device for targeted memory restoration in human patients with intractable illnesses who are suffering from memory deficits. This clinical population is highly relevant to the U.S. military, given that traumatic brain injury frequently results in deficits in retrieving memories formed prior to the injury and/or forming and retaining memories of new experiences following injury onset. The RAM program focuses on restoring declarative memories – the memories of facts and events that can be explicitly recalled. To achieve the goal of declarative memory restoration, the RAMprogram aims to fund the development of new neural interface hardware that can target multi-scale neural underpinnings of declarative memory in human patients with high spatial and temporal resolution. It is anticipated that the data collected via these neural interfaces will be leveraged to develop and validate mathematical algorithms that characterize the neural correlates of successfully (and unsuccessfully) encoding and/or retrieving specific types or attributes of memory. The computational models developed under RAM are envisioned to be integrated into a closed-loop BCI system that can restore specific types or attributes of memory through the use of targeted neural stimulation.

As both SUBNETS and RAM involve the application of new technologies to human clinical populations, DARPA has been working closely with the FDA. These collaborations between DARPA and the FDA have led to new innovation pathways for device development ( U.S. Food and Drug Administration, 2011a and U.S. Food and Drug Administration, 2011b). Continued interaction among government agencies, clinical investigators, and technologists are expected to facilitate the establishment of a variety of new indications, outcome metrics, and endpoints for neuroprosthetic devices.

DARPA is committed to ensuring that the efforts it funds follow all laws and regulations designed to protect humans and animals involved in scientific research. In addition, DARPA has established a panel of individuals with expertise in ethical, legal, and societal implications (ELSI) of neuroscientific research to reflect on and inform DARPA efforts in this domain, and to facilitate communication between DARPA and relevant stakeholder communities including the neuroscience and bioethics communities. ELSI panelists consider the potential downstream implications of agency-funded neuroscientific developments, including those relating to safety, privacy, foreign policy and security (see, e.g., Casebeer, 2013, for a framework discussion), recognizing that some such implications may unfold over years or even decades following DARPA’s initial investments. More broadly, the agency is committed to fostering public awareness and understanding of the potential applications of its efforts to advance neuroscience technologies and the field of BCI.

The future of brain–computer interfaces depends upon multidisciplinary collaborations among neuroscientists, psychologists, clinicians, engineers, and mathematicians, and upon ongoing communication with relevant stakeholder communities including regulators, physicians, and patients. Effectively bringing together these diverse fields and communities will facilitate the development of new tools to measure, quantify and model dynamic, multi-scale brain activity and will ultimately lead to novel therapeutic regimens and training paradigms capable of restoring and improving neural, cognitive, and behavioral function.


The opinions expressed herein belong solely to the authors. They do not represent and should not be interpreted as being those of or endorsed by the Defense Advanced Research Projects Agency, the Department of Defense, or any other branch of the federal government. The durations listed for the DARPA programs mentioned are estimates only; programs may be adjusted at any time due to shifting agency priorities and the availability of funds.

Public release statement

DARPA Distribution Statement “A” (Approved for Public Release, Distribution Unlimited).


Abidian et al., 2012
M.R. Abidian, E.D. Daneshvar, B.M. Egeland, D.R. Kipke, P.S. Cederna, M.G. Urbanchek
Hybrid conducting polymer-hydrogel conduits for axonal growth and neural tissue engineering
Adv Healthc Mater, 1 (6) (2012), pp. 762–767

View Record in Scopus | Full Text via CrossRef | Citing articles (19)
Ames et al., 2014
K.C. Ames, S.I. Ryu, K.V. Shenoy
Neural dynamics of reaching following incorrect or absent motor preparation
Neuron, 81 (2) (2014), pp. 438–451

Article | PDF (2439 K) | View Record in Scopus | Citing articles (6)
Andersen et al., 2012
R.A. Andersen, M.H. Schieber, N. Thakor, G.E. Loeb
Natural and accelerated recovery from brain damage: experimental and theoretical approaches
IEEE Pulse, 3 (2) (2012), pp. 61–65 2011.2181093

View Record in Scopus | Full Text via CrossRef
Armiger et al., 2011
R.S. Armiger, F.V. Tenore, W.E. Bishop, J.D. Beaty, M.M. Bridges, J.M. Burck, et al.
A real-time virtual integration environment for neuroprosthetics and rehabilitation
Johns Hopkins APL Tech Dig, 30 (3) (2011), pp. 198–206

View Record in Scopus | Citing articles (5)
Baghmanli et al., 2013
Z. Baghmanli, K.B. Sugg, B. Wei, B.S. Shim, D.C. Martin, P.S. Cederna, et al.
Biological and electrophysiologic effects of poly(3,4-ethylenedioxythiophene) on regenerating peripheral nerve fibers
Plast Reconstr Surg, 132 (2) (2013), pp. 374–385

View Record in Scopus | Full Text via CrossRef
Becker, 1942
Becker DB. Patent No. US2285885 A. United States; 1942.
Behneman et al., 2012
A. Behneman, C. Berka, R. Stevens, B. Vila, V. Tan, T. Galloway, et al.
Neurotechnology to accelerate learning: during marksmanship training
IEEE Pulse, 3 (1) (2012), pp. 60–63

View Record in Scopus | Full Text via CrossRef | Citing articles (3)
Berg et al., 2013
J.A. Berg, J.F. Dammann, F.V. Tenore, G.A. Tabot, J.L. Boback, L.R. Manfredi, et al.
Behavioral demonstration of a somatosensory neuroprosthesis
IEEE Trans Neural Syst Rehabil Eng, 21 (3) (2013), pp. 500–507

View Record in Scopus | Full Text via CrossRef | Citing articles (16)
Berger et al., 2011
T.W. Berger, R.E. Hampson, D. Song, A. Goonawardena, V.Z. Marmarelis, S.A. Deadwyler
A cortical neural prosthesis for restoring
J Neural Eng, 8 (4) (2011), p. 046017

Full Text via CrossRef
Berka et al., 2004
C. Berka, D.J. Levendowski, M. Cvetinovic, M.M. Petrovic, G.F. Davis, M.N. Lumicao, et al.
Real-time analysis of EEG indices of alertness, cognition and memory acquired with a wireless EEG headset
Int J Human–Comput Interact, 17 (2) (2004), pp. 151–170

View Record in Scopus | Full Text via CrossRef | Citing articles (129)
Biran et al., 2007
R. Biran, D.C. Martin, P.A. Tresco
The brain tissue response to implanted silicon microelectrode arrays is increased when the device is tethered to the skull
J Biomed Mater Res A, 82 (1) (2007), pp. 169–178

View Record in Scopus | Full Text via CrossRef | Citing articles (117)
Birbaumer et al., 2009
N. Birbaumer, A.R. Murguialday, C. Weber, P. Montoya
Neurofeedback and brain–computer interface: clinical applications
Int Rev Neurobiol, 86 (2009), pp. 107–117

Article | PDF (267 K) | View Record in Scopus | Citing articles (41)
Borton et al., 2013
D.A. Borton, M. Yin, J. Aceros, A. Nurmikko
An implantable wireless neural interface for recording cortical circuit dynamics in moving primates
J Neural Eng, 10 (2) (2013), p. 026010

Full Text via CrossRef
Bruneau et al., 2013
E. Bruneau, N. Dufour, R. Saxe
How we know it hurts: item analysis of written narratives reveals distinct neural responses to others’ physical pain and emotional suffering
PLOS ONE, 8 (4) (2013), p. e63085

Full Text via CrossRef
Carmena et al., 2003
J.M. Carmena, M.A. Lebedev, R.E. Crist, J.E. O’Doherty, D.M. Santucci, D.F. Dimitrov, et al.
Learning to control a brain–machine interface for reaching and grasping by primates
PLoS Biol, 1 (2) (2003), p. E42

Casebeer, 2013
W.D. Casebeer
Ethics and the biologized battlefield: moral issues in 21st-century conflict
R.E. Armstrong, M.D. Drapeau, C.A. Loeb, J.J. Valdes (Eds.), Bio-inspired innovation and national security, National Defense University Press, Washington, DC (2013), pp. 293–303

Casebeer and Russell, 2005
W.D. Casebeer, J.A. Russell
Storytelling and terrorism: towards a comprehensive “counter-narrative strategy”
Strateg Insights, 4 (3) (2005), pp. 1–16

Chen et al., 2014
K.H. Chen, J.F. Dammann, J.L. Boback, F.V. Tenore, K.J. Otto, R.A. Gaunt, et al.
The effect of chronic intracortical microstimulation on the electrode-tissue interface
J Neural Eng, 11 (2) (2014), p. 026004

Full Text via CrossRef
Chhatbar and Francis, 2013
P.Y. Chhatbar, J.T. Francis
Towards a naturalistic brain–machine interface: hybrid torque and position control allows generalization to novel dynamics
PLOS ONE, 8 (1) (2013), p. e52286

Full Text via CrossRef
Chi et al., 2012
Y.M. Chi, Y.T. Wang, Y. Wang, C. Maier, T.P. Jung, G. Cauwenberghs
Dry and noncontact EEG sensors for mobile brain–computer interfaces
IEEE Trans Neural Syst Rehabil Eng, 20 (2) (2012), pp. 228–235 2011.2174652

View Record in Scopus | Full Text via CrossRef | Citing articles (38)
Chi et al., 2013
Y.M. Chi, Y. Wang, Y.T. Wang, T.P. Jung, T. Kerth, Y. Cao
A practical mobile dry EEG system for human computer interfaces
D.D. Schmorrow, C.M. Fidopiastis (Eds.), Foundations of augmented cognition, Springer, Heidelberg (2013), pp. 649–655

View Record in Scopus | Full Text via CrossRef
Chuang et al., 2014
A.T. Chuang, C.E. Margo, P.B. Greenberg
Retinal implants: a systematic review
Br J Ophthalmol (2014)

Collinger et al., 2014
J.L. Collinger, M.A. Kryger, R. Barbara, T. Betler, K. Bowsher, E.H. Brown, et al.
Collaborative approach in the development of high-performance brain–computer interfaces for a neuroprosthetic arm: translation from animal models to human control
Clin Transl Sci, 7 (2014), pp. 52–59

View Record in Scopus | Full Text via CrossRef | Citing articles (3)
Collinger et al., 2013
J.L. Collinger, B. Wodlinger, J.E. Downey, W. Wang, E.C. Tyler-Kabara, D.J. Weber, et al.
High-performance neuroprosthetic control by an individual with tetraplegia
Lancet, 381 (9866) (2013), pp. 557–564

Article | PDF (942 K) | View Record in Scopus | Citing articles (119)
Coyle et al., 2013
D. Coyle, J. Principe, F. Lotte, A. Nijholt
Guest editorial: brain/neuronal–computer game interfaces and interaction
IEEE Trans Comput Intell AI Games, 5 (2) (2013), pp. 77–81 2013.2264736

View Record in Scopus | Full Text via CrossRef
Dadarlat et al., 2013
M.C. Dadarlat, J.E. O’Doherty, P.N. Sabes
A learning-based approach to artificial sensory feedback: intracortical microstimulation replaces and augments vision
6th International IEEE EMBS Conference on Neural Engineering, San Diego, CA (2013)

Davis et al., 2013
G. Davis, C. McConnell, D. Popovic, C. Berka, S. Korszen
Soft, embeddable dry EEG sensors for real world applications
D.D. Schmorrow, C.M. Fidopiastis (Eds.), Foundations of augmented cognition, Springer, Berlin/Heidelberg (2013), pp. 269–278

View Record in Scopus | Full Text via CrossRef
Deadwyler et al., 2013
S.A. Deadwyler, T.W. Berger, A.J. Sweatt, D. Song, R.H. Chan, I. Opris, et al.
Donor/recipient enhancement of memory in rat hippocampus
Front Syst Neurosci, 7 (2013), p. 120

Defense Advanced Research Projects and Agency, 2013
Defense Advanced Research Projects Agency
Driving technological surprise: DARPA’s mission in a changing world

Diester et al., 2011
I. Diester, M.T. Kaufman, M. Mogri, R. Pashaie, W. Goo, O. Yizhar, et al.
An optogenetic toolbox designed for primates
Nat Neurosci, 14 (3) (2011), pp. 387–397

View Record in Scopus | Full Text via CrossRef | Citing articles (126)
Dorrance, 1912
Dorrance DW. Patent No. US1042413 A. United States; 1912.
Doud et al., 2011
A.J. Doud, J.P. Lucas, M.T. Pisansky, B. He
Continuous three-dimensional control of a virtual helicopter using a motor imagery based brain–computer interface
PLoS ONE, 6 (10) (2011), p. e26322

Full Text via CrossRef
Fetterhoff et al., 2014
D. Fetterhoff, I. Opris, S.L. Simpson, S.A. Deadwyler, R.E. Hampson, R.A. Kraft
Multifractal analysis of information processing in hippocampal neural ensembles during working memory under Δ9-tetrahydrocannabinol administration
J Neurosci Methods (2014) pii: S0165-0270(14)00264-7

Fetz, 1969
E.E. Fetz
Operant conditioning of cortical unit activity
Science, 163 (3870) (1969), pp. 955–958

View Record in Scopus | Citing articles (183)
Foster et al., 2012
J.D. Foster, P. Nuyujukian, O. Freifeld, S.I. Ryu, M.J. Black, K.V. Shenoy
A framework for relating neural activity to freely moving behavior
Conf Proc IEEE Eng Med Biol Soc, 2012 (2012), pp. 2736–2739 2012.6346530

View Record in Scopus | Full Text via CrossRef | Citing articles (1)
Furman, 1962
B. Furman
Progress in prosthetics
US Department of Education, Health and Welfare, Washington, DC (1962)

Gage et al., 2005
G.J. Gage, K.A. Ludwig, K.J. Otto, E.L. Ionides, D.R. Kipke
Naïve coadaptive cortical control
J Neural Eng, 2 (2) (2005), pp. 52–63

View Record in Scopus | Full Text via CrossRef | Citing articles (56)
Géléoc and Holt, 2014
G.S. Géléoc, J.R. Holt
Sound strategies for hearing restoration
Science, 344 (6184) (2014), p. 1241062

View Record in Scopus | Full Text via CrossRef | Citing articles (5)
Gilja et al., 2011
V. Gilja, C.A. Chestek, I. Diester, J.M. Henderson, K. Deisseroth, K.V. Shenoy
Challenges and opportunities for next-generation intracortically based neural prostheses
IEEE Trans Biomed Eng, 58 (7) (2011), pp. 1891–1899 2011.2107553

View Record in Scopus | Full Text via CrossRef | Citing articles (50)
Graimann et al., 2010
B. Graimann, B. Allison, G. Pfurtscheller
Brain–computer interfaces: a gentle introduction
B. Graimann, B. Allison, G. Pfurtscheller (Eds.), Brain–computer interfaces, Springer, Berlin (2010), pp. 1–27

View Record in Scopus | Citing articles (12)
Halford et al., 2013
J. Halford, C.G. Waters, G. Martz, E. Kutluay, B.C. Dean, W. Soussou
Quality assessment of electroencephalography obtained from a “dry electrode” system in veterans
67th Annual Meeting of the American Epilepsy Society, Washington, DC (2013), p. 3.084

Hampson et al., 2012c
R.E. Hampson, G.A. Gerhardt, V. Marmarelis, D. Song, I. Opris, L. Santos, et al.
Facilitation and restoration of cognitive function in primate prefrontal cortex by a neuroprosthesis that utilizes minicolumn-specific neural firing
J Neural Eng, 9 (5) (2012), p. 056012

Full Text via CrossRef
Hampson et al., 2012a
R.E. Hampson, D. Song, R.H. Chan, A.J. Sweatt, M.R. Riley, G.A. Gerhardt, et al.
A nonlinear model for hippocampal cognitive prosthesis: memory facilitation by hippocampal ensemble stimulation
IEEE Trans Neural Syst Rehabil Eng, 20 (2) (2012), pp. 184–197

View Record in Scopus | Full Text via CrossRef | Citing articles (15)
Hampson et al., 2012b
R.E. Hampson, D. Song, R.H. Chan, A.J. Sweatt, M.R. Riley, A.V. Goonawardena, et al.
Closing the loop for memory prosthesis: detecting the role of hippocampal neural ensembles using nonlinear models
IEEE Trans Neural Syst Rehabil Eng, 20 (4) (2012), pp. 510–525

View Record in Scopus | Full Text via CrossRef | Citing articles (5)
Hampson et al., 2013
R.E. Hampson, D. Song, I. Opris, L.M. Santos, D.C. Shin, G.A. Gerhardt, et al.
Facilitation of memory encoding in primate hippocampus by a neuroprosthesis that promotes task-specific neural firing
J Neural Eng, 10 (6) (2013), p. 066013

Full Text via CrossRef
Hauschild et al., 2012
M. Hauschild, G.H. Mulliken, I. Fineman, G.E. Loeb, R.A. Andersen
Cognitive signals for brain–machine interfaces in posterior parietal cortex include continuous 3D trajectory commands
Proc Natl Acad Sci USA, 109 (42) (2012), pp. 17075–17080

View Record in Scopus | Full Text via CrossRef | Citing articles (20)
He and Bellamkonda, 2008
W. He, R.V. Bellamkonda
A molecular perspective on understanding and modulating the performance of chronic central nervous system (CNS) recording electrodes
W.M. Reichert (Ed.), Indwelling neural implants: strategies for contending with the in vivo environment, CRC Press, Boca Raton, FL (2008) [Chapter 6]. Available from:

Hebert et al., 2013
J. Hebert, J. Olson, M. Morhart, M. Dawson, P. Marasco, T. Kuiken, et al.
Novel targeted sensory reinnervation technique to restore functional hand sensation after transhumeral amputation
IEEE Trans Neural Syst Rehabil Eng (2013) 2013.2294907

Huber et al., 2013
D.J. Huber, D. Khosla, Y. Chen, D.J. Van Buer, K. Martin
Low power multi-camera system and algorithms for automated threat detection
Proc SPIE, 8740 (2013) 874006 13

Hung et al., 2005
C.P. Hung, G. Kreiman, T. Poggio, J.J. DiCarlo
Fast readout of object identity from macaque inferior temporal cortex
Science, 310 (5749) (2005), pp. 863–866

View Record in Scopus | Full Text via CrossRef | Citing articles (270)
Jacobs et al., 2012
M. Jacobs, A. Premji, A.J. Nelson
Plasticity-inducing TMS protocols to investigate somatosensory control of hand function
Neural Plast, 2012 (2012), p. 350574

Johannes et al., 2011
M.S. Johannes, J.D. Bigelow, J.M. Burck, S.D. Harshbarger, M.V. Kozlowski, T. Van Doren
An overview of the developmental process for the modular prosthetic limb
Johns Hopkins APL Tech Dig, 30 (3) (2011), pp. 207–216

View Record in Scopus | Citing articles (25)
Johnson et al., 2013
L.A. Johnson, J.D. Wander, D. Sarma, D.K. Su, E.E. Fetz, J.G. Ojemann
Direct electrical stimulation of the somatosensory cortex in humans using electrocorticography electrodes: a qualitative and quantitative report
J Neural Eng, 10 (3) (2013), p. 036021

Full Text via CrossRef
Judy, 2012
J.W. Judy
Neural interfaces for upper-limb prosthesis control: opportunities to improve long-term reliability
IEEE Pulse, 3 (2) (2012), pp. 57–60 2011.2181026

View Record in Scopus | Full Text via CrossRef | Citing articles (8)
Kar and Krekelberg, 2012
K. Kar, B. Krekelberg
Transcranial electrical stimulation over visual cortex evokes phosphenes with a retinal origin
J Neurophysiol, 108 (8) (2012), pp. 2173–2178

View Record in Scopus | Full Text via CrossRef | Citing articles (6)
Karumbaiah et al., 2013
L. Karumbaiah, T. Saxena, D. Carlson, K. Patil, R. Patkar, E.A. Gaupp, et al.
Relationship between intracortical electrode design and chronic recording function
Biomaterials, 34 (33) (2013), pp. 8061–8074

Article | PDF (3853 K) | View Record in Scopus | Citing articles (14)
Kerr et al., 2012
C.C. Kerr, S.A. Neymotin, G.L. Chadderdon, C.T. Fietkiewicz, J.T. Francis, W.W. Lytton
Electrostimulation as a prosthesis for repair of information flow in a computer model of neocortex
IEEE Trans Neural Syst Rehabil Eng, 20 (2) (2012), pp. 153–160 2011.2178614

View Record in Scopus | Full Text via CrossRef | Citing articles (8)
Khosla et al., 2011
D. Khosla, R. Bhattacharyya, P. Tasinga, D.J. Huber
Optimal detection of objects in images and videos using electroencephalography (EEG)
Proc. SPIE, 8050 (2011) 80501C-1

Khosla et al., 2013
D. Khosla, Y. Chen, D.J. Huber, D.J. Van Buer, K. Kim, S.Y. Cheng
Real-time, low power neuromorphic hardware for autonomous object recognition
Proc. SPIE, 8713 (2013), p. 871313

Full Text via CrossRef
Larson and Cheung, 2012
P.S. Larson, S.W. Cheung
Deep brain stimulation in area LC controllably triggers auditory phantom percepts
Neurosurgery, 70 (2) (2012), p. 398-405 discussion 405-6

Lyle, 1981
R.C. Lyle
A performance test for assessment of upper limb function in physical rehabilitation treatment and research
Int J Rehabil Res, 4 (4) (1981), pp. 483–492

View Record in Scopus | Full Text via CrossRef | Citing articles (411)
Macdonald et al., 2011
J.S. Macdonald, S. Mathan, N. Yeung
Trial-by-trial variations in subjective attentional state are reflected in ongoing prestimulus EEG alpha oscillations
Front Psychol, 2 (2011), p. 82

View Record in Scopus | Full Text via CrossRef | Citing articles (2)
Mahmoudi and Sanchez, 2011
B. Mahmoudi, J.C. Sanchez
A symbiotic brain–machine interface through value-based decision making
PLoS ONE, 6 (3) (2011), p. e14760

Full Text via CrossRef
Makin et al., 2013
J.G. Makin, M.R. Fellows, P.N. Sabes
Learning multisensory integration and coordinate transformation via density estimation
PLoS Comput Biol, 9 (4) (2013), p. e1003035

Full Text via CrossRef
Marsh et al., 2013
B. Marsh, A. Tarigoppula, C. Chen, J. Francis
Sensorimotor reward modulation during motion and observation used to implement an actor-critic brain machine interface
NCM 23rd Annual Meeting (2013)

Marshall et al., 2013
D. Marshall, D. Coyle, S. Wilson, M. Callaghan
Games, gameplay, and BCI: the state of the art
IEEE Trans Comput Intell AI Games, 5 (2) (2013), pp. 82–99 2013.2263555

View Record in Scopus | Full Text via CrossRef | Citing articles (7)
Marx, 2013
V. Marx
Neuroscience waves to the crowd
Nat Methods, 10 (2013), pp. 1069–1074

View Record in Scopus | Full Text via CrossRef
May et al., 2013
T. May, I. Ozden, N. Agha, F. Wagner, B. Brush, D.A. Borton, et al.
Behavioral report of optogenetic stimulation of somatosensory cortex hand representation in non-human primates
Society for Neuroscience Annual Meeting, San Diego, CA (2013) p. 644.12/NN1

National Spinal Cord Injury Statistical Center, 2013
National Spinal Cord Injury Statistical Center
Spinal cord injury facts and figures at a glance
(2013) Birmingham, AL

Neymotin et al., 2013
S.A. Neymotin, G.L. Chadderdon, C.C. Kerr, J.T. Francis, W.W. Lytton
Reinforcement learning of two-joint virtual arm reaching in a computer model of sensorimotor cortex
Neural Comput, 12 (2013), pp. 3263–3293

View Record in Scopus | Full Text via CrossRef | Citing articles (1)
O’Doherty et al., 2011
J.E. O’Doherty, M.A. Lebedev, P.J. Ifft, K.Z. Zhuang, S. Shokur, H. Bleuler, et al.
Active tactile exploration using a brain–machine–brain interface
Nature, 479 (7372) (2011), pp. 228–231

View Record in Scopus | Full Text via CrossRef | Citing articles (110)
O’Donnell, 2012
F.L. O’Donnell
Medical surveillance monthly report
Armed Forces Health Surveillance Center, Silver Spring, MD (2012)

Opris et al., 2014
I. Opris, J.L. Fuqua, G.A. Gerhardt, R.E. Hampson, S.A. Deadwyler
Prefrontal cortical recordings with biomorphic MEAs reveal complex columnar-laminar microcircuits for BCI/BMI implementation
J Neurosci Methods (2014) pii: S0165-0270(14)00197-6

Opris et al., 2012
I. Opris, J.L. Fuqua, P.F. Huettl, G.A. Gerhardt, T.W. Berger, R.E. Hampson, et al.
Closing the loop in primate prefrontal cortex: inter-laminar processing
Front Neural Circuits, 6 (2012), p. 88

View Record in Scopus | Citing articles (2)
Ortiz-Catalan et al., 2012
M. Ortiz-Catalan, R. Brånemark, B. Håkansson, J. Delbeke
On the viability of implantable electrodes for the natural control of artificial limbs: review and discussion
Biomed Eng Online, 11 (2012), p. 33

Full Text via CrossRef
Ozden et al., 2013
I. Ozden, J. Wang, Y. Lu, T. May, J. Lee, W. Goo, et al.
A coaxial optrode as multifunction write-read probe for optogenetic studies in non-human primates
J Neurosci Methods, 219 (1) (2013), pp. 142–154

Article | PDF (3085 K) | View Record in Scopus | Citing articles (6)
Pasquina et al., 2014
P.F. Pasquina, M. Evangelista, A.J. Carvalho, J. Lockhart, S. Griffin, G. Nanos, et al.
First-in-man demonstration of a fully implanted myoelectric sensors system to control an advanced electromechanical prosthetic hand
J Neurosci Methods (2014) pii: S0165-0270(14)00267-2

Pei et al., 2011
X. Pei, D.L. Barbour, E.C. Leuthardt, G. Schalk
Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans
J Neural Eng, 8 (4) (2011), p. 046028

Full Text via CrossRef
Putrino et al., 2014
D. Putrino, Y.T. Wong, A. Weiss, B. Pesaran
A training platform for many-dimensional prosthetic devices using a virtual reality environment
J Neurosci Methods (2014) pii: S0165-0270(14)00098-3

Pohlmeyer et al., 2014
E.A. Pohlmeyer, B. Mahmoudi, S. Geng, N.W. Prins, J.C. Sanchez
Using reinforcement learning to provide stable brain–machine interface control despite neural input reorganization
PLOS ONE, 9 (1) (2014), p. e87253

Full Text via CrossRef
Prasad and Sanchez, 2012
A. Prasad, J.C. Sanchez
Quantifying long-term microelectrode array functionality using chronic in vivo impedance testing
J Neural Eng, 9 (2) (2012), p. 026028

Full Text via CrossRef
Prasad et al., 2014
A. Prasad, Q. Xue, R. Dieme, V. Sankar, R. Mayrand, T. Nishida, et al.
Abiotic-biotic characterization of Pt/Ir microelectrode arrays in chronic implants
Front Neuroeng (2014)

Qian et al., 2009
M. Qian, M. Aguilar, K.N. Zachery, C. Privitera, S. Klein, T. Carney, et al.
Decision-level fusion of EEG and pupil features for single-trial visual detection analysis
IEEE Trans Biomed Eng, 56 (7) (2009), pp. 1929–1937 2009.2016670

View Record in Scopus | Citing articles (8)
Rahman et al., 2013
A. Rahman, D. Reato, M. Arlotti, F. Gasca, A. Datta, L.C. Parra, et al.
Cellular effects of acute direct current stimulation: somatic and synaptic terminal effects
J Physiol, 591 (Pt 10) (2013), pp. 2563–2578

View Record in Scopus | Full Text via CrossRef | Citing articles (22)
Raphael et al., 2009
G. Raphael, C. Berka, D. Popovic, G.K. Chung, S.O. Nagashima, A. Behneman, et al.
I-NET: interactive neuro-educational technology to accelerate skill learning
Conf Proc IEEE Eng Med Biol Soc, 2009 (2009), pp. 4803–4807 2009.5332638

View Record in Scopus | Full Text via CrossRef
Resnik et al., 2013
L. Resnik, S.L. Klinger, K. Etter
The DEKA Arm: its features, functionality, and evolution during the Veterans Affairs Study to optimize the DEKA Arm
Prosthet Orthot Int (2013)

Roset et al., 2013
S.A. Roset, H.F. Gonzalez, J.C. Sanchez
Development of an EEG based reinforcement learning brain–computer interface system for rehabilitation
Conf Proc IEEE Eng Med Biol Soc, 2013 (2013), pp. 1563–1566 2013.6609812

View Record in Scopus | Citing articles (1)
Sajda et al., 2003
P. Sajda, A. Gerson, L. Parra
High-throughput image search via single-trial event detection in a rapid serial visual presentation task
Proc. 1st Inter. IEEE EMBS Conf. on Neural Engineering, IEEE, Capri Island, Italy (2003), pp. 7–10 2003.1196297

View Record in Scopus | Full Text via CrossRef | Citing articles (6)
Sajda et al., 2010
P. Sajda, L.C. Parra, C. Christoforou, B. Hanna, C. Bahlmann, J. Wang, et al.
In a blink of an eye and a switch of a transistor: cortically-coupled computer vision
Proc IEEE, 98 (3) (2010), pp. 462–478

View Record in Scopus | Full Text via CrossRef | Citing articles (41)
Sanchez et al., 2012
J. Sanchez, W. Lytton, J. Carmena, J. Principe, J. Fortes, R. Barbour, et al.
Dynamically repairing and replacing neural networks: using hybrid computational and biological tools
IEEE Pulse, 3 (1) (2012), pp. 57–59 2011.2175640

View Record in Scopus | Full Text via CrossRef | Citing articles (3)
Saxena et al., 2013
T. Saxena, L. Karumbaiah, E.A. Gaupp, R. Patkar, K. Patil, M. Betancur, et al.
The impact of chronic blood–brain barrier breach on intracortical electrode function
Biomaterials, 34 (20) (2013), pp. 4703–4713

Article | PDF (1877 K) | View Record in Scopus | Citing articles (19)
Schiller et al., 2011
P.H. Schiller, W.M. Slocum, M.C. Kwak, G.L. Kendall, E.J. Tehovnik
New methods devised specify the size and color of the spots monkeys see when striate cortex (area V1) is electrically stimulated
Proc Natl Acad Sci USA, 108 (43) (2011), pp. 17809–17814

View Record in Scopus | Full Text via CrossRef | Citing articles (6)
Shenoy et al., 2013
K.V. Shenoy, M. Sahani, M.M. Churchland
Cortical control of arm movements: a dynamical systems perspective
Annu Rev Neurosci, 36 (2013), pp. 337–359

View Record in Scopus | Full Text via CrossRef | Citing articles (28)
Shenoy and Nurmikko, 2012
K. Shenoy, A.V. Nurmikko
Brain models enabled by next-generation neurotechnology: using multiscale and multimodal models
IEEE Pulse, 3 (2) (2012), pp. 31–36 2011.2181021

View Record in Scopus | Full Text via CrossRef | Citing articles (1)
Song et al., 2007
D. Song, R.H. Chan, V.Z. Marmarelis, R.E. Hampson, S.A. Deadwyler, T.W. Berger
Nonlinear dynamic modeling of spike train transformations for hippocampal-cortical prostheses
IEEE Trans Biomed Eng, 54 (6 Pt 1) (2007), pp. 1053–1066

View Record in Scopus | Full Text via CrossRef | Citing articles (68)
Song et al., 2009
D. Song, R.H. Chan, V.Z. Marmarelis, R.E. Hampson, S.A. Deadwyler, T.W. Berger
Nonlinear modeling of neural population dynamics for hippocampal prostheses
Neural Netw, 22 (9) (2009), pp. 1340–1351

Article | PDF (6417 K) | View Record in Scopus | Citing articles (42)
Song et al., 2014
D. Song, B.S. Robinson, R.H. Chan, V.Z. Marmarelis, I. Opris, R.E. Hampson, et al.
Identification of functional synaptic plasticity from spiking activities using nonlinear dynamical modeling
J Neurosci Methods (2014) pii: S0165-0270(14)00348-3

Song et al., 2013
W. Song, C.C. Kerr, W.W. Lytton, J.T. Francis
Cortical plasticity induced by spike-triggered microstimulation in primate somatosensory cortex
PLOS ONE, 8 (3) (2013), p. e57453

Full Text via CrossRef
Stevens et al., 2012
R.H. Stevens, T.L. Galloway, P. Wang, C. Berka
Cognitive neurophysiologic synchronies: what can they contribute to the study of teamwork?
Hum Factors, 54 (4) (2012), pp. 489–502

View Record in Scopus | Full Text via CrossRef | Citing articles (16)
Stinner et al., 2010
D.J. Stinner, T.C. Burns, K.L. Kirk, J.R. Ficke
Return to duty rate of amputee soldiers in the current conflicts in Afghanistan and Iraq
J Trauma, 68 (6) (2010), pp. 1476–1479

View Record in Scopus | Full Text via CrossRef | Citing articles (39)
Tabot et al., 2013
G.A. Tabot, J.F. Dammann, J.A. Berg, F.V. Tenore, J.L. Boback, R.J. Vogelstein, et al.
Restoring the sense of touch with a prosthetic hand through a brain interface
Proc Natl Acad Sci USA, 110 (45) (2013), pp. 18279–18284

View Record in Scopus | Full Text via CrossRef | Citing articles (12)
Thota et al., 2014
A.K. Thota, S. Kuntaegowdanahalli, A.K. Starosciak, J.J. Abbas, J. Orbay, K.W. Horch, et al.
A system and method to interface with multiple groups of axons in several fascicles of peripheral nerves
J Neurosci Methods (2014) pii: S0165-0270(14)00271-4

Thurlings et al., 2010
M.E. Thurlings, J.B. van Erp, A.M. Brouwer, P.J. Werkhoven
EEG-based navigation from a human factors perspective
D.S. Tan, A. Nijholt (Eds.), Brain–computer interfaces, human–computer interaction series, Springer-Verlag, London (2010), pp. 71–86

View Record in Scopus | Full Text via CrossRef | Citing articles (7)
Tien et al., 2013
L.W. Tien, F. Wu, M.D. Tang-Schomer, E. Yoon, F.G. Omenetto, D.L. Kaplan
Silk as a multifunctional biomaterial substrate for reduced glial scarring around brain-penetrating electrodes
Adv Funct Mater, 23 (25) (2013), pp. 3185–3193

View Record in Scopus | Full Text via CrossRef | Citing articles (11)
U.S. Food and Drug Administration, 2011a
U.S. Food and Drug Administration
Driving biomedical innovation: initiatives to improve products for patients
(2011) Available from:

U.S. Food and Drug Administration, 2011b
U.S. Food and Drug Administration
FDA launches medical device innovation initiative [Press release] (2011) Retrieved from

U.S. Government Accountability Office, 2011
U.S. Government Accountability Office
VA mental health: number of veterans receiving care, barriers faces, and efforts to increase access. report to the ranking member
Committee on Veterans’ Affairs, House of Representatives (2011)

Vetter et al., 2004
R.J. Vetter, J.C. Williams, J.F. Hetke, E.A. Nunamaker, D.R. Kipke
Chronic neural recording using silicon-substrate microelectrode arrays implanted in cerebral cortex
IEEE Trans Biomed Eng, 51 (6) (2004), pp. 896–904

View Record in Scopus | Full Text via CrossRef | Citing articles (209)
Vidal, 1973
J.J. Vidal
Toward direct brain-computer communication
Annu Rev Biophys Bioeng, 2 (1) (1973), pp. 157–180

View Record in Scopus | Full Text via CrossRef | Citing articles (250)
Vidal, 1977
J.J. Vidal
Real-time detection of brain events in EEG
Proc IEEE, 65 (5) (1977), pp. 633–641

View Record in Scopus | Full Text via CrossRef | Citing articles (108)
Weiden et al., 2012
M. Weiden, D. Khosla, M. Keegan
Electroencephalographic detection of visual saliency of motion towards a practical brain–computer interface for video analysis. ICMI’12
Proceedings of the 14th ACM International Conference on Multimodal Interaction, ACM, New York, NY (2012), pp. 601–606

Full Text via CrossRef
Zaaimi et al., 2013
B. Zaaimi, R. Ruiz-Torres, S.A. Solla, L.E. Miller
Multi-electrode stimulation in somatosensory cortex increases probability of detection
J Neural Eng, 10 (5) (2013), p. 056013

Full Text via CrossRef
Zalocusky and Deisseroth, 2013
K. Zalocusky, K. Deisseroth
Optogenetics in the behaving rat: integration of diverse new technologies in a vital animal model
Optogenetics, 1 (2013), pp. 1–17

View Record in Scopus | Full Text via CrossRef | Citing articles (3)