From Sight to Synesthesia: What Happens when the Senses can be Rewired?

A Dutch physicist, Eindhoven-based Peter Meijer, has developed a device called vOICe that allows blind people to see, using the aural information entering their ears. The device converts visual information received by a camera into soundscapes, which then gives an aural representation of what the camera sees. The soundscapes are fed into the ears and — with training — the user can learn what visual information the soundscapes represent about the world. Exploiting the plasticity of the neural connections in the brain, the device tricks it into thinking that it is receiving information from the eyes. Consequently, the blind users see a visual representation of what is around them. Meijer developed the concept for his device in 1982, and built the first vOICe prototype in 1991. By 1998, he had developed a portable, but bulky, version. Now the device is compact enough to be practically useful. Only a handful of people have used vOICe, but it is showing [such] sufficient promise that the US National Science Foundation is now funding a project to determine its benefits and develop a protocol for its use. Some more information about vOICe can be found in a recent report in New Scientist. Early trials show that, while imperfect, vOICe is working effectively for its users. This is a breakthrough from a therapeutic point of view. But the technology may prove to have many uses. This raises the possibility that it could be appropriate for types of sensory enhancement. In fact, vOICe claims that the technology is available to be used ‘as neural engineering ingredients in a highly modular, adaptive and preferably non-invasive and unobtrusive brain-computer interface for augmented vision, augmented reality, and augmented cognition.’

Currently, only blind people are using use vOICe technology to experience synthetic vision. If, however, the ‘rewiring’ techniques involved could be harnessed for merging sight and sound in sighted people, so that the brain experiences these two sets of sensory information in a mixed and homogenous way, it may be possible to create new synesthetic experiences. It is important to point out that the successful transference of vOICe-style technology to the other senses does depend partly upon a fuller understanding of the locations of the neural connections in the brain that correspond to particular sensory responses. The technology is limited in what it can achieve at the moment, but the development of vOICe does show that it is possible to neurally rewire the senses. In this respect, it may be possible to develop the technology to produce different sensory results.

Estimates vary as to the incidence of synesthesia — different studies carried out in the UK suggest an incidence of anywhere between 0.05% and 4%.  [See references below.] The precise cause is not yet fully understood, and its genetic basis is complex. The condition is charicterized by the joining together of sensory experiences that would ordinarily be separate or distinct. A synesthete may experience colors when they hear sounds or read words. There are five senses, and on this basis a simple pairing of each with every other, we might think that there are twenty potential forms of synesthesia. This is a misreading of the condition, however, as synesthetic experiences may be constituted by more than two senses. There are numerous studies worldwide investigating the nature and effects of different kinds of synesthesia. In the UK, researchers at the University of Sussex are conducting a major study.

Synesthesia is hard for scientists to understand because those affected often report bizarre associations that are conceptually alien to the rest of us.  For example, J. I. Wannerton reports that “I live in a house that tastes of mashed potato in a fruit gum town” and “My next door neighbors are a mix of yoghurt, jelly beans and a subtle hint of a waxy substance.”

There is disagreement over the classification of synesthesia as a disorder.  Although synesthesia can cause difficulties in numerical and linguistic processing for some of those affected, most synesthetes report that their condition does not interfere with their normal functioning. Some people who have synesthetic experiences derive pleasure from them, and some synesthetes even receive unique benefits or abilities from the condition. There is some evidence to suggest that the ability to associate colors with numbers or words, for example, can assist in memory or enhance mathematical or linguistic ability. Some synesthetes are able to harness these kinds of experiences in order to produce work[s] that combine the logical and the aesthetic in ways that reveals unique or important perceptions that other people are neurologically precluded from having. Several studies of the neurology of synesthetes provide some evidence that synesthesia can cause heightened artistic ability, and a quick glance at Wikipedia shows a long list of famous synesthetic artists.  Ludwig Wittgenstein was also reportedly a synesthete. The insights he offered about how reality might be structured are simultaneously complex, simple, important, subtle and efficient. There is also an aesthetic quality to his writing that is rarely found in technical philosophy.

Bearing all this in mind, there might be advantages in being able to perceive the world in a synesthetic way. But there has never been a way to reliably generate these kinds of experiences. Psychotropic drugs such as LSD or Psilocybin (found in Magic Mushrooms) can cause the user to experience a kind of merging of the senses while under the influence, but these drugs also tend to incapacitate the user, and the experiences are difficult to manage or control. They can also, at times, be terrifying or psychologically dangerous. Equally, these synesthetic experiences and perceived insights can be profoundly valuable, and may change the user’s assumptions about the nature and structure of reality.

vOICe and similar technologies may offer a way to reliably and controllably create new forms of synesthetic experience. Given the existence of numerous types of synesthesia, the various ways in which it may be possible to rewire the brain to merge or homogenize  the senses are considerable. It is difficult for people who are not synesthetic to grasp what it might be like to smell colors or to perceive numbers as sounds, and this is because we tend to think of those distinct kinds of sensory inputs as fundamentally, qualitatively different. synesthetic research might lead us to ask — in view of technologies such as vOICe — whether this is, in fact, the correct way to think about the five senses. If several senses can reveal different aspects of a sensory experience, we might be mistaken about the boundaries of certain normative concepts that define those sensory experiences when they are distinct.

There is a famous quote (variously attributed to Elvis Costello, the actor Martin Mull, Miles Davis and Frank Zappa) that asserts the futility of certain types of art criticism, claiming that ‘writing about music is like dancing about architecture.  Our instinctive response is that dancing is an inappropriate or irrelevant way to express one’s reaction to — or feelings about — architecture. But this response is contingent on our perceptions around what it is like to experience architecture. It seems natural to dance to certain kinds of music because its rhythmical nature inspires that kind of motor response in the body. We don’t tend to see music, so we don’t spend time trying to look at it. When we sit or stand still to listen to music, it is because we want to be enveloped fully in the sound, and experience it as completely as possible. Architecture is silent, and most of us experience it as something to be contemplated primarily with the eyes. And we do tend to concentrate on architecture, standing still or walking slowly, rather than rushing past it and missing out on some aspects of it. If a synesthete has aural sense impressions when seeing certain kinds of shapes or forms, however, then a different kind of bodily response might seem natural.

Of course, most of us would probably find it odd if somebody told us they were dancing in response to seeing a building. This example is only a tangential and incomplete way to explain the challenge to our normative understanding of the senses raised by synesthesia. Nevertheless, it does point towards the root of the problem — the proper categorizations for sensory experiences. In argument, when somebody compares two things that appear to be categorically incomparable, we might point it out by saying something along the lines of ‘that’s like comparing round with blue’. Again, this seems like a rational way to illustrate a mistake in the apprehension of the nature of two kinds of objects. But for a synesthete, this might not hold true. We could, on the basis of some synesthetic experiences, reasonably adopt an idealist kind of position and claim that reality (and our perceptions of it) are identical, because there is no way to prove that this is not the case. If I associate circularity with blueness, that is a fact about which there can be no disagreement.

Technologies such as vOICe, that may enable us to experiment with the homogenization of our senses by rewiring parts of the brain, give support to the claim that statements about our perceptions are neither truth-apt, nor empirically or a priori verifiable. Since, indeed, physical alterations to the brain cause[s] changes in the way in which it apprehends the world, it is impossible to know whether objects in the outside word correspond properly to our perception of them. This claim poses an epistemic predicament with a long philosophical pedigree. One possible response might be to adopt a logical positivist stance and conclude, given the empirical unverifiability of sensory experiences, that all statements about sensory experiences must be meaningless.

David Hume considered the problem that we cannot assume nature will always continue to present itself to us in a uniform way. He dealt with this problem, however, by concluding that it is constitutive of human nature and cognition to believe that nature will present itself uniformly, as this belief helps us to anticipate what the future will be like so that we can make plans to deal with it. Synesthetic experiences complicate this argument, firstly because synesthetes commonly report a regularity in the associations that they make; and secondly because these associations often appear nonsensical to the rest of us. Trying to understand the nature of synesthetic experience reveals tensions between knowledge claims that can’t obviously be resolved by appeal to logic alone. So where do these tensions leave us in relation to the value of having synesthetic experiences?

The answer is that the logical problems that are raised by technologies like vOICe are themselves valuable. The prospect of engineered synesthesia gives a new context to a longstanding philosophical problem, and in this respect it shows how deep this philosophical problem goes. It gives some support to an epistemologically skeptical position without devaluing the qualitative experience that we could derive from new sensory perceptions. There are realms in which we could experience pleasure from a change in the way we perceive certain things, and, in particular, things from which we derive sensations of beauty. To experience synesthetically may be to enhance the aesthetic experience. If we could experience immersion in water or the wind in a synesthetic way, for example, or if we were able to experience normatively ‘visual’ art differently, we would open the door to re-experiencing these things anew through the prism of a new category of experience,[:] as a unified, homogenous and integrated channel of perception.

References

How common is synaesthesia?

Early detection of markers for synaesthesia in childhood populations

Synaesthesia Research

See Also

Expanding the Umwelt: Seeing a New Color in the Rainbow

Art, Neurobiology, and Mescaline

Leave a Reply