Sensual Flux
Multi Sense Virtual Experience
In neuroscience study, you learn about how the nervous system transforms signals emitted from the outside world into meaningful information that organisms can interpret. In the case of people, this includes the mechanisms behind sensation and perception among 5 main categories. Wavelengths of light enter the eye, diffract and bounce around the inside of the eyeball, before ultimately hitting receptor cells in the retina and transmitted into chemical and eletcrial signals the brain can interpret. Taste and smell provide a way for us to detect chemical substances in our environment as we use receptors on the tastebuds of our tongues and the olfactory bulb behind our nose to inquire if an item we come in contact with may be noxious and dangerous or useful to consume. Touch is an particularly robust way for us to differentiate items by texture, pattern, and tempertaure; and propioception enables us to know where our body is in space by comparing the forces acting on it. Waves in the air travel though the parts of our ear to eventually vibrate our eardrums, allowing us to perceive speech, enjoy music, and be wary of approaching dangers. Senses our essential to the way we navigate and place ourselves in our environments, and exploring how we can manipulate them is a really interesting concept to me.
What experience arises when we combine specific percepts together with particular degrees of intenisty? I wanted to approach this question with some of the technologies I am fond of.
I use specific media related sensual stimulations to begin exploring this question in this project. With combination of visual, tactile, and auditory stimuli, I question if people feel some messy aggregation of the three feelings, a new sense that can only be explained by some weighted sum of the three senses together, or something in between. Does this produce some sort of learned response, or a sensation close to synesthesia (the broad term for a sense being activated from the stimulation of a different one)?
I use a custom made VR (virtual reality) scene developed using javascript and associated VR hardware to playback visuals and sound. Tactile stimulation comes from Rehastim electrical muscle stimulation hardware. I used custom made code in python and serial messaging to make the electrical stimulation occur upon interaction with visual objects via grasping them. (I also explore this topic with brain-machine interfaces)
The inspiration behind this project comes from previous experience working with electrical muscle stimulation and the feelings associated with having bodily actions eleictied from a source external to the body.