Decoding Visual Experience from the Human Brain
Yukiyasu Kamitani (ATR Computational Neuroscience Labo., JP)
Abstract
Brain activity can be seen as “codes” that encode mental states. Recent advances in human neuroimaging such as functional magnetic resonance imaging (fMRI) have revealed brain regions that encode specific behavior and cognition. Despite the wide-spread use of human neuroimaging, its potential to read out, or “decode”, mental contents from brain activity has not been fully explored. In this talk, I present methods for decoding visual representations from fMRI activity patterns based on machine learning techniques. I show how early visual features represented in “subvoxel” neural structures could be decoded from ensemble fMRI responses. Decoding of stimulus features is extended to the method for neural mind-reading, which attempts to predict a person’s subjective state using a decoder trained with unambiguous stimulus presentation. We then discuss a modular decoding approach, in which a wide variety of percepts can be decoded by combining the outputs of multiple decoder modules. On the basis of this approach, we were able to reconstruct arbitrary visual images using the decoder trained on fMRI responses to only several hundred random images. Finally, I discuss potential applications of neural decoding to brain-based communications.
Yukiyasu Kamitani is currently the head of Department of Neuroinformatics at ATR Computational Neuroscience Laboratories, Kyoto, Japan, and a professor at Nara Institute of Science and Technology (NAIST). He received B.A. in Cognitive Science from University of Tokyo in 1993, M.S. in Philosophy of Science from University of Tokyo in 1995, and Ph.D. in Computation and Neural Systems from California Institute of Technology in 2001. He continued his research in cognitive and computational neuroscience as a research fellow at Beth Israel Deaconess Medical Center (Harvard Medical School), and as a research staff member at Princeton University. In 2004, he joined ATR Computational Neuroscience Laboratories, where he currently works on neural decoding of human brain signals. He was named Research Leader in Neural Imaging on the 2005 “Scientific American 50.”
