Research

Multisensory Integration

Why are we better at hearing speech in noisy environments when we can also see the lip movements of the speaker? Why does food lose its taste when your nose is stuffed up? These are questions of interest for researchers in the area of multisensory integration. Although the scientific study of multisensory processing has existed since psychology became an experimental discipline at the end of the nineteenth century, little is known about the precise mechanisms underlying multisensory integration in the human brain. The work of putting together all the disparate environmental stimuli into a coherent, integrated picture of the world requires extremely fast and flexible communication between the different brain regions.

EEG recording during a

visuotactile attention task

Pomper et al., 2015, HBM

Enhanced gamma power

during visuotactile motion processing

Krebber et al, 2015, Neuroimage

Working memory load enhances

theta power

Michail et al., 2021, J Neurosci

This basic processing is important as a means of understand the way different parts of the brain communicate with each other. When we look at the output of a human electroencephalogram (EEG) or magnetoencephalogram (MEG), we see apparently random and chaotic patterns of different waves. What we are actually seeing relates to the coordinated synchronized communication of large neural populations. A lot of our work is in decoding the way in which these different oscillatory signals help to order and facilitate multisensory processing. The work of putting together all the disparate environmental stimuli into a coherent, integrated picture of the world requires extremely fast and flexible communication between the different brain regions. We have indeed observed multisensory interactions at very short latencies (Senkowski et al., 2011), suggesting that they are a fundamental property of the brain. Further studies have shown a close interplay between multisensory integration and attention (Talsma et al., 2010). Recently, we have started to apply computational modelling to explain how the brain decides which environmental stimuli belong together, and how this is integrated across time.

Intersensory attention modulates

functional connectivity

Keil et al., 2016, Cortex

Role of neural oscillations

for multisensory processing and attention

Keil & Senkowski, 2018, Neuroscientist

GABA concentration mediates

the relationship between gamma

and multisensory illusion rates

Balz et al., 2016, Neuroimage

Key references: Balz et al., 2016, Neuroimage; Keil & Senkowski, 2018, Neuroscientist; Michail et al., 2021, JN, 2022, Neuroimage; Pomer et al., 2015, HBM; Senkowski et al., 2008, TINS; Talsma et al., 2010, TICS

Schizophrenia

Numerous studies, including our own (Keil et al., 2016; Moran et al., 2019), have suggested that alterations in neural oscillations contribute to the psychopathology in schizophrenia. Although schizophrenia is defined by florid and debilitating symptoms such as hallucinations and intense clinical depression, there are also subtler perceptual distortions, indicating some basic, genetically mediated problems in synaptic transmission. In recent years, we have also examined multisensory processing in patients, but surprisingly found only minor deficits (Balz et al., 2016; Roa Romero et al., 2016). In contrast, we observed that widely intact multisensory processing can compensate for attention deficits in schizophrenia (Moran et al., 2021). Our current research focuses on the neural mechanisms underlying working memory deficits in patients. To this end, we will investigate multivariate activation patterns and neural synchrony in patients and controls.

Patients with schizophrenia show an intact

audiovisual N1 suppression effect

Senkowski and Moran, 2022, Neuroimage:Clin

Multisensory processes can compensate

for attention deficits in schizophrenia

Moran et al., 2021, Cerebral Cortex

Key references: Moran et al., 2021, Cereb Cortex; Senkowski & Gallinat, 2015, Biol Psychiatr; Senkowski & Moran, 2022 Neuroimage:Clin; Wooldridge et al., 2021, Sci Rep

Pain

Our studies have mainly focused on the processing of acute pain (Senkowski et al., 2011; Pomper et al. 2013). Noxious stimuli in our environment are often accompanied by input from other sensory modalities that can affect the processing of these stimuli and the perception of pain. Stimuli from these other modalities may distract us from pain and reduce its perceived strength. Alternatively, they can enhance the saliency of the painful input, leading to an increased pain experience. A main outomce of our research is that stimuli from other modalities interact with pain, so that they either elevate or diminish the processing and perception of pain (Hofle et al, 2012, 2013). We also hypothesized that chronic pain can distort body representation in the brain (Senkowski et al., 2016), which has implications for the development of virtual reality feedback interventions for the multisensory treatment of chronic pain.

Looking at a needle during an

injection increases pain

Hofle et al. 2012, Pain

Visual input shapes

pain-related networks

Senkowski et al., 2014, TICS

Alpha-band modulation in

anticipation of a needle prick

Hofle et al., 2013, Eur J Neurosci

Key references: Hofle et al, 2012, Pain; Senkowski et al., 2011, JN; Senkowski et al., 2014, TICS; Pomper et al. 2013, Neuroimage

Other topics

We wave conducted smaller-scale projects on other topics, including generalized anxiety disorder (Senkowski et al., 2003); a collaborative project with the institute of sexology and sexual medicine (Speer et al., 2020); cochlear implant users (Senkowski et al., 2014); genetic research (Gallinat et al., 2003); functional magnetic resonance spectroscopy (Balz et al, 2018); adult ADHD (Senkowski et al., preprint) and health care related research (Moran et al., 2021). Recently, we started a larger-scale project on memory processing in post-traumatic stress disorder. This project seeks to apply established knowledge of dynamic neural processes underlying memory to its potential dysfunction in people with PTSD. Memory dysfunction is a conspicuous element of PTSD – people’s memories of traumatic experiences are often confused, and overlap each other, compounding the distress of the individual. It seems that general working memory and episodic memory performance, even for non-traumatic memories is impaired in PTSD. We want to test whether these memory deficits can be related to oscillatory features of memory encoding and retrieval, and whether manipulating these signals can actually have an effect on memory performance.

Auditory processing in

cochlear implant users

Senkowski et al. 2014, HBM

Delta modulation during the

auditory continuity illusion

Kaiser et al., 2018, Eur J Neurosci

Studying neural processing of

illusory Kanizsa figures

Senkowski et al., 2005, Neuropsychologia

Key references: Kaiser et al., 2018, EJN; Senkowski et al., 2003, Biol Psychiatr; Senkowski et al., 2022, Cereb Cortex Commun; Speer et al., 2020, Eur Child Adolesc Psychiatr