Research

Multisensory Integration

Why are we better at hearing speech in noisy environments when we can also see the speaker's lip movements? Why does food lose its taste when the nose is stuffed up? These are questions of interest to researchers in the field of multisensory integration. Although the scientific study of multisensory processing has been around since psychology became an experimental discipline at the end of the 19th century, little is known about the precise neural mechanisms underlying multisensory integration. Our research investigates the neural mechanisms underlying multisensory integration, with a focus on neural oscillations.

EEG recording during a

multiensory attention task

Pomper et al., 2015, HBM

Gamma oscillations during

visuotactile motion processing

Krebber et al, 2015, Neuroimage

Working memory load enhances

theta power in a multisensory paradigm

Michail et al., 2021, J Neurosci

When we look at the human electroencephalogram (EEG) or magnetoencephalogram (MEG), we see what appear to be random and chaotic patterns of different waves. What we are actually seeing is highly coordinated, synchronized interactions between large populations of neurons. Much of our work involves deciphering how these different oscillatory signals help to order and facilitate multisensory processing. The work of assembling all the different environmental stimuli into a coherent, integrated picture of the world requires fast and flexible communication between the different regions of the brain. Indeed, we have observed multisensory interactions with short latencies (Senkowski et al., 2011), suggesting that they are a fundamental property of the brain. Other studies have shown a close interplay between multisensory integration and attention (Talsma et al., 2010). Recently, we have started to apply computational modelling to explain how the brain decides which environmental stimuli belong together, and how this is integrated across time.

Intersensory attention modulates

functional connectivity

Keil et al., 2016, Cortex

Role of neural oscillations

for multisensory processing and attention

Keil & Senkowski, 2018, Neuroscientist

GABA concentration mediates

the relationship between gamma

and multisensory illusion rates

Balz et al., 2016, Neuroimage

Schizophrenia

Numerous studies, including our own (Keil et al., 2016; Moran et al., 2019), have suggested that alterations in neural oscillations contribute to the psychopathology in schizophrenia. Although schizophrenia is defined by florid and debilitating symptoms such as hallucinations and severe clinical depression, there are also more subtle perceptual distortions that point to some fundamental, genetically mediated alterations in synaptic transmission. In recent years, we have also investigated multisensory processing in patients, but surprisingly found only minor deficits (Balz et al., 2016; Senkowski and Moran, 2022). In contrast, we observed that largely intact multisensory processing can compensate for attention deficits in schizophrenia (Moran et al., 2021). Our current research focuses on the neural mechanisms underlying working memory deficits in patients. To this end, we will examine multivariate activation patterns and neural synchrony in patients and controls.

Patients with schizophrenia show an intact

audiovisual N1 suppression effect

Senkowski and Moran, 2022, Neuroimage:Clin

Multisensory processes can compensate

for attention deficits in schizophrenia

Moran et al., 2021, Cerebral Cortex

Pain

Our studies have mainly focused on the processing of acute pain (Senkowski et al., 2011; Pomper et al. 2013). Noxious stimuli in our environment are often accompanied by input from other sensory modalities that can affect the processing of these stimuli and the perception of pain. Stimuli from these other modalities may distract us from pain and reduce its perceived strength. Alternatively, they can enhance the saliency of the painful input, leading to an increased pain experience. A main outomce of our research is that stimuli from other modalities interact with pain, so that they either elevate or diminish the processing and perception of pain (Hofle et al, 2012, 2013). We also hypothesized that chronic pain can distort body representation in the brain (Senkowski et al., 2016), which has implications for the development of virtual reality feedback interventions for the multisensory treatment of chronic pain.

Looking at a needle during an

injection increases pain

Hofle et al. 2012, Pain

Visual input shapes

pain-related networks

Senkowski et al., 2014, TICS

Alpha-band modulation in

anticipation of a needle prick

Hofle et al., 2013, Eur J Neurosci

Other topics

We have conducted smaller projects on other topics, including adult ADHD (Senkowski et al., 2023), generalized anxiety disorder (Senkowski et al., 2003), a collaborative project with the Institute of Sexology and Sexual Medicine (Speer et al., 2020), cochlear implant users (Senkowski et al., 2014), genetic research (Gallinat et al., 2003), functional magnetic resonance spectroscopy (Balz et al, 2018), and health service research (Moran et al., 2021). More recently, we initiated a DFG-funded project on memory processing in post-traumatic stress disorder. This project aims to apply established knowledge of the dynamic neural processes underlying memory to its potential dysfunction in people with PTSD. Memory dysfunction is a prominent feature of PTSD – people’s memories of traumatic experiences are often confused and overlapping, exacerbating the individual's distress. It appears that general working memory and episodic memory performance, even for non-traumatic memories, are impaired in PTSD. We want to test whether these memory deficits can be related to oscillatory features of memory encoding and retrieval, and whether manipulating these signals can actually affect memory performance.

Auditory processing in

cochlear implant users

Senkowski et al. 2014, HBM

Delta modulation during the

auditory continuity illusion

Kaiser et al., 2018, Eur J Neurosci

Studying neural processing of

illusory Kanizsa figures

Senkowski et al., 2005, Neuropsychologia