Dongyu Gong

Dongyu Gong

PhD Student at Yale University

Posts by Collection

photography

presentations

The effect of distractor saliency on attentional capture

Published:

More information here

Theeuwes (1991, 1992) found that a salient but task-irrelevant color singleton would increase the response time to the target form singleton, and he proposed that this was because the salient distractor captured attention before it was shifted to the target, which is known as the automatic capture hypothesis. He further suggested the relative saliency of the color singleton would determine whether it could capture attention, but so far there hasn’t been any experiments revealing specifically how different distractor saliency conditions on a continuous spectrum would have an effect on attentional capture. In this research, we examined this question within multiple attention-guiding dimensions, including color, size and orientation (Wolfe & Horowitz, 2004).

Guiding Perception by Memories of Different Timescales

Published:

We reported how working memory (WM) and long-term memory (LTM) traces can guide external spatial attention when retro-cued before a perception task, and how timing has a critical influence on the guidance from WM and LTM.

Selecting and Prioritising Contents in Working and Long-term Memory Guides Recall and Perception

Published:

Although theoretical models have highlighted the close relation between long-term memory (LTM) and attention, we have little understanding of how the processes of orienting, selecting, and prioritising relevant contents in long-term memory unfold. Here we build on findings and methods developed to study internal selective attention in working memory (WM), to show that selection and prioritisation of contents from long-term memory improves recall of stored information and guides perception in a secondary task. We developed a task in which participants hold separate colour-location combinations in LTM and WM. Subsequently, a colour cue (‘retro-cue’) would indicate the identity of the target that would be probed in a memory recall task after a short delay. Critically, in half of the trials, following the delay participants would instead have to complete a perceptual detection task in which stimuli unrelated to memory contents briefly flash on the screen. Our results show that selection and prioritisation of contents from both long-term and working memory not only improve memory recall, but also enhance perceptual sensitivity when the probed location in the perception task is matched with the currently prioritised location in LTM/WM. Our method of studying internal selective attention in long-term memory opens the door to exploring our flexible and adaptive usage of memories in service of behaviour.We reported how working memory (WM) and long-term memory (LTM) traces can guide external spatial attention when retro-cued before a perception task, and how timing has a critical influence on the guidance from WM and LTM.

Focusing attention in long-term and working memory improves recall and guides perception

Published:

Attention can be directed not only to sensory stimuli from the external environment but also to internal representations in memory. While attentional selection in working memory (WM) has been widely studied, we know less about how we focus on contents within long-term memory (LTM). In two experiments, we directly compared the consequences of selecting items in LTM and WM, examining influences on subsequent recall and perception. Participants first learned the features of two LTM items during a learning phase. In the subsequent phase, participants viewed two items with features different from the LTM items and encoded these into WM. After a delay, a retrocue informed participants which item they should select from either the LTM or WM items or, was uninformative. On half of the trials, participants were prompted to recall the retrocued item. Critically, on the other trials, participants performed a perceptual discrimination task on a briefly presented and masked sensory array unrelated to the memory contents. The item to discriminate appeared equiprobably at the four locations, thereby overlapping with retrocued LTM or WM locations on 25% of the trials. In both experiments, informative retrocues facilitated recall speed for LTM and WM contents. Retrocuing items in LTM and WM also incidentally improved perceptual discrimination for visual items coinciding with the location of retrocued items, despite retrocues having no sensory predictive value. Although the behavioural benefits showed a similar pattern, eye tracking suggested functionally dissociable mechanisms for orienting attention in LTM and WM. Significant gaze biases followed retrocues indicating items in WM but not in LTM. Overall, our study introduces an experimental approach for comparing the processes and consequences of attentional orienting in LTM and WM. We observe memory and perceptual benefits for attentional orienting in both memory domains but through partially dissociable mechanisms.

Dissociable neural processes during attentional selection within working memory and long-term memory

Published:

Research has increasingly emphasized the ability to orient attention selectively to prioritise internal contents for retrieval - in both workingmemory (WM) and long-term memory (LTM). However, little is known about the potential degree of overlap between mechanisms for internal attention in WM and LTM. We developed a task for comparing shifts of internal attention in WM and LTM for retrieving equivalent stimulus attributes. Eye tracking and EEG recordings from human participants (N = 30) tracked oculomotor and neural signals triggered by retrospective attention cues (retrocues) that prioritised object features of WM or LTM items on a trial-by-trial basis. The eye-tracking results confirmed our recent observation that gaze biases toward the attended item were more pronounced in WM than LTM. The EEG data revealed striking differences in neural processes following retrocues prioritising WM and LTM items. Replicating previous findings, transient lateralization of alpha power (8-12Hz) at posterior sites occurred for shifts of attention in WM, but no such effects occurred for attention within LTM. Instead, frontal modulation oftheta power (4-8 Hz) occurred during shifts of attention in LTM but not in WM. Further dissociable markers of internal shifts of attention in WM and LTM were observed in the event-related potentials. Our findings show that the robust consequences of orienting selective attention in WM and LTM occur through different routes, thus also providing valuable insights into the long-debated relationship between these two memory systems.

EEG signatures of orienting attention to long-term vs. working memory contents

Published:

Attention can be directed towards internally generated representations of the past, a process known as internal attention. Internal attention can operate over contents of both working memory (WM) and long-term memory (LTM)**, **but far less is known in terms of the neural signatures when orienting attention in LTM compared to WM. To answer this question, we recorded EEG signals from participants engaged in a behavioral task where retrospective attention cues prioritized either a WM item that switched features from trial to trial, or an LTM item that had been previously memorized. Following a brief delay, participants reproduced the cued item. Using multivariate pattern analysis, we were able to decode the timescale of the cued item (WM vs. LTM) during the delay. Furthermore, left vs. right items were better decodable for WM, whereas LTM representations seemed to show less lateralization. Event-related potentials revealed higher contralateral responses at posterior sites for WM items, but no such lateralization for LTM items. Time-frequency analysis indicated alpha power (8-12 Hz) lateralization during a WM delay but not LTM. However, non-lateralized theta power (3-7 Hz) is found to be higher during an LTM delay compared to WM, hinting at distinct retrieval processes for LTM. To sum up, our findings suggest that orienting attention to LTM contents involves different neural mechanisms compared to WM. Selecting an item in LTM does not necessarily bring it back to the same state of a WM item, indicating potentially dissociable representational formats for WM and LTM contents.

publications

research

Effects of Distractor Saliency and Spatial Location on Attentional Capture

Project Leader, Cognitive Neuroscience Lab, Tsinghua University, 2018

  • Background: Previous research found that a salient but task-irrelevant color singleton would increase the response time to the target form singleton, which is known as attentional capture. However, there haven’t been studies systematically examining how different distractor saliency conditions on a continuous spectrum would have an effect on attentional capture. Additionally, there has been evidence showing spatial heterogeneity in the perception of various visual features, but it’s not clear yet whether attentional capture also exhibits spatial heterogeneity across display locations.
  • Designed and programmed experiments, collected and analyzed data with MATLAB, and presented posters at the Psychonomic Society 2019 Annual Meeting and VSS 2020 Virtual Meeting.
  • Examined the effects of distractor saliency defined within multiple attention-guiding dimensions, including color, size and orientation, and established the existence of a certain threshold for distractor saliency to elicit attentional capture.
  • Found that there existed a spatial pattern of attentional capture susceptibility, which was distinctive and stable for each individual.

Reverse Correlating Ensemble Perception

Research Assistant, Perception and Action Lab, Universityof California, Berkeley, 2019

  • Background: There hasn’t been enough evidence that people can extract summary statistical information about abstract social traits (e.g., trustworthiness, dominance, submissiveness and the like) because the space of faces features that can draw specific judgments is infinitely large.
  • Employed a data-driven reverse correlation approach to model the ensemble perception of trustworthiness.
  • Programmed experiments, collected and analyzed data with Reverse-Correlation Image-Classification Toolbox in R.
  • Participants viewed face crowds of different average trustworthiness levels, and then did a two-images-forced-choice (2IFC) classification task where they selected one of the two faces (with noise patterns superimposed on the base image) more representative of the average trustworthiness of the face crowd previously shown.
  • The average of all selected noise patterns constitutes the classification image (CI) and then the CIs for different average trustworthiness levels were statistically compared to determine how much they were different from each other.

Saliency-Specific Mechanism of Distractor Suppression

Visiting Researcher, Department of Experimental and Applied Psychology, VU Amsterdam, 2020

  • Background: Research has shown that interference caused by a salient distractor in visual search tasks can be reduced by suppressing the high-probability location (HPL) of the distractor through implicit learning, while the underlying neural mechanisms and the impact of distractor saliency on suppression effects remain unclear.
  • Designed and programmed experiments, collected and analyzed data with MATLAB, and wrote the paper (published on Attention, Perception, & Psychophysics).
  • Developed a novel paradigm to manipulate saliency of distractors in additional singleton task and examined how distractors of different saliency were suppressed at the same HPL.
  • Inferred the neural mechanisms underlying the saliency-specific mechanism: Spatial probability manipulation elicited attentional modulation of V1 cells that cover the HPL with their classical receptive fields. The attentional modulation is tuned in accordance to the firing rates of the group of V1 cells representing the distractor, which is finally reflected on the saliency-specific reduction of interference when the distractor appears at the HPL.

Relationship between Selective Attention and Ensemble Perception

Research Intern, Visual Attention Lab, Harvard University, 2020

  • Background: Our visual system copes with limited capacity using two different modes of attention, a distributed attention mode extracting the gist of a scene (i.e., ensemble perception) and a focused attention mode selecting only relevant information (i.e., selective attention). These two modes of processing serve different purposes. Still, it is unclear how they work together, whether they conflict with each other, and how cognitive control might play a role in conciliating their different processing demands.
  • Designed and programmed experiments, collected data and conducted analyses with MATLAB.
  • Developed a novel paradigm incorporating the mean orientation discrimination task and target orientation detection task. Introduced a single-task condition (requiring only one mode of processing), a dual-task condition (requiring both modes of processing), and a mixed-task condition (requiring either one or two modes of processing across trials).
  • Discovered that people’s performance in target selection and ensemble discrimination tasks positively correlated in both single-task and dual-task conditions, indicating some shared neural mechanisms underlying selective attention and ensemble perception. The mixed-task condition significantly impaired ensemble discrimination performance rather than target selection performance, suggesting a cognitive control strategy favoring selective attention when faced with conflicts in processing demands.

Guiding Perception by Memories of Multiple Timescales

PhD Student, Brain and Cognition Lab, University of Oxford, 2021

  • Our brain is extraordinary at matching incoming sensory signals to past experiences, which guides selective attention and allows us to behave adaptively and dynamically based on predictions and expectations. Memories of different timescales are involved in guiding perception and performance. For instance, when cycling to work, you are not only using long-term memories (LTM) of the spatial map and cycling route to guide your direction, but also relying on current working memories (WM) of traffic lights, pedestrians and cars on the road to adjust your speed or re-plan the route.
  • However, we still lack a clear understanding of how these memories of different timescales work together to guide adaptive behaviour, and neither do we know the underlying neural mechanisms.
  • This study aims to elucidate how the prospective nature of memory operates in a multi-timescale and interactive way, and what brain processes are involved in doing so.
  • Part of the funding of this research comes from UKRI (UK Research and Innovation). An overview of this research on UKRI website can be found here.

Bayesian Telephone: Memory Consolidation and Recall as Generative Processes

PhD Student, Yale University, 2023

The hippocampus serves a key role in memory acquisition and consolidation, yet it is unknown whether the hippocampus stores raw sensory inputs or merely generative reconstructions of those inputs. In this paper we examined these competing hypotheses of memory representation in the hippocampus. To do so we modeled the hippocampus as a modern Hopfield network and the entorhinal cortex as a variational autoencoder (VAE). We used Mitsuba 3 to generate a Cornell box dataset. In our first model, we passed these scenes directly into our Hopfield network and trained our VAE on the Hopfield network’s output when prompted by stimulus. In the second, the model first probabilistically inferred latent parameters for the observations and generated reconstructed observations which were then passed into the Hopfield network to aid in the training of our VAE. We tested the capacity of our models for generative recall of these scenes and found reliable minimization of reconstruction error during recall in both models. We concluded that either representation scheme or a combination of the two might be at work in the human brain. Future studies should explore implementing features such as forgetting and recall vulnerability in our base model and comparing model performance to human performance on recall tasks.

Github page: Bayesian Telephone: Memory Consolidation and Recall as Generative Processes

NeuroAI

PhD Student, Yale University, 2024

  • Using neural-network models such as RNNs and Transformers to gain mechanistic understandings of psychological phenomena related to attention and memory: How does attention facilitate memory encoding? How is working memory transformed into long-term memory? Why is there a fundamental limit in human working memory capacity? Are working memory and long-term memory represented in the same neural architecture? How do the representational formats of working memory and long-term memory differ? How to explain memory retrieval and false memory?
  • Using theoretical and mathematical tools to understand the inner workings and limitations of Transformer-based large language models (LLMs): How to use formal mathematical tools to assess the reasoning ability of LLMs? How to define the “emergence” of cognitive abilities in LLMs through the lens of statistical physics? What are the unique mathematical properties of LLMs that make them powerful? What are the missing pieces in these models to achieve human-level cognitive abilities? How to mathematically define the amount of intelligence and sentience in deep neural networks like Transformer-based LLMs? If LLMs are not the final solution to artificial general intelligence, what insights can theoretical neuroscience offer on the way of achieving that goal?