|
Sequential Replay of Task States in the Human Hippocampus Science 28 Jun 2019: Sequential replay of nonspatial task states in the human hippocampus Nicolas W. Schuck, et.al. Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany. Max Planck University College London (UCL) Centre for Computational Psychiatry and Ageing Research, Berlin, Germany, and London, UK. Princeton Neuroscience Institute and Department of Psychology, Princeton University, Washington Road, Princeton, NJ 08544, USA. [paraphrase] Sequential neural activity patterns related to spatial experiences are “replayed” in the hippocampus of rodents during rest. We investigated whether replay of nonspatial sequences can be detected noninvasively in the human hippocampus. Participants underwent functional magnetic resonance imaging (fMRI) while resting after performing a decision-making task with sequential structure. Hippocampal fMRI patterns recorded at rest reflected sequentiality of previously experienced task states, with consecutive patterns corresponding to nearby states. Hippocampal sequentiality correlated with the fidelity of task representations recorded in the orbitofrontal cortex during decision-making, which were themselves related to better task performance. Our findings suggest that hippocampal replay may be important for building representations of complex, abstract tasks elsewhere in the brain and establish feasibility of investigating fast replay signals with fMRI. Studies in rodents have shown that hippocampal representations of spatial locations are reactivated sequentially during short on-task pauses, longer rest periods, and sleep. This sequential reactivation, or replay, is accelerated relative to the original experience, related to better planning and memory consolidation, and suppression of replay-related sharp-wave ripples impairs spatial memory. The role of replay in nonspatial decision-making tasks in humans has remained unclear. We instructed participants to perform a nonspatial decision-making task in which correct performance depended on the sequential nature of “task states” that included information from past trials in addition to current sensory information (partially observable states). This ensured that participants would encode sequential information while completing the task. We recorded functional magnetic resonance imaging (fMRI) activity during resting periods before and after the task as well as during two sessions of task performance, and investigated whether sequences of fMRI activation patterns during rest reflected hippocampal replay of task states. Thirty-three participants performed a sequential decision-making task that required integration of information from past trials into a mental representation of the current task state. Each stimulus consisted of overlapping images of a face and a house, and participants made age judgments (old or young) about one of the images. An on-screen cue before the first trial determined whether the age of faces or houses should be judged. From the second trial onward, if the ages in the current and previous trial were identical, the category to be judged on the next trial remained the same; otherwise, the judged category was switched to the alternative. These task rules created an unsignaled “miniblock” structure in which each miniblock involved judgment of one category. No age comparison was required on the first trial after a switch. Miniblocks were therefore at least two trials long and on average lasted for three trials. We showed that fMRI patterns recorded from the human hippocampus during rest reflect sequential replay of task states previously experienced in an abstract, nonspatial decision-making task. Previous studies have relied on sustained fMRI activity patterns in the hippocampus or sensory cortex as evidence for replay, investigated wholebrain magnetoencephalography signals, or studied electroencephalography sleep spindles and memory improvements that are thought to index replay activity. Our study provides evidence of sequential offline reactivation of nonspatial decision-making states in the human hippocampus. Our results further suggest a role for hippocampal replay in supporting the integrity of on-task state representations in the orbitofrontal cortex. Hippocampal replay may support the offline formation or maintenance of a “cognitive map” of the task, deployed through the orbitofrontal cortex during decision-making. The interpretation of our findings as reflecting hippocampal replay was reinforced by systematic comparisons to several control conditions and simulations. Larger sample sizes for the important pre-task resting-state control condition could provide further support. Heart rates were equated between the different off-task conditions (wakeful rest with eyes open, and the instruction phase). More direct measures of vigilance could provide additional insight into the relationship between vigilance and replay. In animal studies, replay has been shown to be sequential and specific to hippocampal place cells. Unlike the majority of previous investigations in animals, the sequences of activation patterns reported here signify the replay of nonspatial, abstract task states. Our results therefore add to a growing literature proposing a substantial role for cognitive maps in the hippocampus in nonspatial decision-making. Our findings are in line with the idea that the human hippocampus samples previous task experiences to improve the current decision-making policy, a mechanism that has been shown to have distinct computational benefits for achieving fast and yet flexible decision-making. Dating back to Tolman, this idea requires a neural mechanism that elaborates on and updates abstract state representations of the current task, regardless of the task modality. The hippocampus and adjacent structures support a broad range of relational cognitive maps, as indicated by hippocampal encoding of not only spatial relations but also temporal, social, conceptual, or general contingency relations. We found that the human hippocampus not only represents these abstract task states but also performs sequential offline replay of these states during rest. One important open question concerns the temporal compression of the observed sequential reactivation. Previous results have indicated reactivation events in humans with a speed of around 40 ms per item. Although we provide evidence that our results could reflect fast sequential replay events with speeds similar to what was found in these reports, we cannot infer the speed of the replay directly from our observations. Our results hint at forward rather than reverse replay, which may suggest that in our experiment, replay was related more to memory function rather than planning because experienced task sequences did not contain natural endpoints or explicit rewards. Alternatively, decoding may have been dominated by the falling slope of hemodynamic responses, which could lead to order inversions. In this case, forward transitions would indicate backward replay. Although our findings clearly suggest asymmetrically directed reactivation, inferences about the direction of replay remain indirect. Last, our results imply a relationship between hippocampal replay and the representation of decision-relevant task states that are thought to reside in the orbitofrontal cortex. The relationship between “offline” hippocampal sequenceness and the fidelity of “online” orbitofrontal task-state representations raises the possibility that the hippocampus supports the maintenance and consolidation of state transitions that characterize the task and are used during decision-making. Given our findings—and recent evidence implicating hippocampal place cells and entorhinal grid cells in signaling nonspatial task-relevant stimulus properties —a crucial challenge is to further specify how flexible, task-specific representations in the hippocampus interact with task representations in other brain regions. Of particular interest are investigations asking whether neural populations in the hippocampus and entorhinal cortex share a common neural code for abstract task states with orbitofrontal and medial prefrontal regions, as suggested by recent studies.
|