Edmund Rolls; Neural Networks and Brain Function
Book	Page		Topic
Rolls & Treves; Neural Networks	6		Different types of activation function (diagram)
Rolls & Treves; Neural Networks	7		A simple learning rule that was originally presaged by Donald Hebb (1949) proposes that synapses increase in strength when there is conjunctive presynaptic and postsynaptic activity.		1
Rolls & Treves; Neural Networks	7		Presynaptic and postsynaptic activity must be present approximately simultaneously (to within perhaps 100-500 ms).		0
Rolls & Treves; Neural Networks	7		Long-term potentiation (LTP) and long term depression (LTD) provide useful models of some of the synaptic modifications that occur in the brain.		0
Rolls & Treves; Neural Networks	8		Long-term potentiation (LTP) is the use-dependent and sustained increase in synaptic strength that can be induced by brief periods of synaptic stimulation.		1
Rolls & Treves; Neural Networks	8		LTP is long-lasting, in that its effect can be measured for hours in hippocampal slices, and in chronic in vivo experiments in some cases may last for months.		0
Rolls & Treves; Neural Networks	8		LTP becomes evident rapidly, typically in less than one minute.		0
Rolls & Treves; Neural Networks	11		Long term depression (LTD) can also occur.		3
Rolls & Treves; Neural Networks	12		A sparse distributed representation is a distributed representation in which a small proportion of the neurons is active at any one time.		1
Rolls & Treves; Neural Networks	13		One advantage distributed encoding is that the similarity between two representations can be reflected by the correlation between the two patterns of activity.		1
Rolls & Treves; Neural Networks	13		Another advantage is distributing encoding is that the number of different stimuli that can be represented can be very large.		0
Rolls & Treves; Neural Networks	13		In the high-order visual cortex in the temporal lobe of primates, the number of faces that can be represented increases approximately exponentially with the number of neurons in the population.		0
Rolls & Treves; Neural Networks	19		After mainly unimodal processing stages, information processing streams converge into a number of areas, particularly the amygdala and orbitofrontal cortex.		6
Rolls & Treves; Neural Networks	19		Convergence areas appear to be necessary for learning to associate sensory stimuli with other reinforcing (rewarding or punishing) stimuli.		0
Rolls & Treves; Neural Networks	19		The amygdala that is involved in learning associations between the sight of food and its taste.		0
Rolls & Treves; Neural Networks	19		Taste is a primary or innate reinforcer.		0
Rolls & Treves; Neural Networks	19		The orbitofrontal cortex is especially involved in rapidly relearning associations when environmental contingencies change.		0
Rolls & Treves; Neural Networks	19		In the orbitofrontal cortex, association learning is also used to produce a representation of flavor, resulting from activation by both olfactory and taste stimuli.		0
Rolls & Treves; Neural Networks	22		Hippocampus receives inputs from both the 'what' and the 'where' systems.		3
Rolls & Treves; Neural Networks	22		By rapidly learning associations between conjunctive inputs from the 'what' and the 'where' systems, the hippocampus is able to form memories of particular events occurring in particular places at particular times.		0
Rolls & Treves; Neural Networks	24		Pattern association memory (diagram)		2
Rolls & Treves; Neural Networks	30		Graceful degradation or fault tolerance.		6
Rolls & Treves; Neural Networks	36		With a pattern associator, very little will be retrieved if too many associations are simultaneously in storage and/or if too little is provided as input.		6
Rolls & Treves; Neural Networks	42		Autoassociative memories, or attractor neural networks, store memories, each one of which is represented by a pattern of neural activity. They can then recall the appropriate memory from the network when provided with a fragment of one of the memories.		6
Rolls & Treves; Neural Networks	43		Architecture of an auto associative neural network (diagram)		1
Rolls & Treves; Neural Networks	46		Perhaps the most important and useful property of autoassociative memories is that they complete an incomplete input vector, allowing recall of a whole memory from a small fragment of it.		3
Rolls & Treves; Neural Networks	46		The memory recalled in response to a fragment is that stored in the memory that is closest in pattern similarity (as measured by the dot product, or correlation).		0
Rolls & Treves; Neural Networks	46		The autoassociative property at pattern associative neural networks is very similar to the properties of human memory.		0
Rolls & Treves; Neural Networks	56		Feature discovery by self-organization.		10
Rolls & Treves; Neural Networks	56		Each neuron in a competitive network becomes activated by a set of consistently coactive, that is correlated, input axons, and gradually learns to respond to that cluster of coactive inputs.		0
Rolls & Treves; Neural Networks	56		We can think of competitive networks as discovering features in the input space where features can be defined it is set of consistently coactive inputs.		0
Rolls & Treves; Neural Networks	56		Feature analyzers respond to correlations in the input space, and their learning occurs via self-organization in the competitive network.		0
Rolls & Treves; Neural Networks	62		Brain systems in which competitive networks may be used for orthogonalization and sparsification.		6
Rolls & Treves; Neural Networks	62		Removal of redundancy by competition is thought to be a key aspect of how sensory systems operate.		0
Rolls & Treves; Neural Networks	63		Neurons that respond to correlated combinations of their inputs can be described as feature analyzers.		1
Rolls & Treves; Neural Networks	63		Neurons that act is feature analyzers perform useful preprocessing in many sensory systems.		0
Rolls & Treves; Neural Networks	63		The power of competitive networks in multistage hierarchical processing is to build combinations of what is found in earliest stages and thus effectively to build higher-order representations.		0
Rolls & Treves; Neural Networks	63		The consequences of sensory stimuli could be fed back to sensory systems to influence the categories formed.  This could be one function of the backprojections in the cerebral cortex.		0
Rolls & Treves; Neural Networks	63		Inputs would consist of forward inputs from a preceding cortical area, and backprojecting axons in the next cortical area, or from a structures as it is the amygdala to call hippocampus.		0
Rolls & Treves; Neural Networks	65		Flavor circuit diagram of taste input and olfaction input.		2
Rolls & Treves; Neural Networks	95		Hippocampus and Memory		30
Rolls & Treves; Neural Networks	107		The hippocampus receives, via the adjacent parahippocampal gyrus and entorhinal cortex inputs from virtually all association areas in the neocortex, including those in the parietal, temporal, and frontal lobes.
Rolls & Treves; Neural Networks	107		The hippocampus has available highly elaborated multimodal information that has already been processed extensively along different, and partially interconnected, sensory pathways.
Rolls & Treves; Neural Networks	107		The provision of a single network for association between any subset of neurons (the CA3 neurons) may be an important feature of what the hippocampus circuitry provides for memory.
Rolls & Treves; Neural Networks	107		Additional inputs to the hippocampus come from the amygdala that and, via a separate pathway, from the cholinergic and other regulatory systems.
Rolls & Treves; Neural Networks	108		To complement the massively convergent set of pathways into the hippocampus, there is a massively divergent set of backprojecting pathways from the hippocampus via the entorhinal cortex to the cerebral neocortical areas that provide inputs to the hippocampus.		13
Rolls & Treves; Neural Networks	108		Within the hippocampus, information is processed along a mainly unidirectional path, consisting of three major stages.		0
Rolls & Treves; Neural Networks	108		Axonal projections mainly from layer 2 of entorhinal cortex reach the granule cells in the dentate gyrus via the perforant path (PP), and also proceed to make synapses on the apical dendrites of pyramidal cells in the next stage, CA3.		0
Rolls & Treves; Neural Networks	108		A different set of fibers projects from the entorhinal cortex (mainly layer 3) directly onto the third processing stage, CA1.		0
Rolls & Treves; Neural Networks	109		Connections within the Hippocampus -- (diagram)		1
Rolls & Treves; Neural Networks	110		The extrinsic axonal projections from CA3, the Schaffer collaterals, provide the major inputs to CA1 pyramidal cells.		1
Rolls & Treves; Neural Networks	110		Information appears to be funneled from the dentate gyrus (DG) through the CA3 bottleneck, and then spread out again into CA1. The output of CA1 returns directly, and via the subiculum, to the entorhinal cortex, from which it is redistributed to neocortical areas.		0
Rolls & Treves; Neural Networks	112		CA3 network as an autoassociative memory.		2
Rolls & Treves; Neural Networks	118		The dentate granule cell stage of hippocampal processing, which precedes the CA3 stage, acts to produce during learning, the sparse yet efficient (i.e. non-redundant) representation in CA3 neurons that is required for the autoassociation to perform well.		6
Rolls & Treves; Neural Networks	120		CA3 autoassociation effect -- several arbitrary patterns of firing occur together on the CA3 neurons, and become associated together to form an episodic or 'whole scene' memory.		2
Rolls & Treves; Neural Networks	120		The amount of information about each episode retrievable from CA3 has to be balanced against the number of episodes that can be held concurrently in storage.		0
Rolls & Treves; Neural Networks	120		The total amount of information that can be stored and that autoassociation memory is approximately constant so that there must be a trade-off between the number of patterns stored and the amount of information in each pattern.		0
Rolls & Treves; Neural Networks	121		The CA1 network as a double set of afferents, one set from the Schaffer collaterals coming from CA3, and a second set from the direct perforant path projections from entorhinal cortex.		1
Rolls & Treves; Neural Networks	121		The existence of the double set of inputs to CA1 appears to indicate that there is a need for a system of projections that makes available, after the output of CA3, some information closely related to that which was originally given as input to the hippocampal formation.		0
Rolls & Treves; Neural Networks	122		The perforant path projection may serve during retrieval to integrate the information-reduced description of the full event recalled from CA3, with the information-richer description of only those elements used as a cue provided by the entorhinal/perforant path signal. This integration may be useful both in the consolidation of longer-term storage, and the immediate use of the retrieved memory.		1
Rolls & Treves; Neural Networks	122		Backprojections to the neocortex, and recall from the hippocampus.		0
Rolls & Treves; Neural Networks	124		Recall of information from the hippocampus to the cerebral cortex.		2
Rolls & Treves; Neural Networks	130		Widespread connectivity of the CA3 recurrent collaterals is important for good retrieval from incomplete cues restricted to one part of the topographically organized inputs.		6
Rolls & Treves; Neural Networks	136		Pattern Association in the Brain -- Amygdala and Orbitofrontal Cortex		6
Rolls & Treves; Neural Networks	138		Emotions associated with different reinforcement contingencies (diagram)		2
Rolls & Treves; Neural Networks	146		Bilateral damage to the amygdala that produces a deficit in learning to associate visual and other stimuli with a primary (i.e. unlearned) reward or punishment.		8
Rolls & Treves; Neural Networks	146		The amygdala is a subcortical region in the anterior part of the temporal lobe.		0
Rolls & Treves; Neural Networks	146		Amygdala receives massive projections from the visual and auditory temporal lobe cortex.		0
Rolls & Treves; Neural Networks	146		Amygdala receives inputs from the inferior temporal visual cortex, but not from earlier stages of cortical visual information processing.		0
Rolls & Treves; Neural Networks	146		The amygdala also receives inputs that are potentially about primary reinforcers, for example taste inputs (from the secondary taste cortex, via connections from the orbitofrontal cortex to the amygdala).		0
Rolls & Treves; Neural Networks	146		The amygdala also receives somatosensory inputs, potentially about the rewarding or painful aspects of touch (from the somatosensory cortex via the insula).		0
Rolls & Treves; Neural Networks	146		Outputs of the amygdala include projections to the hypothalamus.		0
Rolls & Treves; Neural Networks	151		The orbitofrontal cortex is strongly connected to the amygdala.		5
Rolls & Treves; Neural Networks	153		Inputs to the orbitofrontal cortex include many of those required to determine whether a visual or auditory stimulus is associated with a primary reinforcer such as taste or smell.		2
Rolls & Treves; Neural Networks	154		The orbitofrontal cortex is involved in the affective, reward-related, representation of touch.		1
Rolls & Treves; Neural Networks	165		Cortical Networks for Invariant Pattern Recognition		11
Rolls & Treves; Neural Networks	177		There is considerable evidence that the processing time required in each cortical area for useful computation is of the order of 20-30 ms.		12
Rolls & Treves; Neural Networks	177		It is unlikely that the processing time in each cortical area is sufficiently long for a stochastic iterative process, or for temporal encoding and synchronization of multiple different populations of neurons.		0
Rolls & Treves; Neural Networks	177		There must be useful information emanating from the first 15-20 ms of processing in each cortical area in the hierarchy.		0
Rolls & Treves; Neural Networks	178		Learning to identify new objects or images never seen before can occur very rapidly.  Just a very few seconds of seeing a new face or picture will enable us to recognize it later.		1
Rolls & Treves; Neural Networks	189		Motor Systems -- Cerebellum and Basal Ganglia		11
Rolls & Treves; Neural Networks	189		The cerebellum has a remarkably a regular neural network architecture.		0
Rolls & Treves; Neural Networks	189		Basal ganglia are of great interest because they are implicated in motor habit learning, might play a role in interfacing many cerebral cortical areas to systems for behavioral output, or at least in allowing some interaction between different cortical systems competing for output, and because they have considerable regularity in their structure.		0
Rolls & Treves; Neural Networks	189		The cerebellum is involved in accurate control of movements.		0
Rolls & Treves; Neural Networks	189		If the cerebellum is damaged, movements can still be initiated, but the movements are not directed accurately at the target, and frequently oscillate on the way to the target.		0
Rolls & Treves; Neural Networks	189		There is insufficient time during rapid movements for feedback control to operate, and the hypothesis is that the cerebellum performs feedforward control by learning to control the motor commands to the limbs and body in such a way that movements are smooth and precise.		0
Rolls & Treves; Neural Networks	189		The cerebellum is a system for adaptive feedforward motor control.		0
Rolls & Treves; Neural Networks	206		Basal ganglia are parts of the brain that include the striatum, globus pallidus, substantia nigra, and  subthalamic nucleus and are necessary for normal initiation of movement.		17
Rolls & Treves; Neural Networks	206		The basal ganglia receive inputs from all parts of the cerebral cortex, including the motor cortex, and have outputs directed strongly toward the premotor and prefrontal cortex by which they could influence movement initiation.		0
Rolls & Treves; Neural Networks	206		The general connectivity through the basal ganglia is for cortical or limbic inputs to reach the striatum, which then project to the globus pallidus and substantia nigra, which in turn project by the thalamus back to the cerebral cortex.  Within this overall scheme, there are a set of at least partially segregated parallel processing streams.		0
Rolls & Treves; Neural Networks	209		Damage to the striatum produces effects which suggests that it is involved in orientation to stimuli and to initiation and control of movement.		3
Rolls & Treves; Neural Networks	209		Lesions of the dopamine pathways that deplete the striatum of dopamine lead to a failure to orient to stimuli.		0
Rolls & Treves; Neural Networks	209		In humans, depletion of dopamine in the striatum is found in Parkinson's disease, in which there is akinesia, i.e. a lack of voluntary movement, bradykineaia, rigidity and tremor.		0
Rolls & Treves; Neural Networks	227		Cerebral Neocortex		18
Rolls & Treves; Neural Networks	227		In addition to forward connections from one area to the next in the cerebral neocortex, there also is many projections backwards to the preceding cortical area.		0
Rolls & Treves; Neural Networks	227		The neocortex consists of many areas which can be distinguished by the appearance of the cells (cytoarchitecture) and the fibers of axons (myeloarchitecture), but nevertheless, the basic organization of the different neocortical areas has many similarities.		0
Rolls & Treves; Neural Networks	227		Pyramidal cells with cell bodies in different laminate of the cortex not only have different distributions of their dendrites, but also different distributions of their axons.		0
Rolls & Treves; Neural Networks	228		The main information bearing afferents to a cortical area have many terminals in layer 4.(By these afferents, we mean primarily those from the thalamus or from the preceding cortical area.  We do not mean the cortico-cortical back projections, nor the subcortical cholinergic, noradrenergic, dopaminergic, and serotoninergic inputs which are numerically minor, although they are important in setting cortical cell thresholds, excitability, and adaptation.)		1
Rolls & Treves; Neural Networks	228		In primary sensory cortical areas there are spiny stellate cells in a rather expanded layer 4, and the thalamic terminals synapse onto the cells.		0
Rolls & Treves; Neural Networks	228		Primary sensory cortical areas receive their inputs from the primary sensory thalamic nucleus for a sensory modality.		0
Rolls & Treves; Neural Networks	228		Primate striate cortex receives inputs from the lateral geniculate nucleus, which in turn receives from the retinal ganglion cells.		0
Rolls & Treves; Neural Networks	228		Spiny stellate cells are so-called but called they have radially arranged, star-like dendrites. Their axons usually terminate within the cortical area in which they are located.		0
Rolls & Treves; Neural Networks	228		In addition to spiny stellate cell terminals, there are some terminals of thalamic afferents onto pyramidal cells with cell bodies in layers 6 and 3, and also terminals onto inhibitory interneurons such as basket cells, which thus provide for a feedforward inhibition.		0
Rolls & Treves; Neural Networks	232		Inhibitory cells and connections		4
Rolls & Treves; Neural Networks	240		Neural processing could operate through a hierarchy of cortical stages.  Convergence and competition are key aspects of the system.		8
Rolls & Treves; Neural Networks	240		Neurons in each cortical stage respond for 20-30 ms when an object can just be seen.		0
Rolls & Treves; Neural Networks	240		The time from V1 to inferior temporal cortex takes about 50 ms.  There is insufficient time for a return projection from IT to reach V1, influence processing there, and in turn for V1 to project up to IT to alter processing there. Nevertheless, back projections are a major feature of cortical connectivity.		0
Rolls & Treves; Neural Networks	240		The amygdala and hippocampus are stages of information processing at which the different sensory modalities (such as vision, hearing, touch, taste, and smell for the amygdala) are brought together, so that correlations between inputs in different modalities can be detected, but not at prior unimodal cortical processing stages.		0
Rolls & Treves; Neural Networks	243		During recall, neural activity does occur in cortical areas involved in the original processing.		3
Rolls & Treves; Neural Networks	243		Research has shown that when humans are asked to recall visual scenes in the dark, bloodflow is increased in visual cortical areas, stretching back from association cortical areas as far as early (possibly even primary) visual cortical areas.		0
Rolls & Treves; Neural Networks	243		Recall is that function that could be produced by cortical back projections.		0
Rolls & Treves; Neural Networks	243		There are many NMDA receptors on the apical dendrites of cortical pyramidal cells, where the backprojection synapses terminate. These receptors are implicated in associative modifiability of synapses.		0
Rolls & Treves; Neural Networks	243		Plasticity is very evident in the superficial layers of the cerebral cortex.		0
Rolls & Treves; Neural Networks	243		Backprojection synapses in ending on the apical dendrite, quite far from the cell body, might be expected to be sufficient to dominate the cell firing when there is no forward input close to the cell body.		0
Rolls & Treves; Neural Networks	243		When there is forward input to the neuron, activating synapses closer to the cell body than the backprojecting inputs, this would tend to electrically shunt the effects received on the apical dendrite.		0
Rolls & Treves; Neural Networks	243		During the original learning, the forward input would have a stronger effect on the activation of the cell, with mild guidance than being provided by the back projections.		0
Rolls & Treves; Neural Networks	243		As an example of how recall could operate, consider the situation in the visual system the sight of food is forward projected onto pyramidal cells and the higher order visual cortex, and conjunctively there is a  backprojected representation of the taste of the food from, the amygdala to or orbitofrontal cortex.		0
Rolls & Treves; Neural Networks	243		Neurons that have conjunctive inputs from two stimuli set up representations of both, so that later if only one representation is back projected, then the visual neurons originally activated by the sight of that food will be activated.		0
Rolls & Treves; Neural Networks	243		Using neurons with learned, conjunctive inputs, many other low-level details of the original visual stimulus might be recalled.		0
Rolls & Treves; Neural Networks	243		Research has shown that during recall relatively early cortical processing stages are activated in quite posterior visual areas for visual (but not auditory) scenes.  The backprojections are probably acting as pattern associators.		0
Rolls & Treves; Neural Networks	244		Quantitative analysis of the recall that could be implemented through the hippocampal back projections synapses to the neocortex, and then via multiple stages of corticocortical back projections, make it clear that the most important quantitative factor influencing the number of memories that can be recalled is the number of back projecting synapses onto each cortical neuron in the back projecting pathways.		1
Rolls & Treves; Neural Networks	244		An interpretation of why there are in general as many backprojecting synapses between two cortical areas as forward connections. The average number of a back projections needs to be large to recall as many memories is possible, but need not be larger than the number of forward inputs to each neuron, which influences the number of possible classifications that each neuron can perform with its forward inputs.		0
Rolls & Treves; Neural Networks	244		If backprojections are used for recall, it would place severe constraints on their use for functions such as error backpropagation.		0
Rolls & Treves; Neural Networks	244		It will be difficult to use the backprojections in cortical architecture to convey an appropriate error signal from an output layer back to the earlier layers if the backprojection synapses are also to be set up associatively to implement recall.		0
Rolls & Treves; Neural Networks	244		Backprojection architecture could implement semantic priming by using backprojecting neurons to provide a small activation of just those neurons that are appropriate for responding to the semantic category of input stimulus.		0
Rolls & Treves; Neural Networks	244		Attention could operate from higher to lower levels, to selectively facilitate only certain pyramidal cells by using backprojections.		0
Rolls & Treves; Neural Networks	244		Backprojections could produce many of the top-down influences that are common in perception.		0
Rolls & Treves; Neural Networks	245		There are a number of different short-term memory systems, each implemented in a different cortical area.		1
Rolls & Treves; Neural Networks	245		The particular systems considered here implements short-term memory by subpopulations of neurons that show maintained activity in the delay, while a stimulus or event is being remembered.		0
Rolls & Treves; Neural Networks	246		The short-term memories may operate as autoassociative attractor networks.		1
Rolls & Treves; Neural Networks	246		Autoassociation could be implemented by associatively modifiable synapses between connected pyramidal cells within an area, or between adjacent cortical areas in a hierarchy.		0
Rolls & Treves; Neural Networks	246		One short-term memory system is in dorsolateral prefrontal cortex, area 46. This is involved in remembering the locations of spatial responses.		0
Rolls & Treves; Neural Networks	246		Another short-term memory system is implemented in the inferior temporal visual cortex. This memory is for whether a particular visual stimulus (such as the face) has been seen recently.		0
Rolls & Treves; Neural Networks	247		Another short-term memory system is human auditory-verbal short-term memory, which appears to be implemented in the left hemisphere at the junction of the temporal, parietal, and occipital lobes.		1
Rolls & Treves; Neural Networks	249		A sparse distributed representation is a distributed representation in which a small proportion of the neurons is active at any one time.		2
Rolls & Treves; Neural Networks	249		One researcher suggested that 1000 active neurons (called cardinal cells) might represent the whole of a visual scene.  An important principle involved in forming such a representation is a reduction of redundancy.		0
Rolls & Treves; Neural Networks	250		In the higher parts of the visual system in the temporal lobe visual cortical areas, there are neurons that are tuned to respond to faces.		#REF!
Rolls & Treves; Neural Networks	253		An information theory approach makes it clear that there is considerable information in distributed coding of temporal visual cortex neurons.		3
Rolls & Treves; Neural Networks	259		The number of stimuli encodable by the population of neurons might be expected to rise exponentially as the  number of neurons in the sample of the population is increased.		6
Rolls & Treves; Neural Networks	260		Neurophysiological evidence demonstrates that the encoding is distributed and the responses are sufficiently independent and reliable that the representational capacity increases exponentially.		1
Rolls & Treves; Neural Networks	260		Large numbers of stimuli and fine discriminations between them can be represented without having to measure the activity of an enormous number of neurons.		0
Rolls & Treves; Neural Networks	261		Capacity of representation rises exponentially with the number of neurons.		1
Rolls & Treves; Neural Networks	262		Part of the biological significance of exponential encoding capacity is that the receiving neuron or neurons can obtain information about which one of a very large number stimuli is present by receiving the rate of firing of relatively small numbers of inputs from each of the neuronal populations from which it receives.		1
Rolls & Treves; Neural Networks	263		The properties and representation of faces, and of olfactory  and taste stimuli, have been evident when the readout of the information was by measuring the firing rate of neurons, typically over a 500 ms period.		1
Rolls & Treves; Neural Networks