Hawkins - On Intelligence

In an auto associative memory you don't have to have the entire pattern you want to retrieve in order to retrieve it.

Hawkins - On Intelligence

The auto-associative memory can retrieve the correct pattern, as it was originally stored, even though you start with a messy version of it.

Hawkins - On Intelligence

With a sequence of patterns, similar to a portion of a melody, the auto associative memory can remember the entire melody.

Hawkins - On Intelligence

People learn practically everything as a sequence of patterns.

Hawkins - On Intelligence

A neuron collects inputs from its synapses, and combines these inputs together to decide when to output a spike to other neurons.

Hawkins - On Intelligence

A typical neuron can cycle its functions and reset itself in about 5 ms, or around 200 times per second.

Hawkins - On Intelligence

The brain is a parallel “computer. “ It has billions of cells all computing at the same time.

Hawkins - On Intelligence

The brain does not compute the answers to problems. It retrieves the answers from memory.

Hawkins - On Intelligence

The neocortex is not like a computer, parallel or otherwise. Instead of computing answers to problems, the neocortex uses stored memories to solve problems and produce behavior.

Hawkins - On Intelligence

Neocortex stores sequences of patterns.

Hawkins - On Intelligence

Neocortex recalls patterns auto-associatively.

Hawkins - On Intelligence

Neocortex stores patterns in an invariant form.

Hawkins - On Intelligence

Neocortex stores patterns in a hierarchy.

Hawkins - On Intelligence

A story is stored in your head in a sequential fashion and can only be recalled in the same sequence.

Hawkins - On Intelligence

It's almost impossible to think of anything complex that isn't a series of events or thoughts.

Hawkins - On Intelligence

All memories are like this. You have to walk through the temporal sequence of how you do things.

Hawkins - On Intelligence

Your memory of the alphabet is a sequence of patterns.

Hawkins - On Intelligence

All memories are stored in the synaptic connections between neurons.

Hawkins - On Intelligence

An adult human neocortex has an incredibly large memory capacity but we can only remember a few at any time and can only do so in a sequence of associations.

Hawkins - On Intelligence

An autoassociative memory is one that can recall complete patterns when given only a partial or distorted input.

Hawkins - On Intelligence

Autoassociative memory can work for both spatial and temporal patterns.

Hawkins - On Intelligence

During conversation we often can't hear all the words we are in a noisy environment. Our brains fill in what they miss with what they expect to hear.

Hawkins - On Intelligence

Some people complete of the sentences of others aloud, but in our minds all of us are doing this constantly.

Hawkins - On Intelligence

When you see, feel, or hear something, the cortex takes the detailed highly specific input and converts it to an invariant form.

Hawkins - On Intelligence

Memory storage, memory recall, and memory recognition occur at the level of invariant forms.

Hawkins - On Intelligence

An important function of the neocortex is to use its memory to make predictions.

Hawkins - On Intelligence

Your brain does this by combining a memory of the invariant structure of her face with the particulars of your immediate experience.

Hawkins - On Intelligence

The combining of invariant representations and current input to make detailed predictions is exactly what is happening.

Hawkins - On Intelligence

This combining is a ubiquitous process that happens in every region of cortex.

Hawkins - On Intelligence

Three properties of cortical memory (storing sequences, autoassociative recall, and invariant representations) are necessary ingredients to predict the future based on memories of the past.

Hawkins - On Intelligence

The predictions are made in parallel and will just as readily detect an odd texture, a misshapen nose, or an unusual motion.

Hawkins - On Intelligence

What we perceive is a combination of what we sense and of our brain's memory=derived predictions.

Hawkins - On Intelligence

Visual areas make predictions about edges, shapes, objects, locations, and motions.

Hawkins - On Intelligence

Auditory areas make predictions about tones, direction to source, and patterns of sound.

Hawkins - On Intelligence

somatosensory areas make predictions about touch, texture, contour, and temperature.

Hawkins - On Intelligence

Correct predictions result in understanding.

Hawkins - On Intelligence

Prediction is the primary function of the neocortex, and the foundation of intelligence.

Hawkins - On Intelligence

Rodolfo Llinas, I of the Vortex, the capacity to predict the outcome of future events is most likely, the ultimate and most common of all global brain functions.

Hawkins - On Intelligence

There is an entire subfield of mathematics devoted to Bayesian networks.

Hawkins - On Intelligence

Bayesian networks use probability theory to make predictions.

Hawkins - On Intelligence

When listening to people speak, you often know what they're going to say before they finished speaking.

Hawkins - On Intelligence

People tend to use common phrases or expressions in much of their conversation.

Hawkins - On Intelligence

Prediction is not always exact. Rather, our minds work to make probabilistic predictions concerning what is about to happen.

Hawkins - On Intelligence

In Western music, the brain automatically predicts beats, repeated rhythms, completion of phrases, and end of songs.

Hawkins - On Intelligence

Intelligence is measured by the capacity to remember and predict patterns in the world including language, mathematics, physical properties of objects, and social situations.

Hawkins - On Intelligence

105

To recall the appropriate memories, the brain has to retrieve patterns by their similarity to past patterns (auto associative recall).

Hawkins - On Intelligence

105

Memories have to be stored in an invariant form so that the knowledge of past events can be applied to new situations that are similar but not identical to the past.

Hawkins - On Intelligence

113

As many if not more feedback connections in visual cortex as there are feedforward connections.

Hawkins - On Intelligence

115

Patterns received by the first auditory area can vary widely. A word can be spoken with different accents, and different pitches, or at different speeds.

Hawkins - On Intelligence

115

Higher up in the cortex, those low level features don't matter; a word is a word regardless of the acoustic details.

Hawkins - On Intelligence

115

We see the same kind of feedback, prediction, and invariant recall in auditory cortex as we see in the visual system.

Hawkins - On Intelligence

117

The downward flow fills in the current input and makes predictions about what we will experience next.

Hawkins - On Intelligence

119

These predictions can only come about by massive coordination of patterns streaming up and down the cortical hierarchy.

Hawkins - On Intelligence

119

We have an overarching sensory system, sights, sounds, touch, and more combined, all flowing up and down a single multi-branched hierarchy.

Hawkins - On Intelligence

119

You are not born with any of this knowledge; you learned it through the incredibly large capacity of your cortex to remember patterns.

Hawkins - On Intelligence

120

The way the cortex processes downward-flowing sensory predictions is similar to how it processes downward-flowing motor commands.

Hawkins - On Intelligence

120

Sensory patterns simultaneously flow in anywhere and everywhere, and then flow back down in any area of the hierarchy, leading to predictions or motor behavior.

Hawkins - On Intelligence

120

Seeing, hearing, touching, and acting are profoundly intertwined.

Hawkins - On Intelligence

122

V1,V2, and V4, each is a collection of many smaller subregions.

Hawkins - On Intelligence

122

The largest region by far is V1, the primary visual area.

Hawkins - On Intelligence

122

Next would be V2. They are large compared to most regions.

Hawkins - On Intelligence

122

V1 is made up of numerous separate little cortical areas that are only connected to their neighbors indirectly, through regions higher up in the hierarchy.

Hawkins - On Intelligence

123

The cortex now looks similar everywhere.

Hawkins - On Intelligence

123

Pick any region and you will find many lower regions providing converging sensory input.

Hawkins - On Intelligence

123

The receiving region sends projections back to the input regions, telling them what patterns they should expect to see next.

Hawkins - On Intelligence

123

Higher association areas unite information from multiple senses such as vision and touch.

Hawkins - On Intelligence

124

The most important result of this depiction of cortical hierarchy is that every region of cortex forms invariant representations.

Hawkins - On Intelligence

124

Now we can say that invariant representations are ubiquitous. Invariant representations are formed in every cortical region.

Hawkins - On Intelligence

125

Association regions above IT form invariant representations of patterns from multiple senses.

Hawkins - On Intelligence

125

All regions of the cortex form invariant representations of the world beneath them in the hierarchy.

Hawkins - On Intelligence

125

The real world's nested structure is mirrored by the nested structure of the cortex.

Hawkins - On Intelligence

127

The cortex has a clever learning algorithm that naturally finds whatever hierarchical structure exists and captures it.

Hawkins - On Intelligence

127

Higher regions of the cortex are keeping track of the big picture while lower areas are actively dealing with the fast-changing small details.

Hawkins - On Intelligence

128

If the patterns are related in such a way that the region can learn to predict what pattern will occur next, the cortical region forms a persistent representation, or memory, for the sequence.

Hawkins - On Intelligence

129

As information moves up from primary sensory regions to higher levels, we see fewer and fewer changes over time.

Hawkins - On Intelligence

129

In primary visual areas like V1, the set of active cells is changing rapidly as new patterns fall on the retina several times each second.

Hawkins - On Intelligence

129

In visual area IT, self firing patterns are more stable.

Hawkins - On Intelligence

129

Each region of cortex has a repertoire of sequences it knows.

Hawkins - On Intelligence

129

Each cortical region has a name for each sequence it knows.

Hawkins - On Intelligence

129

The "name" is a group of cells whose collected firing represents the set of objects in the sequence.

Hawkins - On Intelligence

129

These cells remain active as long as the sequences playing, and it is this “name” that is passed up to the next region in the hierarchy.

Hawkins - On Intelligence

130

We can imagine region IT at the top of the visual hierarchy relaying to an association area above it, "I am seeing a face."

Hawkins - On Intelligence

130

One region might recognize a sequence of sounds that comprise phonograms.

Hawkins - On Intelligence

130

The next higher region recognizes sequences the phonems to create words.

Hawkins - On Intelligence

130

The next higher region recognizes sequences of words to create phrases, and so on.

Hawkins - On Intelligence

130

By collapsing predictable sequences into named objects at each region in the hierarchy, we achieve more and more stability the higher we go.

Hawkins - On Intelligence

130

This creates invariant representation.

Hawkins - On Intelligence

130

The opposite effect happens as a pattern moves back down the hierarchy: stable patterns get unfolded into sequences.

Hawkins - On Intelligence

131

At this point the unfolding pattern splits and travels down both the auditory section of cortex and the motor section of cortex.

Hawkins - On Intelligence

131

In the final bottom region, each phonem is unfolded into a sequence of muscle commands to make sounds.

Hawkins - On Intelligence

131

The lower you look in the hierarchy, the faster the patterns are changing.

Hawkins - On Intelligence

131

A single constant pattern at the top of the motor hierarchy eventually leads to a complex and lengthy sequence of speech sounds.

Hawkins - On Intelligence

133

The bottom-up inputs to a region of cortex are input patterns carried on thousands or millions of axons.

Hawkins - On Intelligence

133

The number of possible patterns that can exist on even 1000 axons is larger than the number of molecules in the universe.

Hawkins - On Intelligence

134

Both steps, classification and sequence formation, are necessary to create invariant representations, and each region of cortex does them.

Hawkins - On Intelligence

135

You use the context of known sequences to resolve ambiguity

Hawkins - On Intelligence

135

Most of the time you are aware that you are filling in ambiguous or incomplete information from your memories or sequence

Hawkins - On Intelligence

135

By recognizing a sequence of patterns, a cortical region will predict its next input pattern and tell the region below what to expect.

Hawkins - On Intelligence

135

A region of cortex not only learns familiar sequences, it also learns how to modify its classifications.

Hawkins - On Intelligence

136

In cortical regions, bottom-up classifications and top-down sequences are constantly interacting, changing throughout your life.

Hawkins - On Intelligence

136

Forming new classifications and new sequences is how you remember the world.

Hawkins - On Intelligence

136

Another part of the cortical job is to relay the “name,” of the sequence you are seeing to the next level up.

Hawkins - On Intelligence

137

The hierarchy of the cortex ensures that memories of objects are distributed over the hierarchy; they aren't located in a single spot.

Hawkins - On Intelligence

137

Because each region of the hierarchy forms invariant memories, what a typical region of cortex learns is sequences of invariant. representations, which are themselves sequences of invariant memories

Hawkins - On Intelligence

138

Cortical regions vary greatly in size, the largest being the primary sensory areas.

Hawkins - On Intelligence

138

Let's assume that a typical cortical area is the size of a small coin.

Hawkins - On Intelligence

139

The density and shape of the cells in the cortex vary as you move from top to bottom.

Hawkins - On Intelligence

139

These differences define the layers.

Hawkins - On Intelligence

139

Layer 1, the top layer is the most distinct of the six layers. It has very few cells consisting primarily of a mat of axons running parallel to the cortical surface

Hawkins - On Intelligence

139

Layers 2 and 3 looks similar. They contain many, tightly packed. pyramidal cells.

Hawkins - On Intelligence

139

Layer 4 has a type of star-shaped cell.

Hawkins - On Intelligence

139

Layer 5 has regular pyramidal cells as well as a class of extra big pyramidal-shaped cells.

Hawkins - On Intelligence

139

Layers 6, the bottom layer also has several types of unique neurons.

Hawkins - On Intelligence

139

Columns of cells run perpendicular to the layers. You can think of columns as being vertical units of cells that work together.

Hawkins - On Intelligence

139

The layers within each column are connected by axons that run up and down, making synapses along the way.

Hawkins - On Intelligence

139

Columns do not have clear boundaries but their existence can be inferred from several lines of evidence.

Hawkins - On Intelligence

139

The cells within each column are strongly connected.

Hawkins - On Intelligence

140

Activity spreads up and down within a column of cells.

Hawkins - On Intelligence

140

In an embryo, single precursor cells migrate from an inner brain cavity to where the cortex takes take shape.

Hawkins - On Intelligence

140

Each of these cells divides to create about 100 neurons, called a microcolumn.

Hawkins - On Intelligence

140

The human cortex has an estimated several hundred million microcolumns.

Hawkins - On Intelligence

140

Imagine a single microcolumn is the width of a human hair.

Hawkins - On Intelligence

140

The brush like mat is a simplistic model of the coin size cortical region.

Hawkins - On Intelligence

140

And information flows mostly in the direction of the hairs: horizontally in layer 1 and vertically in layers 2 through 6

Hawkins - On Intelligence

140

At least 90% of the synapses on cells within each column come from places outside the column itself.

Hawkins - On Intelligence

140

Some connections arrived from neighboring column.

Hawkins - On Intelligence

140

Other connections come from halfway across the brain.

Hawkins - On Intelligence

140

Vernon Mountcastle argued there is a single cortical algorithm, he also proposed a cortical column is the basic unit of computation in the cortex.

Hawkins - On Intelligence

141

It is believed that the column is the basic unit of prediction.

Hawkins - On Intelligence

141

Converging inputs from lower regions always arrive at layer 4, the main input layer.

Hawkins - On Intelligence

141

Layer 4 cells then send projections up to cells in layer 2 and layer 3 within their column.

Hawkins - On Intelligence

141

Layer 6 cells are the downward projecting output cells from a cortical column and project to layer 1 in the region hierarchically below.

Hawkins - On Intelligence

153

The sum of all these mechanisms allows the cortex to learn sequences, make predictions, and form constant representations, or "names," for sequences.

Hawkins - On Intelligence

168

The neocortex is responsible for all complex motor sequences and can directly control your limbs.

Hawkins - On Intelligence

170

Connections between the hippocampus and the neocortex suggest that the hippocampus is the top region of the neocortex.

Hawkins - On Intelligence

170

The hippocampus occupies the peak of the neocortical pyramid.

Hawkins - On Intelligence

170

The hippocampus not only sits at the top of the cortical pyramid, but it still connects directly to many older parts of the brain.

Hawkins - On Intelligence

170

Think about the information flowing from your eyes, ears, and skin into the neocortex.

Hawkins - On Intelligence

170

Each region of the neocortex tries to understand the input in the in terms of the sequences it knows.

Hawkins - On Intelligence

170

If it does understand the input it does not pass on the details to higher levels of the hierarchy.

Hawkins - On Intelligence

170

If a region does not understand the current input, it passes it up the hierarchy until some higher region does understand it.

Hawkins - On Intelligence

170

A pattern that is truly novel will escalate further and further up the hierarchy until some higher region does understand it.

Hawkins - On Intelligence

170

The net effect is that when you get to the top of the cortical pyramid, what you have left is information that can't be understood by prior experience.

Hawkins - On Intelligence

171

You are left with the part of the input that is truly new and unexpected.

Hawkins - On Intelligence

171

It is these unexplained and unanticipated remainders, the new stuff, that enters the hippocampus and is stored there.

Hawkins - On Intelligence

171

This new, fresh information won't be stored forever. Either it will be transferred into the cortex or it will eventually be lost.

Hawkins - On Intelligence

171

The hippocampus has a heterogeneous structure with several specialized regions. It's good at the unique task of quickly storing whatever pattern it sees.

Hawkins - On Intelligence

171

You can instantly remember a novel event in the hippocampus, but you will permanently remember something in the cortex only if you experience it over and over, either in reality or by thinking of it.

Hawkins - On Intelligence

171

The cortex has a second major pathway for passing information from region to region, up the hierarchy.

Hawkins - On Intelligence

172

The alternate path starts with cells in layer 5 which project to the thalamus and then up to the next higher region of cortex.

Hawkins - On Intelligence

172

As we move up the cortical hierarchy, there is a direct path between two regions and an indirect path through the thalamus.

Hawkins - On Intelligence

173

The alternate pathway through the thalamus is likely the mechanism by which we attend to details that normally we would not notice.

Hawkins - On Intelligence

173

It bypasses the grouping of sequences in layer 2, sending the raw data to the next higher region of cortex.

Hawkins - On Intelligence

174

In this way unusual events quickly rise to your attention.

Hawkins - On Intelligence

174

This is why we can't avoid focusing on deformities and other unusual patterns.

Hawkins - On Intelligence

174

Often however, errors aren't strong enough to open the alternate pathway. This is why we sometimes don't notice if a word is misspelled as we read it.