This post aims to explain the most fundamental difference
between computers and brains. It involves how the two use and instantiate
memory. As you will see, there are two differences. Both computers and brains
have a form of short-term memory that is updated iteratively, but in the brain
the set of items in short-term memory are bound, and integrated to create
composite representations. Computers don’t do this, their short-term memories
are composed of completely disconnected items that do not form any type of
context. Secondly, short-term memory in the brain is used for search. In the
computer it is not. A computer’s short-term memory only speeds up retrieval
from long-term memory. The next state of a computer is not determined by its
short-term memory. It is determined by the code in the program it is running.
The next state of the brain is determined by what is currently active in
short-term memory. This makes its short-term memory qualify as working memory.
Computers don’t have working memory, and until they do they cannot be
conscious, or display human-like intelligence.
The Differences Between Computer and Human Short-term Memory at a Glance:
1) The memory hierarchy in computers is composed of separate modules and information must be transferred between them. In the brain these modules are embedded within one another.
2) Short-term memory in the brain is bound and integrated together to create context. In a computer short-term memories appear as a list and are disconnected from each other.
3) In the brain short-term memory is used to search for the next update. In a computer it merely remains available in case it is called upon.
To retrieve information from long-term memory a computer copies bytes of information (such as 01100101) from the storage drive (HHD, or SSD) to the CPU. These bytes (which comprise data or program instructions) are processed inside the CPU. If it is likely to need this information again soon, it will save it in a form of short-term memory such as CPU cache, or RAM. It saves temporary copies of information that will be used again in smaller storage closer to the CPU. It does this because retrieval from the large capacity long-term memory is very slow. Oversimplifying, you can think of data that is expected to be needed on the order of milliseconds to seconds as being stored in CPU cache (L1, L2, L3). Data expected to be needed on the order of seconds to minutes is stored in RAM. The rest of the data, which may or may not be needed, is stored in long-term memory.
Brains do something very similar. Neurons hold information
in long-term memory and when certain memories are needed the neurons that
encode that information become active. Just as in the computer, retrieval from
long-term memory is slow, so the brain uses short-term memory to potentiate
information that is likely to be used again soon. In the brain there are also
at least two levels of activation. The first level of short-term memory is
sustained firing. A neuron exhibits sustained firing when it continues to
physically fire at the neurons it is connected to at an elevated rate. The
second level is synaptic potentiation. This is where a chemical change to the
neuron’s synapses makes the information that the neuron encodes more readily
available, even after it has stopped firing. Information in sustained firing is
very similar to the information in the CPU’s cache (the store is small but
fast), and information maintained through synaptic potentiation is similar to
information in RAM (the store is larger, but slower).
Computer
|
Brain
|
Time Scale
|
CPU
Register
|
Cortical
Binding
|
Very
short-term working memory (millis.)
|
CPU
Cache (SRAM)
|
Sustained
Firing
|
Short-term
working memory (seconds)
|
RAM
|
Synaptic
Potentiation
|
Short-term
memory (seconds to hours)
|
Virtual
Memory
|
Short-term
Potentiation
|
Short-term
memory (minutes to hours)
|
SSD
|
Commonly
used LTM
|
Accessible
Long-term Memory
|
Hard
Drive
|
Long-term
Memory
|
Long-term
Storage (days to lifetime)
|
Computers are often described as having a “memory
hierarchy,” expressed as a pyramid with different levels of memory storage.
This is also referred to as a caching hierarchy, because there are different
levels of short-term memory with different associated speeds. The levels at the
top are faster, smaller, and more energetically expensive. The same could be
said for human memory, because there are several levels, the levels at the top
are faster, and more metabolically expensive. The table above is my attempt to
compare the levels of these two hierarchies.
Sustained firing and synaptic potentiation together are
referred to as working memory, a psychological construct that overlaps with
attention, and consciousness. Working memory is composed of concepts that are
persistently coactive for a period of time. When you are in the middle of a
task, information about the current step in that task is activated by sustained
firing, and information about the previous steps you just took are activated by
synaptic potentiation. What you are reading right now is made available for
several seconds due to sustained firing. The information in the beginning of
this post, and your thoughts about it, are made available due to synaptic
potentiation. These two stores interact and cooperate, just like CPU cache and
RAM. In both cases they allow the coordination of information processing
resources over time.
A large proportion of the information in short-term memory
(both in brains and computers) is not just data from long-term memory, but also
newly minted information that was just created. The information processing
applied to existing long-term memories creates new intermediate results. In a
computer the intermediate results are often mathematical. The results of a
multiplication might be stored in RAM. In the brain the intermediate results
are the new ideas that intervene between the starting point of a line of
thought, and its ending point.
There are many differences between the short-term memory of
computers and brains though. Information held in a computer’s CPU cache or RAM
is not united or integrated in any way. It is simply made available in case the
instructions in the program it is running call for it. These are simply items
in an unconnected list. In contrast, all of the information that is currently
active in the brain is bound together in a process called neural binding. This
turns a list of items into a composite scene, a scenario, or a situation. It
makes for a conceptual blending of information. This creates mental content to
attend to and be conscious of: declarative, semantic context.
There is no context in computer short-term memory, just a
group of disconnected items. For instance, if I give you four words you will
try to create a dynamic story out of them, but a computer will merely hold
these four words statically. There is very little information integration in
the short-term memory of a computer. In a computer, because the items active in
each state are not bound together in any way there is no connective tissue
within a state. This is the first reason that computers do not have
consciousness, and that it doesn’t “feel” like anything to be a computer. The second
reason has to do with the integration and connective tissue between states.
Both a computer program and the human mind progress
serially. One state of short-term memory leads to the next state. They both
usually start with a problem state, and progress through a chain of
intermediate states toward a goal state. To do this they must update their
short-term memory iteratively. This means that new items must be continually
added, others must be subtracted, but other must remain.
All computers using the Von Neumann architecture routinely
use iteration when updating cache memory. Because these stores are constantly
updated with new information they must evict old information. There are many
replacement policies for determining which information to evict from cache,
including first-in first-out (FIFO) and least recently used (LRU). LRU is based
on the observation that the most recently activated information has the highest
probability of being needed again in the near future (all else being equal).
If a computer program calls an instruction once, there is a
high probability that it will need it again very soon. Computer scientists
refer to this as “temporal locality.” Similarly, if a mammal recalls
information once, there is a high probability that it will need it again soon,
and LRU may also structure the caching behavior of the various stores of
working memory. LRU seems to be inherent in sustained firing and synaptic
potentiation in the sense that activity in the least recently used neurons is
the first to decay. There is a long history in cognitive psychology of studies
showing that humans and animals prioritize recently accessed items, and that
retention diminishes with passing time (Averell & Heathcote, 2011). The LRU
policy may capture the fundamental structure of how physical systems change
over time, so it should not be surprising that it is an organizing principle in
both computers and animals.
The various memory stores in the modern computer’s memory
hierarchy (static RAM, dynamic RAM,
virtual memory, etc.) are all incrementally updated where, second by second,
some data remains and the least used data is replaced (Comer, 2017). Thus, the
updating of computer cache memory is iterative in the same sense it is in the
brain. However, modern computers do not demonstrate self-directed intelligence,
so there must be more to the human thought process than LRU, and iterative
updating. This unexplained factor may be found, not in the way cached memory is
updated, but in the way it is used while active.
What links successive processing states together is very
different in brains and computers. Digital, rule-based computing systems use
cache to speed up delivery of data to the processor. However, the next bytes of
data needed by the CPU are not determined by the contents of the cache itself.
The instruction sequence of a computer’s operations is determined by its
program. The next instruction is dictated by the next line of executable code.
Then what determines the instruction sequence of thought?
Well, mammalian brains use cache (working memory) as parameters to guide
searches for the long-term memories to be accessed next. The next instruction
used by the brain is determined by what is currently in its short-term memory.
In the brain all of the active neurons fire at the inactive neurons they are
connected to, and whatever is activated the most will become part of the next
state. This means that the content active in short-term memory is used (in a
process called spreading activation) to search for the next update to
short-term memory. The various bytes of data within computer cache memory can
certainly be considered coactive, but they are not “cospreading.” That is, they
do not pool their activation energy to search long-term memory for relevant
data as in the brain.
The next state of a computer is determined very
deterministically by its code irrespective of what is held in cache. But the
next state of a brain is determined stochastically by the cache itself.
Computers do not use the contents of their short-term memory to search for the
updates that will be added in the next state. Brains do. Until computers or
computer programs are designed to do this they will not be able to be
conscious, and will not demonstrate human-level intelligence.
No comments:
Post a Comment