Wednesday, October 30, 2019

Comparing Information Processing in the Brain to That in the Computer





There are many interesting comparisons between the way the human brain processes information and the way a computer does it. By looking at the similarities and differences between the two I believe it is possible to understand each much better. The table below introduces a few of these comparisons, and the rest of this post will expound on these and others:

Computer Components Listed Next to Their Corresponding Brain Components


Computer

Brain

Type

Hard Drive

Cortical Structure & Connections

Long-term Memory

Cache

Working Memory

Short-term Memory

CPU Cache

Sustained Firing

Focus of Attention

RAM

Synaptic Potentiation

Short Term Store

CPU

Prefrontal Cortex

Instruction Processing

Programmed Code

Associations / Cortical Search

Instruction Selection

Operating System

Personality

Interface

Motherboard

Peripheral Nervous System

Connecting Structure

BIOS / Firmware

Instincts and Reflexes

Predispositions

Hardware

Anatomy, Wetware

Physical Structures

Software

Associations, Schemas, Beliefs

Mental Structures

Microphone, Camera, Keyboard, Mouse

Ears, Eyes, Touch, Smell, Taste

Input

Monitor, Speakers

Voice, Muscles

Output

GPU

Visuospatial Sketchpad

Internal Video

Sound Card

Phonological Loop

Internal Audio

Virtualization

Empathy

 

Clock Cycles in hz

Oscillations in hz

Cycles per Second

Capacitor

Neuron

Memory Unit

Transistor

Neuron

Fundamental Processing Unit

Logic Gate

Neural Circuit

Secondary Processing Unit


Humans were designed over 100s of millions of years of evolution. Computers were designed over the last 100 years.




Brains learn from every experience they have. Most computers (outside of machine learning) do not learn from experience, although they can save data from operations if you ask them to. Bytes in a program remain unaltered after every use. However, every time a brain’s memory is used it is at least slightly altered.

Brains can never do the same thing, or have the exact same state twice. Asked to perform the same operation a computer will perform each step in an identical manner over and over again with no variation.

A brain is a piece of biology. A computer is a piece of technology. Brains are electrochemical. Computers are electronic and electromechanical. Brains are analog, and computers are digital. Brains are flexible, computers are inflexible.





A computer can be rendered useless if a single microscopic part is damaged. A brain shows graceful degradation and can work near flawlessly in many tasks even after severe damage. For example, cutting one wire in a CPU or motherboard can completely incapacitate a computer. But humans can often still think, move, and speak even after extensive traumatic brain injury.




Humans search for novelty, food, sex, resources, and are driven by curiosity. Computers simply process exactly what we ask them to. They have no choice, no discretion at all. They are incapable of altering their own processing stream.

Both brains and computers are designed to retrieve and activate (recall) specific information on command.

They both store information in long-term memory. They both also retrieve information from long-term memory when it is needed by placing it into short-term memory. When this happens, the information goes from dormant to active and can be used in processing.




Both brains and computers transition from state to state as a function of time. Their present state is based on what long-term information is currently active in short-term memory. The very next state is determined by how the currently active information interacts with preset rules. In computers, these rules come from programmed lines of code. In brains, they come from associative memories.



Both take inputs, combine them with short-term memory, and use this to determine the output. When computers do this they take user input (mouse and keyboard) and reference them against instructions, coded by programmers, to create output (sound and video). Humans, on the other hand, take environmental input and use their memories, encoded by experience, to create behavior.



Humans and computers can both complete tasks. How to complete the task is often described by an algorithm held memory. To go about the task they transition from one state to the next in a serial manner, where each state prompts the next state. This causes them to progress through the algorithm until the task is complete.



A computer employs algorithms created by a programmer. A brain’s algorithms derive from guesswork, memes, trial and error, process of elimination and insight.

They can both multitask. Humans can switch between tasks several times per minute. Computers often switch between tasks thousands of times per second. In both brains and computers, separate tasks can be called “streams.” When you run a program on your computer it looks like its only running one thing at a time, but it is actually running dozens of programs, applications, and services simultaneously. The simultaneity is an illusion, however. Because it can only focus on one task at a time, the CPU actually switches between tasks rapidly, in an effort to keep them all up to speed. During this switching it often stays with a single program for just a few millionths of a second at a time. Your serial conscious mind is much slower, spending at least a few seconds on each thought.



Some computer processors have multiple CPU cores making it possible for them to have several separate information streams running at the same time. However, each core in a CPU (even multithreaded cores) must process information in a strict serial or linear manner. Similarly, thoughts can only involve one serial stream of information at a time. But the brain accomplishes its serial stream of consciousness through massive parallel processing. Your massively parallel unconscious mind performs thousands to millions of operations in a second, and they truly occur simultaneously.



The brain is decentralized. Every memory is distributed over many neurons. One isolated neuron cannot hold a memory all by itself. It holds a tiny fragment of a memory. These fragments combine with other fragments from other neurons to form words, concepts, and ideas. The computer on the other hand stores bits in definite physical locations that either contain a certain amount of electrical charge, designating a 1, or does not contain that charge, designating a 0. Groups of 8 bits create binary words that have meanings that include letters, numbers, and mathematical and logical operations for the CPU to perform.



Because each memory is distributed over many neurons, each neuron must play a role in many completely separate memories. Not so in a computer. A computer’s bits are stored in microscopic capacitors. Each capacitor can only hold one bit each. Its memory is not distributed, it is localized.



A bit can only have one of two states (0 or 1). This is because a transistor can either be on or off. Similarly, a neuron is either firing or not firing.

The brain uses neurons to hold memories. Computers use registers to hold their memories. Each register is made up of capacitors, and each capacitors can hold a bit. There are 8 bits in a byte (e.g. 01101100), and bytes are shuttled from register to register.



The brain uses circuits of neurons to process memories. Computers use transistors and logic gates to process bytes.



Neurons and the information that they encode do not move around, they remain stationary. Despite being set in a fixed point in space, they do send signals to each other, but the informational content held by one neuron is unique to it, and cannot be transferred to another. The registers (made of capacitors) in a computer also do not move, but the bits and bytes of information they hold are constantly and rapidly copied, and transferred from register to register.



The average computer today has billions of capacitors, and around 4 billion transistors. The brain wins in sheer number of processing units. The average human brain has 100 billion neurons. Each of these neurons makes many, many connections (synapses) with other neurons, and typical estimates for the number of synapses in the brain vary between 100 trillion, and a quadrillion.



Neurons and transistors both integrate an input to create an output. The neuron, has many more inputs and outputs than the transistor though. Neurons can have thousands of inputs, and tens of outputs, whereas transistors have two inputs and one output. The elementary logic of a transistor can be made much more complex when transistors are used together to construct logic gates, but even logic gates (AND, NOT, XOR, NAND etc.) do not even have tens of inputs or outputs. However. despite their low connectivity, transistors are much faster.



Transistors can switch billions of times per second, whereas neurons generally fire less than 400 times per second. Even logic gates (made of several transistors) are much faster than neurons.

Information is also carried through space far more quickly in computers. The buses, wires, and channels on a circuit board can move electrons a large fraction of the speed of light (299,792,458 meters per second). An axon can only propagate an action potential as fast as 150 meters per second.



A brain uses “content addressable memory, and spreads activation from active neurons to inactive neurons to search for associated concepts. A computer uses “byte addressable memory” to locate the next byte called upon by the CPU in the database using rows and columns.

The brain has electrical oscillations that organize and coordinate timing between its separate specialized modules (cortical regions). Similarly, the computer uses electrical oscillations in the form of a system clock to organize and coordinate its operations. The brain oscillates up to around 150 times per second (150 hz). A modern computer’s system clock oscillates at frequencies above 3 billion times per second. 



Human brains live 79 years on average. Desktop computers have a lifespan of 3 to 5 years.

The brain never shuts down until death. A computer is regularly turned off. Brains and computers both sleep. Computers have a hibernation setting and humans do not.

Brains don’t crash or hang like computers do. Both can get viruses. Both can malfunction and take damage from overheating.



When one operating system simulates another operating system within it, this is called virtualization. When one person simulates the thoughts of another in their mind, this is called empathy.

Two brand new computers of the same make and model will be identical, two members of the same species will be very different.

A new computer is much like a baby in that it has a basic input/output system (BIOS) that will determine how it will “instinctually” interact with its environment. Both new computers and babies will come to acquire a lot of disparate information over their lifetime, making each very unique.



In the case of trauma for instance, some human memories that are unwanted cannot be forgotten. However, information can be selectively and permanently erased from a computer.

Some things that we would like to remember (like an old phone number) cannot be recalled. Other things (like a new phone number) may be difficult to store at times. A computer doesn’t have this problem, if it is asked to recall, or store a memory on the hard drive, it does it with no problems and no mistakes.


Both brains and computers use instructions. In a sense these come in the form of “if, then” instructions. Both can be thought to have instruction sets made up of all the possible operations they can perform. A computer’s instruction set is its alphabet containing operations like add, divide, recall from RAM, and place in RAM.



Computers are often described as having a “memory hierarchy,” expressed as a pyramid with different levels of memory storage. This is also referred to as a caching hierarchy, because there are different levels of short-term memory with different associated speeds.



The levels at the top are faster, smaller, and more energetically expensive. The same could be said for human memory, because there are several levels, the levels at the top are faster, and more metabolically expensive. The following table is my attempt to compare the levels of these two hierarchies: 

Computer
Brain
Time Scale
CPU Register
Cortical Binding
Very short-term working memory (millis.)
CPU Cache (SRAM)
Sustained Firing
Short-term working memory (seconds)
RAM
Synaptic Potentiation
Short-term memory (seconds to hours)
Virtual Memory
Short-term Potentiation
Short-term memory (minutes to hours)
SSD
Commonly used LTM
Accessible Long-term Memory
Hard Drive
Long-term Memory
Long-term Storage (days to lifetime)

RAM and CPU cache memory, much like working memory, are volatile. This means that the information in them decays rapidly and require energy be maintained. RAM and cache, also like working memory, have a limited, and fixed capacity.



Long-term memory holds everything a person knows, just like the storage drive (HDD or SSD) holds everything the computer knows.



A person’s long-term memory takes longer to access than short-term memory, but has a much larger capacity. Similarly, the storage drive takes much longer to access than RAM or cache, but has a much larger capacity.



Human short-term memory is thought to be able to remember 3-7 items or concepts at a time. Computer short-term memory can hold millions of bytes, but cannot hold any true concepts. Nothing in a computer can hold concepts.

In order to feed more pertinent information into limited-capacity short-term memory the least pertinent information must be removed.

Both brains and computers get rid of the least recently used information to make room for new, incoming information in their short-term memory. This is an eviction policy known as LRU for “least recently used.”

We can take notes when there is too much information to hold in working memory. Similarly a computer can record information in a swap or paging file if there is not enough room in RAM.

Computers and humans use limited resources to perform tasks. They can both reach a processing limit where they are loaded by tasks so much that their performance decreases. In humans this is called cognitive load.

Computers can quantify their processing resources and tell you exactly how much of their resources are currently engaged by ongoing tasks. Humans cannot do this precisely.



Computers can give you detailed up-to-date information about the state of their components, the occupancy of their drives, heat of CPU, memory usage, and others. Humans can make similar reports, though in the form of unquantified sensations. Like people that can state their name and birthday, a computer can display its make, model and manufacture date.



Like us computers have basic needs. They need energy to run. They must be protected from the elements, and from physical harm.

Computers have no emotions, drive, curiosity, thought, interests, imagination, opinions, creativity, philosophies, beliefs, fears, wants, awareness, mental models, or sentience. A computer running AI software can be considered an agent, but not a subjective entity.

Human information processing is much more energy efficient. The largest supercomputers consume millions of watts of electricity, and yet do not have the processing capacity of the human brain. An average home desktop PC uses between 60 and 300 watts. The human brain uses merely 20 watts, less than most incandescent light bulbs.



Computers are inherently adept at logic and arithmetic, humans must learn it for years in school. Even the very first computers from the mid 20th century were much faster at computing than humans. This is how they got their name.

Most modern computers can multiply two 64 bit numbers more than 4 billion times in a second. Can you do this calculation once in an hour? Without a pen and paper humans cannot do this at all.

However, brains are inherently good at working associatively, using abstractions, finding matches, completing patterns, and using analogies. Computers struggle with this. Today even the most sophisticated personal assistants (Siri, Alexa, Google Assistant) cannot hold a conversation past one exchange. Every sentence you say to it is disconnected from every other sentence.



Brains and computers are both adept at “networking,” but only humans are good at socializing. AI chatbots are getting better, but without any comprehension or consciousness, can their chatting really be considered social?

Supercomputers are now approaching human brains in terms of processing speed, and amount of memory. However, they cannot think, even at the level of a two year old.

Even one year olds know things. A computer though does not know anything, just like a thermostat doesn’t know anything. All it can know is the locations of where bits are either 1s or 0s. In fact, any computer in the world, even the most complicated AIs, could be replaced by wooden parts and a hand crank (a mechanical Turing machine). It would amount to a tremendous contraption, but in theory it would work exactly the same.



Computers are now better than humans at checkers, chess, go, Jeopardy, object recognition, language translation, and much else.



Even the best computers and programs have trouble understanding natural spoken language. Today they can transcribe and translate language better than humans, but have zero comprehension. We have to admit though that most humans have trouble understanding programming languages, and very few people can read machine language (1s and 0s).



Computers appear to perform very complex tasks but they are really performing very simple tasks extremely quickly. Because they are exceptionally fast and accurate they can be made to chain together long series of operations to produce useful software and applications. However, each individual thing they do is very simple, the equivalent of elementary school math.

Computers learn new programs immediately after downloading and installing them. Humans can take years to learn new skills.

Computers can do things they were never programmed to do explicitly, but only if that thing is implicit in their design. Humans can do things that were never implicit in their design (like writing a poem).



Animal brains are embodied in the sense that they interact with and live in the world. Desktop computers, phones and tablets are disembodied.

Humans are made of carbon, oxygen and hydrogen. Computers are made from silicon, metal and plastic. But very soon computer semiconductors may be made from carbon nanotubes. This would make computers carbon-based, just like us.



Computers have short-term memory, but it works in a way that is fundamentally different from short-term memory in humans. Computers have no equivalent of working memory. This is probably the most important difference between humans and computers today. Until they are programmed to simulate working memory they probably cannot be conscious.