A Neural Model of Working Memory and Mental Continuity: The Iterative Updating Model

 

The Role of Iteration in Working Memory Updating

The following describes a new theoretical article that I am developing on the brain basis of thought and consciousness. It is an outgrowth of a previously published article that can be found here.

 

How The Concept of Iteration Ties Consciousness to Working Memory

As the stream of thought progresses through time some elements stay the same. Of course, elements are always being added to and subtracted from the stream of thought but working memory ensures that some of these concepts are held online temporarily. This maintenance function keeps your train of thought from being derailed and makes it so that what you are thinking at this moment is highly related to what you will be thinking one second from now. This interrelatedness of consecutive thoughts provides the fabric of consciousness and allows thought to progress.

The maintenance of working memory is made possible by a cellular phenomenon called sustained firing. Sustained firing happens when a neuron is signaled by dopamine to fire repeatedly over the course of several seconds. This type of neural activity happens in neurons in the higher brain areas (i.e. the prefrontal and parietal cortices). Sustained firing is what allows us to keep important concepts in mind so that we can manipulate them, reason with them, and think about them. Without sustained firing we would continuously forget what we were just thinking. We wouldn’t be able to think or reason. This continuity of neural activity allows us to build chains of logical thoughts that progress toward a solution or plan.

Now let’s consider the concept of iteration.

 

Iteration

I organized this article around the term “iteration.” I think that iteration is easy for people to understand because everyone is aware of the concept of product development. Personally, I think of the iPhone iterations. The customer feedback and R&D on the iphone 7 is combined with some of the original features of the iphone 7 to create the newest iteration: the iphone 8. In a similar way the computational products of the previous thought are combined with some of the original elements of that previous thought to create the next thought. I think the word "iterative" captures the meaning of “incremental change” and is a good word to describe the model.

The article addresses the role of iteration in the processing stream of working memory. This is a concept that has not been addressed by contemporary cognitive science. As the article documents, it was considered by William James over 100 years ago. The article describes how, when working memory is updated, elements from the previous state remain active due to sustained firing. This ensures that each state of working memory is a revised iteration of the state before it. This pattern of iterative updating is discussed in detail and it is related to a large number of cognitive and neuroscientific phenomena.

The model views working memory as activated long-term memory. It features a brief sensory store, and an FOA (focus of attention) embedded within a STM store, which in turn is embedded within LTM. Instead of focusing on stimulus reception and the controlled and automatic responses to incoming stimuli, it focuses on ongoing activity within working memory, the repeating iterative pattern inherent in it, and the search and selection of new items to be added to working memory.

The model views each instantaneous state of working memory as two things: 1) a set of solutions to the last state’s search, and 2) a set of parameters for the next search. The active items in working memory spread their activation energy throughout the brain to select the next ensembles to be added to working memory. This is how “search” is performed.

Fig. 1. Flowchart of Iterative Updating

In an iterative process, a thing is modified repetitively to generate a series of updated states. Each state is an iteration as well as the starting point for the next iteration. One way to accomplish iterative modification is to alter a given state by retaining pertinent elements and then subtracting and adding others. In the brain, the items to be added and subtracted may be determined by spreading activation.

I suggest that iteration is fundamental to working memory in that it allows context to be carried from one brain state to the next. This recursive property is instrumental in implementing learned algorithms, in allowing mental continuity (interrelatedness of consecutive mental states), and creating progressive changes to the contents of working memory.

Please take a look at the figures below. These original figures are helpful in conveying the model.

Fig. 2. Atkinson and Shiffrin’s (1968) Multi-store Model

This model depicts environmental stimuli being received by the senses and held in sensory memory. If attended, this stimulus information will enter short-term memory (working memory). If it is not rehearsed, it will be forgotten; if it is rehearsed, it will remain in short-term memory; and if elaborated upon sufficiently, it will be stored in long-term memory from which it can be retrieved later.

Fig. 3. Baddeley and Hitch’s (1974) Multicomponent Model

In this model the short-term store from the Atkinson and Shiffrin model is split into four interacting components that control working memory activity: the visuospatial sketchpad; the phonological buffer; the central executive; and, added later (Baddeley, 2000), the episodic buffer. These components interact with long-term memory, represented by the bottom rectangle.

Fig. 4. Cowan’s (1988) Embedded Processes Model

Short-term storage is an activated subset of long-term storage, and the FoA is an attended subset of short-term storage. Habituated stimuli enter the short-term store but do not enter the FoA.

Fig. 5.  Venn Diagrams of Information Shared Between Successive States of Working Memory

These Venn diagrams depict informational overlap between successive states of working memory. The horizontal axis represents time. The small circles represent information within the FoA, and the large circles represent information within the short-term store. Diagrams 1 and 2 show no Venn overlap between states from different periods; 3 and 4 show overlap in the short-term store only; 5 and 6 show the short-term store of one state overlapping with the FoA of the neighboring state; and 7 and 8 show the FoA of separate states overlapping, suggesting attentive continuity. It may be plausible that Diagrams 1 and 2 roughly depict sampling of cortical activity hours apart, 3 and 4 depict sampling several minutes apart, 5 and 6 depict sampling every minute, and 7 and 8 depict sampling every second. 

Fig. 6.  Hypothetical Depiction of Iteration in Neurons Exhibiting Sustained Firing

Each arc, designated by a lower-case letter, represents the active time span of a neuron exhibiting sustained firing. The x-axis represents time. Dashed arcs represent neurons that have stopped firing, whereas full arcs denote neurons that have not yet stopped firing.

 

	Sensory Memory	Focus of Attention	Short-Term Store	Long-Term Memory	References
Description	Early sensory processing	Attended content	Temporary store of content	Inactive, but retrievable content	Baddeley et al., 2018
Brain Correlate	Firing neuron	Sustained firing	Synaptic potentiation	Long-term potentiation	Christophel et al., 2017
Capacity	Numerous features	1 to 9 items	Very large	Extremely large	Shipstead et al., 2015
Duration	Visual: .25 s. Auditory: 2.5 s.	Up to a minute	Minutes	Days to lifetime	Brydges et al., 2018
Origin	Pre-attentive	Attentive	Recent attention	Memorization	Cowan, 2017
Location	Sensory cortex	Association cortex	Cerebral cortex	Cerebral cortex	Eriksson et al., 2015
Maintenance	Not possible	Continued attention	Rehearsal	Repetition, mnemonics	Chia et al., 2018
Departure	Decay, highly volatile	Replacement	Forgetting	Irretrievability	Constantinidis et al., 2018

Table 1

Fig. 7. Schematic of Iterative Updating in the FoA Where Items are Displaced, Sustained, and Newly Activated

Items are designated by uppercase letters. White spheres indicate active items and black spheres indicate inactive ones. In time 1, item A has already been deactivated, and B, C, D, and E are coactivated, echoing the pattern of activity shown in Figure 4. When coactivated, these items spread their activation energy, resulting in the convergence of activity onto a new item, F. At time 2, B has deactivated; C, D and E have remained active; and F became active. At time 3, D exits the FoA before C indicating that the order of entry does not determine the order of exit. As with other figures in this article, this figure is an emblematic abstraction.

Iteration	Repetition of a computational procedure applied to the product of a previous state, used to obtain successively closer approximations to the solution of a problem.
Working Memory	A mechanism dedicated to maintaining selected representations available for use in further cognitive processing.
Working Memory Updating	Changes in the contents of working memory occurring as processing proceeds through time.
Iterative Updating	A partial shift in the contents of working memory that occurs during updating as some contents are added, others are removed, and others are repeated.

Table 2. Definition of key terms

 

Fig. 8. Illustration of Four Possible State Transitions in the Iterative Function of the FoA

In the first transition at time 1, there are four active items (white spheres). In time 2, one of these four items has been replaced (1/4); that is, one white sphere becomes black (inactive) and a different black sphere becomes white (active). Thus, 25% of items have been updated between time 1 and 2 without any change in the total number of active items. Other figures in this article feature this 25% updating; however, in a store with four items, updating can occur in three other ways. The other transitions in this figure depict 50%, 75%, and 100% updating.

Fig. 9. Two Rates of Updating Carried Out Over 4 Time Periods

In the first scenario, 71% (5÷7) updating is carried out over four different time periods. In the second scenario, 29% (2÷7) updating is carried out. This comparison delineates the difference between unfocused, minimally overlapping thought (loose iterative coupling) and highly focused, closely overlapping thought (tight iterative coupling). To better illustrate this point, the capacity of the FoA is depicted here as seven items after Miller (1956). The Venn diagrams to the right of each diagram illustrate the percentage of iterative updating in the FoA using the style of Figure 6. 

Fig. 10. Intermittent Noniterative Updating Marks a Boundary between Thoughts

In both diagrams most of the updating is occurring at a rate of 20% (1/5). In the first diagram, there are three intermittent updates of 80% (4÷5). In the second there is only one intermittent update of 80%. This comparison delineates the difference between four brief thoughts occurring in quick succession and two more prolonged thoughts. The first strategy would result in small islands of associative connections among coactive items. The second strategy would result in less fragmented, longer sequences of associative continuity.

 

Fig. 11. Two Successive Instants of Coactive Assemblies in the FoA

The engrams for attended items of information B, C, D, E and F are each composed of many assemblies of neurons (represented by lowercase letters) active in association areas. In time 1, the assemblies for B, C, D, and E are active, whereas in time 2 the assemblies for C, D, E and F are active. Time 2 is an iterative update of time 1.

Fig. 12. A Schematic for Polyassociationism

Spreading activity from each of the assemblies (lowercase letters) of the four items (uppercase letters) in the FoA (B, C, D, and E) propagates throughout the cortex (represented by the field of assemblies above the items). This activates new assemblies that will comprise the newest item (F) to be added to the FoA in the next state. The assemblies that comprise items B, C, D, and E are each individually associated with a very large number of potential items, but, as a group, they are most closely associated with item F.

Fig. 13. The Iterative Updating Model

The FoA, the short-term store, and the hippocampus, as well as sensory and motor cortices, all contribute to the spreading activation that will select the next item(s) from long-term memory to be added to working memory. At time 1, two (K and L) of a potential five items are converged upon and will update the FoA in time 2. Five items are shown in gray within the short-term store but in actuality this store can hold many more.

Fig. 14. Depiction of Progressive Imagery Modification

In time 1, items B, C, D, and E, held active in association areas, all spread their activation energy to early visual cortex where a composite topographic map (sketch) is built that is based on prior experience with these items. In time 2, salient features introduced by the map from time 1 spread activation energy up the cortical hierarchy converging on the assemblies for item F. B drops out of activation, and C, D, E, and F diverge back onto visual cortex. The process repeats creating a progressive series of related images.

Fig. 15. Merging Subproblems in Working Memory

Each horizontal row is a snapshot of items coactive in the FoA at a specific time period. An original problem is activated, and iterative updating is used to reach a subsolution. This subsolution is saved in the short-term store, and a related subproblem is introduced into the FoA. This subproblem iterates until a second subsolution is generated. Relevant items from the first subsolution are combined with those from the second subsolution and iterated to generate a final solution.

 

Fig. 16. The Iterative Updating Model for Imagery and Behavior

Iteratively updated items in working memory interact with sensory cortices to construct progressive mental imagery and interact with motor cortices to construct progressive behavior. In the next state, the items in the FoA will undergo partial replacement. The parameters used in sensory and motor cortices will reflect this change making their output an advancement on their previous output.

Fig. 17. Iterative Inhibition

An original problem is activated in time 1 (B, C, D), and their spreading activity triggers a new item at time 2 (E). Executive processes determine that E is not a suitable behavioral parameter and E is inhibited. With E unavailable, B, C, and D continue to spread activation energy that converges on F (at time 2). The same iterative inhibition occurs with F (at time 4). G is then activated, and iterative updating continues.   

Fig. 18. Artificial Neural Network Implementation of Iterative Updating

Each enclosed set of circular nodes represents a specialized neural network wired to receive a different modality of input. Networks at the bottom (left) of the hierarchy take input of a single modality from the environment. Others take input from multiple neural networks below them in the hierarchy. Spreading network activity would oscillate between the top and bottom of the hierarchy while also allowing recurrent feedback between and within networks, creating a serial cognitive cycle. This figure features 24 networks, each with 19 nodes. An actual build would necessitate dozens of networks, each with millions of nodes. 

Fig. 19.  Venn Diagrams of Working Memory in Different Systems

These diagrams depict informational overlap between states of working memory. The diagrams on the left use the format from Figure 6 and those on the right use the format from Figure 7. Diagram 1 shows zero overlap between working memory at time 1 and that at time 2. This would make it more difficult for system 1, a hypothetical mouse, to make associations between events that are separated by the delay. For example, calling this mouse’s name and feeding it 10 seconds later may not condition it to come when called, whereas feeding it one second later might. Training an AI should involve a maturational process where the system begins learning with a very limited working memory span (e.g., Diagram 1), and gradually develops a superhuman capacity for working memory span (Diagram 4) as formative experiences accumulate. 

 Figure 20. A Hypothetical Example of How Iterative Updating Could Be Found Using Electrodes

Single-cell recording from a large number of cells in association cortex could produce an activity profile exhibiting iterative updating. In this simplified figure, the x-axis represents time in seconds and the y-axis presents the recorded activity of 30 individual neurons, each of which remains active for four seconds. Five neurons that become active each second. Each group of five neurons that begin and end their period of activity at the same time is taken to constitute an individual ensemble, or item, of working memory. Brackets at the bottom of the figure indicate which item each group of neurons belongs to. This profile coincides precisely with the pattern introduced in Figure 8. Searching new and existing data for this kind of iterative pattern could provide strong support for the present model. 

Term	Definition
Iteration	Repetition of a computational procedure applied to the product of a previous state, used to obtain successively closer approximations to the solution of a problem.
Working Memory	A mechanism dedicated to maintaining selected representations available for use in further cognitive processing.
Working Memory Updating	Changes in the items held in working memory occurring as processing proceeds through time.
Iterative Updating	A shift in the contents of working memory that occurs during updating as some items are added, others are removed, and others are maintained
Coactive	A group of items that are active in the same instantaneous state
Cospreading	A group of coactive items that combine their spreading activation energy to search the same global network
Multiassociative Search	A type of search where all of the coactive, cospreading contents converge in parallel on the update for the next state
State-spanning Coactivity (SSC)	Sustained coactivity exhibited by a set of two or more items that span two or more consecutive brain states
Incremental Change in State-spanning Coactivity (icSSC)	The process in which a set of items exhibiting SSC undergoes a change in group membership, where some items remain in SSC and others are deactivated and replaced
Rate of Updating	The proportion of items that are updated as a function of time
Mental Continuity	The recursive interrelatedness of consecutive mental states made possible by iterative updating
Iterative Compounding	Search results from a previous state are reused to update the system, incorporating them into the present search, prolonging their influence
Progressive Modification	The logical or algorithmic progress in information processing made possible by iteration, continuity, or compounding
Iterative Thread	A chain of iteratively updated states that underlies a line of thought
Merging of Subsolutions	When select contents from two separate instances of iteration, or separate lines of thought, are coactivated in a new state and used together for multiassociative search

Table 3

This article applied iterative updating to the traditional model of working memory items. However, it can similarly be applied to a large number of compatible frameworks that model item-like constructs including: ACT-R’s “symbols” (Anderson, 1996), adaptive resonance theory’s “templates” (Grossberg, 2013), global workspace theory’s “processes” (Baars, 2005), the pattern recognition theory of mind’s “pattern recognizers” (Kurzweil, 2013), hierarchical temporal memory’s “time-based patterns” (Hawkins et al., 2007), SPAUN’s “semantic pointers” (Eliasmith, 2013), SOAR’s “operators” (Laird, 2012), LIDA’s “codelets” (Baars & Franklin, 2007), OpenCog’s “atoms” (Goertzel, 2014), Fuster’s “cognits” (2005), Hofstadter’s “simmballs” (2007), Edelman’s “neuronal groups” (2004), Minsky’s “agents” (1986), and Damasio’s “convergent-divergent zones” (1989). These models, and many others, provide detailed mechanistic explanations for critical cognitive components underspecified by the present model (Table 4).

Fig. 21. Imagery and Behavior in the Iterative Updating Model

Iteratively updated items in working memory interact with sensory cortices to progressively construct mental imagery, and they interact with motor cortices to progressively construct behavior. In the next state, the items in working memory will undergo partial replacement. The parameters used in the sensory and motor cortices will reflect this change, making their output an advancement on their previous output, resulting in a capacity to support purposeful action. Related cognitive processes are included as arrows. 

Fig. 22. Schematic representation of ongoing iteration in the FoA and short-term memory store

This graphic expands on Figure 13, incorporating a larger number of the present model’s theoretical features: 1) the number of items coactive in the FoA (white spheres) at any one point in time varies between three and five; 2) the percentage of updating in the FoA varies between 25% and 100%; 3) items that have exited the FoA are capable of reentering the FoA; 4) the order of entry into the FoA does not determine the order of exit; and 5) items that exit the FoA briefly enter the short-term store (gray spheres) before deactivating completely (black spheres).

1.       Items of working memory correspond to temporarily activated groups of neural assemblies that encode long-term memory information.

2.       These items enter the FoA from which they move toward unattended short-term memory and then into inert long-term memory.

3.       Items remain active in working memory as long as their neural components demonstrate persistent activity in the form of sustained firing (FoA) or synaptic potentiation (non-FoA short-term memory).

4.       The active items in the FoA and the short-term store serve as search parameters for the next additions to working memory by spreading their activation energy throughout the cortex.

5.       The newly activated items of working memory are added to the remaining active items from the previous state to form an updated set of search parameters.

6.       This iterative updating process ensures that the next search is not an entirely new search but rather a modified version (updated iteration) of the previous search.

7.       Iteration may allow progressive modification, implementation of learned algorithms, and mental continuity.

Table 4.

Information processing features of the iterative updating model

         1) Mouse

Previous Title:

Thought Is Structured by the Iterative Updating of Working Memory

Fig. 14. Conditional Dependencies Between Consecutive Events

Each arc represents the span of time since an event occurred. S’s represent stimuli, R’s represent responses, and other capital letters represent items. To provide an illustrative example, the variables named above could correspond to the following events: S1 = friend, S2 = enemy, S3 = approach, S4 = depart, R1 = act friendly, R2 = act aggressive, R3 = wait, R4 = follow, A = foraging alone, B= feel hungry, C= find berries, D = not poisonous, Z = poisonous, Y = friend approaching, R5 = eat, R6 = don’t eat, R7 = share berries, R8 = eat berries before friend arrives.

Fig. XX Reiterating Through an Earlier Sequence

A set of six items is held in working memory, then iteratively updated over the next three time steps, creating a series of four related states. This activity, occurring from t1 through t4, might be considered a self-contained thought. Starting at t5, attention shifts completely as an unrelated thought takes place using an entirely different set of items. From t9, the first sequence is reiterated as before. This might happen when someone revisits an earlier thought, such as when rehashing a plan of action, retracing a set of previous steps, or retelling a story. 

Fig. XX Revisiting the Endpoint of an Earlier Iterative Sequence and Continuing It

Six items are modified over the first three time steps, creating a thought composed of four related states. Attention shifts completely at t5 and an unrelated thought occurs. Starting at t9, attention shifts back to the items from t4, and they are iterated without using any of the items from t5 through t8. This might happen when someone picks up a thought where it left off and continues to think about the issues from the last point at which they were considered. 

Fig. XX Merging Subsolutions in Working Memory

An original problem is activated at t1. Iterative updating is used to reach a subsolution at t4. This subsolution is saved in the short-term store, and a related subproblem is introduced at t5. This subproblem iterates until a second subsolution is generated at t8. At t9, relevant items from the first subsolution are combined with those from the second subsolution and iterated to generate a final solution at t12. This might happen when two nearby thoughts are mutually informative and elements of each can be used to draw a higher-order inference.

Fig. XX Revisiting the Midpoint of an Earlier Iterative Sequence and Altering It

Six items are modified over seven time steps, creating a line of thought composed of eight related states. At t9, attention shifts back to a point in the middle of this sequence. This set or subproblem from t4 is then iterated without including any of the items that were introduced from t5 through t8. This creates an alternate branch and a “forking” of the iterative sequence. This might happen when someone decides to assume a previous intermediate step in a problem-solving sequence and solve the problem in a different way.

Fig. XX Dopamine Reduces the Rate of Iterative Updating in Working Memory

A set of six items is held in working memory, then 67% (4÷6) updating is carried out over four time periods. At t5, the rate of updating is reduced to 17% (1÷6). This might happen when a person encounters a novel set of stimuli that causes the brain to release dopamine (at t4) and switch from default mode to attentive processing. The activity of the items from t5 is sustained, and the concepts are anchored upon giving them more processing priority so that greater focus can be brought to bear on them.