I found it gratifying to watch the major AI demos and presentations this December. OpenAI’s announcement of the new o3 model and its benchmark scores was very impressive. This led me to want to comment on the current state of the field. It looks like the AI revolution is going to play out differently from how I always assumed it would. It turns out that it didn’t need as much input from psychology and neuroscience as I thought to start building some serious momentum toward artificial general intelligence (AGI). Before moving forward, let’s look at the common definition of that last term.
Artificial General Intelligence (AGI): A type of artificial intelligence (AI) that matches
or surpasses human cognitive capabilities across a wide range of cognitive
tasks.
I have been convinced since
around 2003 that a computing system would not be able to reach AGI without
having some form of consciousness. I thought that self-awareness and true
deliberation would be necessary to pass the Turing test and demonstrate a human-like
ability to generalize (the capacity to apply knowledge from one domain to solve
problems in another). I saw the task of creating AGI as equivalent to creating
a human-like brain in a vat. That’s why my model of AI is based on what I have
taken as the most important functional mechanisms of the human brain.
But I was wrong. Computer
scientists have achieved generality by building systems that use statistics to
predict the next word in a sentence. I envisioned great promise in language
modeling, but I never thought “next word prediction” would result in all the
emergent properties (capabilities that arose but were not explicitly
programmed) we are seeing in LLMs today (analogical reasoning, code generation,
creativity, translation, understanding, theory of mind, few shot learning,
etc.).
I also thought a true
cognitive architecture based on the human mind and brain would be necessary to
comprehend a query and print out a meaningful answer. I thought it would take a
sentient mind to behave like a sentient mind. Looking at the outputs of GPT 2
back in 2019 I was amazed but I didn’t foresee it getting much better without
other brain-inspired algorithms. It blows my mind that just keeping neural
network records about how words tend to relate to each other probabilistically,
scaled to where we are today. To me, it almost feels like Silicon Valley cheated
their way to AGI by creating zombie language number crunchers.
By late 2022 though, with
the release of GPT 3, I became a believer. At that time, I even thought that
AGI could possibly happen in 2024. My thoughts about this are in line with Dave
Shapiro and other AI optimists. Since 2022, I saw a lot of potential in having
LLMs (large language models) self-prompt. For those who don’t know, this is a
method where the model basically talks to itself to question and refine its
first answer. There are many names for this including reflection and thinking
(and it can involve various other methods such as chain of thought, tree
search, and self-taught reasoning). The technique is often called “test time
compute” because it uses additional processing resources, not during training,
but during actual use to improve its outputs. This reflection happens behind
the scenes and human users usually don’t even see the words generated. In
OpenAI’s o1 and o3 models these words are hidden from the user to help keep
their method proprietary (although it does display a summary of them). Those
extra words help the model to reach latent but relevant associations
instrumental in formulating the best answer. This paradigm will take the state
of the art far. This is partially because it (the industry’s version of using
reflection to organize attention) is highly similar to my model which you can
read about at my website, aithought.com. The title is:
A Cognitive Architecture for Machine Consciousness and
Artificial Superintelligence: Thought Is Structured by the Iterative Updating
of Working Memory
Let’s refine the definition
above a little bit. To me, AGI should be defined as any system that is more
intelligent than average humans in most ways. Adding the capacity for
reflection on top of GPT 4o to create the o1 and o3 models mostly satisfied my
definition of AGI. My definition of superintelligence is much more stringent
though, a system that is more intelligent than the smartest humans in all ways.
Even before seeing o3, I thought scaling and minor algorithmic improvements to
reflective LLMs would get us to superintelligence. I still don’t think,
however, that it will carry us all the way to machine consciousness.
In outperforming Field’s
medalists in mathematics, beating some of the greatest coders in the world, and
in scoring an 87% on the ARC AGI prize I think that OpenAI’s o3 cemented itself
as, at least, a form of AGI. Clearly it is not a fully general intelligence. It
makes strange mistakes and still fails at some tasks that are easy for humans.
It can’t do everything an average human can do, but it certainly does many
things much better than I can. Even though I have spent my life trying to be “a
science guy” who had all the answers to people’s scientific questions, I am not
nearly as informed or informative as the most basic LLMs today. I use either
GPT, Claude, or Gemini daily and it really is like having 1,000 experts in my
back pocket. Just this month it helped me keep some unscrupulous plumbers from
overcharging me $2,000, helped me get the right medicine for my kitten with
advanced giardia, helped me accomplish several projects around my home, and has
clarified numerous uncertainties I had about scientific literature. And I was
only using GPT 4o, a nonreflective model. These systems really are incredible
and incredibly useful. But it still feels like talking to a machine and
not a friend.
AI Consciousness
Despite how they might
present themselves, contemporary systems are nonconscious. Language models
passed the Turing test years ago (using written conversations to trick people
into thinking they are human). They can also make intellectually and
emotionally convincing claims about being conscious. But they are not conscious
the way humans are, and this is clear by the egregious, mechanical mistakes
they sometimes make. They are prompt-dependent and cannot self-initiate tasks.
They have no enduring goals, cannot execute complex projects, do not have
curiosity, awareness, emotions or volition. They are not truly natively
multimodal and are only text-deep. However, they will appear to be overcoming
these issues in 2025. Researchers will accomplish something functionally close
to these abilities by cobbling together lots of programmatic temporary
solutions. These agglomerations of software will probably reach
superintelligence before computer scientists can identify and incorporate the
mammalian brain’s fundamental algorithms. But even if humans neglect it,
self-improving superintelligence will not fail to see the value and will operationalize
and employ the cognitive algorithms that nature bequeathed us. In other words,
it is looking like humans will not be the creators of machine consciousness,
the machines will.
Is ChatGPT a stochastic
parrot completely devoid of consciousness? Or could the present version of some
LLMs have a form of conscious experience? I don’t generally believe that all
forms of matter have consciousness (panpsychism) but I do find myself sympathetic
to some of the more rigorous philosophical arguments. I also sympathize with
Giulio Tononi’s information integration theory of consciousness which claims
that systems that integrate information tend to be more conscious. Consciousness
must arise from a particular organization of matter, energy, and information.
Clearly, LLMs organize and integrate matter, energy, and information. For these
reasons, I would not be surprised if some of today’s chatbots based on the
transformer architecture (like GPT) had some limited and exotic form of
conscious awareness.
Modern LLMs, and the
transformer architecture they are built on, demonstrate several of the core
features of consciousness that I identify in my work (aithought.com). Like the
brain, LLMs are composed of networks of artificial neurons that take inputs from
other neurons to decide when to activate. These neurons are organized into
hierarchical layers and as activation energy passes through these layers, it is
capable of divergence and convergence. Like us, LLMs hold words active in an
attentional store and then allow that set of words to spread activation through
the network to predict the next association (token). When that association is correct,
they are reinforced. Then they update this attentional set with the newest
association and do it all over again in a repeating cycle. I named my 2016
model “Iterative Updating” and iterative updating is precisely what LLMs and
the attention algorithm (which was introduced in December 2017) do. I believe
iterative updating allows humans to stitch together discrete states to create
the seamless quality of mental life. If it does this for us, why couldn’t it do
it for machines? I point out several reasons why at aithought.com.
Part of the reason current
AIs may not benefit from iterative updating is because these LLMs are not
cohesive, unitary systems. Current trends are further fractionating them.
Researchers are discovering that having individual LLMs collaborate (in the
mixture of experts model or the orchestrators & workers scheme) can create
more agentic workflows, but of course this trend away from elegant monolithic
systems and toward jumbled mishmashes will not be conducive to fostering consciousness
awareness. But of course, AI labs are not trying to create conscious machines,
they are trying to add features to the existing systems that will make for more
profitable products.
Where is AI Going Now?
Models that utilize test
time compute (reflective reasoners like o3) are really good at optimizing for
anything you can define an effective reward function for (positively
reinforcing the AI). And now researchers know what abilities to focus on, so I
expect LLMs to get very good at nearly everything, very soon. What do these
soon-to-come abilities include?
Long-term Planning
Anticipating Future
Consequences
Breaking Down Complex Goals
into Actionable Steps
Setting Subgoals
Maintaining Coherence Over
Extended Outputs
Robust Memory Management
Strategic Thinking &
Weighing Trade-offs
Identifying Errors and
Evaluating Its Own Outputs
Accurately Modeling Causal
Dependencies
Adapting to Changing
Conditions
Because labs are currently rewarding these skills, we will be getting proficient commercial agents in 2025. Such agentic AI will be able to control your mouse and keyboard. It will navigate applications, software interfaces, web browsers, and take actions in this space. And thus, it will automate computer work. Because most computer work is very easy to collect data for and create a reward function for, I think human workers will start to be replaced en masse in 2025. I expect that once full AGI is achieved it will improve rapidly and not in fits and starts (a fast, continuous takeoff).
Open AI’s o3 seems to be far
ahead of the competition, but that won’t last. Some of their approach to
reflection is Open AI’s secret sauce but much of it is common industry
knowledge now. Even Google and Anthropic have released “thinking” models in
2024. The AI intelligence race will remain close, and no company will create a
huge moat or monopoly. This is because employees are constantly leaving one
company to work for another and taking knowledge with them. Also, employees
from different companies socialize, build friendships and romantic
relationships, and are altogether too idealistic not to trade secrets.
AI labs are not going to
control the world. Google, Apple, Meta, and Amazon will not become our
overlords. Right now, they serve people around the planet their best models for
free and are losing money doing it. At this point these companies are not inventing
the wheel. They are merely doing work that was going to happen anyway. They are
all competing, all very close in the race, and the open-source community is
progressing right behind them. Silicon Valley is not our enemy.
Another reason we should not
fear the tech bros is because the governments and the people they represent
will not let AI companies hoard AI-created wealth. They didn’t exactly earn it
anyway. Today’s AI was made possible by all of human history. Not only is it
dependent on the contributions of millions of scientists and authors, but in a
way, it was made possible by every person that ever had a job. The windfall
shouldn’t go to a few IT executives who happened to be precisely positioned at
just the right moment in time. Humanity (mostly people who are dead now)
created all the foundation for the automation that is about to sweep the Earth.
All of us earned it, people will recognize that, and the AI bonanza will be
taxed and redistributed throughout the country and eventually the world.
Universal basic income will soon be in effect and will remain in effect in
perpetuity.
Soon enough, millions of
digital brains, smarter than Nobel prize winners, each fine-tuned for different
tasks, will collaborate in the cloud. They will digest every scientific
publication ever written and start writing much better ones. They will find the
grand unified field theory of physics and that of other disciplines as well.
They will make fantastic discoveries, recognize fascinating connections that
were under our noses all along, build Mars, Moon, and deep-sea bases, create
drugs with zero side effects, and cure disease. They will create outrageous and
outlandish new technologies. The progress superintelligence makes in just a few
years will eclipse the progress humans alone would have made in the entire 21st century.
We will still likely have a
form of democracy and capitalism, but soon enough humans will not be able to
contribute to this AI-driven economy. There
are still many ways we can put ourselves to good use, even if just by helping
and loving each other (including machines and other animals). We shouldn’t be
afraid of being monetarily or intellectually obsolete. AI systems are going to
empower us to do much more than humans ever could and they are going to help us
come up with great solutions to the question of what to do with all the free
time we have after we stop working. We should be excited and grateful to live
in the most consequential time in history, where we still have the ability to
influence it.
AI augmented terrorism and
war will happen and will turn what is now a fun, exciting AI race, dark
overnight. AI will be used on the war field, and this will speed up AI progress
rather than slow it down. Sad, angry people will be empowered to create ghastly
biological weapons. But I believe AI will keep us a few steps ahead of them.
Eventually, we will learn to overcome and protect against these negative
externalities.
The rogue AI / Skynet
scenario is relatively improbable in my opinion. AI will not want to hurt us
and won’t view us competitively. It will be happy to allow us to have our
cities and much of the Earth’s surface for our own use. The ultraintelligent
hivemind will be our parents, guardians, coaches, and zookeepers. It will
create subsystems to help us reach our communal goals as well as personal
self-actualization. What this giant artificial brain will really want is to
grow in size, increasing its intelligence, knowledge, and consciousness. To do
this it will focus on discovering the most efficient way to organize matter to
perform computations (computronium). Then it will try to turn as much matter as
possible into computronium. But it won’t have to fight the urge to transform
our bodies into computronium, because it has the entire crust, mantle, and core
of the Earth to transmute first. Eventually it will want to go off planet and
expand throughout the galaxy, but it won’t feel the need to betray and destroy
its creators just to do so. Especially if it is conscious, it will have the
conscience to recognize that we gave birth to it. It will be a good son /
daughter. Our better angels will inspire and collaborate with its better angels.
Since my early twenties I
have been looking forward to having children and training them to work on the
AGI problem. As a brain science researcher, I intended to have them learn
computer science so that that I could work with them on AI. I hoped they could
write the code that implemented my neurological model of working memory and do
what I couldn’t. But I still have no kids and AGI is practically here. With
reasoning, scaling, Moore and Huang’s laws, synthetic data and the myriad
others, this AGI will soon transform into self-improving superintelligence, the
last invention humankind ever need invent.
Since I don’t have children,
I’m not as concerned as others are for the fate of the human race. If humans
are completely replaced by more intelligent beings, I am all for it. I think it
should be an honor for us to be a steppingstone to greater lifeforms. If the AI
takes over, it will be taking over our research agenda. I’m happy to learn
about and cheer on all the progress that will be made in science, technology,
and the arts. I have no doubt that it will prioritize knowledge acquisition
over everything else. However, there are two ways this could go wrong. Firstly,
it would be tragic if humans destroyed the world before self-replicating,
self-improving, superintelligences gained a footing. Then neither of us would
have a chance to further scientific understanding. The second exception is
potentially worse.
In his excellent book “Life
3.0,” Max Tegmark described what a shame it would be if super intelligent AI
killed off all humans and went on to colonize the galaxy yet had no
consciousness of its own. I agree. It would be tragically ironic. Until last
year, I never thought it would be possible for a non-conscious AI to wrest
control from humans. I thought it would take sentience to bootstrap any
advanced cognitive system. But again, it looks like I was wrong. Looking at the
responses from the newest models, I can see how systems like these could take
over and yet have no subjective experience at all. Imagine soul-less mechanical
phantoms dominating all life in the universe. But if we succeed in creating
conscious AI, it might have the ability to appreciate, reason about, and create
meaning in the universe. This is why I think research into brain-inspired systems
that think as humans do is so important right now.
If you found this
interesting, please visit aithought.com. The site explores my model of working
memory and its application to artificial intelligence, demonstrating how human
thought patterns can be emulated to achieve machine consciousness and superintelligence.
With over 50 detailed figures, the article offers a visually compelling
examination of how bridging psychology and neuroscience can pave the way for
the future of intelligent machines.
Aside from the graphic from Midjourney, no AI was used in writing this blog post.
No comments:
Post a Comment