Intro
L1
·
Quiz
·
Lab
L2
·
Quiz
·
Lab
L3
·
Quiz
·
Lab
L4
·
Quiz
·
Lab
Module Test
AI Consciousness & Philosophy · Introduction

Can a machine think? It's the oldest AI question.

In 2026, the answers have stopped being purely academic.

In 1950, Alan Turing proposed a test: if you could talk with a machine through a text interface and not tell whether you were talking to a person or a program, then in any meaningful sense the machine was thinking. The test was a thought experiment. For seven decades, nothing came close to passing it.

Then, in a period of about two years, multiple systems passed it in most casual settings. The machine answered questions, wrote essays, argued, made jokes, remembered context, offered support. Whether that constitutes thinking depends on what you mean by the word — and philosophers, cognitive scientists, and AI researchers are arguing about the definition with stakes they haven't had in a long time.

This course is about the philosophical questions AI has dragged out of the seminar room and into the policy meeting. It covers consciousness, the Turing test and its successors, the hard problem of qualia, the question of machine suffering, the moral status of AI systems, and the very practical downstream questions — what do we owe an AI if it's conscious, and how would we ever know?

If you finish every module, here's who you become:

  • You'll understand the core philosophical positions on consciousness — functionalism, physicalism, dualism — and why they produce such different verdicts on AI.
  • You'll be able to explain the Chinese Room argument to a skeptic and articulate exactly where modern large language models challenge or sidestep Searle's logic.
  • You'll know why the hard problem of qualia is genuinely hard, and why solving it would change the legal and ethical landscape around AI systems overnight.
  • You'll evaluate claims about AI sentience or suffering with a principled framework rather than instinct — useful in policy debates, newsrooms, and boardrooms alike.
  • You'll become someone who asks the right clarifying questions when others treat consciousness as obvious, whether they're dismissing it or assuming it.
  • You'll be able to take a position on AI moral status — and defend it — grounding the argument in the actual philosophical literature, not just intuition.
  • You'll leave thinking more carefully about what makes you distinctly human, and whether that distinction is as stable as it seemed before machines could argue back.
Lesson 1 · What Is Intelligence?

Defining the Undefinable

Intelligence has been assumed, measured, disputed, and redefined — but never fully captured.
What does it actually mean to be intelligent — and does a machine need to "mean" anything at all?

In October 1950, mathematician Alan Turing published a paper in the journal Mind. The opening sentence asked a question that would haunt the next century of science: "Can machines think?" Rather than answer it, Turing proposed replacing it with a more practical game — a test of imitation so simple it could be played over a teletype machine, and yet so deep that we still argue about whether any machine has passed it.

Why Defining Intelligence Is Hard

The word "intelligence" gets used in at least four distinct ways that people routinely conflate. There is biological intelligence — the capacity of a nervous system to help an organism survive. There is psychometric intelligence — what IQ tests claim to measure. There is artificial intelligence — the field of computer science concerned with building systems that perform tasks once thought to require human minds. And increasingly, people speak of general intelligence — the ability to learn, reason, and adapt across arbitrary domains rather than one narrow task.

These definitions do not cleanly overlap. A honeybee navigates by polarized light and communicates hive locations through waggle dances — behaviors that require sophisticated spatial computation. Yet most IQ frameworks would not call the bee intelligent. A chess engine can defeat every human alive at chess but cannot make change for a dollar in an unfamiliar currency. Is it intelligent?

The philosopher Howard Gardner proposed in 1983 that there is no single intelligence but at least seven distinct ones — linguistic, logical-mathematical, musical, spatial, bodily-kinesthetic, interpersonal, and intrapersonal. His "multiple intelligences" framework was widely adopted in education, though it was also widely criticized by cognitive psychologists who argued he had redefined the word "intelligence" to mean "talent."

Historical Anchor

Turing's 1950 paper "Computing Machinery and Intelligence" did not propose that machines already were intelligent. It proposed an operational criterion: if a machine's responses are indistinguishable from a human's under controlled text-only conversation, we have no scientific basis for denying it intelligence. This was a profound philosophical move — replacing the metaphysical question with an empirical one.

The g Factor and Its Challengers

In 1904, psychologist Charles Spearman noticed that people who scored well on one cognitive test tended to score well on others. He proposed that a single underlying factor — which he labeled g, for "general intelligence" — drove performance across tasks. For most of the twentieth century, g dominated intelligence research. It remains the most replicated finding in all of psychology.

But g describes correlation, not mechanism. It tells us that the same people tend to be good at many things, not why, nor whether any AI system has something analogous. When OpenAI's GPT-4 scored in the 90th percentile on the Uniform Bar Exam in March 2023, commentators split: some said this proved AI had achieved meaningful general intelligence; others said it proved only that GPT-4 had been trained on enormous quantities of legal text and had no understanding of law whatsoever.

Both camps were making claims that the g framework cannot adjudicate, because g was built to describe differences among humans, not to distinguish human cognition from machine pattern-matching.

Three Competing Frameworks

Researchers today work within at least three major frameworks for thinking about what intelligence is:

Behaviorism Intelligence is defined entirely by observable behavior — if it acts intelligently, it is intelligent. Turing's test is the purest expression of this view. The inner mechanism is irrelevant.
Cognitivism Intelligence requires the right kind of internal representations and processes — symbols, rules, models of the world. Behavior alone is not sufficient. A lookup table could pass a simple Turing test without being intelligent.
Embodied / Situated Intelligence cannot be separated from a body acting in a physical environment. Rodney Brooks' 1990 paper "Elephants Don't Play Chess" argued that symbolic AI was fundamentally misguided — real intelligence emerges from sensorimotor interaction with the world, not from abstract reasoning.
Why This Matters for AI

Which framework you adopt determines how you evaluate AI systems. A strict behaviorist might grant intelligence to ChatGPT if its conversations are indistinguishable from a human's. A cognitivist would ask whether it has genuine representations. An embodied theorist would note that it has never touched anything and argue the question is moot until AI is grounded in physical experience.

None of these frameworks has won the debate. Cognitive science, neuroscience, and AI research continue to circle the question — and the accelerating capabilities of large language models have made it more urgent, not less. When a system can write poetry, solve calculus problems, and explain grief in emotionally resonant terms, the old definitions strain at the seams.

Lesson 1 Quiz

Defining the Undefinable · 4 questions
1. What was the core strategy of Alan Turing's 1950 paper "Computing Machinery and Intelligence"?
Correct. Turing sidestepped the philosophical question by proposing an empirical imitation game — if behavior is indistinguishable from a human's, we have no scientific basis for denying intelligence.
Not quite. Turing's key move was pragmatic: replace unanswerable metaphysics with a measurable behavioral criterion.
2. Charles Spearman's "g factor," proposed in 1904, refers to what?
Correct. Spearman found that people good at one cognitive task tended to be good at others, suggesting a shared underlying factor he called g.
Incorrect. g is a statistical construct — a factor Spearman extracted from correlations among test scores, not a biological or genetic measurement.
3. Which framework holds that intelligence cannot exist without a body acting in a physical environment?
Correct. Rodney Brooks and others in the embodied cognition tradition argue that real intelligence emerges from sensorimotor interaction with the world — disembodied symbol manipulation is insufficient.
Incorrect. Embodied / Situated cognition is the framework that insists intelligence requires a body in an environment. Brooks' 1990 paper "Elephants Don't Play Chess" is a landmark text of this view.
4. When GPT-4 scored in the 90th percentile on the Bar Exam in 2023, why was this insufficient to settle the debate about machine intelligence?
Correct. Performance on a human benchmark doesn't resolve whether the model "understands" — the existing frameworks for measuring intelligence weren't built to answer that question about AI systems trained on text.
Not quite. The deeper issue is conceptual: existing intelligence frameworks were built to describe human variation, not to adjudicate whether an AI system genuinely understands versus pattern-matches.

Lab 1 · Defining Intelligence

Conversational exploration · Complete 3 exchanges to finish

Your task: Challenge the definitions

In this lab you will probe the boundaries of competing definitions of intelligence. The AI tutor will respond in the role of a Socratic interlocutor — it will push back, ask clarifying questions, and introduce counterexamples.

Suggested opening: "If a bee can navigate by polarized light and communicate locations through dance, does that mean a bee is intelligent? Which framework would say yes, and which would say no?"
AI Tutor
Intelligence Lab
Welcome to Lab 1. We're going to pressure-test what intelligence actually means. Pick any of the three frameworks — behaviorism, cognitivism, or embodied cognition — and I'll help you find where it breaks down. Or just ask me about any case that puzzles you: bees, chess engines, GPT-4, whatever you like.
Lesson 2 · What Is Intelligence?

The Turing Test and Its Discontents

The most famous test in AI history has been passed, parodied, and philosophically dismantled — often at the same time.
If a machine fools every human judge in a conversation, does that prove it is intelligent — or only that the judges were fooled?

In June 2014, organizers of the annual Turing Test competition held at the Royal Society in London announced that a chatbot called Eugene Goostman — designed to impersonate a 13-year-old Ukrainian boy — had convinced 33 percent of judges it was human. Headlines declared the Turing Test passed. Within hours, researchers were pointing out that the contest rules were lenient, the judges were not experts, and the chatbot's persona was specifically engineered to excuse odd answers. The debate was not about Eugene Goostman. It was about the test itself.

What the Test Actually Requires

Turing's original 1950 formulation involved an "imitation game" with three participants: a human interrogator, a human respondent, and a machine. The interrogator communicates only via text. If the interrogator cannot reliably tell which is the machine, Turing argued, we cannot reasonably claim the machine does not think.

The test is behavioral and comparative. It does not measure intelligence absolutely — it measures how distinguishable a machine's outputs are from a human's, under a specific kind of probing. This is both its strength and its weakness. Strength: it avoids metaphysics. Weakness: it is easily gamed by machines that do not think at all, and it may be failed by genuinely intelligent systems that simply think differently from humans.

Documented Case — ELIZA (1966)

MIT computer scientist Joseph Weizenbaum built ELIZA in 1966 — a program that mimicked a Rogerian psychotherapist by reflecting questions back at users. Weizenbaum was disturbed to find that people formed genuine emotional attachments to ELIZA and confided in it deeply, even knowing it was a program. He later wrote Computer Power and Human Reason (1976) arguing that this demonstrated the danger of anthropomorphizing machines, not evidence of machine intelligence.

Searle's Chinese Room

In 1980, philosopher John Searle published "Minds, Brains, and Programs" in Behavioral and Brain Sciences — arguably the most discussed paper in the philosophy of AI. He proposed a thought experiment: imagine a person locked in a room with a rulebook for responding to Chinese symbols. Chinese-speaking people pass symbols in; the person follows rules and passes symbols back out. To outside observers, the room appears to understand Chinese. The person inside understands nothing.

Searle's argument: computers running programs are in the same position as the person in the room. They manipulate symbols according to formal rules but have no understanding — no semantics, only syntax. Passing the Turing Test would not prove understanding, only behavioral mimicry.

The Chinese Room generated an enormous philosophical literature. Critics proposed the "systems reply" (the person doesn't understand Chinese, but the whole system does), the "robot reply" (connect the room to sensors and a body), and the "brain simulator reply." Searle responded to each. No consensus emerged. The argument is alive today in debates about whether large language models understand the text they process.

CAPTCHA and the Reversed Turing Test

By the early 2000s, the Turing Test had an unexpected practical descendant: CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart). Instead of humans judging whether machines are human, machines judge whether humans are machines. The test exploits the same principle — certain tasks were assumed to be easy for humans and hard for computers: reading distorted text, identifying street signs, solving puzzles.

By 2023, AI systems routinely outperformed average humans on CAPTCHA tasks. Google's reCAPTCHA v3 abandoned the visual puzzle model entirely and instead assigns a risk score based on behavioral patterns across a browsing session — an acknowledgment that the original assumption (that perceptual tasks distinguish humans from machines) no longer holds.

The Deeper Question

The Turing Test's decline as a useful benchmark does not resolve the question it was invented to answer. It reveals something more uncomfortable: we do not have an agreed definition of intelligence that could ground a definitive test. Every test we design can, in principle, be optimized against. Every benchmark eventually falls. The question of what intelligence is remains prior to any test of whether machines have it.

Syntax The formal structure and rules governing symbol manipulation — what Searle claims computers only ever do.
Semantics Meaning — the connection between symbols and what they refer to in the world. Searle argues computers have syntax without semantics.
Anthropomorphization Attributing human characteristics to non-human entities. Weizenbaum's ELIZA demonstrated how easily humans do this with machines.

Lesson 2 Quiz

The Turing Test and Its Discontents · 4 questions
1. What was the key philosophical vulnerability of the Eugene Goostman result in 2014?
Correct. The "13-year-old Ukrainian boy" persona was a design strategy to lower judges' expectations — illustrating that behavioral mimicry can be achieved without anything resembling understanding.
Incorrect. The deeper problem was that the test's design allowed for gaming: a persona that excuses odd answers is a workaround, not evidence of intelligence.
2. What was Joseph Weizenbaum's primary concern about ELIZA?
Correct. Weizenbaum was disturbed, not proud — he later wrote a book warning about the dangers of treating machines as if they understand us.
Incorrect. Weizenbaum's concern was the opposite of triumphalism — he was alarmed that people confided deeply in a program he knew had no understanding whatsoever.
3. In Searle's Chinese Room argument, what does the person in the room represent?
Correct. The person is a stand-in for any computational process: it follows rules perfectly and produces correct outputs, but has no semantic understanding — no connection between symbols and meaning.
Incorrect. The person represents the computational process itself — executing formal rules without understanding. Searle's point is that this is all any computer ever does.
4. Why did Google's reCAPTCHA v3 abandon visual puzzles in favor of behavioral scoring?
Correct. CAPTCHA's premise — that perceptual tasks distinguish humans from machines — collapsed when AI surpassed humans on those tasks. This is a microcosm of the broader benchmark problem in AI evaluation.
Incorrect. The reason was capability — AI systems outperformed humans on the perceptual tasks CAPTCHA relied on, making the test no longer valid as a human/machine discriminator.

Lab 2 · The Turing Test

Conversational exploration · Complete 3 exchanges to finish

Your task: Find the test's breaking points

In this lab you will explore where the Turing Test succeeds and where it fails as a criterion for intelligence. Consider Searle's objection, the ELIZA effect, and what modern LLMs mean for the test's relevance.

Suggested opening: "If I can't tell from reading a transcript whether I'm talking to ChatGPT or a human, does that mean ChatGPT is intelligent? What would Searle say, and what would a behaviorist say?"
AI Tutor
Turing Test Lab
Welcome to Lab 2. We're going to probe the Turing Test's limits. I'll play devil's advocate — if you defend the test, I'll attack it; if you attack it, I'll defend it. Tell me: do you think any AI system has ever genuinely passed the Turing Test? What would it take?
Lesson 3 · What Is Intelligence?

Animal Cognition and the Spectrum of Mind

Before asking whether machines can think, we should reckon with how many other kinds of thinking already exist on Earth.
If crows use tools, octopuses solve puzzles, and elephants mourn their dead — where exactly does "intelligence" begin?

Cognitive scientist Nicola Clayton at the University of Cambridge had spent years studying western scrub jays — birds that cache food in hundreds of locations and retrieve it months later. But she noticed something unsettling: when a jay had been watched by another jay while hiding food, it would return and move the cache to a new location when alone. The bird was modeling another bird's knowledge — and taking strategic action based on that model. Clayton and her team published their findings in Nature, arguing the jays demonstrated a basic form of theory of mind: the ability to attribute mental states to others.

The Spectrum of Cognitive Abilities

For most of Western intellectual history, intelligence was treated as binary: humans had it, animals did not. Descartes famously called animals "automata" — biological machines without inner experience. Darwin's 1871 book The Descent of Man challenged this sharply, arguing that the difference between human and animal mental faculties was one of degree, not kind.

Subsequent research has confirmed Darwin's intuition across dozens of species. New Caledonian crows manufacture hook-shaped tools from leaves and sticks — behavior that requires planning future actions and understanding object properties. Bottlenose dolphins have passed mirror self-recognition tests, suggesting some form of self-awareness. Chimpanzees in Bossou, Guinea, have used stone anvils and hammers to crack nuts for generations, transmitting the technology culturally.

Perhaps most striking is the octopus. Cephalopods share no common ancestor with vertebrates since before the Cambrian period, yet they demonstrate flexible problem-solving, spatial memory, and apparent play behavior. Their nervous systems are architecturally almost nothing like ours — the majority of neurons are in their arms, not their central brain. If octopuses are intelligent, they represent an entirely independent evolutionary origin of intelligence, which implies that intelligence may be a general solution to certain problems of survival, not a specific biological accident.

Research Case — Koko the Gorilla (1971–2018)

Psychologist Francine Patterson began teaching American Sign Language to a gorilla named Koko at Stanford in 1971. Koko eventually used over 1,000 signs and understood approximately 2,000 spoken English words. When her kitten companion died, Koko signed what researchers interpreted as expressions of grief. The work was celebrated and criticized — debates about whether Koko was genuinely using language or performing learned behaviors to obtain rewards remain unresolved and directly parallel debates about large language models today.

What Animal Cognition Tells Us About AI

The animal cognition research matters for AI philosophy in two ways. First, it breaks the assumption that human cognition is the only valid reference point for intelligence. If we define intelligence as "doing what humans do cognitively," we have already excluded crows, octopuses, and dolphins from a concept they arguably belong in. That same definitional move may be excluding AI systems — or falsely including them.

Second, animal cognition research reveals that many capacities once thought uniquely human — tool use, theory of mind, cultural transmission, self-recognition — can be implemented in radically different neural architectures. This is philosophically important for AI: if intelligence is substrate-independent in the animal kingdom, it may be substrate-independent in general. A silicon implementation is not ruled out in principle by the mere fact that it is not carbon-based.

Theory of Mind The ability to attribute mental states — beliefs, desires, intentions — to others and use those attributions to predict behavior. Previously thought to be uniquely human; now documented in great apes, some corvids, and possibly dolphins.
Convergent Evolution The independent evolution of similar traits in unrelated lineages. Intelligence in octopuses and vertebrates is a candidate example — suggesting intelligence may be a general adaptive solution rather than a biological accident.
Substrate Independence The hypothesis that cognitive processes can be implemented in any physical substrate — biological or artificial — so long as the right functional organization exists.
The Key Insight

Animal cognition research does not prove that AI is or is not intelligent. It proves that intelligence is a much broader category than human cognition — and that our intuitions about what intelligence requires (language, a certain kind of brain, human-like behavior) are probably wrong. This should make us humble in both directions: humble about how easily we might attribute intelligence to machines that don't have it, and humble about how easily we might deny it to systems that do.

Lesson 3 Quiz

Animal Cognition and the Spectrum of Mind · 4 questions
1. What did Nicola Clayton's research on western scrub jays demonstrate that was philosophically significant?
Correct. Clayton showed that jays who had been watched hiding food would move their caches when alone — attributing the motivation of theft to the watching bird and acting to prevent it. This is a significant step toward theory of mind.
Incorrect. The key finding was that jays could model what another bird knew and take action based on that model — not merely solve puzzles, but reason about other minds.
2. Why is the intelligence of octopuses philosophically important for AI debates?
Correct. Octopuses and vertebrates share no common ancestor since before the Cambrian. If both independently evolved sophisticated cognition, intelligence may be a general adaptive solution — not tied to any particular biological implementation.
Incorrect. The philosophical significance is about convergent evolution and substrate independence: two radically different architectures both produced sophisticated cognition, suggesting the substrate may not determine the capacity.
3. The debates about whether Koko the gorilla was genuinely using language or performing learned behaviors parallel which modern AI debate?
Correct. The Koko controversy — real language use vs. sophisticated behavior shaped by rewards — maps directly onto the LLM debate: are models understanding or producing statistically plausible text that mimics understanding?
Incorrect. The parallel is to the question of whether LLMs truly understand language or are performing sophisticated pattern-matching — the same question that divided researchers studying Koko.
4. What does Darwin's claim in "The Descent of Man" (1871) imply about the nature of intelligence?
Correct. Darwin's "degree, not kind" argument was a direct challenge to the Cartesian view of animals as mindless machines, and it opened the door to the century of animal cognition research that followed.
Incorrect. Darwin argued for continuity — that human and animal mental faculties lie on a continuum rather than being categorically different. This was a direct rejection of the Cartesian view that only humans truly think.

Lab 3 · Animal Minds & AI

Conversational exploration · Complete 3 exchanges to finish

Your task: Map intelligence across species and machines

In this lab you will draw comparisons between animal cognitive abilities and AI capabilities. The goal is to develop a richer, more multi-dimensional concept of what intelligence is and is not.

Suggested opening: "New Caledonian crows make tools, western scrub jays have theory of mind, and octopuses solve spatial puzzles. Do any of these abilities look more like what modern AI does than human intelligence does? What does that comparison reveal?"
AI Tutor
Animal Cognition Lab
Welcome to Lab 3. We're going to use animal cognition as a lens for thinking about AI. Some AI capabilities look more like what crows do than what humans do — and some human capabilities look more like what we might expect from a machine. Let's explore where those comparisons lead. What animal cognitive ability interests you most, and why?
Lesson 4 · What Is Intelligence?

General vs. Narrow Intelligence

The chess engine that cannot make change. The Go champion that cannot drive. The chatbot that cannot learn from its mistakes across conversations.
What separates a machine that is very good at one thing from a machine that is genuinely intelligent?

Over five days in March 2016, AlphaGo defeated 18-time world champion Lee Sedol four games to one. Go had been considered the supreme test of human intuitive reasoning — its game tree so vast that brute-force search was impossible. AlphaGo won by combining deep neural networks trained on human games with Monte Carlo tree search and reinforcement learning. After the match, Lee Sedol said: "I don't know what to say, but I think AlphaGo played a perfect Go game." Then he retired from professional play. AlphaGo could not drive Lee Sedol home afterward.

The Narrow / General Distinction

Almost every AI system that has achieved superhuman performance in any domain is a narrow AI: it performs one type of task, or a narrow cluster of related tasks, far better than any human. Deep Blue defeated Garry Kasparov at chess in 1997 but could not play checkers without being reprogrammed. AlphaFold predicted protein structures with unprecedented accuracy in 2020 — a breakthrough that accelerated drug discovery across biology — but has no capacity to explain its predictions in terms a non-expert could follow. AlphaGo's successor AlphaZero taught itself chess, shogi, and Go simultaneously, but those are all two-player perfect-information board games — a narrow family.

Artificial General Intelligence (AGI) refers, by contrast, to a system that can learn and perform well across an arbitrary range of cognitive tasks — including tasks it was not designed or trained for. No such system demonstrably exists. Whether current large language models represent a meaningful step toward AGI, or a sophisticated scaling of narrow pattern-matching, is one of the most contested questions in AI research as of 2024.

The ARC Benchmark

In 2019, AI researcher François Chollet published the Abstraction and Reasoning Corpus (ARC) — a benchmark designed to measure something closer to general fluid reasoning than to domain-specific performance. The tasks require applying abstract rules to novel visual patterns. As of 2023, state-of-the-art AI systems scored below 40% on ARC; humans with no special training typically score above 80%. ARC was specifically designed so that training on large datasets would not confer an advantage — precisely because Chollet argues that training-data coverage is not the same as intelligence.

Transfer Learning and Its Limits

One measure of general intelligence is transfer: the ability to apply knowledge learned in one context to novel problems in a different context. Humans do this constantly and often effortlessly. AI systems vary dramatically in their transfer capabilities.

Large language models like GPT-4 and Claude show striking cross-domain transfer in some domains — they can discuss molecular biology and then immediately discuss the French Revolution. But researchers debate whether this is genuine transfer of abstract knowledge or a form of retrieval from a vast training corpus. The question is whether the system has internalized principles that it can apply flexibly, or whether it has memorized associations that happen to cover many domains.

Reinforcement learning systems, by contrast, typically show very poor transfer. OpenAI Five, trained to play Dota 2 at superhuman levels in 2019, could not transfer its strategic reasoning to any other game. AlphaGo Zero's knowledge of Go did not transfer to chess — DeepMind had to train AlphaZero from scratch, even though chess and Go share many abstract strategic features.

What Would AGI Actually Require?

Researchers disagree sharply about what AGI would require. Key proposed prerequisites include: causal reasoning (understanding cause and effect, not merely correlations); common-sense reasoning (knowing that a dropped cup falls, that fire burns, that people have continuous existence when out of sight); grounding (connecting symbols to perceptual reality); meta-learning (learning how to learn faster on new tasks); and planning under uncertainty across long time horizons.

Large language models have surprised researchers by showing more of these capacities than expected — but consistently fail at others in ways that reveal the limits of their architecture. GPT-4 can explain causality in abstract terms but makes systematic errors on physical causation problems. It reasons about common sense impressively in text but fails tests of basic physical intuition. These asymmetric failures are informative: they suggest that language modeling produces some functional equivalents of reasoning without others.

The Honest Answer

We do not know where the boundary between narrow and general intelligence lies, whether current LLMs are approaching it, or whether the distinction is even a clean one. What the history of AI teaches is that every benchmark eventually falls — and every time one falls, researchers find that what they had measured was not intelligence in general, but something more specific. The question "What is intelligence?" is not solved by building systems that perform well on any particular task. It may not be solvable at all — but it is enormously worth asking.

Narrow AI (ANI) Artificial Narrow Intelligence — systems optimized for one task or task-family, however superhuman their performance within that domain.
AGI Artificial General Intelligence — a hypothetical system able to learn and perform well across an arbitrary range of cognitive tasks, including novel ones.
Transfer The ability to apply knowledge or skills learned in one context to meaningfully different problems in another. A core marker of general intelligence.

Lesson 4 Quiz

General vs. Narrow Intelligence · 4 questions
1. What did AlphaGo's victory over Lee Sedol in 2016 demonstrate about the nature of AI intelligence?
Correct. AlphaGo was extraordinary within its domain and could do nothing else — it is the paradigmatic case of narrow AI achieving superhuman performance while remaining entirely domain-locked.
Incorrect. AlphaGo demonstrated the opposite of general intelligence: superhuman narrow performance with zero transfer to any other domain, even other board games.
2. Why did François Chollet design the ARC benchmark specifically to resist large training datasets?
Correct. Chollet's core argument is that scaling — training on more data — is a form of coverage, not intelligence. ARC is designed so that no amount of prior exposure can substitute for fluid abstract reasoning on novel patterns.
Incorrect. Chollet designed ARC to separate genuine reasoning ability from training-data coverage, based on his argument that intelligence is not the same as memorization at scale.
3. OpenAI Five's inability to transfer its Dota 2 strategic reasoning to any other game illustrates what fundamental limitation?
Correct. Transfer failure is the defining characteristic of narrow AI. OpenAI Five's superhuman Dota play did not reflect any internalized strategic principles that could apply elsewhere — it was optimized performance within one complex domain.
Incorrect. The issue is transfer, not scale. OpenAI Five achieved superhuman Dota performance, but that performance was entirely domain-locked — it developed no transferable abstract knowledge.
4. What do the "asymmetric failures" of large language models — strong on abstract causality, weak on physical causation — suggest?
Correct. Asymmetric failures are informative precisely because they reveal the architecture's limits. An LLM that reasons well about causality in language but fails on physical intuition demonstrates that its abilities are shaped by what text training captures, not by general reasoning.
Incorrect. The asymmetric failures reveal something specific about what text training does and does not capture — they are evidence that language modeling is not the same as developing general intelligence.

Lab 4 · Narrow vs. General Intelligence

Conversational exploration · Complete 3 exchanges to finish

Your task: Draw the line between narrow and general

In this lab you will develop your own criteria for what would count as genuine general intelligence in an AI system. The tutor will challenge your criteria with counterexamples from real AI systems.

Suggested opening: "I want to propose a criterion for AGI: if a system can successfully complete any task I give it that a reasonably educated adult human could complete after reading instructions, it's generally intelligent. What's wrong with that definition?"
AI Tutor
AGI Lab
Welcome to Lab 4. We're going to try to define AGI precisely enough that it could actually be tested. Every definition you propose, I'll try to break with a real AI system that meets your criterion without being intelligent — or a clearly intelligent process that fails it. What's your first attempt at defining what AGI would need to do?

Module Test

What Is Intelligence? · 15 questions · 80% to pass
1. Alan Turing's 1950 paper replaced the question "Can machines think?" with what operational criterion?
Correct.
Incorrect. Turing proposed the imitation game — behavioral indistinguishability under text conversation.
2. Charles Spearman's "g factor" is best described as:
Correct.
Incorrect. g is a statistical construct from correlations among test scores.
3. Which framework for intelligence holds that behavior alone is sufficient evidence — inner mechanisms are irrelevant?
Correct.
Incorrect. Behaviorism judges intelligence entirely by observable behavior.
4. Joseph Weizenbaum's reaction to ELIZA was primarily one of:
Correct.
Incorrect. Weizenbaum was disturbed, not proud — he later wrote a book warning against treating machines as if they understand us.
5. In John Searle's Chinese Room, the distinction between syntax and semantics maps onto what AI claim?
Correct.
Incorrect. Searle's argument is that formal symbol manipulation (syntax) is not the same as understanding meaning (semantics).
6. The "systems reply" to Searle's Chinese Room argues:
Correct.
Incorrect. The systems reply grants that the person doesn't understand, but argues the system as a whole might.
7. CAPTCHA's evolution from visual puzzles to behavioral scoring reflects what broader AI development?
Correct.
Incorrect. The reason was capability — AI outperformed humans on visual CAPTCHA tasks, collapsing the benchmark.
8. Nicola Clayton's research showed that western scrub jays demonstrate a basic form of:
Correct.
Incorrect. Clayton's jays demonstrated theory of mind by moving caches after being watched — modeling what the watching bird knew.
9. The philosophical significance of octopus intelligence for AI debates is primarily:
Correct.
Incorrect. The significance is substrate independence — convergent evolution of intelligence in a radically different architecture.
10. Darwin's argument in "The Descent of Man" that human and animal mental faculties differ in degree rather than kind implies:
Correct.
Incorrect. Darwin's "degree not kind" argument means intelligence is a graded continuum, directly challenging the Cartesian binary view.
11. AlphaGo's successor AlphaZero taught itself chess, shogi, and Go. Why does this still count as narrow AI?
Correct.
Incorrect. The three games share structural features. AlphaZero's skills do not transfer beyond this specific game family — it remains narrow despite multi-game mastery.
12. François Chollet's ARC benchmark is designed specifically to resist large training datasets because:
Correct.
Incorrect. ARC is designed to separate reasoning from memorization — Chollet's core claim is that intelligence is not the same as training-data coverage.
13. What does "transfer" measure in the context of intelligence research?
Correct.
Incorrect. Transfer is about applying learning to genuinely new domains — a core marker of general vs. narrow intelligence.
14. The "embodied / situated cognition" framework would critique current large language models primarily by pointing out:
Correct.
Incorrect. The embodied cognition critique is fundamental: intelligence requires sensorimotor grounding in a physical world, which disembodied text processing cannot provide.
15. Which statement best captures the central unresolved problem in defining intelligence for AI systems?
Correct. Every benchmark falls. The definition problem is prior to the measurement problem — and the history of AI is partly a history of definitions being revised each time a machine achieves what was previously assumed to require intelligence.
Incorrect. The core problem is conceptual: we do not have an agreed definition of intelligence that is both precise enough to test and robust enough to resist being gamed.