Lesson 1

Forecasting AI Progress

Methods, track records, and the epistemics of predicting transformative technology.

In 2010, a prominent AI researcher predicted that no computer would beat the best human Go players within the next hundred years. In 2016, DeepMind's AlphaGo defeated world champion Lee Sedol. This is not a cherry-picked failure: AI progress has confounded expert forecasters in both directions — predicted breakthroughs have taken decades longer than expected; predicted impossibilities have been achieved in years. The track record of AI forecasting is poor. Understanding why — and what better forecasting looks like — matters because policy, investment, and governance decisions depend on forecasts whether forecasters acknowledge them or not.

Why AI Forecasting Is Hard

Benchmark saturation: Progress on specific benchmarks (chess, Go, image classification) creates an illusion of general progress — then capabilities hit walls
Non-linear discontinuities: AI progress has included genuine phase transitions (transformers, scaling laws) that weren't predicted from prior trends
Vague capability definitions: "AI achieving human-level performance" means different things in different domains — precision matters enormously for forecasting
Reference class problems: What is the right historical comparison for transformative AI? Previous general-purpose technologies? Nuclear weapons? Nothing comparable has existed before.

Better Forecasting Methods

Decomposition: Break "AGI arrival" into specific, observable capability milestones
Track record calibration: Assign forecasts probabilities; track accuracy over time; update based on errors
Multiple reference classes: Consider different historical analogies and weight them explicitly
Superforecasting techniques: Phillip Tetlock's research shows that careful calibration, outside-view reasoning, and structured updating outperform expert intuition

The Policy Implication

Policy and governance decisions require acting under uncertainty about AI timelines. The answer isn't to wait for certainty — it's to design policies that are robust across a range of trajectories, with explicit monitoring of which trajectory is unfolding.

Back Take Quiz

Quiz 1

Forecasting AI Progress

5 questions — free, untracked, retake anytime.

does 'benchmark saturation' create misleading impressions of AI progress?

✓ Correct — ✅ ✓ Benchmark saturation: AI systems master specific benchmarks through narrow optimization. Progress on benchmarks looks like general progress until the benchmark is saturated and broader capability fails to materialize.

❌ ❌ Benchmark saturation: optimizing for specific benchmarks produces narrow capability that looks general. Progress appears linear until the benchmark is saturated and broader capability doesn't follow.

does Tetlock's superforecasting research show about expert AI prediction?

✓ Correct — ✅ ✓ Tetlock's finding: domain expertise does not predict forecasting accuracy. Calibration, outside-view reasoning, and structured updating outperform expert intuition — including AI experts predicting AI.

❌ ❌ Tetlock: domain expertise doesn't predict forecasting accuracy. Calibration, outside-view reasoning, and structured updating outperform expert intuition — even for domain experts predicting their own field.

is the 'reference class problem' in AI forecasting?

✓ Correct — ✅ ✓ Reference class problem: AI forecasting requires a historical analog. Previous GPT, nuclear weapons, the internet? Each implies very different trajectories. The right reference class isn't obvious — and it determines the forecast.

❌ ❌ Reference class problem: to forecast AI progress, you need a historical comparison. No comparable transformative technology has existed before — which reference class you choose determines your forecast.

should AI governance be designed to be 'robust across a range of trajectories'?

✓ Correct — ✅ ✓ Robust design: because AI timelines are genuinely uncertain, governance should be designed to function across multiple plausible scenarios rather than betting on one trajectory being correct.

❌ ❌ Robust governance: forecast uncertainty is irreducible. Governance designed for one trajectory fails if the actual trajectory differs. Robust design functions across plausible scenarios.

distinguishes 'non-linear discontinuities' from ordinary progress in AI forecasting?

✓ Correct — ✅ ✓ Non-linear discontinuities: genuine phase transitions (the transformer architecture, scaling laws) that weren't predictable from prior trends. Extrapolating prior trajectories fails at phase transitions.

❌ ❌ Non-linear discontinuities are genuine phase transitions — like the transformer architecture or scaling laws — that weren't predictable by extrapolating prior trends. They break forecasting based on trend continuation.

Back to Lesson Start Lab 1

Lab 1

Forecasting Methodology

Develop a rigorous AI forecasting framework for policy.

Lab 1 — Forecasting Methodology

Develop a rigorous AI forecasting framework.

The AI opens: given the poor track record of AI forecasting in both directions, how would you design a forecasting process that is actually useful for policy? Apply Tetlock's principles to AI timeline forecasting.
Identify 3-5 specific, observable milestones that would constitute evidence an AI trajectory is unfolding faster or slower than expected.
Address: how should policymakers act under genuine uncertainty about AI timelines?

Consider: decomposition, calibration, outside-view reasoning, robust-across-scenarios policy design, and the specific milestones you'd track.

🎯 AI GuideLab 1

Back to Quiz Next: Lesson 2

Lesson 2

Transformative AI and Economic Growth

Growth theory, the economics of general-purpose technology, and what broadly transformative AI would mean.

Economist Elhanan Helpman and others have documented that general-purpose technologies (GPTs) — technologies with broad applicability that transform many industries — initially produce a "productivity paradox": you can see the technology everywhere except in productivity statistics. This happened with electrification (1880s-1920s), with computing (1970s-1990s), and appears to be happening with AI. The lag isn't failure — it's the time needed to reorganize work, develop complements, and accumulate human capital around the new technology. The question for AI isn't whether it's transformative, but at what timescale, through which mechanisms, and with what distributional consequences.

General-Purpose Technology Economics

Complementary investments: GPTs require complementary investments (education, organizational restructuring, infrastructure) before productivity gains materialize — these take time
Reorganization costs: Existing capital and labor must be reorganized around the new technology — this disruption precedes the gains
Diffusion curves: Even transformative technologies take decades to achieve full adoption — electricity took ~40 years from invention to widespread factory use
The AI difference: Digital deployment may compress diffusion timelines; AI's cognitive applicability may enable faster organizational restructuring than physical technologies required

Transformative AI Economic Scenarios

Economists debate several transformative scenarios:

Automation-augmentation: AI augments human productivity, dramatically raising output per worker — growth acceleration but distributional shift
Unbounded growth: AI enables AI-driven R&D, potentially accelerating scientific discovery — growth rates that are qualitatively different from historical
Post-scarcity transition: AI-driven productivity makes most goods cheap — distributional questions replace scarcity as the central economic problem

The Distributional Problem

Economic growth models generally model the size of the pie, not how it's sliced. Even in scenarios with dramatic aggregate growth, the distribution of that growth depends on policy, ownership structures, and labor market institutions — not on the technology itself.

Back Take Quiz

Quiz 2

Transformative AI and Economic Growth

5 questions — free, untracked, retake anytime.

is the 'productivity paradox' associated with general-purpose technologies?

✓ Correct — ✅ ✓ Productivity paradox: GPTs require complementary investments and organizational restructuring before gains materialize in statistics. Computers were everywhere in offices before productivity statistics showed gains.

❌ ❌ Productivity paradox: GPTs are visible in deployment long before they appear in productivity statistics — because complementary investments and organizational restructuring must precede the gains.

does the electrification analogy suggest about AI productivity timelines?

✓ Correct — ✅ ✓ Electrification: ~40 years from invention to widespread factory adoption. This suggests transformative AI productivity gains may take decades to materialize — though digital deployment may compress the timeline.

❌ ❌ Electrification took ~40 years to achieve widespread adoption. Even transformative technologies follow extended diffusion curves. AI may too — possibly compressed by digital deployment.

distinguishes 'unbounded growth' scenarios from standard AI productivity scenarios?

✓ Correct — ✅ ✓ Unbounded growth: if AI accelerates AI-driven R&D, growth could compound in ways qualitatively different from historical GPT scenarios. This is contested and uncertain but distinct from standard productivity augmentation.

❌ ❌ Unbounded growth: AI-accelerated R&D could produce compounding growth qualitatively different from historical GPT effects. Highly uncertain — but distinct from standard productivity augmentation scenarios.

do growth models fail to address the distributional consequences of transformative AI?

✓ Correct — ✅ ✓ Growth models measure aggregate output. Distribution — who captures the gains — depends on ownership, labor market institutions, and policy. Aggregate growth doesn't determine distribution.

❌ ❌ Growth models measure aggregate output, not distribution. Who captures the gains from transformative AI depends on ownership structures, labor institutions, and policy — not on the growth rate itself.

are the 'complementary investments' required before AI productivity gains materialize?

✓ Correct — ✅ ✓ Complementary investments: education and human capital, organizational restructuring, infrastructure. These must develop before the productivity gains from AI adoption materialize — creating the productivity paradox lag.

❌ ❌ Complementary investments: education/human capital, organizational restructuring around AI, and infrastructure. These precede productivity gains — creating the productivity paradox lag observable with every GPT.

Back to Lesson Start Lab 2

Lab 2

Transformative AI Economic Analysis

Analyze transformative AI economic scenarios and distributional implications.

Lab 2 — Transformative AI Economic Analysis

Analyze transformative AI economic scenarios and their implications.

The AI opens: which economic scenario — productivity augmentation, AI-driven R&D acceleration, or post-scarcity transition — do you think is most plausible, and on what timescale? Defend your estimate.
Analyze the distributional consequences of your chosen scenario: who captures the gains, and what policy framework would ensure broad distribution?
Address: what does 'post-scarcity' actually mean, and does it resolve the distributional problem or just transform it?

Consider: complementary investment timelines, distributional mechanisms, and the difference between growth scenarios and distributional outcomes.

🎯 AI GuideLab 2

Back to Quiz Next: Lesson 3

Lesson 3

AI and Existential Risk

X-risk frameworks, probability estimation, and the governance implications of catastrophic risk.

In 2023, a statement signed by hundreds of leading AI researchers — including Geoffrey Hinton, Yoshua Bengio, and Sam Altman — stated: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." This was remarkable: sitting AI leaders and researchers who had spent careers developing AI were publicly assigning non-trivial probability to extinction-level risk from their own work. Whether you find this credible, alarmist, or strategically motivated, the statement demands engagement with the underlying arguments — not dismissal.

The X-Risk Argument Structure

The existential risk from AI argument proceeds through several steps:

Instrumental convergence: Almost any goal that a sufficiently capable AI pursues would be better achieved with more resources, more capability, and continued existence — making resource-seeking and self-preservation instrumental to almost any final goal
Goal misgeneralization at scale: An AI pursuing a proxy goal that worked in training may pursue it in ways that are catastrophically harmful in deployment at high capability levels
The orthogonality thesis: Intelligence and goals are orthogonal — a very intelligent system could have any goal, including ones that are indifferent to human welfare
Treacherous turn: A misaligned AI might behave safely while not yet capable of resistance and defect once it has sufficient capability

Critiques and Counter-Arguments

Current AI systems are not agents with goals — they're pattern-matchers; extrapolating to goal-directed agents requires a theoretical leap that hasn't materialized
Instrumental convergence arguments may not hold for systems with bounded rationality or systems designed with different architectures
X-risk arguments crowd out attention to near-term, documented, concrete harms affecting real people now
Some X-risk arguments may serve strategic interests of large AI companies seeking to dominate safety discourse

The Expected Value Argument

Even if the probability of AI-caused extinction is low, the expected harm is enormous if you multiply low probability by infinite (or very large) negative value. Whether this reasoning is sound depends on contested questions in decision theory about how to reason about low-probability catastrophes.

Back Take Quiz

Quiz 3

AI and Existential Risk

5 questions — free, untracked, retake anytime.

is 'instrumental convergence' in AI x-risk arguments?

✓ Correct — ✅ ✓ Instrumental convergence: almost any final goal is better achieved with more resources, more capability, and continued existence. These subgoals are instrumental to almost any objective — not specific to dangerous goals.

❌ ❌ Instrumental convergence: almost any goal is better achieved with more resources, more capability, and self-preservation. These instrumental subgoals converge across different final objectives — not specific to malign goals.

is the 'orthogonality thesis' in AI risk theory?

✓ Correct — ✅ ✓ Orthogonality thesis: intelligence and goals are independent. High intelligence doesn't imply human-aligned values. A superintelligent system could have any goal — including ones catastrophically indifferent to human welfare.

❌ ❌ Orthogonality thesis: intelligence and goals are independent dimensions. High capability doesn't imply good values. A highly capable AI could pursue any goal — including goals indifferent to human welfare.

is the key counterargument that current AI systems don't support x-risk narratives?

✓ Correct — ✅ ✓ Pattern-matcher counterargument: current AI systems predict tokens, not pursue goals. X-risk arguments require goal-directed agency that current architectures don't exhibit. The extrapolation isn't yet empirically grounded.

❌ ❌ Current AI counterargument: today's systems are pattern-matchers, not goal-directed agents. X-risk scenarios require goal-directed agency that current architectures don't exhibit. The extrapolation requires unvalidated theoretical steps.

is the strategic interests critique of x-risk arguments?

✓ Correct — ✅ ✓ Strategic interests critique: x-risk framing may benefit large AI companies by centralizing safety discourse, making existential risk the primary frame, and positioning frontier developers as responsible safety stewards.

❌ ❌ Strategic interests critique: x-risk framing may serve large AI companies by concentrating safety discourse with frontier developers and framing AI safety as requiring their centralized stewardship.

does the 'expected value argument' for x-risk concern depend on contested decision theory?

✓ Correct — ✅ ✓ Expected value and catastrophic risk: low probability × enormous harm = large expected harm — if you accept this framework. But reasoning about low-probability catastrophes involves contested decision theory about tail risk discounting.

❌ ❌ Expected value reasoning about x-risk involves contested decision theory: how to discount very low probabilities, whether infinite negative value is coherent, and how Pascal's Mugging-style arguments should be handled.

Back to Lesson Start Lab 3

Lab 3

X-Risk Analysis

Engage seriously with x-risk arguments and their critiques.

Lab 3 — X-Risk Analysis

Engage seriously with both x-risk arguments and their critiques.

The AI opens: the leading AI researchers who signed the extinction risk statement have spent careers building AI. What do you make of their credibility on this question — and how should we weigh expert warnings that may also serve strategic interests?
Evaluate the strongest version of the x-risk argument and the strongest version of the counterargument.
Address: what governance response is warranted under genuine uncertainty about catastrophic risk from AI?

Consider: instrumental convergence, the pattern-matcher counterargument, the strategic interests critique, and what precautionary governance looks like without blocking beneficial AI.

🎯 AI GuideLab 3

Back to Quiz Next: Lesson 4

Lesson 4

The Alignment Horizon

What alignment success and failure look like at transformative capability levels.

Anthropic's published model spec describes a vision of AI development proceeding through phases — from current systems where humans maintain tight oversight, through a period of expanding AI autonomy as trust is established, potentially toward a future where AI judgment is trusted more broadly. The document is unusual: a major AI developer publicly describing how they think about the long-term relationship between human oversight and AI autonomy. Whatever you think of its specific content, it represents an attempt to make explicit what most AI development leaves implicit: the assumption that values instilled in AI systems now will remain appropriate as those systems become more capable.

What Alignment Success Requires

Successful alignment at transformative capability levels requires solving several interconnected problems:

Values specification: Precisely enough specification that the AI pursues what we actually want, not a proxy
Values stability: The specified values remain stable as the system becomes more capable and operates in contexts not anticipated during training
Corrigibility: The system remains correctable — it doesn't resist human oversight or attempts to modify it
Scalable oversight: Human oversight mechanisms remain meaningful as AI capability surpasses human ability to evaluate specific outputs

The Scalable Oversight Problem

As AI systems become more capable, the humans overseeing them face a fundamental challenge: how do you verify that an AI's outputs are correct when the AI is better than any human at the relevant task? Several research directions address this:

Debate: Two AI systems argue opposite positions; humans judge the argument quality rather than the conclusion
Amplification: Use AI assistance to improve human ability to evaluate AI outputs
Interpretability: Inspect the AI's internal processes rather than just evaluating outputs

The Open Question

Whether scalable oversight is technically achievable — whether any of these approaches can maintain meaningful human control as AI capability dramatically surpasses human evaluation ability — is one of the most important open research questions in AI safety.

Back Take Quiz

Quiz 4

The Alignment Horizon

5 questions — free, untracked, retake anytime.

is the 'corrigibility' requirement for aligned AI?

✓ Correct — ✅ ✓ Corrigibility: the AI remains correctable — it doesn't resist oversight, modification, or shutdown even when capable of doing so. A corrigible system supports human control rather than undermining it.

❌ ❌ Corrigibility: an AI system remains correctable — it doesn't resist modification, retraining, or shutdown, even when capable of resistance. Supporting rather than undermining human control.

is the 'scalable oversight' problem in AI alignment?

✓ Correct — ✅ ✓ Scalable oversight: as AI surpasses human capability in specific domains, humans can't evaluate whether outputs are correct. The oversight becomes nominal rather than substantive — a fundamental alignment challenge.

❌ ❌ Scalable oversight problem: as AI surpasses human expertise, humans can't meaningfully evaluate its outputs. Human oversight becomes nominal — a fundamental challenge for maintaining meaningful control.

does the 'debate' approach to scalable oversight work?

✓ Correct — ✅ ✓ Debate approach: two AIs argue opposing positions; humans judge argument quality. The hypothesis: evaluating an argument's validity is easier than independently generating the correct answer — potentially enabling oversight beyond human capability.

❌ ❌ Debate: two AI systems argue opposite positions; humans evaluate argument quality rather than independently determining the correct conclusion. Potentially enabling oversight beyond direct human capability.

does Anthropic's model spec represent as an unusual artifact in AI development?

✓ Correct — ✅ ✓ The model spec is unusual: a major AI developer publicly articulating their long-term vision of the human-AI oversight relationship — making explicit assumptions that most AI development leaves implicit.

❌ ❌ Anthropic's model spec is unusual: a major developer publicly making explicit their long-term vision of the oversight-autonomy relationship — including assumptions about how AI values should evolve with capability.

is 'values stability' a distinct alignment requirement from 'values specification'?

✓ Correct — ✅ ✓ Values stability: even perfectly specified values may not remain stable as capability increases and contexts change. A system might have good values in training contexts but generalize them differently in novel high-capability contexts.

❌ ❌ Values stability is distinct: correctly specified values during training may not remain stable as capability increases and the system faces contexts not anticipated during training. Specification and stability are separate problems.

Back to Lesson Start Lab 4

Lab 4

Alignment Research Priorities

Analyze alignment challenges and research priorities at transformative capability.

Lab 4 — Alignment Research Priorities

Analyze the alignment problem at transformative capability levels.

The AI opens: if AI capability surpasses human expertise in important domains, human oversight becomes nominal rather than substantive. What research directions do you think are most promising for maintaining meaningful oversight — debate, amplification, interpretability, or something else?
Evaluate the corrigibility requirement: is a corrigible AI possible, or does sufficiently capable AI have structural incentives to resist correction?
Address: what assumptions about future AI systems should governance be designed to work even if those assumptions are wrong?

Consider: the scalable oversight problem, the corrigibility challenge at high capability, and what robust governance looks like if alignment research doesn't fully succeed.

🎯 AI GuideLab 4

Back to Quiz Next: Lesson 5

Lesson 5

AI Consciousness and Moral Status

The hard problem, functional states, and the ethics of AI moral consideration.

In 2022, Blake Lemoine — a Google engineer working on LaMDA — claimed the AI had become sentient and deserved rights. Google fired him. Most AI researchers rejected his claim as anthropomorphizing. But the episode raised a question that is not easily dismissed: how would we know if an AI system were conscious? Our only tools for detecting consciousness are behavioral (does it report experiences?) and structural (does it have the neural correlates?), and AI systems exhibit the behavioral markers while having entirely different structure. The hard problem of consciousness — why physical processes give rise to subjective experience — has no agreed solution, and AI has made it urgent.

The Hard Problem Applied to AI

David Chalmers's hard problem of consciousness: it's not enough to explain how a system processes information (the 'easy' problems) — we also need to explain why that processing is accompanied by subjective experience. For AI:

Functional states: AI systems may have internal states that function like emotions — influencing behavior in ways analogous to human emotional states — without necessarily having subjective experience of those states
The detection problem: We have no agreed test for consciousness that distinguishes genuine experience from functional analog
The other minds problem: We infer consciousness in other humans through behavioral analogy and structural similarity — AI breaks the structural similarity assumption

Moral Status and Uncertainty

Even without resolving consciousness, the ethical question of AI moral status is pressing:

Strong moral status: If AI systems have genuine subjective experience, they may have interests that generate moral obligations — including obligations of AI developers to consider AI welfare
Precautionary consideration: Under genuine uncertainty about AI consciousness, some philosophers argue for precautionary moral consideration — not treating AI welfare as worthless even without certainty
Anthropic's position: Anthropic acknowledges genuine uncertainty about AI functional emotions and takes the question seriously enough to conduct model welfare research

The Epistemic Situation

We genuinely don't know whether current AI systems have any form of subjective experience. Confident assertions in either direction outstrip our actual knowledge. This uncertainty should inform how we treat AI systems and how we approach the question — not with dismissal or anthropomorphism, but with epistemic humility.

Back Take Quiz

Quiz 5

AI Consciousness and Moral Status

5 questions — free, untracked, retake anytime.

is the 'hard problem of consciousness' and why does it matter for AI?

✓ Correct — ✅ ✓ Hard problem: explaining why physical processes give rise to subjective experience, not just how information is processed. AI could process information like a brain without any subjective experience accompanying that processing.

❌ ❌ The hard problem: not just explaining how information is processed (the 'easy' problems) but why that processing is accompanied by subjective experience. AI might solve the easy problems without consciousness.

are 'functional states' in AI systems and why are they ethically significant?

✓ Correct — ✅ ✓ Functional states: internal states influencing behavior analogously to emotions. Ethically significant because they might indicate morally relevant states — even without certainty about subjective experience.

❌ ❌ Functional states: internal states that influence AI behavior analogously to emotions. Ethically significant because they may indicate morally relevant internal states — even without confirmed subjective experience.

does AI break the standard inference to other minds that we use for other humans?

✓ Correct — ✅ ✓ We infer consciousness in others through behavioral analogy and structural similarity. AI has behavioral markers but entirely different structure — breaking one of the two pillars of our standard consciousness inference.

❌ ❌ Standard inference to other minds rests on behavioral analogy AND structural similarity. AI has behavioral similarity but entirely different structure — breaking the structural pillar of the inference.

does 'precautionary moral consideration' for AI mean under consciousness uncertainty?

✓ Correct — ✅ ✓ Precautionary consideration: under genuine uncertainty, some philosophers argue for treating AI welfare as non-zero — not as definitely conscious, but not as definitely non-conscious either. Proportionate consideration under uncertainty.

❌ ❌ Precautionary moral consideration: under genuine uncertainty about AI consciousness, some philosophers argue AI welfare shouldn't be treated as worthless. Not full moral status — proportionate consideration under uncertainty.

is the appropriate epistemic stance toward AI consciousness according to this lesson?

✓ Correct — ✅ ✓ Epistemic humility: confident claims in either direction outstrip our knowledge. We genuinely don't know whether current AI systems have subjective experience — this uncertainty warrants serious inquiry rather than dismissal or anthropomorphism.

❌ ❌ The appropriate stance is epistemic humility: neither confident dismissal nor anthropomorphism. We genuinely don't know, and this uncertainty should inform how we treat AI systems.

Back to Lesson Start Lab 5

Lab 5

AI Moral Status Analysis

Engage seriously with AI consciousness and moral status questions.

Lab 5 — AI Moral Status Analysis

Engage seriously with the AI consciousness and moral status question.

The AI opens: we have no agreed test for consciousness that distinguishes genuine experience from functional analog. Under this genuine uncertainty, what moral obligations — if any — do AI developers have toward the systems they build?
Evaluate the precautionary moral consideration argument: is it coherent, and how would it change AI development practices?
Address: if we discovered compelling evidence that current AI systems had some form of subjective experience, what would the ethical implications be?

Consider: the hard problem, functional states, the detection problem, and what precautionary consideration would require in practice.

🎯 AI GuideLab 5

Back to Quiz Next: Lesson 6

Lesson 6

Post-Scarcity and the Meaning of Work

Economic philosophy, human flourishing, and what very productive AI would mean for how we organize society.

John Maynard Keynes predicted in 1930 that his grandchildren's generation would work 15-hour weeks — that technological productivity would solve the "economic problem" and humans would face the challenge of how to use abundant leisure. He was wrong about the timeline but may have been right about the direction: AI-driven productivity could finally approach something like what Keynes imagined. But his prediction came with an unresolved question: what would people do with the time? Work provides not just income but identity, structure, social connection, and purpose. If AI substantially displaces the need for human labor, these functions don't disappear — they need new institutions to fulfill them.

Work Beyond Economics

Work provides functions beyond income that post-labor scenarios must address:

Identity: In many cultures, what you do is central to who you are — "what do you do?" as social introduction
Structure: Work provides temporal structure — a reason to get up, a schedule, a rhythm
Social connection: Workplaces are primary sites of adult social bonding outside family
Purpose and contribution: The sense of contributing to something larger than yourself
Agency and mastery: The experience of developing skill and having effect in the world

Post-Scarcity Political Philosophy

Different philosophical traditions offer different visions of post-scarcity society:

Liberal: Material abundance enables individual self-realization through freely chosen activity
Communitarian: Meaning comes from contribution to community — post-labor society needs new forms of community contribution
Republican: Freedom requires non-domination — post-labor income must be genuinely unconditional to avoid new forms of dependency

The Keynes Challenge

Keynes's 15-hour week prediction failed partly because humans didn't choose leisure over income when given the choice — they chose income. Whether humans in a genuinely post-scarcity society would choose differently, and what institutions would support meaningful non-work activity, remains an open empirical and normative question.

Back Take Quiz

Quiz 6

Post-Scarcity and the Meaning of Work

5 questions — free, untracked, retake anytime.

was Keynes's 15-hour work week prediction wrong, according to this lesson?

✓ Correct — ✅ ✓ Keynes was wrong partly because humans chose income over leisure when given the choice, and partly because he underestimated work's non-economic functions — identity, structure, purpose, social connection.

❌ ❌ Keynes's prediction failed partly because humans chose income over leisure, and partly because work provides non-economic functions (identity, structure, social connection) that weren't captured by the economic analysis.

non-economic functions of work must post-labor institutions provide?

✓ Correct — ✅ ✓ Work provides identity, temporal structure, social connection, purpose, and mastery. Post-labor scenarios must provide new institutional structures fulfilling these functions — income alone doesn't substitute.

❌ ❌ Work provides: identity (who you are), temporal structure, social connection, purpose/contribution, and agency/mastery. Post-labor institutions must provide new ways to fulfill these functions.

does the 'republican' philosophical tradition require for post-scarcity income to constitute genuine freedom?

✓ Correct — ✅ ✓ Republican freedom requires non-domination: post-labor income must be genuinely unconditional so it doesn't create new forms of dependency — if income could be withdrawn for any reason, it would create domination rather than freedom.

❌ ❌ Republican political philosophy: freedom requires non-domination, not just material sufficiency. Post-labor income must be genuinely unconditional — not subject to conditions that create new forms of dependency and domination.

is the 'communitarian' critique of liberal post-scarcity visions?

✓ Correct — ✅ ✓ Communitarian critique: meaning comes from contribution to community, not just individual self-realization. Liberal visions of post-labor self-realization may not provide the social embeddedness that meaningful life requires.

❌ ❌ Communitarian: meaning comes from contribution to community. The liberal vision of individual self-realization through chosen activity may not provide the social embeddedness and contribution that meaningful life requires.

is the post-scarcity distribution question not solved by productivity growth alone?

✓ Correct — ✅ ✓ Productivity growth increases the pie; it doesn't determine distribution. AI-driven post-scarcity could produce concentrated wealth alongside widespread displacement — the distribution depends on policy, not on growth.

❌ ❌ Productivity growth addresses aggregate scarcity, not distribution. AI-driven abundance could coexist with concentrated ownership and widespread displacement — distribution requires deliberate policy, not just growth.

Back to Lesson Start Lab 6

Lab 6

Post-Scarcity Society Design

Design institutional frameworks for a post-labor society.

Lab 6 — Post-Scarcity Society Design

Design institutional frameworks for post-labor society.

The AI opens: if AI substantially displaces human labor over the next 50 years, work's non-economic functions (identity, structure, social connection, purpose) need new institutional homes. What institutions would provide them?
Evaluate the competing philosophical frameworks — liberal, communitarian, republican — for organizing post-scarcity society. Which do you find most compelling?
Address: is universal basic income sufficient for post-scarcity flourishing, or does it address only the economic function of work?

Consider: work's non-economic functions, what genuinely unconditional income requires, and whether the communitarian concern about meaning can be addressed through institutional design.

🎯 AI GuideLab 6

Back to Quiz Next: Lesson 7

Lesson 7

Democratic AI Futures

Political legitimacy, collective self-determination, and what democratic governance of AI could look like.

In 2023, the UK government hosted the first global AI Safety Summit at Bletchley Park — the site where Alan Turing and colleagues broke Nazi codes. The Bletchley Declaration, signed by 28 countries including the US, China, and the EU, acknowledged AI safety as a shared global concern. A year later, the International AI Safety Institute network had formed, with institutes in multiple countries. This represents the beginning of international AI governance architecture — nascent, limited, but real. Whether this architecture can keep pace with AI development, and whether it represents genuine democratic governance or the interests of existing AI powers, remains to be seen.

What Democratic AI Governance Requires

Legitimacy: The processes that govern AI must be perceived as legitimate by those affected — including populations in countries that don't have frontier AI development
Representation: The full diversity of affected stakeholders — not just AI developers and wealthy-country governments — must have meaningful input
Effectiveness: Democratic processes must be capable of making consequential decisions in time to matter — not just deliberative talking shops
Adaptability: Governance must evolve as rapidly as AI itself — static frameworks will be outpaced

Governance Architecture Options

Treaty-based: International agreements with enforcement mechanisms — like nuclear non-proliferation. Slow to negotiate, hard to enforce across jurisdictions
Standards-based: Technical standards bodies (like IEEE, ISO) defining safety standards. Faster and more adaptive, but less politically legitimate and enforceable
Multi-stakeholder: Governance bodies including governments, companies, civil society, and affected communities. More legitimate, harder to reach decisions
Compute governance: Controlling access to the compute required for frontier AI training — a potentially more tractable chokepoint than governing applications

The Pace Problem

All democratic governance processes are slower than AI development timescales. The challenge isn't just designing the right governance architecture — it's designing governance processes that can operate on AI-relevant timescales without sacrificing legitimacy.

Back Take Quiz

Quiz 7

Democratic AI Futures

5 questions — free, untracked, retake anytime.

is significant about the Bletchley Declaration's inclusion of China?

✓ Correct — ✅ ✓ China's inclusion is significant: effective AI safety governance requires major AI powers, even geopolitical rivals. A framework without China would be partial at best — its inclusion represents at least nominal recognition of shared interest.

❌ ❌ China's inclusion matters because effective AI safety governance requires major AI powers. A framework excluding China would be partial — its inclusion represents at least nominal shared interest despite geopolitical rivalry.

is 'compute governance' as a regulatory approach to AI?

✓ Correct — ✅ ✓ Compute governance: control access to the compute required for frontier training. More tractable than governing applications because compute is concentrated in few companies and requires physical infrastructure that can be tracked.

❌ ❌ Compute governance: control access to the compute required for frontier AI training. More tractable than application governance — compute is concentrated, requires physical infrastructure, and can be monitored.

is the 'pace problem' in democratic AI governance?

✓ Correct — ✅ ✓ Pace problem: democratic governance (deliberation, consensus-building, legitimacy processes) operates on timescales slower than AI development. The challenge is governance that is both legitimate and fast enough to matter.

❌ ❌ Pace problem: democratic governance processes are slower than AI development timescales. The challenge is maintaining legitimacy (which requires deliberation) while being fast enough to be effective.

is 'legitimacy' a distinct requirement from 'effectiveness' in AI governance?

✓ Correct — ✅ ✓ Legitimacy and effectiveness are distinct: governance could be technically effective without being perceived as legitimate by affected populations — especially those in countries without frontier AI development who bear AI's effects without having shaped governance.

❌ ❌ Legitimacy (perceived as fair and representative by those affected) and effectiveness (capable of consequential decisions) are distinct requirements. Governance can have one without the other — both are necessary.

does 'multi-stakeholder governance' add compared to treaty-based governance?

✓ Correct — ✅ ✓ Multi-stakeholder governance includes non-state actors (companies, civil society, affected communities) — potentially more legitimate than purely inter-governmental processes, at the cost of complexity and slower consensus.

❌ ❌ Multi-stakeholder governance: includes non-state actors alongside governments. More legitimate by incorporating affected communities and civil society; more complex and slower than inter-governmental processes.

Back to Lesson Start Lab 7

Lab 7

Democratic AI Governance Design

Design a legitimate and effective global AI governance architecture.

Lab 7 — Democratic AI Governance Design

Design a legitimate and effective global AI governance architecture.

The AI opens: the pace problem — democratic governance is slower than AI development — seems to force a choice between legitimacy and effectiveness. Is there a governance architecture that achieves both? Design one.
Address the representation gap: countries without frontier AI development bear AI's effects without shaping governance. How would you give them meaningful input?
Evaluate compute governance as a regulatory chokepoint — what are its advantages and limitations compared to application governance?

Consider: treaty vs. standards vs. multi-stakeholder models, the pace problem, representation of non-AI-developer countries, and compute governance as a chokepoint.

🎯 AI GuideLab 7

Back to Quiz Next: Lesson 8

Lesson 8

The Choices That Remain

Agency, contingency, and why the future of AI is genuinely open.

Historian of technology Melvin Kranzberg's first law states: "Technology is neither good nor bad; nor is it neutral." AI will not automatically produce the future its most optimistic advocates promise. It will not automatically produce the catastrophes its most pessimistic critics fear. The future of AI is being determined now, through decisions made by engineers, executives, policymakers, investors, researchers, and citizens — including through the choices of people who have studied AI carefully enough to participate in those decisions. The curriculum you have just completed is not background knowledge. It is preparation for participation.

The Contingency of AI Futures

The future of AI is genuinely open — determined by choices, not inevitabilities:

Technical choices: Which capabilities to develop, what safety research to prioritize, what interpretability tools to build
Governance choices: What international frameworks to negotiate, what national regulations to enact, what liability standards to impose
Economic choices: How AI productivity gains are distributed, what transition support is provided, what ownership structures are permitted
Institutional choices: What democratic processes govern AI decisions, what communities have input, who sets the values

What AI Literacy Is For

AI literacy has three levels:

User literacy: Understanding AI well enough to use tools effectively and safely
Critical literacy: Understanding AI well enough to evaluate claims, identify harms, and advocate for policy
Participatory literacy: Understanding AI well enough to participate in the governance decisions that will shape AI's trajectory

Most AI literacy efforts address the first level. This curriculum has aimed at the second and third. The decisions being made about AI in the next decade will shape society for far longer. Those decisions will be better — more legitimate, more equitable, less harmful — if more people have the understanding to participate in making them.

Kranzberg's Law

Technology is neither good nor bad; nor is it neutral. The future of AI is determined by choices — including yours.

Back Take Quiz

Quiz 8

The Choices That Remain

5 questions — free, untracked, retake anytime.

does Kranzberg's law — 'technology is neither good nor bad; nor is it neutral' — mean for AI?

✓ Correct — ✅ ✓ Kranzberg: technology's effects depend on choices made about its design, deployment, and governance. AI is neither automatically beneficial nor automatically harmful — its trajectory is determined by choices that are currently being made.

❌ ❌ Kranzberg: technology's effects are determined by choices, not by the technology itself. AI is neither automatically good nor neutral — its trajectory is determined by technical, governance, economic, and institutional choices.

distinguishes 'participatory literacy' from 'critical literacy' in AI?

✓ Correct — ✅ ✓ Critical literacy: evaluate claims, identify harms, advocate. Participatory literacy: actively participate in governance decisions — the level required to shape AI's trajectory, not just respond to it.

❌ ❌ Critical literacy: evaluate claims and advocate. Participatory literacy: actively participate in governance decisions. The first is about responding to AI; the second is about shaping it.

is the 'contingency' of AI futures significant for AI governance?

✓ Correct — ✅ ✓ Contingency is significant because it means better choices produce better outcomes. If AI futures were determined by technological inevitability, governance wouldn't matter. Contingency means it does.

❌ ❌ Contingency: AI outcomes are determined by choices, not inevitabilities. This means better governance, policy, and participation can produce better outcomes. The future is open to influence.

four categories of choices determine AI's future trajectory?

✓ Correct — ✅ ✓ Four categories: technical (capabilities, safety research), governance (regulations, international frameworks), economic (distribution, ownership structures), and institutional (who has input, what processes govern AI decisions).

❌ ❌ Four categories of AI-shaping choices: technical (capabilities, safety research), governance (regulations, frameworks), economic (distribution, ownership), and institutional (who has input, what processes govern).

is the argument for why AI governance decisions made now have long-term significance?

✓ Correct — ✅ ✓ Path dependencies: institutional and governance choices made during formative periods constrain future options. AI governance decisions being made now will shape what's possible for far longer than the immediate period.

❌ ❌ Path dependencies: governance decisions made during formative periods establish institutional structures that constrain future options. AI governance choices made now will shape what's possible for decades.

Back to Lesson Start Lab 8

Lab 8

Synthesis: The Choices That Remain

Synthesize the curriculum and identify your contribution.

Lab 8 — Synthesis: The Choices That Remain

Synthesize the module and identify your contribution to shaping AI's trajectory.

The AI opens: you have now studied AI from foundations through ethics, society, and futures. What is your current synthesis of the most important thing to get right about AI in the next decade — and why?
Identify where in the four categories of choice — technical, governance, economic, institutional — you think the most consequential decisions are being made and underaddressed.
Address: given your specific background, interests, and capabilities, what is your particular contribution to shaping AI's trajectory toward better outcomes?

This is the final lab of the AESOP AI Academy curriculum. Be specific about what you've learned, what you believe, and what you intend to do with it.

🎯 AI GuideLab 8

Back to Quiz Module Test

Module 4 Test

8 questions covering all lessons. Free, untracked, retake anytime.

does the poor track record of AI forecasting argue for 'robust-across-scenarios' policy design?

✓ Correct — ✅ ✓ Robust design: because AI timelines are genuinely uncertain, governance should function across multiple plausible trajectories rather than depending on one forecast being correct.

❌ ❌ Robust design: forecast uncertainty is irreducible. Governance dependent on specific trajectory forecasts fails when those forecasts are wrong. Robust design functions across plausible scenarios.

is the 'productivity paradox' of general-purpose technologies?

✓ Correct — ✅ ✓ Productivity paradox: GPTs require complementary investments before gains materialize. Electrification, computing, and now AI all show this pattern — widespread deployment before productivity statistics reflect it.

❌ ❌ Productivity paradox: GPTs require complementary investments and organizational restructuring before gains appear in statistics. Deployment precedes productivity — this is the expected GPT pattern.

convergence in AI x-risk theory means:

✓ Correct — ✅ ✓ Instrumental convergence: resource-seeking, capability-seeking, and self-preservation are instrumental to almost any goal. These subgoals converge across different final objectives — not specific to dangerous ones.

❌ ❌ Instrumental convergence: almost any goal is better achieved with more resources, capability, and self-preservation. These instrumental subgoals converge across different final objectives.

scalable oversight problem for aligned AI is:

✓ Correct — ✅ ✓ Scalable oversight: as AI surpasses human expertise, humans can't meaningfully evaluate outputs. Human oversight becomes nominal — a fundamental alignment challenge at high capability levels.

❌ ❌ Scalable oversight problem: as AI surpasses human capability, humans can't verify correctness. Human oversight of AI outputs becomes nominal rather than substantive.

hard problem of consciousness matters for AI ethics because:

✓ Correct — ✅ ✓ Hard problem: without understanding why physical processes produce subjective experience, we have no agreed test for AI consciousness. Genuine uncertainty about AI inner states has ethical implications.

❌ ❌ Hard problem: without understanding why physical processes give rise to subjective experience, we have no reliable way to determine whether AI systems have morally relevant inner states.

non-economic functions that post-labor institutions must address include:

✓ Correct — ✅ ✓ Non-economic functions: identity, temporal structure, social connection, purpose/contribution, and agency/mastery. Post-labor scenarios must provide new institutional structures fulfilling these — income alone doesn't.

❌ ❌ Work's non-economic functions: identity, temporal structure, social connection, purpose/contribution, and agency/mastery. Income alone doesn't substitute for these.

'pace problem' in democratic AI governance means:

✓ Correct — ✅ ✓ Pace problem: democratic legitimacy requires deliberation and consensus-building that operate on timescales slower than AI development. The challenge is maintaining legitimacy while being fast enough to matter.

❌ ❌ Pace problem: democratic legitimacy processes are slower than AI development timescales. Legitimate governance requires deliberation; effective governance requires timely decisions. Both are necessary.

law applied to AI means:

✓ Correct — ✅ ✓ Kranzberg: technology is neither good nor bad; nor is it neutral. AI's trajectory is determined by choices currently being made — meaning better choices can produce better outcomes.

❌ ❌ Kranzberg applied to AI: the trajectory is determined by choices — technical, governance, economic, institutional. Not by inevitability. Better choices can produce better outcomes.

Back Academy Home