1. Which of the following best illustrates the "benchmark laundering" hype language pattern from Lesson 1?
Right. Benchmark laundering is specifically the practice of using a test score as a proxy for general professional capability. The gap between "passed a test" and "can practice law" is enormous — law involves judgment, client relationships, procedural navigation, and contextual reasoning under uncertainty that standardized tests don't measure. Using the test score to imply the general capability is the laundering.
The first option is inevitability framing. The second is demo vs. deployment. The fourth is a form of social proof manipulation. Benchmark laundering specifically involves using a measurable test result to imply a general capability it doesn't actually demonstrate. Which option does that?
2. The lesson says AI power concentration risk "has near-term evidence." What distinguishes it from purely speculative risks?
Exactly. The current industry concentration is a measured, observable fact — not a prediction. The long-term concern about what that implies for democratic governance is the speculative layer built on top of a real foundation. That's what gives this risk more weight than pure conjecture.
The distinction is about what's currently observable vs. what's predicted. Concentration is observable; its downstream governance effects are extrapolated. The lesson doesn't claim documented harm or official classification — it cites measured market structure.
3. A company says their hiring AI is "audited for fairness." What would you need to know to evaluate whether that audit is meaningful?
"Audited for fairness" is a phrase that can mean almost anything. Who conducted it (independent or internal?) matters. Which fairness definition (equal accuracy, demographic parity, equalized odds?) matters — there are dozens of mathematically incompatible fairness definitions. Which demographic groups were examined matters. Whether results were disclosed matters. The phrase alone tells you nothing.
"Audited for fairness" is a hollow claim without specifics. Fairness has multiple mathematical definitions that are often mutually incompatible. An internal audit using a favorable fairness metric, examining only the easiest-to-satisfy demographic splits, and never publishing results tells you nothing useful. You need who, which definition, which groups, and what was found.
4. What does "coverage diversity" mean in the context of building an AI source stack?
Coverage diversity means your source stack includes different source types — investor, journalist, academic, practitioner — so that their different incentive structures create a more complete and balanced picture than any single type could provide.
The lesson's definition of coverage diversity is specifically about source type diversity — not format, geography, or topic angle. The goal is to prevent monoculture where all your sources share the same incentives and therefore the same systematic biases.
5. A peer in your class says "I read that AI can now pass the CPA exam." You ask where they read it. They say "I saw it on Twitter." What should your confidence level be in this claim?
A Twitter attribution with no traceable source puts you at the far end of the amplification chain. The claim might be real — an AI possibly has passed some version of the CPA exam — but you have no way to evaluate the original result from this citation. Very low confidence, treat as hypothesis to investigate, not a fact to repeat.
The correct confidence is very low — not zero (it might be true) but not moderate (you have nothing to evaluate it against). Social media posts are typically the telephone-game endpoint of a chain that may have started with a real study. Without being able to trace it back, you can't know if the original claim was accurate, and you certainly can't know if the Twitter version is faithfully representing it.
6. What does it mean when a language model "hallucinates"?
Correct.
Hallucination refers to generating plausible but factually wrong or fabricated content — a structural property of generative models.
7. What does "benchmark overhang" mean?
Benchmark overhang happens when models are tuned specifically to perform well on evaluation datasets — sometimes to the point where the test score measures the optimization, not the underlying capability. A model can ace a benchmark through specific training without being generalizable. This is why benchmark scores should never be the sole measure of AI capability.
Benchmark overhang describes models optimized for test scores in ways that don't reflect real-world performance. Think of it like teaching to the test — the score goes up, the underlying capability doesn't necessarily follow. The benchmark becomes a measure of benchmark optimization rather than the thing you actually care about.
8. "Expected value reasoning modified by reversibility" suggests that for low-probability but catastrophic, irreversible AI risks, you should:
Right. This is the intellectually honest landing point: you don't need to be a doom believer to rationally support governance investment in low-probability catastrophic risks. The cost-benefit math works differently for irreversible outcomes.
The framework isn't dismissive of speculative risk nor does it equate it with near-term harm. It provides a rational basis for precautionary action without requiring certainty about catastrophic outcomes.
9. Model drift most directly threatens which aspect of an AI system's performance claims?
Model drift is a deployment problem, not a research or benchmark problem. A model that performed accurately at launch can degrade as the world changes and the input data drifts away from the training distribution. This is why post-deployment monitoring matters — the launch accuracy figure is only the starting point.
Model drift affects deployed system performance, not the original paper's results. The paper was accurate when it was written. The deployment accuracy at month 18 is a different question, and drift is why those two numbers can diverge significantly. Ongoing monitoring is the only way to catch drift before it causes serious failures.
10. What is the "access journalism" problem in AI coverage?
The lesson is careful to describe this as structural, not corrupt. Journalists don't make explicit deals. But the incentive structure — losing access if you write something unflattering — creates systematic pressure toward positive framing that operates even without conscious awareness.
The lesson explicitly distinguishes access journalism from explicit agreements or corruption. It's a structural pressure: writing unflattering coverage risks losing future access, so favorable framing becomes the path of least resistance even without any overt arrangement.
11. "AI will automate 40% of knowledge work by 2028" is a classic example of which manipulation pattern?
Right. "Will" converts a probabilistic model with many contested assumptions into a stated fact. The Inevitability Frame does exactly that.
The key tell is "will automate" — that's the Inevitability Frame converting a forecast into a certainty. The claim might be sourced from a single study too, but the primary manipulation is the certainty framing of a speculative projection.
12. The EU AI Act classifies AI hiring tools as "high risk." What does this classification require?
Correct. "High risk" under the EU AI Act triggers transparency and human oversight requirements — not prohibition. This is one of the most significant regulatory frameworks currently in force.
The EU AI Act doesn't ban high-risk applications — it regulates them with transparency and oversight requirements. The distinction between prohibition and regulation matters for understanding what legal protections actually exist.
13. Which of the following correctly pairs an AI risk with the lesson category it belongs to?
Correct. Specification gaming in medical AI is exactly the bridge category the lessons describe — it has a theoretical mechanism grounded in current system behavior and is increasing in relevance as AI autonomy expands in healthcare. Near-term enough to matter, structural enough to shape field practices.
Review the framework: superintelligence in 2025 is speculative (not confirmed near-term); hiring discrimination is near-term documented harm; voice cloning fraud is near-term documented harm. The pairings in the other options misapply the categories.
14. The Mata v. Avianca case (SDNY, 2023) is relevant to AI hallucinations because:
Correct.
Mata v. Avianca is the case where attorneys were sanctioned for submitting AI-fabricated citations — one of the clearest documented professional consequences of AI hallucination in a high-stakes domain.
15. Priya's cover letter approach works well because:
Content from her, polish from AI. That's the right sequencing — the thinking and specific details are hers, which is why the letter was authentic and worked.
The key is who supplies the thinking. She reviewed and selected; she didn't just submit. The details were all hers.
16. The Andreessen Horowitz "Techno-Optimist Manifesto" was published in October 2023. What financial context makes it most important to read it as something beyond a philosophical essay?
Right. The manifesto functions simultaneously as philosophy and investor relations — defending an investment thesis by arguing against the regulations that would most constrain it. That's not a disqualification; it's a context that changes how you should weigh the arguments.
The specific relevant context is the investment fund and the manifesto's arguments against regulation of those investments. Personal wealth and track record are less central than the direct financial stake in the specific argument being made.
17. Content Credentials (C2PA standard) and Google's SynthID are described in the lesson as:
Correct. The lesson's exact language: "Use them as one data point, not a verdict." A negative result from a detector isn't proof of authenticity — it's a piece of evidence to combine with primary source checking and contextual analysis.
The lesson is explicit that these tools aren't definitive. They're useful signals, not verdicts. Treating them as conclusive is itself a form of the automation complacency the lesson warns against.
18. You're reading a McKinsey report claiming AI will create $4.4 trillion in annual economic value. You find the original report and see that the number comes from McKinsey's own proprietary economic model, not external data. What is the correct characterization of this evidence?
Proprietary economic models from firms with commercial interests in the outcome sit near the bottom of the evidence hierarchy for a reason. The model assumptions aren't independently reviewable, the incentives point toward large impressive numbers, and the methodology can't be replicated. Treat it as one data point with wide uncertainty bands, not a finding.
A proprietary model from a consulting firm with incentive to project large AI values is weak-to-moderate evidence at best. The methodology can't be independently examined, the assumptions can't be tested, and the firm profits from advising companies on AI adoption. That doesn't make the number wrong — it makes it unverifiable and incentive-shaped. Wide uncertainty bands are appropriate.
19. You're reading a research paper. The abstract claims strong positive results. The most important section to read next is:
The limitations section is where researchers are required to disclose what their findings don't support — it's almost never quoted in coverage and almost always the most useful part for evaluating how far the claims actually extend.
The limitations section is the part almost never quoted in media coverage and almost always the most informative for evaluating a claim. Researchers are required to describe what their methodology doesn't support — which is exactly what you need to know to assess whether the headline accurately represents the finding.
20. The Anthropic founding story involves people who left OpenAI over safety concerns and then built a competing commercial AI company. The lesson says this logic is "coherent but also unfalsifiable as a personal justification." Which real-world behavior would most directly test whether safety concern or competitive motivation is driving their decisions?
Yes. This is the "does it cost anything" test applied to a company rather than a framework. Public statements, research citations, and team size ratios are all consistent with safety being primarily a brand positioning exercise. A documented case of declining commercial deployment due to safety evaluation — at genuine cost — is what would distinguish real constraint from elaborate rationalization.
Public statements and research outputs are consistent with safety being primarily brand positioning. Team size ratios measure investment, not actual constraint. The diagnostic question is whether safety evaluation has produced a commercially costly "no" — that's what would make the commitment credible as a real constraint on behavior.