L1
·
Quiz
·
Lab
L2
·
Quiz
·
Lab
L3
·
Quiz
·
Lab
L4
·
Quiz
·
Lab
Module Test
Module 8 · Lesson 1

The Information Half-Life Problem

What you learned about AI six months ago may already be wrong — and that's actually fine, if you build the right habits now.
How do you stay calibrated on a technology that changes faster than any single course can track?

Priya graduated with a marketing degree in May 2024. During her senior year she'd taken an AI tools workshop, learned to use ChatGPT for copy drafts, and felt reasonably confident walking into interviews. By August, her first manager asked whether she'd "tried Perplexity for research briefs" and mentioned that the agency had moved its entire content workflow to a tool called Claude. Priya had never heard of either. The job didn't go badly — she was smart and adaptable — but she spent her first month feeling like she'd missed a memo. She hadn't. The memo just hadn't existed yet when she graduated.

That feeling — of being confidently current and then suddenly not — is one of the most disorienting experiences of working in or adjacent to AI right now. And it's not going to stop. The pace of AI development means that specific tool knowledge has a short shelf life. What doesn't expire is your ability to evaluate new claims, contextualize new tools, and decide quickly what deserves your attention.

Why AI Information Decays So Fast

Most domains have a relatively stable knowledge base with incremental updates. Tax law changes annually. Medical guidelines shift over years. Even software frameworks have multi-year lifecycles. AI — particularly the generative AI space — doesn't operate like this. Major capability jumps, new model releases, and significant price or access changes happen on timescales measured in weeks and months, not years.

Consider what changed between January 2023 and January 2025: the entire category of accessible image generation, video generation, and real-time voice interaction went from expensive novelty to commodity. Models that cost dollars per query in 2023 became free or nearly free. Entire companies built on a single API found their value proposition eroded. This isn't a pace you can manage by reading one book or taking one course — it requires an ongoing practice.

The practical consequence is that information has a half-life. Claims about which AI tools are "best," which capabilities are "impossible," and which companies are "leading" all degrade. Your job isn't to memorize the current state — it's to build a reliable process for updating your picture of that state over time.

The Half-Life Principle

Specific AI tool rankings: half-life roughly 3–6 months. Capability claims ("AI can't do X"): half-life roughly 6–18 months. Underlying concepts about how LLMs work, how incentives shape AI development, how to evaluate hype: half-life of several years. Invest accordingly.

The Three Failure Modes Your Peers Are Stuck In

Here's what we're all navigating right now, honestly. Most people in the 18–25 age range fall into one of three patterns when it comes to staying current on AI:

Passive absorption. They scroll LinkedIn and Twitter, see AI posts from tech influencers, and form impressions based on whatever gets the most engagement. The problem is that engagement-optimized content is systematically biased toward extreme claims — either "AI will take your job tomorrow" or "AI is completely overhyped and nothing works." Neither is useful calibration.

One-time deep dive, then nothing. They took a course, watched a documentary, or read a book, then considered themselves informed. This works for stable fields. For AI, it means your mental model is frozen at the moment you stopped actively learning, and that model gets less accurate with every passing month.

Total avoidance. The noise is overwhelming, the claims are contradictory, and it all feels like too much, so they opt out entirely. This one is genuinely understandable — the signal-to-noise ratio in AI coverage is abysmal. But avoidance means you're ceding your own assessment to whoever shapes the narrative around you.

None of these are stupid approaches — they're the predictable responses to a genuinely difficult information environment. The alternative isn't to follow fifty newsletters and read every preprint. It's to build a small, deliberate, sustainable monitoring system.

What a Calibrated Radar Actually Looks Like

The metaphor of a "radar" is useful here: you're not trying to observe everything in the sky at once — you're scanning for things that warrant closer attention. A well-built AI radar has three components:

Signal sources. A small number of trusted, high-quality sources you check regularly. Not comprehensive — curated. Two or three sources that have demonstrated calibrated judgment over time, not just reach or confidence.

Evaluation filters. A set of questions you ask when you encounter a new claim. Is this a researcher, a journalist, an investor, or a marketer making this claim? What's their incentive? What evidence supports this, and how recent is it? These filters don't take long to apply once they're habitual.

A regular review cadence. A specific time — even once a month — when you deliberately update your understanding rather than just absorbing passively. This can be as short as thirty minutes of reading with active note-taking. The regularity matters more than the duration.

Practical Takeaway

Before you leave this lesson: identify one information source you currently rely on for AI news and ask honestly — do they have skin in the game? Who funds them? Have their past predictions been accurate? You don't need to abandon the source, but knowing the answer changes how you read it.

Building the Habit Before You Need It

The worst time to build an AI radar is when you're already behind — when you're about to interview for a job that lists AI skills, or when your team has just adopted a new tool you've never heard of. The best time is right now, when there's no acute pressure, and you can build the habit slowly and deliberately.

One concrete starting point: pick one substantive AI-related article per week and read it with a specific question in mind — not "what is this saying" but "what would I need to believe for this to be true, and do I believe those things?" That question alone changes reading from passive intake to active evaluation. It takes the same amount of time. It produces much better calibration.

The goal isn't to become an AI expert. The goal is to never be caught completely off-guard by a major development in a domain that is now, whether you signed up for it or not, going to shape your career environment for the next twenty years.

Information half-life
The time it takes for a specific claim or piece of knowledge to become significantly less accurate or relevant due to new developments in a fast-moving field.
Calibrated judgment
Confidence in a belief that is proportional to the actual evidence for it — neither overconfident nor underconfident relative to what's known.
Signal-to-noise ratio
In information environments, the proportion of content that is genuinely informative versus content that is loud, high-engagement, or widely shared but not actually reliable.

Lesson 1 Quiz

The Information Half-Life Problem · 5 questions
1. Which type of AI knowledge has the longest half-life and is therefore most worth investing in deeply?
Frameworks for evaluation last because they apply regardless of which specific tools or models exist. Tool rankings, prompting tricks, and pricing all change on timescales of months.
That's the type of information most likely to be outdated quickly. The lesson distinguishes between knowledge that decays fast (specific tools, rankings, pricing) and knowledge with a longer shelf life (how to evaluate claims, how incentives work).
2. Your roommate says they stay current on AI by following a popular tech influencer on Instagram who posts daily. What's the most precise problem with this as a primary strategy?
Exactly. The problem isn't social media per se — it's that platforms reward content that generates strong reactions, which means extreme and sensationalist takes about AI get amplified regardless of accuracy. This creates systematic bias, not random error.
The lesson's critique is more specific than "social media bad." The problem is the incentive structure: engagement-optimization rewards extreme claims, which distorts the information toward both "AI will save/destroy everything" poles and away from calibrated middle-ground analysis.
3. What does the lesson mean by a "regular review cadence" as part of an AI radar?
Right. The key distinction is deliberate vs. passive. Reading incidentally is different from sitting down with the specific intent of updating your mental model. The lesson emphasizes that regularity matters more than duration — even monthly works.
The lesson explicitly says regularity matters more than frequency, and that even a monthly thirty-minute deliberate session is more useful than daily passive scrolling. The cadence is about intentionality, not volume.
4. You're reading an article claiming that a new AI model is "a quantum leap beyond anything previously possible." You apply the evaluation filter from the lesson. What's the first question you should ask?
The lesson lists this as the foundational evaluation filter: researcher, journalist, investor, or marketer? Each has different incentives that shape what they say and how they say it. Technical questions come later — but the incentive question shapes how you read everything else.
Technical specs and competitor reactions are worth examining eventually, but they don't address the source credibility issue. The lesson's first filter is incentive-mapping: who benefits from you believing this claim?
5. Which of the following reading approaches does the lesson recommend for staying calibrated?
This is the specific question the lesson recommends. It converts reading from passive intake to active evaluation without requiring more time — just a different mental posture. It forces you to identify underlying assumptions and test whether you actually accept them.
The lesson recommends a specific active-reading question that works on any source: "what would I need to believe for this to be true, and do I believe those things?" This reframes the same reading time as critical analysis rather than information absorption.

Lab 1: Auditing Your Information Diet

You'll map your current AI information sources and stress-test whether they're actually giving you calibrated signal.

Your Role: Information Auditor

Your lab partner is a sharp peer who has been thinking about AI information quality for a while. They're going to push you to be honest about where your current understanding of AI actually comes from — and whether those sources are producing calibrated judgment or just confidence.

This lab works best if you're actually honest rather than giving the "right" answers. The point is to map your real information environment.

Start by telling your lab partner: where do you currently get most of your information about AI developments — social media, specific newsletters, podcasts, conversations with people you know, or something else? Be specific if you can.
Lab Partner
Information Audit
Hey. Let's do a real audit here — not the version where you list impressive-sounding sources. Where does your actual understanding of what AI is doing right now come from? Don't overthink it. Just tell me where you most recently got information that changed or updated how you think about AI.
Module 8 · Lesson 2

Reading the Landscape: Sources That Actually Help

Not all AI coverage is created equal. Here's how to find the fraction that's actually worth your time — and what makes it trustworthy.
If most AI content is noise, what does the reliable signal actually look like?

Marcus is a junior at a state university studying computer science. He's genuinely interested in AI — not just as a career path but intellectually. So he does what motivated people do: he subscribes to newsletters. By November 2024 he's on twelve of them. He spends about forty minutes every morning going through his inbox. Three of the newsletters are run by VC firms. Four are written by founders of AI companies. Two are written by journalists who cover tech broadly. One is run by an academic researcher. One is a curated link aggregator. And one is from an org he can't quite place — something about "responsible AI futures."

Here's what Marcus notices after a few months: the VC newsletters are consistently bullish on AI's near-term capabilities and consistently skeptical of regulation. The founder newsletters are even more so. The journalists are more variable but tend toward whichever angle makes the best story that week. The researcher's newsletter is denser, slower, and more nuanced — but also more reliably accurate when Marcus checks the claims against what actually happened later. He didn't notice this pattern until he was looking for it.

Why Source Type Predicts Bias Direction

This is one of the most useful heuristics in AI media literacy: you can predict the direction of a source's bias with reasonable accuracy just by knowing what kind of organization produces it. This doesn't mean those sources are lying — it means their incentives reliably shape what they cover, how they frame it, and what they leave out.

Investors and VCs need their portfolio companies and the overall sector to look promising. Their public communications are essentially long-form investment theses. When they write about AI capabilities, they're also, consciously or not, making the case for the sector they've bet on.

AI company communications — press releases, CEO posts, official blogs — are marketing materials with technical vocabulary. This doesn't make them useless. But the framing "our model achieves X on benchmark Y" requires you to know whether benchmark Y is actually meaningful, whether competitors were tested fairly, and whether the capability generalizes outside the test condition.

Tech journalists are incentivized by engagement, which rewards recency and strong takes. Their coverage is often the best available on breaking news but frequently lacks the sustained critical lens to evaluate whether the breaking development actually matters as much as reported.

Academic researchers have their own biases — toward their own area of focus, toward certain methodologies, sometimes toward prestige — but their work is subject to peer critique in ways that investor and corporate communications are not. The caveat: academic AI research is often months or years behind the commercial frontier.

The Bias-Direction Table

Investor content: bullish on AI capabilities, skeptical of regulation, optimistic timelines. Company comms: capability emphasis, competitive framing, benchmark focus. Tech journalism: recency bias, story-framing bias, access journalism risk. Academic research: methodological conservatism, publication lag, depth over timeliness.

The Signals That Indicate a Reliable Source

Reliable doesn't mean neutral — it means that a source has demonstrated, over time, that it will tell you when it got something wrong, that it distinguishes between what is known and what is speculated, and that its past predictions have a track record you can actually check.

Some specific things to look for:

Explicit uncertainty language. Sources that use phrases like "the evidence suggests," "we don't yet know," and "earlier reporting on this was incorrect" are demonstrating calibration. Sources that consistently state things with high confidence regardless of how uncertain the underlying situation actually is are performing confidence, not reporting it.

Distinguishing between demos and deployment. A huge amount of AI hype lives in the gap between a demo (controlled, optimized, cherry-picked) and actual deployment (messy, variable, failure-prone). Reliable sources make this distinction explicitly. Unreliable ones show you the demo video and imply the deployment is equivalent.

Accuracy on previous claims. The single most useful signal for a source's reliability is whether their past confident claims turned out to be right. This takes effort to check but pays significant dividends. If a newsletter told you in 2023 that AGI was "eighteen months away," that's information about how much to trust their next prediction.

Building a Minimal Viable Source Stack

You don't need twelve newsletters. You probably need three to five sources that cover different angles: one that has good access to practitioners and tracks actual deployment (not just announcements), one that covers policy and regulatory developments, one that gives you academic-level depth on capabilities and limitations, and one that does critical journalism on AI industry claims.

The specific sources that fill these slots will change over time — publications get acquired, researchers shift focus, journalists change beats. What matters more than the specific list is that you're intentionally maintaining coverage diversity across these angles rather than accidentally following five sources that all share the same priors.

One practical exercise: take the sources you currently follow and categorize them by type. If they're all investor-adjacent or all skeptic-adjacent, you're getting a monoculture. A well-calibrated picture of AI requires input from people who have very different relationships to the technology and different stakes in its development.

Practical Takeaway

List your current top three AI information sources right now. For each one, identify: (1) what type of entity produces it, (2) what direction their incentive bias runs, and (3) one specific claim they made in the past year that you could check against what actually happened. You don't have to do all three today — but committing to checking even one past prediction is a high-leverage exercise in source calibration.

The Access Journalism Trap

There's a specific failure mode in AI coverage worth naming separately: access journalism. When a reporter gets exclusive interviews with AI company leadership, early access to new models, or briefings before public announcements, they gain reporting advantages — but they also take on implicit obligations. Biting the hand that feeds you access means losing the access.

This produces a structural dynamic in AI journalism: the reporters with the best access to the companies building AI tend to publish the most positive coverage of those companies. This isn't corruption in any simple sense — it's the predictable result of an incentive structure. Being aware of it means treating "exclusive access" pieces differently than independent analysis. The access piece is valuable for what the company wants you to know. The independent analysis is valuable for what the company doesn't.

Access journalism
Reporting that depends on maintaining good relationships with powerful sources, which creates structural pressure to avoid coverage those sources would find unfavorable.
Demo vs. deployment gap
The difference between a controlled, optimized demonstration of AI capability and actual real-world deployment, which is typically messier, less reliable, and more limited.

Lesson 2 Quiz

Reading the Landscape: Sources That Actually Help · 5 questions
1. A prominent venture capital firm publishes a detailed report arguing that AI will achieve human-level reasoning within two years. What is the most important context for reading this claim?
Exactly the right framing. You're not dismissing the claim outright, but you're reading it through the lens of incentive structure. VCs need the sector to look promising. That doesn't make the claim false, but it means you should seek corroborating evidence from sources without that same incentive.
The lesson is clear: you can predict the direction of bias from the source type. VCs are incentivized to be optimistic about AI capabilities. That doesn't mean you ignore their reports, but it means you apply the appropriate discount and seek independent corroboration.
2. Which of these signals most strongly indicates a reliable AI information source?
Right. Explicit uncertainty language and a track record of acknowledging errors are genuine signals of calibrated reporting. Frequency, access, and social sharing say nothing about accuracy — they're measures of reach and relationships, not reliability.
The lesson identifies explicit uncertainty language ("we don't yet know," "earlier reporting was incorrect") and a verifiable track record as the key signals of reliability. Frequency and access can actually correlate with worse accuracy, as the access journalism section explains.
3. What specifically is the "demo vs. deployment gap" and why does it matter for evaluating AI claims?
Correct. The demo is controlled and cherry-picked; deployment is variable and failure-prone. A lot of AI hype lives in this gap — a capability shown in a demo video may work only under ideal conditions, or only for the specific use case shown, or only some percentage of the time. Reliable sources make this distinction; unreliable ones imply equivalence.
The demo vs. deployment gap is specifically about the difference between a controlled demonstration (optimized, curated) and actual deployment (messy, variable). The benchmark point is related but different. The key lesson is that demos systematically overstate what deployment looks like.
4. You discover that a journalist who covers AI got early access to GPT-5 before the public release and published a glowing review. Based on the lesson, what concern does this raise?
The lesson calls this the "access journalism trap." It's not about corruption — it's about incentive structure. A reporter who writes something OpenAI or Anthropic finds deeply unflattering may find their access reduced for the next launch. That's a structural pressure toward positive framing, independent of the journalist's intentions.
The access journalism concern isn't about technical competence or paid advertising — it's more structural than either. When access to sources depends on maintaining good relationships with those sources, there's systematic pressure toward favorable coverage. The lesson specifically distinguishes this from simple dishonesty.
5. What does the lesson recommend as the most useful check on a source's past reliability?
The lesson gives this directly: if a newsletter confidently predicted AGI in 18 months and that didn't happen, that's concrete information about how to weight their next prediction. Track records are more informative than credentials, follower counts, or citation patterns because they're about actual accuracy over time.
The lesson explicitly recommends checking whether past confident claims turned out to be accurate. Followers and citations measure influence, not accuracy. Credentials are relevant context but don't guarantee calibrated judgment — what matters is the demonstrated track record of prediction.

Lab 2: Source Stack Builder

You'll design a minimal, high-quality AI information source stack and defend your choices under pressure.

Your Role: Information Architect

Your lab partner is helping you build a real, usable AI information source stack — not a fantasy reading list, but something you'd actually maintain. They'll ask hard questions about your choices and push back on coverage gaps.

The goal: a set of 3–5 sources that give you diverse angles on AI without taking over your life.

Start by proposing your source stack. Name 3 to 5 sources you'd actually use to stay current on AI over the next year — include the type of source each one is (newsletter, podcast, researcher, journalist, etc.) and what angle it covers. Then your lab partner will probe your choices.
Lab Partner
Source Stack
Alright, let's build something real. Walk me through the sources you'd put in your stack — 3 to 5 is the target. Tell me what each one is, what type of entity produces it, and what gap in your AI understanding it's supposed to fill. I'll push back on anything that looks like a blind spot or a coverage overlap.
Module 8 · Lesson 3

Evaluating New AI Claims in Real Time

A new "breakthrough" drops every few weeks. Here's a repeatable process for deciding what's real, what's overstated, and what you can safely ignore.
When something new hits your feed claiming to change everything — what do you actually do with it?

It's a Tuesday morning and Zoe — a 21-year-old design student with a side hustle doing social media content for small businesses — wakes up to seventeen notifications about something called "Sora 2." Her clients are already texting: "Should we be using this?" Twitter is split between people calling it "the end of professional video" and people calling it "another demo reel that won't work in production." A LinkedIn post from someone with 80,000 followers says her clients should "pivot their entire content strategy immediately." Her friend Maya, who works at a creative agency, texts: "don't panic, we're waiting to see how it actually performs."

Zoe has about twenty minutes between classes to form an opinion that she'll need to deliver credibly to clients by end of day. She doesn't have time to read the research paper. She doesn't have time to run her own tests. She needs a fast, reliable process for triage — for deciding whether this is a real development that changes her work, a real development that doesn't affect her yet, or hype that will mostly fade within a week.

The Five-Minute Triage Protocol

You can't deep-dive every AI announcement. But you can apply a consistent triage process that takes less than five minutes and gives you a defensible initial position. Here's how it works:

Step 1: Who released it and what do they gain? A company releasing its own model and announcing it's revolutionary has obvious incentives. A third-party researcher testing that model and publishing results has different incentives. An academic lab replicating results has different incentives still. The announcement source alone tells you how much skepticism to bring.

Step 2: What's the specific capability claim, and what is it being compared to? "Better than GPT-4" tells you almost nothing without knowing what was measured, how it was measured, and whether the thing being measured is actually what matters for your use case. "Reduces hallucination rate by 60% on medical question-answering benchmarks" tells you something specific that you can reason about.

Step 3: Has anyone without a stake tried it yet? Early reports from people who have no affiliation with the company — even informal Twitter or Reddit posts from developers who clearly know what they're doing — are often more informative than official communications. Absence of these reports, especially for a widely-hyped release, is itself a data point.

Step 4: Does this change the inputs or outputs of something I actually do? Even if a claim is entirely accurate, it may not affect your work at all, or may not affect it yet. A new video model at $500/minute of generation is a real technology development, but if you're a freelance content creator it's not yet your problem. Relevance filtering is a legitimate part of triage, not avoidance.

Step 5: Set a review date, not an immediate verdict. For genuinely uncertain new developments, the best answer is often "I'll form a real opinion in three weeks when the initial hype cycle has played out and independent evaluations exist." This is not a cop-out — it's appropriate calibration. The pressure to have an instant take is real but not well-served by guessing.

What Zoe Actually Told Her Clients

"This looks technically significant but I'm waiting for independent testing before recommending any workflow changes. I'll update you in two weeks with an actual assessment. Don't pivot anything yet." That's a completely defensible and professionally credible answer that requires zero time to research.

Hype Cycle Patterns You Can Recognize

The AI hype cycle has recurring patterns that are worth knowing because they let you calibrate your response before you have all the details. Here's the pattern that plays out most consistently:

Day 1–3 (announcement): Company announcement or leak. Extreme takes in both directions. "This changes everything" vs. "just another incremental improvement." Influencers and investors share enthusiastically. This is the lowest signal-to-noise moment. Anything you conclude here has high probability of needing revision.

Week 1–2 (initial testing): Developers, researchers, and power users who get access start sharing real results. This is where the gap between demo and deployment starts becoming visible. Capability limitations, failure cases, and real performance data begin emerging. This is when the picture starts to clarify.

Week 3–6 (consolidation): More systematic independent evaluation, academic commentary, and media retrospectives. By this point the actual capability delta — how much better is this really than what existed before — is usually assessable. This is the minimum point at which you should form a strong opinion about a new release.

Most people's AI information diet is heavily weighted toward Day 1–3 content, which is also the worst-calibrated. Deliberately waiting even a couple of weeks before forming a firm opinion will improve your accuracy substantially.

When Fast-Moving Claims Affect Real Decisions

Sometimes you can't wait for the consolidation phase. If you're being asked to make a tool adoption decision at work, if a client needs immediate guidance, or if you're about to make an educational investment based on a capability claim, the stakes are real and the timeline is short.

In these cases, the five-minute triage protocol still applies, but you're prioritizing different signals: independent developer reports over official claims, conservative estimates over optimistic ones, and your own direct test of the tool over any documentation. If you can run a quick test yourself in a domain you understand well, that personal test is often worth more than ten reviews from people whose specific use case differs from yours.

One peer pattern worth flagging: a lot of people in our age range feel pressure to appear current and enthusiastic about new AI tools, which creates a bias toward early adoption and positive framing. There's social cost in being the person who says "let's wait and see." But "let's wait and see" is usually the right answer for major decisions, and building the confidence to say it is genuinely useful — professionally and personally.

Practical Takeaway

The next time an AI announcement hits your feed and someone asks your opinion, try this answer: "Based on the announcement alone it sounds promising, but I want to see independent testing before I have a strong view. Check back with me in two weeks." Practice saying that confidently. It's a more credible answer than either enthusiastic endorsement or dismissal based on no direct evidence.

The Compound Interest of Good Calibration

Here's the long-game argument for building this habit: people who consistently apply good triage develop a track record. Over time, your colleagues, clients, and network notice whether your recommendations pan out. The person who excitedly endorsed three tools that didn't work as advertised loses credibility. The person who said "let's wait" and was later vindicated gains it.

Calibration compounds. Good judgment about one AI claim makes it easier to make good judgments about the next one because you start to recognize the patterns. You learn which types of claims from which types of sources have historically been accurate, and you build genuine expertise not about AI per se but about how AI claims work and fail. That expertise is durable in a way that specific tool knowledge isn't.

Triage protocol
A fast, repeatable process for sorting incoming information claims into categories — worth immediate action, worth monitoring, safe to ignore — without requiring deep investigation of every item.
Hype cycle
The predictable pattern by which new technology announcements generate extreme coverage, followed by early testing that reveals limitations, followed by more calibrated assessment. In AI, this cycle often completes within weeks to months.

Lesson 3 Quiz

Evaluating New AI Claims in Real Time · 5 questions
1. A major AI company releases a new model on a Monday and announces it "scores 20% higher than the best competing model." It's Tuesday. Your manager asks whether your team should adopt it. What's the most calibrated response?
This is the triage protocol in action. You're not dismissing the claim, but you're correctly identifying that "20% better" requires knowing what was measured, and that independent testing hasn't happened yet. Asking for a two-week window is a calibrated response, not avoidance.
The lesson's five-step triage protocol is designed for exactly this situation. The announcement is just Step 1 — you need to know what was specifically claimed, whether it's relevant to your use case, and what independent testing reveals before making an adoption decision.
2. According to the hype cycle pattern described in the lesson, when is the earliest point at which you should form a strong opinion about a major new AI release?
The lesson specifies Week 3–6 as the minimum. Week 1–2 starts to clarify the picture but is still early. Day 1–3 is explicitly described as "the lowest signal-to-noise moment." Academic publication timelines would be too slow for practical decision-making.
The lesson is specific: the consolidation phase (Week 3–6) is the minimum point for forming strong opinions about new releases. This is when the actual capability delta becomes assessable, not just claimed. Earlier is still too noisy; waiting for academic publication is usually too slow for practical purposes.
3. What is the most informative early signal about whether a new AI capability claim is genuine, according to the triage protocol?
The lesson specifically points to unaffiliated developers sharing real results — even informal posts from people who clearly know what they're doing — as more informative than official communications or journalist coverage. Their incentives aren't aligned with making the tool look good.
The triage protocol's third step is looking for reports from people without a stake in the outcome. Official announcements have obvious promotional incentives. Journalist coverage is valuable but often comes with access dynamics. Social share counts measure virality, not accuracy. Independent developers are the key signal.
4. Your coworker is excited about a new AI video tool and wants to rebuild your team's content workflow around it. You've only seen the company's demo reel. What's the most important thing to do before evaluating their proposal?
The lesson makes this point explicitly for high-stakes decisions: your own direct test on a task representative of your specific use case is often worth more than external reviews from people whose work differs from yours. The demo reel shows the best case; your own test shows your case.
For real decisions with real stakes, the lesson recommends your own direct test over documentation or others' reviews, because your specific use case matters more than general capability claims. The demo reel is systematically optimized to look good; your test shows actual performance in your context.
5. The lesson identifies a peer behavior pattern worth noting. Which is it?
The lesson names this directly: there's social cost in being the "wait and see" person when peers are enthusiastic adopters. That social pressure creates systematic bias toward early adoption and positive framing, which is worth being aware of in yourself and others.
The lesson specifically identifies premature enthusiasm — not excessive caution — as the dominant peer pattern. The social reward for appearing current and open to new tech creates pressure toward early adoption even when "wait and see" is the more rational position.

Lab 3: Live Triage Simulation

A new AI announcement just dropped. You have five minutes and a decision to make. Work through the triage protocol in real time.

Your Role: The Person Being Asked Right Now

Your lab partner is going to feed you a realistic AI announcement scenario and put you on the spot — exactly like a manager, client, or peer would in a real situation. Your job is to apply the five-minute triage protocol out loud, step by step. Your partner will push on your reasoning and surface gaps.

The goal isn't to give the "right" answer about whether the fictional tool is good — it's to demonstrate that your reasoning process is sound.

Tell your lab partner which domain you work in or are studying (or pick one: content creation, software development, healthcare, finance, education, or any other). They'll then give you a fictional AI announcement relevant to that domain and ask for your triage assessment.
Lab Partner
Triage Simulation
Alright, I'm going to put you through a live triage scenario. First tell me: what domain are you in or studying — content creation, software development, healthcare, finance, education, or something else? I'll build the scenario around what's actually relevant to you, and then you'll walk me through your five-step triage out loud.
Module 8 · Lesson 4

Your Long-Term AI Compass

The endgame isn't staying current — it's developing the kind of judgment that survives a technology you can't fully predict. Here's what that actually looks like built into a life.
What does it mean to be well-calibrated about AI not just this year, but over a career?

Think about what the AI landscape looked like in 2019. GPT-2 had just been released and OpenAI called it "too dangerous to release fully" — a framing that, in retrospect, was either genuinely cautious or a masterclass in PR, depending on who you ask. Most people using computers professionally had never interacted with an AI system in any meaningful way. The idea of a college student using AI to help draft an essay was a hypothetical, not a weekly occurrence.

Now think about someone who was 17 in 2019 — entering college just as the pandemic hit, graduating into a world where AI was already reshaping hiring, creative work, and how knowledge work gets done. They didn't choose to enter an AI-inflected world. The world just became that while they were trying to figure out their major. The question for that person — and for everyone reading this — isn't whether AI will continue changing things. It will. The question is whether your relationship to that change is active or passive, calibrated or reactive.

What Actually Compounds Over a Career

We've talked a lot in this module about staying current. But there's a different and more important question: what do you want to have built, in terms of knowledge and judgment, ten years from now? Because what you build over the next decade isn't just a set of current skills — it's a way of thinking that will shape how you navigate whatever comes after the current moment.

Here's what compounds over time and stays useful regardless of what specific AI technologies exist:

A reliable mental model of incentive structures. Understanding who benefits from AI hype, who has incentives to understate capabilities, and how those incentives shape public discourse — this knowledge doesn't expire. The specific actors change, but the incentive map stays structurally similar.

A track record of calibrated predictions. Every time you make an explicit prediction about an AI development and then check whether you were right, you're building a personal calibration database. Over years, this becomes a genuine signal about your own cognitive biases and blind spots. It's uncomfortable in the short term and extremely valuable in the long term.

Comfort with deep uncertainty. AI development involves real epistemic uncertainty — we don't know how capable these systems will become, we don't fully understand how they work, and experts disagree profoundly about near-future trajectories. People who can hold genuine uncertainty without either collapsing into nihilism or papering it over with false confidence are more useful in this domain than people who can't.

Building a Personal AI Position

A "personal AI position" is a set of considered views about how AI is likely to develop and what that means for your specific life — your career field, your creative work, your financial decisions. This is different from following the news. It's an active synthesis that you update deliberately as new evidence comes in.

A personal position has a few components. It includes your best current assessment of AI capabilities in your specific domain — not in general, but in the specific applications that are relevant to you. It includes your view on the timeline for those capabilities to mature or change. And it includes explicit acknowledgment of what you're most uncertain about and what evidence would change your view.

The reason to build this explicitly rather than leaving it as a vague intuition is that explicit positions are updatable. When you've written down "I think AI writing assistance will be table stakes for marketing roles within two years," you can check that belief against reality, notice when evidence undermines it, and update it. Vague intuitions don't update cleanly — they shift imperceptibly and often in ways you don't notice.

Your peers who are doing this well — and some are — tend to be the ones who are most useful to talk to about AI. They don't just share hot takes; they share reasoning. They know what would change their mind. Those qualities are rarer than they should be and significantly more valuable professionally than simply being enthusiastic about new tools.

What a Personal AI Position Looks Like

"In my field (UX design), I think AI tools will significantly change the speed at which wireframes and user research synthesis get done within 18 months, but won't substantially change the value of user interview skills and cross-functional communication for at least 3–4 years. The evidence that would change my view: widespread deployment of reliable automated user research synthesis that produces outputs better than a skilled junior researcher." That's a position. It's specific, it's falsifiable, and it's updatable.

The Practice of Deliberately Updating Your Beliefs

There's a specific cognitive habit that separates people who get better at evaluating AI over time from people who just accumulate more opinions: the practice of explicitly updating beliefs when evidence warrants it, and being willing to say out loud "I was wrong about this."

This is harder than it sounds. The social dynamics around AI are intense — there are strong tribal affiliations between AI skeptics and AI boosters, and both sides treat changing your position as evidence of weakness rather than evidence of learning. This is exactly backwards. In a genuinely uncertain and fast-moving domain, the people who update most readily when evidence changes are the ones building the most accurate picture over time.

One concrete practice: keep a short log of your AI-related beliefs and predictions. Once a month, review the log and note where reality has diverged from what you expected. Don't be harsh on yourself about being wrong — being wrong in interesting ways is how you learn the actual patterns. Be curious about why you were wrong. What did you misread about the incentives? What did you not know about how the technology actually works? These post-mortems are more useful than any amount of general AI reading.

Practical Takeaway

Right now, write down three specific predictions about AI development relevant to your life in the next 12–18 months. Include what evidence would confirm or disconfirm each prediction. Calendar a reminder to check these in six months. This single practice, maintained consistently, will do more for your calibration over time than almost anything else in this module.

What You're Actually Building

Everything in this module — the information half-life concept, the source stack, the triage protocol, the personal position — is in service of a single thing: developing the kind of relationship with AI that puts you in a position of active judgment rather than passive reaction.

You're going to spend your career in an environment shaped significantly by AI. That's not a prediction anymore — it's already the case. The question is whether you navigate that environment with a reliable compass or just respond to whatever the loudest signal happens to be at any given moment.

The people who will do best in this environment aren't necessarily the ones who know the most about AI right now. They're the ones who have built the habits that keep their knowledge accurate and their judgment well-calibrated over years. That's what this module has been about. Not a snapshot of the current state — a practice for staying clear-eyed in a domain that will keep moving.

You've put in the work. Now the real project is maintaining it.

Personal AI position
A set of explicit, considered beliefs about how AI development is likely to affect your specific field and life — specific enough to be falsifiable and designed to be actively updated as evidence accumulates.
Belief updating
The practice of explicitly revising your views when new evidence warrants it, tracking where your previous expectations were wrong, and using those misses to improve future predictions.

Lesson 4 Quiz

Your Long-Term AI Compass · 5 questions
1. According to the lesson, what is the most durable type of knowledge to build about AI over a career?
The lesson identifies incentive-structure knowledge and a track record of calibrated predictions as the most durable assets — because they don't depend on any specific technology remaining current. Technical architecture knowledge and tool familiarity both decay; the ability to reason about incentives and update beliefs based on evidence doesn't.
The lesson argues that tool experience and technical specs are among the fastest-decaying AI knowledge. What compounds over a career is understanding incentive structures (which are structurally stable) and developing a personal calibration record (which improves with deliberate practice).
2. What distinguishes a "personal AI position" from just following AI news closely?
The lesson emphasizes that explicit, written-down positions are updatable in ways that vague intuitions aren't. The key elements are specificity, domain relevance, and falsifiability — knowing in advance what evidence would change your view. That last piece is what makes it a genuine belief rather than a preference.
The lesson's definition of a personal AI position centers on explicitness and falsifiability, not volume of reading or speed of opinion formation. The key differentiator is that you've specified in advance what would change your view — which is what makes updating possible and meaningful when evidence comes in.
3. The lesson describes the social dynamics around changing your AI-related views as problematic. What specific pattern does it identify?
The lesson is precise about this: both tribes — skeptics and boosters — treat position-changing as capitulation rather than calibration. This is exactly backwards in a fast-moving, uncertain domain. People who update readily when evidence changes are building more accurate models, not demonstrating weakness.
The lesson specifically identifies that both AI skeptics AND boosters exhibit this pattern — it's not one-sided. Both treat changing positions as a form of tribal betrayal rather than evidence of good thinking. In a genuinely uncertain domain, this dynamic actively penalizes the people doing the best epistemic work.
4. What does the lesson recommend as a concrete practice for improving your calibration over time?
The lesson recommends this specific practice: write down predictions, review them monthly, and be curious (not harsh) about where you were wrong. The value is in the post-mortem — figuring out what you misread about incentives or technology. That process produces real calibration improvement over time.
The lesson prescribes a specific personal practice: maintain a prediction log and review it monthly to find where reality diverged from expectations. The post-mortem process — understanding why you were wrong, not just that you were — is where the actual learning happens.
5. The lesson ends by arguing that the people who navigate AI best over a career are not necessarily those with the most current AI knowledge. What quality does it say matters more?
This is the module's closing argument. Current knowledge is a snapshot; the ability to maintain accurate and well-calibrated knowledge over time as the field moves is the durable advantage. Habits — triage protocols, source stacks, prediction logs, belief updating — are what produce that durable ability.
The lesson explicitly closes with this argument: the relevant advantage isn't the snapshot of current knowledge but the practice that keeps knowledge accurate over time. Habits beat credentials and networks because habits compound in a way that one-time investments don't.

Lab 4: Building Your Personal AI Position

You'll draft and stress-test a real personal AI position statement for your specific field — explicit, falsifiable, and actually yours.

Your Role: Position Builder

Your lab partner is going to help you draft a real personal AI position — not a vague feeling, but a specific set of beliefs about how AI will affect your field over the next 18 months, with explicit statements about what evidence would change your mind. They'll push you to be specific and will surface any claims that are too vague to be updatable.

After you've drafted your position, they'll put it under pressure — playing devil's advocate, offering counterevidence, and testing whether your reasoning holds up.

Start by telling your lab partner: what field or career path are you in or headed toward? Then offer a first draft of one or two specific beliefs you hold about how AI will change that field in the next 18 months. Include, for each belief, one thing that would change your mind if you saw it.
Lab Partner
Position Building
Let's build something real here. I want you to give me an actual position — not "AI will change a lot" but something like "AI writing tools will make entry-level copywriting positions scarce within 18 months, and the evidence that would change my mind is if I see sustained hiring in that category despite widespread tool adoption." Tell me your field and your first draft of a position. I'll be honest if it's too vague to be useful, and I'll push back on the reasoning if I think it's shaky.

Module 8 Test

Building Your Ongoing AI Radar · 15 questions · Pass at 80%
1. What does the concept of "information half-life" mean in the context of AI knowledge?
Correct. Information half-life describes how quickly specific AI knowledge loses accuracy. The lesson distinguishes between fast-decaying knowledge (rankings, pricing) and slow-decaying knowledge (evaluation frameworks, incentive structures).
Information half-life refers to how quickly specific claims become inaccurate — not corporate or product lifecycles. The key insight is that different types of AI knowledge decay at very different rates.
2. Which of the following has the shortest information half-life?
Benchmark rankings change constantly as new models are released — sometimes within weeks. Conceptual frameworks, incentive analysis, and underlying dynamics all have much longer useful lifespans.
The lesson establishes that specific rankings and tool comparisons are among the fastest-decaying AI knowledge. Frameworks for evaluation and structural dynamics like incentive mapping are among the slowest.
3. Someone says they stay informed about AI by scrolling Twitter for thirty minutes every morning. What is the most precise critique of this approach?
The problem is structural and directional, not just noisy. Engagement rewards extreme takes — catastrophe or utopia, not calibrated middle ground. The resulting diet is systematically skewed, not randomly inaccurate, which is actually harder to correct for.
The critique isn't about time investment or technical depth — it's about systematic directional bias. Social media's engagement incentives push toward extreme claims, which means the resulting picture is predictably distorted in a specific direction.
4. A venture capital firm publishes an annual AI report predicting rapid capability growth. A university AI safety lab publishes their own report predicting slower, more uncertain progress. How should you use both?
Both sources have value and both have biases. VCs are incentivized toward optimism; academic safety labs may have biases toward caution and toward their own research areas. The right move is reading both with awareness of their biases and seeking independent evidence on the specific claims that matter most to you.
Averaging biased sources doesn't cancel out bias — it just produces a different estimate without examining the underlying evidence. And "find a neutral source" is impossible; all sources have some perspective. The right approach is reading with bias-awareness and checking specific claims independently.
5. What is the "access journalism" problem in AI coverage?
The lesson is careful to describe this as structural, not corrupt. Journalists don't make explicit deals. But the incentive structure — losing access if you write something unflattering — creates systematic pressure toward positive framing that operates even without conscious awareness.
The lesson explicitly distinguishes access journalism from explicit agreements or corruption. It's a structural pressure: writing unflattering coverage risks losing future access, so favorable framing becomes the path of least resistance even without any overt arrangement.
6. In the five-step triage protocol, what is Step 4?
Step 4 is relevance filtering — even accurate claims may not affect your specific work. The lesson notes this is a legitimate part of triage, not avoidance. A real capability that won't affect your workflow for a year doesn't warrant the same response as one that changes your work this week.
The five steps are: (1) source and incentive, (2) specific capability claim, (3) independent testing reports, (4) relevance to your actual work, (5) set a review date. Step 4 specifically addresses whether a genuine development actually matters to you specifically.
7. According to the hype cycle pattern, when does the gap between AI demo performance and deployment performance typically become visible?
Weeks 1–2 is when real usage begins and failure cases start emerging. Day 1–3 is too early (still announcement noise); weeks 3–6 is when the consolidated picture appears. But the initial gap between demo and deployment starts becoming visible in the first two weeks of actual use.
The lesson specifies that demo vs. deployment gaps start becoming visible in weeks 1–2 as unaffiliated developers and power users report real results. Day 1–3 is still announcement phase. The full picture consolidates at weeks 3–6, but the gap first appears earlier.
8. What makes a "personal AI position" different from a vague intuition, according to the lesson?
The key distinction is falsifiability — knowing in advance what would change your view. Vague intuitions shift imperceptibly; explicit positions with specified update conditions can be genuinely revised when evidence warrants it. That's what makes them useful over time.
The lesson's definition hinges on explicitness and falsifiability, not research depth or public sharing. An explicit position that specifies update conditions can be revised cleanly when evidence comes in; a vague intuition drifts without you noticing.
9. You had predicted in January that AI code assistants would reduce junior developer hiring by 30% by year's end. By December, junior developer job postings have actually increased 15%. What is the most productive response?
The lesson explicitly recommends curiosity rather than harsh self-judgment about wrong predictions. The productive question is: what did I misread about the incentives, technology, or market dynamics? That post-mortem is more valuable than the prediction itself.
Dismissing contradicting evidence, over-generalizing from one miss, and abandoning prediction-making are all worse responses than the targeted post-mortem. The lesson recommends being curious about specifically what your reasoning missed — not punishing yourself or giving up on the practice.
10. What does "coverage diversity" mean in the context of building an AI source stack?
Coverage diversity means your source stack includes different source types — investor, journalist, academic, practitioner — so that their different incentive structures create a more complete and balanced picture than any single type could provide.
The lesson's definition of coverage diversity is specifically about source type diversity — not format, geography, or topic angle. The goal is to prevent monoculture where all your sources share the same incentives and therefore the same systematic biases.
11. A new AI model is released with impressive benchmark scores. Your manager wants to know if it's better for your team's specific use case. According to the lesson, what's the most reliable way to find out?
The lesson makes this point explicitly for high-stakes decisions: your personal test on tasks representative of your actual use case is often more informative than any external review, because your specific situation may differ significantly from reviewers' situations.
For domain-specific questions about whether a tool fits your specific workflow, the lesson recommends personal testing over documentation or others' reviews. Your use case may be meaningfully different from anyone else's, and only a direct test reveals performance in your context.
12. Which of these is the best example of a reliable AI source signal, as described in the lessons?
Track record of accurate specific predictions is the most direct signal of reliability the lessons identify. Reach, access, and insider status are all proxies at best. An accurate past prediction that can be checked against subsequent reality is direct evidence of calibrated judgment.
The lessons consistently return to track record as the gold standard. Subscriber counts measure reach. Former executive status means industry knowledge but also potential bias. Early access implies access journalism dynamics. Only verified accuracy of past claims is direct evidence of reliable judgment.
13. The lesson argues that "comfort with deep uncertainty" is a valuable skill in the AI domain. What does this specifically mean?
The lesson describes this as holding uncertainty without collapsing into nihilism (giving up on having views) or papering it over with false confidence. Both of those are failure modes. Comfort with uncertainty means staying engaged and forming calibrated beliefs even when the honest answer includes significant uncertainty.
The lesson's definition is about epistemic stance, not technical knowledge or disengagement. The failure modes are nihilism (it's all too uncertain, I have no views) and false confidence (I'll project certainty I don't have). The goal is genuine engagement with real uncertainty acknowledged.
14. You've been following an AI researcher on Twitter whose posts are consistently more accurate than other sources you follow. They have relatively few followers compared to larger AI influencers. What does this situation tell you?
The lessons consistently separate reach from reliability. A smaller but demonstrably accurate source is more valuable for calibration than a large source with poor accuracy. This is the whole point of building a curated stack rather than just following popular sources.
The lessons make this point clearly: engagement and accuracy are independent variables. Social media rewards reach-generating content, which may have little correlation with accuracy. A small but demonstrably reliable source is a genuine asset in your source stack regardless of follower count.
15. Which habit, if maintained consistently over several years, does the lesson identify as most likely to compound into genuinely valuable AI judgment?
The prediction log practice — making explicit predictions, reviewing them, and doing post-mortems on errors — is the habit the lesson most clearly identifies as compounding over time. It builds a personal calibration database and surfaces your specific cognitive biases in ways that general reading cannot.
All of those practices have value, but the lesson specifically identifies the prediction-and-review cycle as the highest-compounding habit. It's the only practice that directly measures and improves your calibration accuracy over time, rather than just increasing your exposure to AI information or people.