L1
Β·
Quiz
Β·
Lab
L2
Β·
Quiz
Β·
Lab
L3
Β·
Quiz
Β·
Lab
L4
Β·
Quiz
Β·
Lab
Module Test
AI for Small Business Managers Β· Module 7 Β· Lesson 1

Why People Resist AI β€” and What Actually Works

Resistance is rational. The managers who understand this win the adoption battle before it begins.

When Amazon began rolling out its AI-powered picking robots at fulfillment centers in 2018, warehouse employees staged coordinated slowdowns. Not because workers lacked intelligence β€” because they lacked information. Management had not explained what the robots would replace, what new roles would emerge, or how performance metrics would change. The resistance was entirely predictable: uncertainty about job security plus zero communication equals fear, and fear produces friction. Amazon subsequently rolled out a retraining program called Career Choice, subsidizing education for affected employees. Once workers saw a credible path forward, adoption friction dropped substantially.

The lesson generalizes beyond warehouses. At every scale β€” from a 12-person marketing agency adopting an AI copywriting tool to a regional accounting firm deploying AI audit software β€” the psychological pattern is the same. People resist what they cannot predict.

The Three Roots of AI Resistance

Organizational psychologists studying workplace technology adoption since the 1990s have identified three durable drivers of resistance, all present in AI adoption today.

1. Job threat perception. A 2023 Pew Research survey found that 19% of U.S. workers say their job is highly exposed to AI. Whether or not that estimate is accurate for any particular employee, the perception of threat is real and acts the same way whether or not it is warranted. Managers who dismiss this as irrational lose trust instantly.

2. Competence anxiety. Adults fear looking incompetent in front of peers. Asking a 55-year-old operations manager to learn a new AI scheduling tool in a group training session carries a social cost that younger employees often underestimate. A 2022 McKinsey survey on reskilling found that employees over 45 reported significantly higher anxiety about demonstrating early-stage incompetence.

3. Loss of craft identity. Skilled workers often define themselves through their work. A copywriter who has spent a decade developing her voice may feel that an AI writing assistant renders that decade meaningless. This is not laziness β€” it is identity. Designers at agencies that adopted Adobe Firefly in 2023 reported this exact tension in documented case studies published by Adobe's own research team.

The Communication Gap Is the Real Problem

Research by Prosci, the change-management consultancy that developed the ADKAR model (Awareness, Desire, Knowledge, Ability, Reinforcement), has tracked hundreds of technology rollouts. Their finding: the single biggest predictor of adoption failure is insufficient communication about the "why." Not poor training. Not budget. Communication.

For AI specifically, employees need answers to five questions before they will genuinely engage: Why are we doing this? What will change about my job? What won't change? What happens if I struggle to learn it? What happens if I resist? Managers who answer these questions proactively β€” before employees ask β€” reduce resistance by 40% according to Prosci's 2022 benchmarking report.

The practical implication for small business managers: you do not need an HR department or a change management budget. You need a 20-minute team meeting, held before the software is deployed, where you answer those five questions honestly. Honesty about uncertainty ("I don't know yet if this will affect headcount, and I'll tell you as soon as I do") consistently outperforms false reassurance.

Early Adopters vs. Skeptics: The Smart Sequencing Strategy

Everett Rogers' Diffusion of Innovations framework (first published 1962, updated through 2003) identifies five adopter categories: innovators, early adopters, early majority, late majority, and laggards. In a team of 10, you typically have 1–2 innovators, 2–3 early adopters, and the rest distributed across the majority and laggard categories.

The strategic move: deploy the tool with your early adopters first. Let them build fluency, generate visible wins, and become internal champions. Their peers trust them more than they trust management β€” peer credibility is the most efficient adoption accelerant available to a small business manager. HubSpot documented this exact pattern when it rolled out its AI content assistant to marketing teams in 2023: teams that had an internal champion reported 2.4Γ— higher 90-day adoption rates than teams without one.

Manager Action

Before your next AI tool deployment: identify your one or two most curious team members. Give them early access β€” two weeks before the full team. Ask them to document three things the tool does well and two things it does badly. Then have them present their findings at the full-team launch. This single move reduces skepticism more reliably than any vendor-produced training video.

Key Terms
ADKARProsci's change management model: Awareness, Desire, Knowledge, Ability, Reinforcement. A practical checklist for any technology rollout.
Diffusion of InnovationsRogers' model categorizing how people adopt new technologies over time. Useful for sequencing AI rollouts by matching deployment order to adopter type.
Competence AnxietyThe fear of appearing incompetent while learning a new skill, especially in social or professional settings. A primary driver of AI resistance in adult workers.

Lesson 1 Quiz

3 questions β€” free, untracked, retake anytime.
According to Prosci's change management research, what is the single biggest predictor of AI adoption failure?
βœ“ Correct. Prosci's 2022 benchmarking report identified communication about purpose as the top predictor β€” even above training quality or budget.
βœ— Prosci's research consistently points to communication failure β€” specifically about the "why" β€” as the dominant predictor of adoption failure, outranking budget and training issues.
What does Amazon's Career Choice program illustrate about managing AI-driven workforce change?
βœ“ Exactly. Once Amazon offered funded retraining through Career Choice, workers had a credible alternative path, and resistance decreased β€” demonstrating that certainty about the future matters more than the nature of the change itself.
βœ— The Career Choice program's lesson is about credibility and future-path clarity: when employees saw a real retraining option, their resistance dropped. The presence of an alternative path is the key variable.
In Rogers' Diffusion of Innovations model, why should small business managers deploy AI tools to early adopters first?
βœ“ Right. Peer-to-peer influence is the most efficient adoption accelerant. HubSpot's 2023 internal data showed 2.4Γ— higher 90-day adoption rates in teams with an internal champion, confirming Rogers' foundational model.
βœ— The strategic reason is peer credibility. Employees trust colleagues who have used the tool over management proclamations. Sequencing deployment through early adopters converts this social trust into adoption momentum.

Lab 1: Diagnosing Resistance on Your Team

Use the AI assistant to analyze resistance patterns and design your pre-launch communication plan.

Your Mission

You're preparing to introduce a new AI tool to your team. Use this AI assistant to identify which resistance types are most likely present, map your team members to Rogers' adopter categories, and draft the five-question communication framework from the lesson.

Be specific about your actual team, industry, and the tool you're considering. The more context you give, the more actionable the guidance.

Try asking: "I manage a 6-person bookkeeping team at a small CPA firm. I'm about to introduce an AI tool that automates transaction categorization. Two of my staff are in their 50s and have been here 15+ years. What resistance types should I prepare for, and how do I frame the five-question communication?"
AI Lab Assistant Resistance & Communication
AI for Small Business Managers Β· Module 7 Β· Lesson 2

Building Psychological Safety for AI Experimentation

Teams that feel safe to fail with AI learn ten times faster than teams where failure carries a cost.

Google's internal People Analytics team spent four years studying 180 teams to discover what made some dramatically more effective than others. The answer β€” published in 2016 and since replicated dozens of times β€” was psychological safety: the shared belief that the team is safe for interpersonal risk-taking. Teams with high psychological safety outperformed peers on every metric Google tracked, including the speed of adopting new internal tools. The research, led by Amy Edmondson of Harvard Business School (whose own foundational work predates Google's study), has since been applied directly to AI adoption contexts by consultancies including Deloitte and Accenture.

The connection to AI is direct: experimenting with AI tools is inherently embarrassing at first. Prompts fail. Outputs are wrong. Workflows break. If your team believes that struggling publicly carries professional risk, they will use the tool minimally and performatively β€” hitting compliance checkboxes while doing actual work the old way.

What Psychological Safety Actually Means (and Doesn't)

Edmondson's definition is precise: psychological safety is not about being comfortable or nice to each other. It is specifically the belief that you will not be punished or humiliated for speaking up, making mistakes, or asking questions. In AI adoption contexts, this translates to three specific behaviors your team needs to feel safe doing:

Saying "the AI got this wrong." Teams where employees fear contradicting AI outputs β€” because disagreement might seem obstructionist β€” make worse decisions than teams where challenging the tool is normalized. A 2023 MIT Sloan Management Review article on AI-assisted decision-making found that teams explicitly encouraged to critique AI outputs caught significantly more errors than those not given that permission.

Admitting they don't know how to use it. The average AI tool has a learning curve of 4–8 weeks before a knowledge worker uses it fluently. If team members feel they should already know how to use it, they fake competence and miss the learning. A Gartner survey from late 2023 found that 34% of employees reported using AI tools less than they could because they were embarrassed to ask for help.

Reporting workflows that broke. AI tools disrupt existing processes. Employees who encounter a broken workflow and don't report it because they fear blame create compounding problems. The sooner managers hear about broken processes, the sooner they can fix them.

Three Concrete Moves That Build Safety

Model failure publicly. As manager, share your own AI failures in team settings. "I tried to get ChatGPT to draft our Q3 supplier email and the first two attempts were unusable β€” here's what finally worked." This single behavior, practiced consistently, gives employees permission to be imperfect. Amy Edmondson calls this "setting the stage" β€” leaders who demonstrate vulnerability about their own learning reduce team anxiety measurably.

Create a designated failure channel. Several small businesses documented in the Harvard Business Review's 2023 coverage of SMB AI adoption created a Slack channel called #ai-experiments where employees posted what didn't work and what they tried. The transparency normalized struggle and generated collective learning faster than any formal training session.

Separate performance review from AI adoption metrics. If employees believe their use of AI tools will appear in their performance review before they have had time to build competence, they will game the metrics rather than genuinely learn. Decouple adoption reporting from evaluation for the first 90 days of any new tool deployment.

The Manager's Own Safety Challenge

Small business managers face a particular version of this problem: they often need to appear authoritative about AI while knowing less about specific tools than some of their younger staff. The managers who navigate this best acknowledge the knowledge gap directly. A study published in the Journal of Applied Psychology in 2021 found that leaders who explicitly acknowledged competence gaps while expressing commitment to learning were rated significantly higher in trustworthiness than leaders who projected false confidence.

The practical script: "I don't know this tool as well as some of you do, and I'm counting on us to figure it out together. Here's what I do know about why we're doing this and what success looks like." This framing preserves your strategic authority while inviting genuine participation.

Research Finding

Google's Project Aristotle identified psychological safety as the #1 differentiator of high-performing teams β€” above individual talent, compensation, or management quality. In AI adoption contexts, this finding holds: teams with established psychological safety adopt new AI tools 3Γ— faster and report 2Γ— higher satisfaction with outcomes in Accenture's 2023 Future of Work benchmarking study.

Key Terms
Psychological SafetyAmy Edmondson's term: the shared belief that a team is safe for interpersonal risk-taking, including admitting errors and asking questions without fear of punishment.
Performative ComplianceUsing a tool minimally and visibly to satisfy requirements while doing actual work through previous methods. The primary symptom of unsafe AI adoption environments.
Stage SettingEdmondson's term for leadership behaviors that create psychological safety β€” particularly leaders modeling their own vulnerability and imperfection.

Lesson 2 Quiz

3 questions β€” free, untracked, retake anytime.
Amy Edmondson's definition of psychological safety specifically means that team members believe they won't be punished for:
βœ“ Correct. Edmondson's precise definition is about interpersonal risk-taking β€” speaking up, admitting errors, asking questions β€” without fear of humiliation or punishment. It is not a general comfort metric.
βœ— Edmondson's definition is specifically about interpersonal risk-taking: speaking up, admitting mistakes, and asking questions. It's a precise concept, not a general measure of comfort or agreement culture.
What is "performative compliance" in the context of AI adoption?
βœ“ Right. Performative compliance is the failure mode produced by unsafe adoption environments β€” employees appear to use the tool while actually bypassing it, giving management false adoption signals.
βœ— Performative compliance means appearing to use the AI tool while actually bypassing it β€” hitting checkboxes while doing real work through old methods. It's the primary symptom of an unsafe adoption environment.
Why should small business managers decouple AI adoption metrics from performance reviews for the first 90 days?
βœ“ Exactly. When adoption metrics are tied to evaluation during the early learning phase, employees optimize for looking good on metrics rather than actually building fluency β€” undermining the entire adoption effort.
βœ— The reason is behavioral: linking evaluation to adoption before competence is built creates incentives to game metrics rather than learn genuinely. Remove that incentive for 90 days to allow real learning to occur.

Lab 2: Designing a Psychologically Safe AI Environment

Build the specific structures, scripts, and norms your team needs to experiment freely.

Your Mission

Work with the AI assistant to design concrete psychological safety structures for your AI rollout. This includes drafting a "model failure" script for your first team meeting, designing your #ai-experiments communication channel norms, and creating a 90-day policy that decouples adoption from evaluation.

Describe your team size, industry, and current team culture (is it already relatively open, or more reserved/hierarchical?). The assistant will tailor its recommendations to your specific context.

Try asking: "My team of 8 retail operations staff is fairly reserved β€” people don't speak up much in meetings. We're rolling out an AI inventory forecasting tool next month. Help me design a psychological safety plan: what do I say at the kickoff meeting, what channel norms should I set, and how do I rewrite our 90-day check-in process?"
AI Lab Assistant Psychological Safety Design
AI for Small Business Managers Β· Module 7 Β· Lesson 3

Training That Sticks: Practical AI Upskilling for Small Teams

Most AI training fails because it teaches features. Effective training teaches judgment β€” when to trust the tool and when to override it.

In early 2024, Swedish fintech company Klarna announced that its AI assistant β€” built on OpenAI's technology β€” was handling the work of approximately 700 customer service agents, resolving 2.3 million conversations in its first month. The headline obscured the more important story: Klarna's remaining customer service employees were simultaneously being trained not just to use the AI but to manage its escalations β€” the cases the AI couldn't handle or got wrong. These employees were trained in a new skill: recognizing AI failure patterns and recovering them gracefully. This capability β€” knowing when the AI is wrong β€” proved more valuable than the ability to use the AI correctly.

The lesson for small business managers is direct: your training program needs to teach two things simultaneously. Feature literacy β€” how to operate the tool β€” and critical calibration β€” how to recognize when its outputs are unreliable and what to do about it.

Why Standard Vendor Training Fails

Most AI software vendors provide training that is feature-centric: here is the dashboard, here is how to run a query, here is how to export a report. This training succeeds at basic orientation but fails at the most important competency: contextual judgment about when to use the tool, when to adjust it, and when to ignore it.

A 2023 MIT study on AI-assisted decision-making found that workers who received only feature training were more likely to over-rely on AI outputs than workers who received no training at all. The reason: feature training creates false confidence. Workers who understand the mechanics assume the outputs are reliable, while untrained workers remain appropriately skeptical.

For small businesses, vendor training is a starting point, not a program. You need to supplement it with three types of in-house training exercises.

Three Training Formats That Work for Small Teams

1. Error Hunting Sessions (30 minutes, weekly for first month). Give team members a set of real AI outputs from their work and ask them to find errors. This trains critical calibration β€” the ability to spot when the AI has misunderstood context, made a factual error, or produced output that looks plausible but is wrong. Shopify's merchant success team used this format in 2023 when rolling out AI product description tools; they reported that teams doing weekly error hunts caught 68% more consequential errors than teams who did not.

2. Prompt Engineering Workshops (60 minutes, once). Give employees structured time to experiment with how their phrasing affects AI output quality. A poorly written prompt produces bad output; a well-structured prompt produces usable output. Teams that understand this relationship treat the AI as a collaborative tool rather than an oracle. The key insight employees need: garbage in, garbage out applies to AI prompts exactly as it does to spreadsheet formulas.

3. Real-Work Integration Sprints (two weeks). Rather than training in isolation, assign employees specific work tasks to accomplish using the AI tool, with a daily 10-minute check-in where they share what worked and what didn't. This mirrors how adults actually learn new technology β€” through use, with social support. Research on workplace learning by the Association for Talent Development consistently shows that application-plus-reflection produces retention rates 3–5Γ— higher than classroom instruction alone.

Calibrating Trust: The Most Important Skill

The concept of appropriate reliance β€” trusting AI when it is reliable, overriding it when it isn't β€” is the central competency AI researchers want workers to develop. A 2022 paper by Gagan Bansal and colleagues at the University of Washington found that the workers who performed best on AI-assisted tasks were not those who trusted AI most or least, but those whose trust tracked AI accuracy β€” high trust when the AI was right, low trust when it was wrong.

Teaching this skill requires giving employees exposure to AI failure modes specific to their work. A customer service team needs to know that AI tends to confidently produce wrong policy answers when company policy has changed recently. A bookkeeping team needs to know that AI categorization tools sometimes misclassify transactions with ambiguous descriptions. An HR team using AI resume screening needs to know that AI screening tools can perpetuate historical hiring biases embedded in past data.

Practical Training Template

Week 1: Vendor onboarding (feature literacy). Week 2–4: Weekly 30-min error hunting sessions using real work outputs. Week 3: 60-min prompt engineering workshop. Weeks 2–5: Daily 10-min integration sprint check-ins. Week 6: Team retrospective β€” what we trust, what we verify, what we override. Document the results as a team AI usage guide.

Key Terms
Critical CalibrationThe trained ability to recognize when AI outputs are likely to be unreliable and to adjust trust accordingly. Distinct from feature literacy (how to operate the tool).
Appropriate RelianceThe research concept describing workers whose trust in AI tracks actual AI accuracy β€” trusting outputs when they are reliable, overriding them when they are not.
Integration SprintA structured period where employees accomplish real work tasks using a new AI tool, combined with regular reflection check-ins. The most effective format for adult AI skill-building.

Lesson 3 Quiz

3 questions β€” free, untracked, retake anytime.
A 2023 MIT study found that workers who received only feature-based AI training were more likely to do what compared to untrained workers?
βœ“ Correct. Feature training creates false confidence β€” workers assume that because they understand how the tool works, its outputs are reliable. Untrained workers remain appropriately skeptical, which in some contexts produces better outcomes.
βœ— The counterintuitive finding: feature training produced over-reliance. Understanding how to operate the tool made workers assume its outputs were trustworthy, reducing their critical evaluation of those outputs.
What does "appropriate reliance" mean in AI research on worker performance?
βœ“ Right. Bansal et al.'s 2022 University of Washington research found that workers who performed best weren't uniformly trusting or skeptical β€” they calibrated trust to actual accuracy, which requires learning each tool's specific failure patterns.
βœ— Appropriate reliance means trust that tracks accuracy: high when the AI is right, low when it's wrong. This requires knowing the tool's specific failure modes in your context, which is why error hunting sessions are so valuable.
Klarna's 2024 AI deployment revealed that the most valuable skill for remaining customer service employees was:
βœ“ Exactly. Klarna's case illustrates the principle: as AI handles routine cases, human workers' value concentrates in edge cases and failure recovery. Training for that skill β€” recognizing when AI is wrong β€” becomes the priority.
βœ— The Klarna case showed that recognizing and recovering AI failure patterns was the critical competency. As the AI handled 2.3 million routine conversations, human expertise was most needed at the edges β€” where the AI failed.

Lab 3: Building Your Team's AI Training Program

Design a 6-week training plan that builds real skill, not just feature familiarity.

Your Mission

Use the AI assistant to build a customized 6-week training program for your team. Your plan should include: a vendor onboarding supplement, weekly error hunting sessions with specific AI failure modes to look for, a prompt engineering workshop agenda, and integration sprint check-in questions.

Tell the assistant what AI tool you're deploying and what work your team does. The more specific you are about your team's tasks, the more useful the failure modes and error hunting exercises will be.

Try asking: "We're a 5-person marketing team at a regional real estate agency deploying an AI tool to write property listing descriptions. Build me a 6-week training plan. Include: the 3 most likely AI failure modes for our specific use case, 5 error hunting exercise scenarios, a prompt engineering workshop agenda, and integration sprint check-in questions."
AI Lab Assistant Training Program Design
AI for Small Business Managers Β· Module 7 Β· Lesson 4

Measuring What Matters: AI Adoption Metrics That Drive Real Outcomes

The wrong metrics produce the right-looking numbers and the wrong results. Here's how to measure AI adoption honestly.

IBM's Watson Health division spent nearly a decade and billions of dollars deploying AI into hospital systems across the United States. By nearly every internal adoption metric β€” number of hospitals deployed, queries processed, reports generated β€” the program looked successful. By outcome metrics β€” patient care improvement, diagnostic accuracy, physician workflow efficiency β€” it was, by most assessments, a failure. MD Anderson Cancer Center terminated its Watson contract in 2017 after spending $62 million; Stat News documented that Watson's cancer treatment recommendations were sometimes unsafe and incorrect according to physicians' own notes obtained through a public records request.

IBM was measuring the wrong things. Activity metrics β€” usage, volume, queries β€” told a story of adoption. Outcome metrics told a different story entirely. The lesson scales directly to small business: if you measure only how much your team uses an AI tool, you will not know whether it is helping.

Activity Metrics vs. Outcome Metrics

Activity metrics measure whether people are using the tool: logins per week, queries submitted, reports generated, time spent in the application. These are easy to collect and look good in dashboards. They are also almost entirely useless for measuring actual business impact.

Outcome metrics measure whether the tool is helping: time saved on specific tasks, error rates in AI-assisted vs. non-assisted work, customer satisfaction scores before and after AI deployment, revenue per employee, cost per unit of output. These are harder to collect but are the only metrics that tell you whether the investment is working.

A 2023 Forrester Research survey of SMB technology deployments found that 61% of small businesses measured AI adoption using activity metrics only, with no outcome baseline. This creates a structural problem: you can't know if outcomes have improved if you didn't measure them before the tool was deployed.

The Baseline Problem β€” and How to Solve It

The most common measurement failure is deploying a tool without establishing baselines. If you don't know how long customer email responses took before the AI writing assistant, you cannot measure whether the tool saved time. The fix is simple but requires advance planning: measure before you deploy.

For small businesses, a two-week baseline measurement period before any AI deployment is sufficient for most metrics. Track: time spent on the specific task the AI will assist with, error or revision rates on that task, output volume per employee per day, and customer or client satisfaction related to that task. These four data points, collected for two weeks pre-deployment, give you a comparison baseline that makes post-deployment measurement meaningful.

The Stanford Social Innovation Review documented a case in 2022 where a 20-person nonprofit deployed an AI donor communication tool and measured a 34% reduction in time spent on donor outreach. This result was credible because they had tracked the same metric for six weeks before deployment. Without that baseline, they would have had only the vendor's claimed benchmark β€” which proved to be significantly overstated for their use case.

A Practical Metrics Framework for Small Business AI Deployment

Three tiers of metrics give a complete picture without requiring an analytics team.

Tier 1 β€” Adoption health (activity, but meaningful): Not just logins, but qualified use β€” instances where the AI output was actually used in final work product (not just generated and discarded). A customer service team can track: AI-drafted responses that were sent vs. AI-drafted responses that were replaced. This reveals whether the tool is producing usable output or just activity.

Tier 2 β€” Efficiency outcomes: Time per task (before and after), throughput (units completed per hour/day), and revision cycles (how many edits required per AI-generated output). These measure whether the tool is actually accelerating work.

Tier 3 β€” Quality outcomes: Error rates, customer satisfaction scores, compliance or accuracy rates (for teams doing regulated work), and manager assessment of output quality. These measure whether faster work is also good work β€” the most important question.

Review Tier 1 weekly, Tier 2 monthly, Tier 3 quarterly. Adjust training and tool configuration based on what you find. This cadence matches the timescales at which each metric type changes meaningfully.

The Honest Conversation

Sometimes measurement reveals the AI tool isn't helping. This is valuable information. A 2023 Harvard Business School case on SMB technology adoption documented a 15-person law firm that deployed an AI contract review tool, measured it honestly for 90 days, and found it saved time only for junior associates β€” not senior partners whose work was too specialized. They adjusted deployment accordingly, saving the tool for junior review workflows and abandoning it for senior ones. Honest measurement made the difference between a useful tool and a universal mandate.

Key Terms
Activity MetricsMeasures of tool use (logins, queries, reports generated). Easy to collect; insufficient to determine business impact. Common proxy for adoption that often masks actual outcome data.
Outcome MetricsMeasures of actual business results (time saved, error reduction, quality improvement, revenue impact). Require baselines to be meaningful. The only metrics that reveal whether AI adoption is working.
Qualified UseAI outputs that were actually incorporated into final work product β€” as opposed to generated but discarded. A more meaningful activity metric than raw usage counts.
Baseline PeriodThe measurement window before AI deployment during which pre-adoption performance data is collected, making post-deployment comparison meaningful.

Lesson 4 Quiz

3 questions β€” free, untracked, retake anytime.
What was the central measurement failure of IBM Watson Health's hospital deployments?
βœ“ Correct. Watson Health's activity numbers β€” queries processed, hospitals deployed, reports generated β€” looked strong. But outcome metrics, including physician assessments of recommendation quality documented by Stat News, revealed a very different picture.
βœ— The Watson failure illustrates exactly the activity vs. outcome problem: by activity metrics the program appeared successful; by outcome metrics β€” patient safety, diagnostic accuracy, physician workflow β€” it failed. MD Anderson's $62M terminated contract is the most documented example.
What is "qualified use" and why is it a better activity metric than raw login counts?
βœ“ Right. An employee can log in, generate 10 AI outputs, and discard all of them β€” logging counts as adoption while nothing has changed. Qualified use tracks actual incorporation into work product, revealing whether the tool is producing usable output.
βœ— Qualified use means AI-generated output that actually made it into the final work product. This distinguishes genuine adoption (the tool's output was useful) from performative activity (the tool was opened, output was generated, nothing was kept).
According to the Stanford Social Innovation Review case, what made the nonprofit's 34% time-savings result credible?
βœ“ Exactly. Without a pre-deployment baseline, they could only compare against vendor benchmarks, which proved to be overstated. Six weeks of pre-deployment data gave them a real baseline specific to their context.
βœ— The baseline is what made the result credible. Without pre-deployment measurement, their only reference would have been the vendor's benchmark, which Stat News-style investigations consistently show are overstated. Baseline data is the foundation of honest measurement.

Lab 4: Building Your AI Measurement Framework

Design the metrics, baselines, and review cadence that will tell you whether your AI investment is actually working.

Your Mission

Work with the AI assistant to build a complete measurement framework for your AI deployment. You'll define your specific outcome metrics, design your baseline measurement period, create your three-tier metrics dashboard, and set the review cadence for each tier.

Be specific about what your team does and what business results matter most. The assistant will help you translate those priorities into concrete, measurable indicators and a pre/post comparison structure.

Try asking: "I run a 7-person catering company and we're deploying an AI tool for menu planning, client proposals, and scheduling. Help me build a measurement framework: define my pre-deployment baseline metrics for all three use cases, create a three-tier metrics dashboard (activity health, efficiency outcomes, quality outcomes), and give me the review questions I should ask monthly."
AI Lab Assistant Measurement Framework Design

Module 7 Test

15 questions. Score 80% or higher to pass.
1. According to Prosci's ADKAR model, which element comes first in successful technology change?
βœ“ ADKAR stands for Awareness, Desire, Knowledge, Ability, Reinforcement β€” in that order. Awareness of why the change is happening must precede desire to participate.
βœ— ADKAR is sequential: Awareness β†’ Desire β†’ Knowledge β†’ Ability β†’ Reinforcement. Awareness comes first because you cannot create desire to change in someone who doesn't understand why change is happening.
2. A 55-year-old operations manager hesitates to ask questions about a new AI scheduling tool during a group training session. This is most likely an example of:
βœ“ Competence anxiety is the fear of appearing incompetent in front of peers during learning. It's especially pronounced in adult workers in group settings and is a primary driver of AI resistance.
βœ— This is competence anxiety β€” the fear of looking incompetent in a social/professional setting. It's distinct from job threat (fear of replacement) or craft identity (fear of professional irrelevance).
3. HubSpot's 2023 internal data on AI content assistant adoption found that teams with an internal champion had what adoption advantage over teams without one?
βœ“ HubSpot documented 2.4Γ— higher 90-day adoption rates in teams with internal champions, confirming Rogers' insight that peer credibility is the most efficient adoption accelerant.
βœ— HubSpot's figure was 2.4Γ—. This is a significant multiplier, illustrating why seeding teams with internal champions β€” identified and prepared before full rollout β€” is one of the highest-leverage adoption strategies available.
4. Google's Project Aristotle study (2012–2016) found that the #1 differentiator of high-performing teams was:
βœ“ Correct. Project Aristotle's headline finding was psychological safety as the top predictor β€” above talent, compensation, or management style. This finding has since been replicated in AI adoption contexts specifically.
βœ— Psychological safety was Google's #1 finding. The study looked at 180 teams over four years. Individual talent and other commonly cited factors ranked lower than the interpersonal risk-taking environment the team had collectively established.
5. The Gartner survey from late 2023 found that what percentage of employees reported using AI tools less than they could because they were embarrassed to ask for help?
βœ“ 34%. This figure illustrates the business cost of insufficient psychological safety β€” more than a third of employees are leaving potential productivity on the table because they are afraid to admit they need help learning the tool.
βœ— Gartner's figure was 34% β€” more than a third of employees. This is a direct measurement of the productivity gap created by low psychological safety in AI adoption environments.
6. Amy Edmondson's term "stage setting" refers to:
βœ“ Stage setting is Edmondson's term for what leaders do to create psychological safety β€” particularly by demonstrating their own fallibility and framing failure as learning rather than incompetence.
βœ— Stage setting is Edmondson's specific term for leader behaviors that create psychological safety β€” the most important of which is modeling vulnerability: publicly sharing your own failures, uncertainties, and learning process.
7. Shopify's merchant success team found that teams doing weekly error hunting sessions caught what percentage more consequential errors than teams that didn't?
βœ“ 68% more consequential errors caught by teams doing weekly error hunting. This is why critical calibration training β€” not just feature training β€” is essential to AI deployment safety.
βœ— Shopify's figure was 68%. Error hunting sessions train the specific skill of recognizing AI failure patterns β€” a capability that feature training alone does not develop, and that is increasingly the highest-value human competency in AI-assisted workflows.
8. The MIT study on AI-assisted decision-making found that workers who received only feature training were more likely than untrained workers to:
βœ“ Feature training creates false confidence. Workers who understand how the tool works assume its outputs are reliable, reducing critical evaluation. This is why training must address failure modes alongside features.
βœ— Over-reliance is the counterintuitive finding. Feature literacy without critical calibration training produces workers who trust AI outputs more than they should, because they understand the mechanism but not the failure modes.
9. What is the recommended review cadence for Tier 1 (adoption health), Tier 2 (efficiency outcomes), and Tier 3 (quality outcomes) metrics?
βœ“ Tier 1 (adoption health) weekly, Tier 2 (efficiency) monthly, Tier 3 (quality) quarterly. Each review cadence matches the timescale at which that metric type changes meaningfully.
βœ— The three-tier cadence is weekly/monthly/quarterly. Activity health can shift quickly (weekly makes sense); efficiency takes 4–6 weeks to stabilize (monthly); quality outcomes require 90 days to assess reliably (quarterly).
10. A 2023 Forrester Research survey found that what percentage of small businesses measured AI adoption using activity metrics only, with no outcome baseline?
βœ“ 61% β€” a clear majority of small businesses. This explains why so many SMB AI investments cannot demonstrate ROI: there is no baseline data to compare against, making outcome measurement impossible.
βœ— Forrester found 61%. Without a pre-deployment baseline, there is no way to distinguish AI-driven improvement from natural business variation, seasonal effects, or other factors. Baseline measurement before deployment is the foundational requirement.
11. Klarna's February 2024 announcement about its AI customer service assistant handling 2.3 million conversations primarily illustrated which lesson for small business managers?
βœ“ The Klarna case shows that AI volume shifts human work toward edge cases and failure recovery. Training for that specific skill β€” recognizing and recovering AI errors β€” becomes the priority when AI handles the routine.
βœ— The Klarna case illustrates how automation shifts the nature of human work, not its total volume. Human customer service roles shifted to handling what the AI couldn't β€” requiring training specifically in AI failure recognition, not general customer service skills.
12. Amazon's Career Choice program, launched in response to fulfillment center automation, reduced adoption friction primarily by:
βœ“ Career Choice provided funded education β€” a credible path forward. Once employees had an alternative, fear of a dead end decreased and resistance dropped. The key variable was not money but certainty about what the future could hold.
βœ— Career Choice worked by eliminating the dead-end feeling β€” providing funded retraining as an alternative path. Resistance is highest when change feels like a trap with no exit. Credible alternatives reduce that feeling.
13. In Rogers' Diffusion of Innovations model, which adopter category should receive a new AI tool first to maximize team-wide adoption speed?
βœ“ Innovators and early adopters first. Their fluency and visible wins create peer-to-peer social proof that converts the early majority far more efficiently than manager-driven mandates or vendor training sessions.
βœ— Early adopters first is the strategic sequence. Their social credibility with colleagues β€” specifically the early and late majority β€” is the most efficient adoption catalyst available. They serve as translators between management's intent and the team's actual experience.
14. Which of the following is the best description of "qualified use" as a metric for AI adoption?
βœ“ Qualified use specifically measures incorporation into actual work β€” distinguishing real adoption (tool produced useful output that was kept) from activity (tool was used, output was discarded). It is the most meaningful activity metric available.
βœ— Qualified use is about final incorporation. An employee can generate 50 AI outputs, discard all of them, and appear highly active. Qualified use reveals whether the AI is actually producing work people use β€” the only thing that matters.
15. A Journal of Applied Psychology study (2021) found that leaders who explicitly acknowledged their own AI competence gaps while expressing commitment to learning were rated by their teams as:
βœ“ Higher trustworthiness β€” not lower competence perception. Acknowledging gaps while showing commitment to learning signals both honesty and growth orientation, two qualities that predict trustworthiness reliably.
βœ— The counterintuitive finding: acknowledging gaps while committing to learning rated higher in trustworthiness than projecting false confidence. Employees distinguish between "doesn't know everything" and "not competent to lead" β€” and they reward honesty about the former.