The AI ethics board met monthly. It had representatives from legal, technical, product, and ethics teams. It reviewed new AI use cases. It produced thoughtful memos.
It could not block a product launch. Its recommendations were advisory. The product team could — and sometimes did — proceed without endorsement. The governance structure existed. The authority didn't.
Corporate AI governance encompasses the internal structures, processes, policies, and accountabilities organizations use to manage AI development and deployment. It is distinct from regulatory compliance — though the two interact. An organization can be regulatory compliant and have poor internal AI governance. It can also have excellent internal governance while operating in a regulatory vacuum.
Corporate AI governance answers questions that external regulation often does not: Who inside this organization is responsible when an AI system fails? What approval is needed before a new AI application goes to production? How does the organization know whether its AI is performing as intended? Who has authority to override an AI system recommendation — and under what conditions?
Organizations have developed several structural approaches to AI governance:
AI Ethics Boards or Committees: Cross-functional groups (technical, legal, ethics, business) with authority to review and approve (or reject) AI use cases. Vary enormously in authority — some can block product launches, others provide only advisory opinions.
Responsible AI Teams: Dedicated internal units focused on AI safety, bias testing, documentation, and model risk management. Some report to the CTO, others to legal or compliance, others to a Chief AI Officer or Chief Ethics Officer.
Model Risk Management: Borrowed from financial services, where model risk management is well-developed and often required by regulators. Involves model documentation, independent validation, ongoing performance monitoring, and approved model inventories.
AI Incident Response: Processes for identifying, escalating, investigating, and remediating AI system failures — analogous to cybersecurity incident response but focused on AI-specific failure modes including bias, performance drift, and out-of-distribution behavior.
Most large tech companies now have published AI ethics principles and governance commitments. Far fewer have governance structures with real authority to enforce them. The gap between stated principles and internal accountability mechanisms is where AI governance most frequently fails in practice.
Choose a company that uses AI consequentially — a major tech company, bank, healthcare system, or government agency.
Assess their internal AI governance: (1) What governance structures do they have in place? (2) What evidence exists about whether those structures have real authority? (3) What governance gaps are visible from public information?
The company announced its AI Ethics Advisory Board with a press release and a list of distinguished external advisors. Three months later, the board had met once. Six months later, it had been quietly dissolved.
This is not a hypothetical. Variations of this story have played out at multiple major technology companies. Understanding why requires looking at what governance authority actually existed.
The AI ethics board concept — a cross-functional body that reviews AI use cases and advises on ethical implications — has been widely adopted and widely criticized. Understanding why requires looking at specific cases.
Google's Advanced Technology External Advisory Council (2019): Announced with significant fanfare. Dissolved within two weeks after member controversies and employee protests. The speed of its collapse illustrated what can happen when governance bodies are created without careful stakeholder consultation.
Microsoft's AI Ethics Committee: Microsoft has maintained longer-running internal AI ethics governance, with Responsible AI teams and documented review processes. But its governance faced questions when the company dissolved its ethics and society team amid layoffs in 2023, raising concerns about the stability of governance commitments during financial pressure.
Meta's Oversight Board (content moderation): Not AI-specific, but instructive. An independent board with authority to overturn content decisions — and Meta has sometimes complied, sometimes not, and sometimes changed policies in ways that circumvent board purview.
Research and practitioner experience suggest several factors distinguish ethics governance that influences decisions from governance that is primarily performative:
Real authority: The body can block or require modification of AI deployments, not merely advise. Independence: Members with genuine independence from product and revenue pressure, including external members or board-level oversight. Early integration: Governance review happens during development, not as a final gate that creates pressure to approve what is already built. Transparency: Decisions, reasoning, and outcomes are documented and accessible — at least internally. Follow-through accountability: When governance bodies make recommendations, there is a mechanism to track whether recommendations were implemented.
Internal AI ethics governance faces a structural challenge: the people conducting governance reviews are paid by the organization whose decisions they are reviewing. This creates pressure — subtle or explicit — to approve use cases that support business goals. Independent external members can reduce but not eliminate this dynamic.
You are designing an AI ethics governance structure for a company that makes AI-powered hiring tools used by hundreds of employers.
Specify: governance structure, membership and independence, authority (advisory vs. binding), scope, review process, transparency, and accountability mechanisms. Then identify the three most likely ways your governance structure would fail in practice.
When the AI hiring tool ranked the candidate poorly and she lost the opportunity, she could not find out why. The employer said the algorithm was the vendor's. The vendor said configuration was the employer's. Legal said they couldn't comment on algorithmic processes.
No one was accountable. Not because accountability was impossible — because accountability mechanisms had not been built.
Accountability in AI systems requires more than good intentions or stated principles. It requires mechanisms — specific processes, responsibilities, and consequences — that create actual accountability relationships between AI systems, their developers, deployers, and affected people.
Documentation and auditability: Systematic recording of design decisions, training data, performance metrics, testing results, and deployment decisions — sufficient to reconstruct why a system behaves as it does. This is the foundation of accountability, but far from sufficient on its own.
Performance monitoring: Ongoing measurement of AI system behavior in production, with thresholds and escalation processes for performance degradation or unexpected behavior. Without monitoring, accountability is retrospective-only.
Redress mechanisms: Processes by which people affected by AI decisions can understand those decisions, challenge them, and receive review or correction. The EU AI Act requires these for high-risk systems; most voluntary frameworks recommend them; many deployed AI systems lack them.
Clear ownership: Designated individuals or teams with specific accountability for AI system performance and outcomes — not diffuse organizational responsibility that means no one is responsible. In practice, accountability often disappears in the gap between AI developers (who built it), AI operators (who configured it), and deployers (who use it).
Consequence mechanisms: Actual consequences for AI governance failures — not just policy violations, but reputational, financial, or career consequences that create incentives for accountability-conscious behavior.
In complex AI deployment chains — foundation model provider, application developer, system integrator, deployer — accountability tends to diffuse. Each party points to others when something goes wrong. Effective accountability requires mechanisms that resist this diffusion: explicit responsibility assignment, contractual accountability, and governance that spans organizational boundaries.
Choose a real AI deployment chain: a bank using a vendor's credit scoring model, a hospital using a diagnostic AI trained by a different company, or a government agency using AI built by a contractor.
Map the accountability structure: who is responsible for what? Where does accountability diffuse? What specific mechanisms are missing that would close the accountability gaps?
The AI worked. In the sense that it made decisions quickly and at scale. It had been tested — accuracy metrics looked fine.
What had not been tested was whether the decisions were fair, whether the training data reflected the outcomes the organization actually wanted, or whether the concerns raised internally had anywhere to go.
Understanding where corporate AI governance fails requires examining specific cases — not to assign blame, but to identify structural patterns that recur across organizations and contexts.
Amazon developed a machine learning tool to screen resumes, then discovered it was systematically downrating resumes that included the word "women's" and graduates of all-women's colleges. The system had learned that male-dominated historical hiring patterns were "good" hiring patterns. The tool was ultimately scrapped. The governance failure: the model was trained on a decade of historical hiring data without systematic bias testing, and deployed in consequential decisions before its bias was discovered in use.
IBM's Watson for Oncology — intended to recommend cancer treatment plans — was found by physicians to frequently produce unsafe or incorrect recommendations. Internal IBM documents revealed concerns were raised early and not addressed. The governance failure: clinical validation was insufficient, concerns raised by clinical advisors were not escalated effectively, and commercial pressure to maintain the product outweighed internal safety concerns.
Across cases, several patterns recur: Training data assumptions — using historical data without examining whether historical patterns reflect desired future outcomes. Validation shortcuts — inadequate testing of AI behavior before deployment, especially in edge cases and demographic subgroups. Escalation failure — internal concerns raised but not reaching decision-makers with authority to act. Commercial pressure override — safety and governance concerns deprioritized when they conflict with launch timelines or revenue goals.
None of these failures required new technology to prevent. They required governance: processes for bias testing, escalation pathways for concerns, authority structures that could override commercial pressure, and accountability for outcomes. The missing ingredient was not capability — it was governance.
Choose one of the two cases from Lesson 4 (Amazon hiring algorithm or IBM Watson Oncology), or a different AI failure you know well.
Conduct a governance post-mortem: (1) What governance mechanisms were missing? (2) What specific process, if in place, would most likely have caught the problem? (3) What would need to change organizationally for that mechanism to have real authority?