AI Is Not a Legal Shield

What Two Real Cases Reveal About AI Governance

What Two Real Cases Reveal About AI Governance

Joe Mariano said something at the Gartner Digital Workplace Summit and it should be on a poster in every AI governance committee in the country. “AI is a tool. It is not a legal shield.” Two recent cases prove him right. In both, an organization deployed an AI system, the system did exactly what it was designed to do, the organization got sued, and the organization lost. Not because the model misbehaved. Because nobody on the human side was watching. These are not abstract risks. They are the first two real precedents we have for AI liability in the enterprise, and they both turned on the same thing: a monitoring and data gap that the legal system treated as the company’s responsibility, not the AI’s. If you are a leader thinking about AI governance, you are no longer thinking about it in theory. You are thinking about it inside the ruling that other companies have already lost.

The Air Canada Chatbot Case

In November 2022, Jake Moffatt visited Air Canada’s website to book a last-minute flight to attend his grandmother’s funeral. He asked the airline’s AI chatbot whether bereavement fares were available. The chatbot told him yes, that he could book the flight at full fare and apply for a bereavement refund within ninety days of travel. He booked. He flew. He filed for the refund. Air Canada denied the claim. The actual policy required bereavement-fare requests before travel, not after. The chatbot had hallucinated a refund window that did not exist. Moffatt took the airline to the British Columbia Civil Resolution Tribunal. Air Canada’s argument was the part that should make every AI governance lead pay attention. The airline argued that the chatbot was, in their words, “a separate legal entity that is responsible for its own actions.” The tribunal rejected that argument flatly. In Moffatt v. Air Canada, 2024 BCCRT 149, the tribunal ruled that Air Canada was responsible for all information on its website, regardless of whether it came from a static page or a chatbot, and ordered the airline to pay damages. The decision is short, the legal reasoning is clean, and the precedent is simple: if your AI tells a customer something false, your company said it. The chatbot does not have its own lawyer. It does not have its own bank account. It does not have legal standing. It is a tool you deployed, and the output is your output. Air Canada’s failure was not that the chatbot hallucinated. Hallucination is a known property of generative systems, and any organization deploying one in a customer-facing context should plan for it. The failure was that nobody checked. There was no monitoring layer, no review pipeline, no human in the loop verifying that high-stakes policy claims matched the airline’s actual policy. The model behaved as models behave. The organization behaved as if the model would not.

The iTutorGroup EEOC Settlement

In August 2023, the U.S. Equal Employment Opportunity Commission settled a case against iTutorGroup, a tutoring company that had used an AI-driven recruitment system to screen applicants for tutor positions. The system was configured to automatically reject women aged 55 and over and men aged 60 and over. More than two hundred qualified applicants were filtered out before any human ever saw their applications. The EEOC argued, and iTutorGroup agreed in a consent decree, that the company had violated the Age Discrimination in Employment Act. iTutorGroup paid $365,000 in damages and committed to anti-discrimination training and oversight changes. It is widely cited as the first EEOC enforcement action targeting algorithmic discrimination, and it set the regulatory tone for what was to come. The interesting thing about this case is that the AI did not malfunction. It did exactly what its rules told it to do. Somewhere in the configuration, somebody had set age thresholds. The AI applied them. Hundreds of times. What was missing was the question of whether anybody should have set those thresholds in the first place. There was no review of the screening logic against employment law. There was no monitoring of who was being filtered out and why. The data quality, the rule design, the oversight layer, all of it sat inside an AI deployment that nobody thought needed governance because the AI itself was working. That is the iTutorGroup pattern, and it is more dangerous than the Air Canada pattern because it does not look like an AI failure. It looks like an AI success.

The Pattern: Monitoring Gaps, Not Bad Models

Joe Mariano walked through both of these cases at Gartner DWS 2026, and the framing he landed on is worth repeating: the failures here were not in the technology layer. They were in the layer above the technology, where humans decide what the AI is allowed to do, what data it sees, and who is watching when it is doing it. The Air Canada chatbot worked as a generative chatbot works. It produced a plausible answer to a question it did not have grounded knowledge to answer. The failure was that the airline deployed it on a high-stakes policy page without a verification pipeline. The iTutorGroup recruiter worked as a rules-based filter works. It applied the configuration it was given. The failure was that the configuration had been set by humans without legal review, and there was no monitoring on the output to flag the discriminatory pattern. Both failures, in other words, traced to the same place: the human-decision system around the AI was not designed. The technology was deployed faster than the governance scaffolding around it could catch up, and the legal exposure that resulted was real. This is the part most AI governance conversations skip. They focus on the technology, on which model, on which vendor, on which compliance certifications, when the actual exposure lives in the workflow. Who reviews high-stakes outputs before they go to customers. Who audits the rules the AI is using. Who is watching for patterns that look fine inside the model but look discriminatory in aggregate. The Gartner data backs this up. There are over 1,000 proposed AI rules and regulations worldwide right now, and not one of them has the same definition of AI. The regulatory landscape is going to get harder, not easier. Companies that are still treating AI governance as a policy document, rather than as an active facilitation problem inside their organization, are going to keep producing the next Air Canada and the next iTutorGroup.

Why AI Does Not Absorb Accountability

There is a comforting fiction that some leaders are still telling themselves about AI deployment, which is that the model carries some of the liability. It does not. Across multiple jurisdictions, in multiple legal frameworks, the rulings are converging on the same answer: the deploying organization is accountable for the output, full stop. This makes sense the moment you say it out loud. The AI did not sign a contract with the customer. The AI did not file a lawsuit. The AI did not get sued. Your company did all three of those things. The model is a tool the company chose to deploy, and the output of that tool is the company’s output, the same way that an internal email written by a junior employee is the company’s email. What this means in practice is that the conversation about AI governance has to move from “is the model trustworthy” to “is our deployment of the model accountable.” Those are different questions. A trustworthy model deployed without governance is still a liability. An imperfect model deployed with rigorous governance is, in many cases, fine. The trustworthy-but-ungoverned configuration is what produced both cases above. The Air Canada chatbot was, by industry standards, a perfectly normal AI product. The iTutorGroup recruiter was, by configuration standards, perfectly capable of being used legally. Neither model was the problem. The deployment around it was.

Four Governance Patterns That Actually Work

If the technology is not the gap, what closes the gap? Mariano’s session offered four patterns, and they map cleanly to what we see when we walk into client governance work. Brain-first deployment. Before AI is brought into a workflow, the team uses human judgment to define the goal, the boundary, and the success criteria. The AI is then brought in to assist a human-defined process, not to replace the human-definition step. Air Canada skipped this. The chatbot was deployed on a policy page without anyone defining what counted as an acceptable policy answer. Human in the loop for quality control. Some volume of AI output gets reviewed by a human before it goes to a customer or a decision. The exact percentage depends on the stakes, but the principle is non-negotiable: zero human review on high-stakes outputs is a deployment, not a governance posture. iTutorGroup ran a recruitment AI with apparently no auditing of the rejection pattern. That is the failure case. Data quality management. AI systems accessing wrong, stale, or biased data will produce wrong, stale, or biased outputs with full confidence. Both Air Canada and iTutorGroup had data quality problems at the root. The chatbot was answering questions about a policy it had not been grounded in. The recruiter was applying rules that had not been audited against current law. Neither case was a model problem. Both were data problems wearing model clothing. Continuous skill and process maintenance. Governance is not a one-time training. It is an ongoing practice, with periodic reviews, audits, and skill refreshes for the people running the system. The model evolves. The regulations evolve. The use cases evolve. A governance framework that was designed twelve months ago and has not been touched since is, by definition, stale. These four patterns are not novel. They are the basic discipline of any high-stakes deployment, applied to AI. What is new is that the legal system is now treating them as the standard of care, and organizations that ignore them are losing the cases.

The Real Move: Treat Governance as Facilitation

Here is the move most organizations miss. AI governance is not a document. It is a set of ongoing agreements between security, legal, business, and operations about what the AI can do, who is watching, and what happens when something goes wrong. Those agreements have to be negotiated. They cannot be written by one team and handed to the rest. This is where the work gets uncomfortable, because it requires the same cross-functional conversation that most organizations are structurally bad at. Legal does not want to talk to Engineering. Security does not want to talk to Marketing. The business unit that wants to deploy the chatbot does not want to slow down for a review. And so the governance conversation never happens, and the deployment goes out, and somebody loses a tribunal. The organizations getting this right are the ones that treat AI governance as a facilitated, recurring practice, not a sign-off process. They have a standing forum, with the right people, that meets often enough to keep up with what is being deployed. They produce decisions, not policy documents. They review the deployments that have shipped. They ask, every time, what would happen if this output ended up in front of a regulator or a tribunal. That is the New Friction. AI eliminated the old friction, which was execution time. The new friction is the human-decision layer that has to keep up with what AI now lets you ship. Organizations that do not invest in that layer ship faster, get sued more, and lose the cases. Organizations that do invest in it ship slightly slower, ship better, and stay out of the tribunal. If you are building or refreshing your AI governance posture right now, the question is not which model you trust. The question is which decisions your organization can keep up with, and which conversations you are willing to have to keep up with them. That is the work. That is the entire work. If your organization is in the middle of that conversation, or trying to start one, that is where Voltage Control comes in. Read our New Friction primer for the full framework, or reach out if you want to talk about where your governance posture is stuck.

Frequently Asked Questions

Can companies be held liable for AI mistakes?

Yes. Both the Air Canada and iTutorGroup cases establish that the deploying organization is responsible for AI output, regardless of whether the output came from a human or an AI system. Air Canada explicitly argued that the chatbot was a separate legal entity. The tribunal rejected the argument. Across jurisdictions, the legal direction is consistent: the company that deploys the AI owns the consequences.

What happened in the Air Canada chatbot case?

A customer asked Air Canada’s chatbot about bereavement fares. The chatbot hallucinated a refund policy that did not exist. The customer relied on it, booked the flight, and was denied the refund. He took the airline to the British Columbia Civil Resolution Tribunal, which ruled in Moffatt v. Air Canada, 2024 BCCRT 149, that the airline was responsible for the chatbot’s output. Air Canada paid damages.

How do organizations govern AI systems effectively?

Effective AI governance is a recurring facilitation practice, not a static policy document. The organizations doing this well bring legal, security, business, and operations into a standing forum that meets often enough to keep up with deployments, audits real outputs, and produces decisions. The four operating patterns are brain-first deployment, human in the loop for quality control, data quality management, and continuous skill maintenance.

What is AI accountability in the workplace?

AI accountability is the principle that the organization deploying the AI is responsible for what the AI produces. That responsibility cannot be delegated to the model, the vendor, or the AI system itself. It lives with the humans who decided to deploy the system, configured it, fed it data, and chose how much oversight to give it.

Who is responsible when AI makes wrong decisions?

The deploying organization. In every major AI liability case to date, including Air Canada and iTutorGroup, the courts and tribunals have held the company responsible for the AI’s output. The model is treated as a tool, and tool failures attach to the operator, not to the tool.