Vishvajit Pathak16 min read
Nearly half of CEOs think their AI investment is break-even at best, and only about 5% call it a real win. That is the Gartner survey number that opens the ThinkCast episode "Start With AI Literacy: Why Most Leaders Are Missing the Point," with Distinguished VP Analyst Mandi Bishop. It is a brutal scorecard for a technology that has pulled more executive attention and capital than anything since cloud.
Bishop's diagnosis is the part worth sitting with. The bottleneck was never the technology. Most people were never taught to think alongside AI, to question it, or to catch it when it produces something confident and wrong. The tools shipped. The literacy did not.
We agree, and we would go further. We have shipped 80+ products across 12 countries since 2019, a growing share of them AI-native. The pattern is consistent. The teams that got real value from AI were not the ones with the most model access or the biggest platform contract. They were the ones who knew how to interrogate an output before they trusted it.
That gap between spend and value is what the rest of this piece is about. The research names the problem. We can tell you what it looks like inside a delivery pipeline, and what a CIO should do about it on Monday.
Buying and scaling tools does not deliver AI enablement, because the tool is not the differentiator. As Bishop puts it: "No matter what technology you're using, that's not the differentiator. The people are." An enterprise can roll out Copilot to 20,000 seats and move its break-even needle by roughly nothing.
Bishop's analogy stuck with us. She compares an unused AI tool to a kitchen gadget you never opened: "It doesn't matter how many cookbooks I have," she says, the air fryer "is still in its box somewhere in my house." Owning the capability is not the same as having the understanding or the motivation to use it well.
We see the boxed-air-fryer problem constantly. A client buys enterprise AI seats. Usage spikes for two weeks, then collapses. The tool worked. The enablement never happened, so people drifted back to the workflow they trusted.
Here is what that looks like across the three failure modes we run into most:
| What the enterprise bought | What they expected | What actually happened |
|---|---|---|
| Copilot / assistant licenses, org-wide | Broad productivity lift | Two-week usage spike, then drop-off. No sustained adoption. |
| A RAG or "chat with your docs" platform | Trusted internal answers | People stopped checking answers, then stopped trusting the tool after one bad one. |
| A model API budget for product teams | Faster shipping | Token bills climbed while output quality stayed flat, because nobody owned prompt discipline. |
None of those failures are tool failures. They are literacy failures wearing a procurement badge. The spend is real, the enablement is missing, and the CEO survey result follows.
Here is the part worth adding. The tool-buying trap is seductive precisely because a purchase order is easy and a behavior change is hard. A license renews itself. Literacy does not. A 20,000-seat rollout books as one line on the capital plan; the behavior change behind it never shows up on any invoice, which is exactly why it gets skipped.
Bishop breaks AI literacy into three pillars: critical thinking, prompt discipline, and output discernment. That taxonomy is right, and each pillar maps cleanly onto an engineering discipline we already run on production AI work. Here is the pillar, then what it looks like when it is real inside a delivery pipeline.
Critical thinking is the pillar most at risk of eroding, and the research backs that worry. It degrades with AI use unless it is continuously cultivated. That is not speculation. A Microsoft Research study presented at CHI 2025 surveyed 319 knowledge workers across 936 real tasks and found that higher confidence in generative AI correlated with less critical thinking applied to its outputs, per the published paper.
Read that finding again, because it is the whole game. The more your people trust the AI, the less they check it. Confidence and scrutiny move in opposite directions.
In production, we do not fix that with a memo asking people to "stay critical." We fix it with eval discipline: a small golden set of known-good examples, deterministic checks, and a scoring pass that runs before anything ships. Critical thinking, made into a gate. When a model output has to clear a rubric your team wrote, skepticism stops being a personality trait and becomes a step in the pipeline. We break down how we build that in our guide to LLM eval suites for production AI.
The quotable version: critical thinking that lives only in someone's head erodes; critical thinking encoded as an eval survives the confidence trap. A rubric does not get tired, does not get impressed by a fluent answer, and runs the same way on task 5 and task 936.
Prompt discipline is where AI literacy stops being abstract and starts showing up on the invoice. Weak prompting ties directly to cost. Sloppy prompts force costly iteration, and getting the prompt right the first time minimizes token spend, a volatile cost factor. Vendors will eventually pass model costs down into embedded enterprise AI, which makes this a budget problem with a shelf life.
We live this every month. Weak prompt discipline is one of the quiet ways an AI feature's bill climbs while its quality stays flat.
A prompt that specifies role, format, and business context gets the answer in one pass. A prompt that gives only the task triggers three rounds of "no, more like this," and every round is billable tokens. The literacy gap is a line item.
The production discipline here is deliberate prompt structure plus cost controls: output caps, prompt caching for stable context, and cascading easy calls to cheaper models. We rank the levers in our playbook on cutting LLM token costs. Prompt discipline is the first one, because it is free and it compounds.
Output discernment is the skill of knowing when a confident answer is wrong, and it is the one to take most seriously. The risk it counters has a name: automation bias, the human tendency to over-trust a good-enough output. The example is ordinary and scary. A Copilot email summary that is 90% right gets forwarded as if it were 100% right, and the missing 10% causes harm three steps downstream.
Bishop's healthcare case-worker example makes it concrete. Unreviewed AI summaries in a case file degrade the quality of a decision, and that decision affects a real person. The output looked fine. Nobody discerned the gap.
The countermeasure is an exercise Bishop calls the "Hallucination Hunt." You give someone a prompt and its output, and ask whether they can spot the subtle fallacy, validate it against a non-AI primary source, correct the prompt, and reach the right outcome. It is tool-agnostic and broken into small, measurable steps with a rubric.
We love this because it is exactly how we train discernment on delivery teams. Output discernment is not a warning label, it is a repeatable drill. "Validate against a primary source" is the same instinct that separates a production RAG system from a demo, which we cover in enterprise RAG architecture. The ones that stay accurate have guardrails and evaluation, not just retrieval.
Automation bias is not a soft-skills gap, it is a governance exposure that belongs on the same risk register as access control and audit logging. It is usually filed as a workforce issue. We would file it, for an enterprise audience, as a control gap: an unreviewed AI output that flows into a regulated decision is a finding waiting to happen.
The behavior is well documented outside the AI hype cycle. Human-factors researchers Raja Parasuraman and Dietrich Manzey defined automation bias two decades ago as the tendency to under-check an automated aid and follow it into both errors of omission and commission, per their 2010 review in the journal Human Factors. Their finding that matters most for a CIO: the bias shows up in expert users too, and instructions alone do not remove it. You cannot brief your way out of it. You have to build a check into the workflow.
Think about where the boxed-air-fryer story ends in a regulated business. A case worker forwards an AI summary. A finance analyst pastes an AI-generated reconciliation. A support agent acts on an AI answer about a customer's coverage. Each one is a decision with a paper trail, and each one now has an unvalidated model in the loop.
Bishop's line, "you can't manage what you can't measure," is why this becomes a leadership blind spot. If nobody measures whether outputs were validated before they were used, the org cannot see the risk until it materializes as an incident. The bias is invisible right up until it is expensive.
The governance move is to treat output validation as a control, the same way we treat deployment approvals or PHI access logging. Who reviewed the AI output? Against what source? Recorded where? On regulated builds, we already wire that discipline into the pipeline, the same posture we describe in SOC 2 and HIPAA-ready engineering. An AI output that touches a regulated decision should leave the same evidence trail as any other privileged action.
The quotable version: automation bias is what happens when discernment is optional. Governance is what makes it mandatory.
AI literacy sticks when it is embedded in how people already work, and here is the part most leaders skip: this needs no new tool and no new budget line. It gets skipped because leaders are looking for something to purchase. And it is the part that actually moves the CEO survey number.
Bishop's mechanics are refreshingly low-tech. Communities of practice. Peer-to-peer teaching where people measure each other. Gamified team meetings where the Hallucination Hunt becomes a recurring game. And critically, embedding literacy into the agile sprints and product sprints that teams already run, so it is not a separate initiative competing for calendar space.
This maps directly onto how we deliver. We do not run "AI literacy" as an offsite. We run it in the sprint, because that is where the work is.
Here is the practical shape of it, drawn from both the research and our delivery method:
The warning on sustained adoption is the reason step 4 matters most. People adopt an AI tool for a week or two, then drop it. Value comes from sustained adoption, and sustained adoption comes from embedded literacy, not from a launch email. Over time, Bishop notes, literacy becomes a career driver rather than a mandate, and that is when it self-perpetuates.
Across our own 80+ builds since 2019, the checkpoint that survived was never the one we announced in a kickoff deck. It was the one that rode an existing ritual: the pull request review, the sprint demo, the retro. A literacy drill bolted onto a ceremony the team already keeps has near-zero adoption cost. A standalone "AI training" competing for a fresh calendar slot dies by the second sprint.
One point here we would not have thought to include, and it lands: "This is not just an employer problem," Bishop says, "this is a societal problem. It's all of us." AI literacy is not only a corporate training checkbox. It is a civic capability, and the enterprise happens to be one of the biggest places it gets built.
The first step is not a purchase, it is a baseline. Asked what a leader should do first, Bishop's answer is one word: "Baseline." Measure where your people actually are on aptitude, attitude, and adoption before you spend another dollar on tooling or training.
We would endorse that without reservation, and add the lighter-weight version she offers for leaders who want to move this week. Look at your collaboration channels. Open Teams or Slack and find the people already talking about AI, sharing prompts, comparing outputs, arguing about a hallucination. Those are your champions, and they are free.
Tap those influencers to host peer-to-peer channels and run the first Hallucination Hunts. You are not building a program from zero. You are surfacing a community that already exists and giving it a mandate.
The race-car analogy is the right mental model for sequencing. You learn to steer, then to work the pedals, then to read traffic, before you race. You do not start at Formula 1 speed. So do not roll out an org-wide AI mandate before you know whether your people can steer.
The CIO move, in one line: baseline the aptitude, surface the champions already in your channels, and embed the three pillars into sprints you already run. No new platform required.
AI literacy is the ability to think alongside AI, question its outputs, and catch it when it is wrong. Gartner defines it through three pillars: critical thinking, prompt discipline, and output discernment. It is a workforce capability, not a tool feature.
Gartner reports nearly half of CEOs call their AI investment break-even at best, with only about 5% seeing a real win. The cause is usually low AI literacy, not weak technology. Buying more tools does not close an enablement gap.
No. Licenses give people a capability, not the skill or motivation to use it well. Mandi Bishop of Gartner compares an unused AI tool to an air fryer still in its box. Value comes from embedded literacy and sustained adoption, not seat count.
Automation bias is the tendency to over-trust good-enough AI outputs. Human-factors research shows it affects expert users and resists training. It matters because an unreviewed output that flows into a regulated decision becomes a governance risk, so treat output validation as a control with an evidence trail.
Embed it in work people already do: communities of practice, peer-to-peer teaching, and literacy drills run inside existing agile sprints. This needs no new budget and survives because it rides on rituals the team already keeps.
It is a Gartner exercise where you give someone a prompt and its output, then ask them to spot the subtle error, validate against a non-AI primary source, correct the prompt, and reach the right result. It is tool-agnostic and scored with a rubric.
Baseline your team's AI aptitude, attitude, and adoption before spending on tools or training. As a lighter start, scan your Teams and Slack channels for people already discussing AI and tap them as peer-to-peer champions.
Research suggests it can. A Microsoft Research study at CHI 2025 surveyed 319 knowledge workers and found higher confidence in generative AI correlated with less critical thinking applied to its outputs. Literacy has to be continuously cultivated to counter that drift.
Gartner is right that AI literacy, not tooling, is the enterprise AI bottleneck. Our production experience says the fix is the same discipline that separates a shipped AI product from a demo: evals for critical thinking, prompt structure for cost, and output validation for discernment, all wired into the work rather than bolted onto it.
If you are rolling out AI across an enterprise and the usage keeps spiking then collapsing, the answer is not another platform. It is enablement built into delivery, with validation treated as a control from the first sprint.
We build AI-native products with that discipline in place from day one, and we have done it 80+ times since 2019. Talk to our engineering team about standing up production AI where literacy and governance are the default, not an afterthought.
Co-Founder, MarsDevs
Vishvajit started MarsDevs in 2019 to help founders turn ideas into production-grade software. With deep expertise in AI, cloud architecture, and product engineering, he has led the delivery of 80+ software products for clients in 12+ countries.
Get more insights like this
Join founders, CTOs, and engineering leaders who receive our engineering insights weekly. No spam, just actionable technical content.
