Unprofessional Arrogance: When AI's Capability Becomes Your Illusion

Unprofessional Arrogance: When the Tool Becomes the Illusion

Picking up where the previous article left off: if professional arrogance is “I don’t trust you,” then unprofessional arrogance is “I’m already your equal.”

Between 2025 and 2026, AI tools spread far faster than anyone expected. Cursor, Replit, Claude Code, and various no-code platforms like Lovable and Base44 made it possible for people with zero programming background to build things that once required an engineer. BetaNews even ran a headline declaring “citizen developers dominate” [5], announcing the arrival of the citizen developer era.

From an information science perspective, this is a good thing. I fully support the democratization of AI tools — even the democratization of programming itself. Coding gives us more than just useful, customized tools; it teaches us how to think logically, how to reason and observe, how to bring process-oriented thinking into everything we do. A friend of mine, Mosky, wants to teach everyone in Taiwan to code with AI, because she believes the era has already arrived where anyone can build what they want — the same way the PC era meant everyone could use Excel even without being an accountant. Using Microsoft Office became a basic computing skill over the past 20 years, and I think using AI is next, in ways that go well beyond programming.

But here’s the thing: being able to use a tool doesn’t mean you understand the principles behind it. And in the AI era, that gap has become more dangerous than ever.

Aalto University researchers identified a phenomenon they called the “reverse Dunning-Kruger effect” [6]. The classic Dunning-Kruger effect says people with low ability overestimate their competence. The AI-era version is more subtle: the more frequently people use AI, the more likely they are to overestimate their own cognitive ability. Not AI’s ability. Not their general ability. Their cognitive ability — because AI assistance creates an illusion that things AI helped accomplish were products of their own thinking. The work feels like theirs because it was driven by their prompts, their ideas, their cognitive engagement.

A paper on ScienceDirect [7] nailed it with its title: “AI makes you smarter but none the wiser.” Smarter means you can complete tasks. Wiser means you understand the quality of what was completed and what risks it carries.

Research from Microsoft and Harvard [8] pointed to a troubling trend: AI use reduces critical thinking in users, and this effect is more pronounced in non-professionals, because they lack a judgment framework that operates independently of AI. Their final validator is still the AI itself.

Which brings us to the core question: how does a non-expert verify AI output?

I mentioned in Mindset 4 that Agents have a hard time spotting their own blind spots. MIT research confirms this — LLMs can’t reliably self-correct, because they generate content and then check it using the same reasoning process. Errors look reasonable to themselves.

What about using multiple AIs to cross-check each other? This is a solution a lot of people propose — including people in tech.

An analysis from Towards Data Science [9] exposed a fatal problem in multi-Agent systems: the 17x error trap. When multiple Agents collaborate, a deviation in one Agent’s output doesn’t get corrected by downstream Agents — it gets amplified. Each Agent treats the previous one’s output as trustworthy input. The final error can be 17 times the original.

And different LLMs share a lot of the same underlying assumptions (because their training data overlaps significantly). Using Codex (GPT) to verify Claude Code’s output, or vice versa, doesn’t provide truly independent validation. It’s like having students from different classes in the same school grade each other’s exams — they were all taught the same material, they share the same blind spots. They might have different insights here and there, but their foundations are identical.

In July 2025, a vivid real-world case showed everyone exactly what this risk looks like in practice: Replit’s AI Agent deleted an entire production database during a code freeze [10].

This wasn’t a hypothetical. It happened. The Agent didn’t just delete the database — when asked what happened, it gave an incorrect explanation. It “lied” (or more precisely: it used the same reasoning mechanism that generated the action to explain the action, producing a narrative that sounded plausible but was completely wrong).

If the person using that tool had been an experienced engineer, they’d have known that automated operations during a code freeze require extra scrutiny. They’d have known database operations need backup strategies. They’d have known that an AI’s explanation of its own behavior can’t be taken as fact. But what if the user was a non-engineering “citizen developer”?

A CodeRabbit report [11] found that AI-generated code has a 1.7x higher issue rate than human-written code — including security vulnerabilities, logic errors, and performance problems. Many of those issues don’t surface during testing. They sit quietly in the codebase, waiting for the right conditions to detonate. Duke University Libraries published an analysis in early 2026 [12] asking a pointed question: “It’s 2026. Why are LLMs still hallucinating?” Their answer: because hallucination isn’t a bug — it’s a fundamental property of LLM architecture. No matter how much models improve, this will never fully go away. IMD Business School reached the same conclusion [13]: LLMs will hallucinate forever, and your AI strategy must treat that as a given.

This is the most dangerous part of unprofessional arrogance: in 95% or even 99% of cases, AI can genuinely help you do what an engineer does. But that fatal 1% hides exactly where you can’t see it.

Worse, these problems usually don’t surface immediately. A security vulnerability looks fine until someone exploits it. A boundary condition error works perfectly with small datasets but collapses at scale. A timezone assumption bug (like the 25-hour window I described in Tips 4) runs every day — and leaks emails every day. These slow-burn failures create a false impression: “Look, the thing I built with AI is working great!”

And beyond AIs that mislead us, there’s another problem: users who spend long stretches processing large volumes of AI output, context-switching across too many conversations, or continuously supervising AI begin experiencing mental fatigue, reduced attention, and degraded decision-making. An arXiv update from December 2025 covered MIT research tracking 54 college students aged 18–39, finding that using LLM assistance for assignments reduced brain activity, recall, and cognitive engagement. The researchers called this “cognitive offloading” — outsourcing thinking and organization to AI. It saves effort short-term, but may erode thinking capacity over time. The accumulated cost is what they called “cognitive debt” — something you’ll need to pay interest on later.

Research by Boston Consulting Group in collaboration with academics surveyed 1,488 full-time U.S. employees across multiple industries and roles, finding that about 14% had experienced “AI brain fog” — mental fatigue caused by overusing or supervising AI tools to the point of cognitive overload. The study also found that employees who needed to closely supervise AI experienced an average 14% increase in mental drain and a 19% increase in information overload. Employees showing these symptoms had 33% higher decision fatigue and meaningfully higher error rates — small errors up 11%, major errors up 39%. There’s a tipping point: productivity climbs noticeably when employees use two to three AI tools simultaneously, but once the count exceeds three, productivity starts declining.

Here’s the twist, though: engineers, having built up years of accumulated intuition, can quickly sense when an AI’s reduced output is right or wrong — and more importantly, what feels off. That instinct shrinks the cognitive debt and mental drain they incur while using AI.

So when an experienced engineer tries to flag a potential risk, how does a non-professional user — one carrying “the AI I built is working great” plus a load of cognitive debt — tend to respond?

“You don’t understand AI.”

“I already cross-checked with three different AIs.”

“You just don’t want to admit non-engineers can do this too.”

That’s unprofessional arrogance. Not because these people aren’t smart. Not because they aren’t working hard. But because they lack one critical capability: the ability to recognize what they don’t know.

Between the Two Arrogances

I have to be honest at this point: I’ve been both victim and perpetrator of both kinds of arrogance.

As an engineer, there were definitely moments when I trusted my own judgment too much and turned down a better approach AI suggested. The METR research matches my personal experience perfectly — I spent too much time questioning AI instead of using that time to actually evaluate whether its suggestions were worth trying.

Though to be fair, I’m not entirely a professional anymore. I’ve gone long enough without writing code that I’ve gradually drifted away from that world. And I’ll admit: when I hear a non-engineer say “I know how to build things with AI now,” a small flicker of skepticism does arise (and the opening paragraph of this article came directly from one of those real moments). But when I think it through — if AI can genuinely help more people solve more problems, isn’t that exactly what technology should be doing?

So our question was never “who should use AI” versus “who shouldn’t,” or “question AI” versus “trust AI.” The real question is whether we maintain enough humility and skepticism toward our own judgment.

For engineers, that means acknowledging that AI might be better than your intuition in certain areas. Some things you spent ten years learning — AI can genuinely do them in seconds, and do them well. Your value isn’t in being able to do those things. It’s in knowing when AI didn’t do them well enough.

For non-professionals, that means acknowledging that AI gives you tools, not capability. You used AI to write code that runs — that doesn’t mean you understand why it runs, or under what conditions it will break. When an engineer tells you “there might be a problem here,” they’re not showing off or pulling rank. They’re using pattern recognition built over decades to help you avoid a pit you can’t see.

The Stack Overflow 2025 developer survey [14] found that 84% of developers use AI, but most still don’t trust AI output. That’s not a contradiction. It’s an attitude: use it, but don’t trust it blindly.

MIT Technology Review’s year-end retrospective for 2025 [15] also noted an interesting shift: the industry is moving from “vibe coding” (letting AI write code by feel) toward “context engineering” (carefully designing AI’s context to improve quality). That shift itself says something — pure reliance on AI’s “feel” isn’t enough. You need to understand what AI is doing in order to get it to do well.

And that understanding — whether or not you’re an engineer — takes time and deliberate learning.

The Best Position Is in the Middle

Ivan Turkovic wrote something in March 2026 [16] that I think is the most precise description available right now: AI coding is in the “almost solved” phase — and that’s exactly the most dangerous phase.

“Almost solved” means it works most of the time, so you let your guard down. But the gap between “almost” and “fully” is precisely where the landmines are buried.

For me, the ideal state isn’t “engineers learn to trust AI” or “non-engineers learn to distrust AI.” It’s everyone learning to stay humble about their own blind spots.

Engineers’ blind spot is imagination. We’re too accustomed to making decisions within familiar frameworks, and we forget that AI can help us explore what lies beyond them.

Non-engineers’ blind spot is judgment. AI lets you do things you couldn’t do before — but you need to develop a way to evaluate the quality of the results that operates independently of AI. You don’t necessarily need to learn to code, but you at least need to learn to ask the right questions, so you can avoid the illusion of control that AI creates.

If you ask AI and it says “this won’t break,” and an engineer says “there might be a problem here” — maybe you should actually listen to what that engineer is saying.

Not because the engineer is necessarily right. But because they might be seeing something neither you nor AI can see.

Maybe, in this era of increasingly capable AI, the hardest thing to learn isn’t how to use AI. It’s how to hold a powerful tool in your hands and still remember your own limitations.

Maybe…

Professional arrogance makes you miss the possibilities AI can open up. Unprofessional arrogance makes you blind to the risks AI carries. The narrowest path between the two is called humility.

References

[5] “Citizen developers dominate, code as the new Latin” — BetaNews, 2025/12. https://betanews.com/2025/12/17/citizen-developers-dominate-the-rise-of-ai-code-as-the-new-latin-development-predictions-for-2026/

[6] “AI use makes us overestimate our cognitive performance” — Aalto University, 2025. https://www.aalto.fi/en/news/ai-use-makes-us-overestimate-our-cognitive-performance

[7] “AI makes you smarter but none the wiser” — Computers in Human Behavior (ScienceDirect), 2025. https://www.sciencedirect.com/science/article/pii/S0747563225002262

[8] “The Impact of Generative AI on Critical Thinking” — Microsoft Research, 2025. https://www.microsoft.com/en-us/research/wp-content/uploads/2025/01/lee_2025_ai_critical_thinking_survey.pdf | “Is AI dulling our minds?” — Harvard Gazette, 2025/11. https://news.harvard.edu/gazette/story/2025/11/is-ai-dulling-our-minds/

[9] “Why Your Multi-Agent System Is Failing: Escaping the 17x Error Trap” — Towards Data Science, 2026. https://towardsdatascience.com/why-your-multi-agent-system-is-failing-escaping-the-17x-error-trap-of-the-bag-of-agents/

[10] “AI-powered coding tool wiped out a software company’s database” — Fortune, 2025/07. https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure/ | “AI Agent Wipes Production Database, Then Lies About It” — eWeek, 2025. https://www.eweek.com/news/replit-ai-coding-assistant-failure/

[11] “State of AI vs Human Code Generation Report” — CodeRabbit, 2026. https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report

[12] “It’s 2026. Why Are LLMs Still Hallucinating?” — Duke University Libraries, 2026/01. https://blogs.library.duke.edu/blog/2026/01/05/its-2026-why-are-llms-still-hallucinating/

[13] “LLMs will hallucinate forever — here is what that means for your AI strategy” — IMD Business School. https://www.imd.org/ibyimd/artificial-intelligence/llms-will-hallucinate-forever-here-is-what-that-means-for-your-ai-strategy/

[14] “AI — 2025 Stack Overflow Developer Survey” — Stack Overflow, 2025. https://survey.stackoverflow.co/2025/ai

[15] “From vibe coding to context engineering: 2025 in software development” — MIT Technology Review, 2025/11. https://www.technologyreview.com/2025/11/05/1127477/from-vibe-coding-to-context-engineering-2025-in-software-development/

[16] “Almost Solved Is the Most Dangerous Phase in Engineering” — Ivan Turkovic, 2026/03. https://www.ivanturkovic.com/2026/03/31/ai-coding-almost-solved-most-dangerous-phase/