AI and the Dunning-Kruger club
Summary
When you don’t have skill, AI looks like magic. If we are to derive productivity gains from AI, we must value hard-earned skills, treat AI use as a means, not an end, and call out slop each time we see it.
I often wonder how much of the generative cacophony leads to real productivity. Don’t get me wrong. I have no doubt that it speeds up output. I’m not a coder, but I have a Claude Code project with a rudimentary user-side harness that handles much of the coding on a personal project. Win!
But the speed of an individual’s outputs has nothing to do with the quality of those outputs or a productive outcome. It is, indeed, possible to produce trash at speed. It’s also possible to produce high-quality outputs at speed and do nothing for a product or business outcome.
Many data points allude to the phenomenon I’m describing. How about I paint one of those pictures? The Google Play Store and the Apple App Store each have over 2.5 million apps. It’s fair to say that “there’s an app for that” is a true statement for most problems. Is there scope for new app ideas? Perhaps, but they’re all in the “build a better mousetrap” category. Agentic AI has led to a proliferation of new apps. The graph of new releases is through the roof. Guess what’s happening in parallel? App reviews and the number of apps with meaningful usage are tanking!
A explosion of apps and an implosion of usage and quality (credit John Burn-Murdoch)
Well-capitalised scaleups aren’t doing much better either. Take the poster children of the Indian market — Zomato, Swiggy, Ola, Cleartrip, Rapido and Zepto. Amongst these, only Zomato is profitable and that too by the skin of its EBITDA teeth. No exaggeration there, mind you. Zomato’s net profit is $2.6 million; not much to write home about.
By the way, if my argument doesn’t sound appealing, just look around you. Which application has improved by orders of magnitude? Sorry, adding a chat interface to prompt the app to generate a random output doesn’t count. My favourite example is Google Slides. It was the worst presentation app before AI and remains the worst even now. Surely, if AI is all we say it is, Gemini should have helped Google get on par with PowerPoint, Keynote, or at least Canva?
So yes, AI is making people like me look better than I’d look if I had to rely on my skills. But it’s doing precious little for productivity, and the industry is noticing. A recent article summarises one of the problems at the heart of the GenAI productivity paradox.
“When a company encourages people to use GenAI widely, two groups of people pay the most attention: poor performers and average performers, who together tend to make up well more than half of any organisation. These people start to use GenAI to summarise their meetings, write their emails, and build their presentations. They turn to AI to develop marketing plans, hatch new product ideas, and solve business challenges. Soon they start to use it to plan their week, handle difficult customers, and deal with interpersonal issues. Their raw output goes up, so they think the quality of their work does, too.”
David Rock Sat, the author of that article, hits at the heart of the problem with generative AI, though he loses the plot towards the end of the piece. Nevertheless, the problem he’s talking about is the Dunning-Kruger effect; at the lower end of skill, people’s confidence is highest when their competence is lowest. This leads to David Dunning’s quip,
“The first rule of the Dunning-Kruger club is you don’t know you’re a member of the Dunning-Kruger club.”
Think of it this way. If you were to ask a hundred random drivers if they were above average, the majority would say yes. Of course that’s a statistical impossibility. That statistical impossibility becomes even starker in high-end knowledge work. Sturgeon’s law tells us that 90% of everything is crap. By association, 90% of all practitioners (from novices to gurus) are crap. Yet, those 90% don’t think of themselves as crap, do they? And that’s the problem. Many of us AI users are feeling smug about an AI-generated graphic, a button-pressed video, or a vibe-coded application, while sitting atop Mount Stupid.
Atop Mount Stupid, AI is pure magic!
The less skill you have, the more AI feels like magic. The more AI you use, the less you connect with the fundamental skills. The more AI you use, the more you atrophy even the skills you have. The more your skills atrophy, the more you risk climbing back atop Mount Stupid.
And I have a theory. In a corporation, which category of people is least in touch with fundamental skills? If you’re thinking “executives”, you're dang right! And who has the most influence? Executives! I’m not surprised that the last FOSE report calls out the executive-engineer perception gap.
‘A recurring, almost universal complaint was that boards and CEOs often believe “a product manager dumps a PRD into the magic machine and perfectly working software comes out”.’
Executives don’t have the time to struggle and learn fundamental skills. Their best value is in being fine-tuned decision engines. But here’s the problem with decision engines. They’re based on historical patterns. What if the patterns have no parallels in the present? That’s when you need humility, not overconfidence. Then again, there’s the first rule of the Dunning-Kruger club. Who’s going to break it to the bosses?
Generative AI is a wonderful, interesting technology; however, it has a long way to go before it drives productivity across industry. For starters, we need to think of it like normal technology, not magic. It has to be a means to an end, not the end itself. Second, we have to recognise that it isn’t a cheap alternative to hard-won skill, unless you’re looking for the average of average outputs. And third, we need a truth-telling environment where it’s OK to speak up if the emperor ain’t got no clothes on. Call out the slop. Chasten the boosters. Calm down the frenzy and experiment without FOMO. Perhaps then, after critical thinking is fashionable again, we may see some productivity gains from the stochastic parrots.