Remember when the easiest way to spot AI was counting fingers on a Midjourney prompt? Those days are mostly behind us. As Large Language Models (LLMs) get better at mimicking human output, the "tells" have become increasingly subtle. Enter the "LLM Smell"—a concept borrowed from software engineering's "code smells." It’s that nagging feeling that a piece of text or code, while technically functional, is just... off.
From Code Smells to Prose Smells
In programming, a code smell isn't necessarily a bug; it's a structural weakness that suggests deeper issues. Tech writers like Shiv (of Shiv After Dark) have started applying this to generative writing. When using LLMs to "polish" a math blog, for instance, the result often loses its soul, replaced by a generic, overly-earnest texture. It’s the linguistic equivalent of a beige room.
On platforms like Medium and Hacker News, these smells manifest as repetitive enthusiasm—think "I know!" or "You are so right!"—and a lack of the "lazy" human errors that actually give writing its character. When every sentence is perfectly balanced and every sentiment is relentlessly positive, the "AI smell" starts to waft off the screen.
The High Cost of Machine Gaslighting
It’s not just about aesthetics; "smelly" AI can be a productivity killer. Research featured on arXiv suggests that LLM performance on coding tasks is highly correlated with the quality of human-written code in similar scenarios. If the task is complex, the "smell" of poor logic intensifies.
Developers are also warning about "AI gaslighting." This occurs when an LLM confidently leads a user down a 30-minute rabbit hole for a problem that should have taken 30 seconds to fix. Even in specialized fields like healthcare, the "smell test" is becoming a vital part of evaluating whether a model is truly comprehending a task or just hallucinating with high fluency. Recent evaluations show that while LLMs have high recall in detecting issues, they often suffer from low precision, making our human "vibe check" more important than ever.
The Future of the Sniff Test
As we move toward a world of "agentic" coding harnesses and automated social media replies, our digital noses are getting sharper. We're learning to sniff out the astroturfing and the 100x engineer hype. Ultimately, identifying an LLM smell isn't about hating the technology; it's about knowing when to step back in and provide the human touch that a machine simply cannot replicate.
Sources
Media
