What Is AI Currently Bad At?
Knowing what AI is bad at is as important as knowing what it's good at — because the expensive failures happen when buyers overestimate AI's current capabilities and deploy it where it reliably fails.
- basics
- ai fundamentals
- limitations
- failure modes
Most of the expensive AI failures I've seen weren't from bad technology. They were from people deploying AI in categories where it reliably fails, because no one had told them clearly what those categories are. Vendors don't lead with limitations. Demo environments are curated. And so buyers go live with implementations built on an inflated picture of what the technology can do.
Here's the corrective. These are the things AI is genuinely bad at right now, with enough specificity to be useful.
Novel reasoning and reliable math
Novel reasoning is the most important one. AI is very good at pattern matching against its training data. Give it a problem that has a close analogue in the millions of documents it was trained on, and it will handle it with remarkable fluency. Give it a genuinely novel problem, one that has no close training analogue, and it will fail in ways that look like success. It will produce a confident, coherent-sounding response that applies the wrong pattern. The failure is invisible unless you already know the answer. For most business workflows this isn't an immediate concern, because most business tasks are variations on problems that have been solved before. But in domains where your competitive advantage depends on novel judgment — creative strategy, novel negotiation positions, product decisions without precedent — AI is not the judgment layer. It's a research assistant at best.
Reliable math is next, and it surprises people. General-purpose large language models make arithmetic and algebraic errors at a rate that should concern anyone deploying them for financial calculation without a verification mechanism. This isn't a minor edge case; it's a consistent failure mode. If you're building a workflow where AI needs to compute something, the right architecture adds a dedicated calculation tool or asks the model to generate code that runs the calculation rather than computing it directly in the language layer. AI generating Python that calculates, then running it, is far more reliable than AI doing the calculation in prose.
Long documents and unsignaled uncertainty
Context degradation over long documents is underestimated. The advertised context windows are real: many current models can technically process 100,000 words or more in a single session. But "can process" and "processes reliably" are different claims. Performance degrades as documents approach context limits. The model's attention effectively compresses across a long document, and sections that appear later receive less precision than sections that appear early. For a 200-page vendor contract, sections in the final quarter will have meaningfully higher error rates than sections in the first quarter. If you're building a document analysis workflow, test it on your longest documents, not your median ones. That's where it will fail first.
Uncertainty signaling is a quiet failure mode with large consequences. AI doesn't reliably know what it doesn't know, and it doesn't reliably say so. It produces confident-sounding outputs regardless of whether the underlying information is solid or fabricated. A human expert in over their head will often signal discomfort, hedge, ask for clarification. An AI will not do this reliably unless specifically architected to do so. This is why output review matters at consequential decision points, and why "the AI said so" is not an audit trail.
Consistency and chained logic
Consistency in unstructured situations is a practical limitation that shows up in implementation. AI in a workflow with well-defined inputs and clear output specifications behaves predictably across thousands of runs. AI given wide latitude with variable inputs behaves inconsistently across runs. The same prompt with slightly different framing can produce meaningfully different outputs. This isn't a reason to avoid AI, but it's a reason to invest in prompt engineering and input standardization before deployment rather than after. The implementations that underperform are usually the ones where the input definition was treated as an afterthought.
Multi-step logical chains are the last failure mode worth naming explicitly. AI is good at one-step reasoning: given this input, what's the most likely output? It's less reliable at three-step, four-step, and five-step deductive chains where an error in step two compounds through steps three, four, and five. Workflows that require AI to make a sequence of inferential judgments before reaching a conclusion will accumulate errors that wouldn't appear in any individual step tested in isolation. If your workflow requires chained logic, test the whole chain, not the individual steps.
What to ask before the build
What this means practically: before you bring a workflow to an implementation agency, ask them which of these failure modes your candidate workflow is exposed to. The good agencies have answers. They'll tell you how they handle the math layer, how they manage context for long documents, how they define and test for "consistent behavior" in your specific input environment. An agency that hasn't thought about failure modes before the build isn't the right agency for the build. That's the question to ask before the proposal, not after the go-live.
If you want that question answered for your specific situation, the Forge Playbook does it. Answer a few questions about your business and we'll put together a tailored outline of which workflows are worth automating and what a realistic budget looks like for each. Free, no obligation, takes about three minutes.
