A Cost-Based Guide to Choosing the Right Approach
We talk about “AI” as if it were a single thing. It isn’t. A rules engine, a fraud model, and a large language model all get labeled AI, yet they differ radically in cost, risk, predictability, and governance. Treating them as interchangeable is how organizations end up with expensive, fragile systems that solve the wrong problem in the most complicated way possible.
The current rush to build everything around LLMs is a case study in solution-first thinking. Yes, large language models are impressive. They’re also the most expensive option in your AI toolbox, with meaningful risks around hallucination, data leakage, and non-deterministic behavior. For many problems, they’re overkill. For some, they’re actively wrong.
Most AI failures don’t come from bad algorithms. They come from choosing the wrong class of AI for the problem.
The Three Questions That Matter More Than Accuracy
Before evaluating any AI approach, ask three questions that cut through the hype:
What does it cost to create? This includes data preparation, expertise required, experimentation cycles, and infrastructure. Some approaches require internet-scale datasets and millions in compute. Others require a domain expert and a whiteboard.
What does it cost to use? Training a model once is different from running it millions of times. API fees, inference compute, and scaling economics vary dramatically across AI categories. An approach that’s cheap to build might be expensive to operate, and vice versa.
How predictable is the behavior? Can you explain why the system made a specific decision? Will it give the same answer twice? In regulated environments, predictability often matters more than capability.
These three questions map directly to the fundamental tradeoffs in AI system design. Get them wrong, and you’ll optimize for the wrong things.
Deterministic AI: Cheap, Boring, and Underrated
The least glamorous AI approaches are often the most valuable: rules engines, expert systems, and classical statistical models.
Rules engines encode human logic directly. No training data required. Near-zero runtime cost. Completely explainable. When a compliance check fails, you can point to the exact rule that triggered it. When pricing logic needs to change, you update the rule and deploy.
Expert systems add inference capabilities to rule-based approaches. They capture domain knowledge in structured ways that support consistent decision-making. Diagnostic systems, configuration tools, and troubleshooting guides often work better as expert systems than as machine learning models.
Classical statistical models (regression, Bayesian methods, probabilistic approaches) require modest amounts of labeled data and produce interpretable results. They’re mathematically grounded, data-efficient, and well-understood. For forecasting, risk modeling, and econometric analysis, they often outperform more complex approaches while remaining explainable.
The common thread: low creation cost, near-zero usage cost, and high predictability.
In regulated environments, boring AI often outperforms intelligent AI. A deterministic rules engine that you can audit, explain, and modify beats a sophisticated model that produces better average outcomes but occasionally fails in ways you can’t predict or explain.
The next time someone proposes an ML solution, ask whether a rules engine would work. You might be surprised how often the answer is yes.
Predictive AI: Where Most Real Value Lives
Traditional machine learning (random forests, gradient boosting, SVMs, and similar approaches) represents the sweet spot for most business prediction problems. Fraud detection, churn prediction, credit scoring, demand forecasting: these are the workhorses of enterprise AI.
The economics are straightforward. Creation cost dominates: you invest in labeled data, feature engineering, and model training. Once trained, usage cost is modest. You can run millions of predictions for pennies.
Supervised learning on structured data is a solved problem. The tooling is mature. The accuracy on tabular data is excellent. The failure modes are well-understood.
The real risk isn’t the algorithm. It’s drift. Models trained on historical data degrade as the world changes. The fraud patterns of 2023 aren’t the fraud patterns of 2025. Effective ML operations means monitoring for drift and retraining before performance degrades.
Unsupervised approaches (clustering, anomaly detection, dimensionality reduction) require no labels but produce results that are harder to validate. They’re discovery-oriented: useful for segmentation and pattern finding, but the business value is often indirect.
Shallow neural networks offer more expressiveness than classical ML while remaining more interpretable than deep learning. For many pattern recognition problems, they hit the right balance of capability and complexity.
The key insight for predictive AI: you’re buying a one-time creation cost and amortizing it over millions of low-cost predictions. This is the opposite of the LLM economic model.
Perceptual AI: Expensive to Build, Expensive to Run
Deep learning for vision, speech, and video represents a different cost profile entirely. CNNs, RNNs, and specialized transformers achieve state-of-the-art accuracy on unstructured data, but the economics are steep.
Training requires large datasets (often weakly labeled), significant GPU infrastructure, and extensive experimentation. This isn’t a weekend project. It’s a sustained investment in data, compute, and expertise.
Inference costs scale with volume. Running a vision model on every image, or a speech model on every audio clip, adds up quickly. The infrastructure requirements don’t disappear after training.
Reinforcement learning pushes these economics further. Training RL agents requires massive experimentation in simulated or real environments. The learning process is unstable, the results are hard to explain, and safety considerations add complexity. When it works (robotics, control systems, game playing), the results can be remarkable. But the path to getting there is expensive and uncertain.
The decision framework for perceptual AI is simple: these systems are justified only when humans cannot do the task at the required scale, or when the task is genuinely beyond human capability. If you’re automating something humans can do, make sure the accuracy gains justify the operational costs.
Generative AI: Usage Cost Is the Business Model
Large language models represent a fundamental shift in AI economics. The creation cost (pretraining on internet-scale data) has been abstracted away. You don’t pay to train GPT-4; you pay to use it.
This shifts AI from a capital expense problem to a variable cost problem. Every token in, every token out, is a charge. At low volumes, this feels cheap. At scale, it becomes existential.
Consider a customer service application processing 10 million conversations per month. At typical token pricing, you’re looking at significant monthly costs that scale linearly with usage. Compare this to a traditional ML model where inference costs are negligible after the initial training investment.
The economics of different LLM approaches vary:
Foundation models offer broad capability but high per-use costs. You’re renting general intelligence.
Fine-tuned LLMs improve domain accuracy and can reduce prompt length (lowering token costs), but add lifecycle complexity. You now own a model that needs versioning, monitoring, and periodic retraining.
Retrieval-Augmented Generation (RAG) reduces hallucination by grounding responses in retrieved documents. This adds system complexity and retrieval infrastructure, but produces more auditable outputs. For enterprise knowledge applications, RAG is often the right architecture.
Agentic systems orchestrate LLMs with tools and workflows. They handle multi-step tasks and unlock significant productivity gains, but compound the non-determinism problem. When an agent chains multiple LLM calls with tool use, the behavior becomes increasingly unpredictable. The security blast radius expands with each capability you add.
Beyond economics, generative AI introduces governance challenges that don’t exist with deterministic or predictive systems:
Hallucination means the system can produce confident, plausible, and completely wrong outputs. This isn’t a bug that will be fixed; it’s inherent to how these models work.
Data leakage means information from prompts can influence future outputs or be extracted by adversarial queries. Sensitive data requires careful architecture.
Non-determinism means the same input can produce different outputs. For many business processes, this is unacceptable.
None of this means LLMs are wrong. It means they require different governance than the AI systems that came before.
Why Hybrid Systems Win in the Real World
The most resilient AI systems are not the most intelligent ones, but the most constrained.
Hybrid architectures combine multiple AI approaches, using each where it’s strongest:
Rules for guardrails. Hard constraints that must never be violated get encoded as rules. An LLM can draft a response, but a rules engine validates it before delivery. A machine learning model can recommend an action, but business logic determines what’s permissible.
ML for prediction. When you need to score, rank, or forecast based on structured data, traditional ML offers the best accuracy-to-cost ratio. The model does the heavy statistical lifting; other components handle the rest.
LLMs for interface and orchestration. Natural language understanding, content generation, and workflow coordination are genuine LLM strengths. Use them where language is the problem, not where precision is the requirement.
This layered approach provides multiple benefits. You get explainability where you need it (the rules layer). You get accuracy where it matters (the ML layer). You get flexibility where it helps (the LLM layer). And when something goes wrong, you can isolate which layer failed.
In regulated domains, hybrid architectures are often the only defensible option. Pure LLM systems struggle to meet auditability requirements. Pure rules systems lack the flexibility to handle natural language input. Combining them gives you both.
The architectural complexity is real. You’re now operating multiple systems with different monitoring, versioning, and operational requirements. But for high-stakes decisions, this complexity is the cost of doing it right.
A Decision Framework for AI Selection
When evaluating AI approaches for a specific problem, work through these questions:
| Question | If Yes, Prefer |
|---|---|
| Must decisions be auditable and explainable? | Rules engines, classical ML |
| Need prediction at scale on structured data? | Supervised ML |
| Need natural language understanding or generation? | LLM with RAG |
| Operating in a regulated domain? | Hybrid with deterministic guardrails |
| Task requires human-like perception (vision, speech)? | Deep learning |
| Problem has clear optimization criteria? | Classical optimization or RL |
The goal is to find the cheapest, most predictable approach that solves the problem. Start simple. Add complexity only when simpler approaches demonstrably fail.
Stop Buying AI, Start Buying Capabilities
AI is not a monolith. It is a toolbox. The tools range from deterministic logic that costs nothing to run, to generative models that charge by the token. They range from fully explainable decision trees to neural networks that nobody fully understands.
Organizations that succeed are the ones that choose the cheapest, most predictable tool that solves the problem. They resist the urge to deploy intelligence where logic would suffice. They treat LLMs as a capability to deploy strategically, not a default architecture for everything.
The question isn’t whether your organization should use AI. It’s which AI, for which problem, at what cost, with what governance. Get that right, and AI becomes a competitive advantage. Get it wrong, and you’ve built an expensive system that’s hard to explain, harder to audit, and impossible to predict.
The boring choice is often the right one.
© 2026 Chris Bollerud, Bollosoft. Unauthorized reproduction prohibited.