LLM-First or Code-First? Where Intelligence Belongs in Production AI

How to decide what the model should handle, what your code should handle, and how to connect the two.

A few years ago, the architecture of an AI application look like: send a prompt to a large language model -> get a response -> show it to the user. That’s not the whole story anymore now a days. The models are asked to interpret intent, retrieve information, pick tools, call APIs, make plans, and run multi-step workflows.

That shift has split the field into two – LLM-First or Code-First

In an LLM-first architecture, the model is in central and decides what happens next. It reads the request, chooses a tool, decides the order of operations, checks intermediate results, and changes course when it needs to.

In a code-first architecture, software/code stays in charge of sequencing, business rules, validation, permissions, and execution. The LLM here is like a specialist the code calls when language understanding or generation is needed.

People love to argue about which is better. I think that’s the wrong argument. The better question is where each kind of intelligence belongs. The strongest production systems I’ve seen are rarely purely one or the other. They mix probabilistic reasoning with deterministic control, and they do it deliberately.

Why LLM-First Is So Appealing

Traditional software work beautifully when you can spell out the requirements. For example, a user picks a product, enters an amount, and submits a payment. You define the allowed states, the validation rules, the error conditions, and the transaction sequence in code. Done.

Natural language doesn’t cooperate like that. Imagine a user typing: “Find the transactions that look unusual, explain what happened, and tell me what I should investigate first.”

There is no fixed path through that request. Here the system has to decide what “unusual” means, figure out which data matters, maybe call several tools, weigh the response, and write an explanation a person can use. No team of engineers/no code is going to anticipate every phrasing and every combination of asks ahead of time.

It is where an LLM plays a vital role, acting as a flexible reasoning layer between human language and your deterministic services. It’s also why agents are getting so much attention. Google Cloud’s agentic AI architecture guidance describes an agent as an application where an AI model acts as the reasoning engine, while tools let it reach external systems and data.

Anthropic’s Building Effective AI Agents guidance makes a distinction I keep coming back to. In a workflow, models and tools follow paths that your code defines. In an agent, the LLM directs its own process and decides how to use its tools. The same guidance recommends starting with the simplest architecture that solves the problem, instead of adding agentic complexity by reflex. I’d underline that advice twice.

The Limits of “Let the Model Decide”

A model can reason about what should happen. Reasoning is not the same as enforcing a rule.

For example take a financial workflow. An LLM might be great at understanding “send the same amount I sent last month to the same vendor.” But should it also decide whether the transfer is authorized, calculate regulatory limits, verify account ownership, override a security policy, and execute the transaction?

Probably not. These jobs are deterministic, testable, auditable, and enforceable, and that’s exactly what traditional software is good at. The risk grows as models get access to tools. OWASP’s generative AI security guidance flags excessive agency as a significant risk: giving an LLM-based system more functionality, permissions, or autonomy than its task needs. A strange or manipulated model output is one thing when it produces text. It’s a much bigger thing when the model can act in the real world.

None of this means models should never take actions. It means model autonomy should be bounded by deterministic authority.

Code-First Still Matters

With AI moving so fast, it’s easy to feel that conventional engineering has gone out of style. I’d argue the opposite. AI makes good deterministic systems more important, not less.

Code is still the right choice whenever a task demands exact repeatability. Authentication is the simplest example. A model should not “reason” about whether someone has admin privileges. Your application should ask an authoritative identity and access-management system. The same goes for monetary calculations, entitlement checks, data validation, regulatory constraints, transaction limits, schema validation, and anything irreversible. These need explicit contracts, not best guesses.

This lines up with broader governance thinking. The NIST AI Risk Management Framework asks organizations to manage AI risk across design, development, deployment, and use. Its companion Generative AI Profile adds that generative systems may need extra oversight, documentation, review, and controls, depending on the risk involved.

So I find it helpful to split every design decision into two questions:

What should happen?

and

What is allowed to happen?

An LLM can often help with the first. Deterministic systems should usually own the second.

The Hybrid Pattern: Reason Probabilistically, Execute Deterministically

For most enterprise applications, the practical answer is a hybrid. The LLM works as an interpretation and reasoning layer. Deterministic services work as the execution and enforcement layer.

Here’s an example. Say an AI assistant helps developers spin up temporary API-testing environments, and a developer types: “Give me a sandbox for the customer onboarding workflow.”

The LLM can interpret that, work out which workflow is likely meant, read the documentation, and suggest which APIs are probably relevant. But actually creating the environment shouldn’t hinge on free-form generated text. Code can confirm the requested APIs exist, validate their contracts, check authorization, enforce resource limits, generate an approved configuration, and run the deployment.

The rough division looks like this:

• The LLM: understand, reason, classify, propose, summarize.
• The code: validate, authorize, calculate, persist, enforce, execute.

Each side does the work it’s best at, and neither is asked to fake the other’s strengths.

Boundaries Matter More as Agents Get More Powerful

This separation becomes more important as we move from assistants to agents. An assistant that gives a bad answer inconveniences someone. An agent with write access to production can cause a much bigger mess.

The fix isn’t necessarily to strip out autonomy. It’s to add autonomy gradually while keeping explicit control points in place. Google’s guidance on multi-agent systems recommends pairing dynamic AI behavior with deterministic security controls, observability, clearly defined autonomy, and human oversight for business-critical scenarios.

Human approval can also be built into the workflow itself instead of being an informal safety net. Microsoft’s agent framework documentation, for instance, supports tool calls that pause until a person explicitly approves the requested operation.

The principle is simple: the higher the consequences of an action, the stronger the deterministic controls around it should be.

Five Questions to Ask Before Handing a Task to an LLM

When I’m deciding whether a component should be LLM-first or code-first, I run through these:

1. Does the task have one objectively correct answer? If so, lean toward deterministic code. Tax calculations, permissions, and schema validation shouldn’t change because a model read them differently today.
2. Does it involve ambiguous language or unstructured information? If so, an LLM may add real value.
3. What happens if the model gets it wrong? The right architecture for a meeting summary is very different from the right architecture for initiating a payment.
4. Can the output be validated independently? LLM-generated plans get much safer when deterministic rules can check the resulting action before it runs.
5. Does this really need an agent? If you already know the steps, a regular workflow with a few targeted LLM calls is usually simpler, cheaper, easier to test, and easier to operate.

That last question deserves extra attention. Agents are powerful precisely because they can handle situations where you can’t predict every step. But if you can predict the steps, turning them into an open-ended reasoning problem often adds variability without adding intelligence.

Reliability Is an Architectural Property, Not a Prompt

Plenty of teams start out trying to improve reliability almost entirely through prompt engineering. Prompts matter, but they can’t carry the whole load.

A production system should assume that model output will sometimes be incomplete, malformed, unexpected, or just wrong. The OWASP Top 10 for LLM applications lists risks like prompt injection and improper output handling, which reinforces a key habit: treat model output as untrusted input to downstream systems, not as instructions to run automatically.

That changes the question you ask. Instead of “How do I write a prompt that always makes the model follow the rule?”, ask “How do I design the system so the rule can’t be broken even when the model makes a mistake?”

That’s a software architecture problem, not a prompting problem. A prompt can tell an agent not to perform an unauthorized action. An authorization service can actually stop it. Those two controls are not equivalent.

Beyond the Debate: Intent-First Systems

Thinking about all this, I’ve come to believe the LLM-first versus code-first debate points toward a third idea: intent-first architecture.

In an intent-first system, the application starts by understanding what the user is trying to accomplish. That’s where an LLM is most valuable, because people are rarely precise about what they want. From there, the system steadily converts that fuzziness into structured, deterministic operations.

A request like “Help me resolve the customer’s payment issue” might turn into a pipeline: understand intent, retrieve the transaction, identify the failure reason, recommend a fix, request approval, execute the approved operation.

Some of those stages benefit from language-model reasoning. Others should be fixed services. The architecture isn’t defined by whether AI or code “wins.” It’s defined by where uncertainty is acceptable.

The Bottom Line

As models improve, it will be tempting to give them control over bigger and bigger slices of the stack. Sometimes that will be the right call. In other systems, the most sophisticated design will be the one that deliberately gives the model less authority.

Production AI engineering, in the end, is about putting intelligence at the right boundary. Use language models where interpretation, reasoning, synthesis, and adaptation create value. Use deterministic software where consistency, authorization, precision, and enforcement matter. Then connect the two through narrow, observable, well-tested interfaces.

The future of enterprise AI probably isn’t purely LLM-first or purely code-first. It’s LLM where uncertainty calls for intelligence, and code where certainty calls for control.

That distinction may matter far more than which model you pick.