How an AI agent actually works: build the loop behind every framework

Strip away every framework, and an AI agent is one loop: show the model the conversation so far, let it either answer or request a tool, run the tool, append the result, repeat. That’s it. LangChain, CrewAI, the harness inside Claude Code — all wrap this loop with conveniences. Build it once bare and you’ll never be intimidated by an agent framework diagram again.

Step 1 — The anatomy of one turn

%% caption: The agent loop — every framework is a variation of these four boxes
flowchart LR
    H["Conversation history<br/>prompt + past turns"]:::system --> M{"Model decides:<br/>answer, or call a tool?"}:::agent
    M -- "tool request" --> T["Your code runs the tool"]:::workflow
    T -- "result appended<br/>to history" --> H
    M -- "final answer" --> D(["Done"]):::outcome

The model never executes anything — it only emits text saying what it wants. Your code decides whether to honor that request. Keep that split in your head; it’s also where all agent security lives.

Step 2 — Mock model, real loop

As in our orchestration tutorial, we use a deterministic mock so the loop is what you’re debugging. Create agent.py:

import json

TOOLS = {
    "get_weather": lambda city: f"18°C and clear in {city}",
    "get_time": lambda city: f"14:32 in {city}",
}

class MockModel:
    """Emits tool requests as JSON, then a final answer — deterministically."""

    def next_action(self, history: list[dict]) -> dict:
        tool_results = [m for m in history if m["role"] == "tool"]
        if not tool_results:
            return {"type": "tool_call", "tool": "get_weather",
                    "args": {"city": "Zurich"}}
        if len(tool_results) == 1:
            return {"type": "tool_call", "tool": "get_time",
                    "args": {"city": "Zurich"}}
        facts = "; ".join(m["content"] for m in tool_results)
        return {"type": "answer", "content": f"Here's Zurich right now: {facts}"}

def run_agent(model, user_message: str, max_turns: int = 10) -> str:
    history = [{"role": "user", "content": user_message}]
    for turn in range(1, max_turns + 1):
        action = model.next_action(history)

        if action["type"] == "answer":
            return action["content"]

        tool = TOOLS.get(action["tool"])
        if tool is None:
            result = f"error: unknown tool {action['tool']!r}"
        else:
            result = tool(**action["args"])

        print(f"[turn {turn}] {action['tool']}({action['args']}) -> {result}")
        history.append({"role": "tool", "content": result})

    return "Stopped: hit the turn limit without a final answer."

if __name__ == "__main__":
    print(run_agent(MockModel(), "What's it like in Zurich right now?"))

Run it:

python agent.py

You’ll see two tool turns, then the synthesized answer. Read run_agent again — those ~20 lines are the agent. Everything else in the ecosystem is packaging.

Step 3 — Notice the three safety valves

Even this toy loop ships the three controls every production agent needs:

1. max_turns — agents must have a budget, or a confused model loops forever.
2. The tool allowlist — TOOLS.get(...) means the model can request anything but only run what you registered. The unknown-tool branch feeds the error back instead of crashing — models correct themselves when shown their mistakes.
3. The history log — the full trajectory is right there to print, store, or audit. Frameworks call this “tracing”; you just built it with a list.

Step 4 — Swap in a real model

The loop doesn’t care where next_action gets its decision. To use a real model, write a class with the same method that calls a chat API with tool definitions attached. The version below uses only the standard library and the OpenAI-compatible /chat/completions format. Most servers and providers speak that format, including llama.cpp, Ollama and vLLM locally and several hosted APIs, so one class covers all of them. Save it as real_model.py next to agent.py:

import json, os, urllib.request

from agent import run_agent

TOOL_SCHEMAS = [
    {"type": "function", "function": {
        "name": name, "description": desc,
        "parameters": {"type": "object",
                       "properties": {"city": {"type": "string"}},
                       "required": ["city"]}}}
    for name, desc in [("get_weather", "Current weather for a city"),
                       ("get_time", "Current local time in a city")]
]

class OpenAICompatibleModel:
    """next_action() backed by any /v1/chat/completions endpoint."""

    def __init__(self, base_url: str, model: str, api_key: str = ""):
        self.base_url, self.model, self.api_key = base_url, model, api_key
        self.calls = []  # the tool calls we returned, in order

    def next_action(self, history: list[dict]) -> dict:
        # Rebuild the chat transcript: each tool result follows the call that produced it.
        messages, calls = [], iter(self.calls)
        for m in history:
            if m["role"] == "user":
                messages.append(m)
            else:
                call = next(calls)
                messages.append({"role": "assistant", "content": None, "tool_calls": [call]})
                messages.append({"role": "tool", "tool_call_id": call["id"],
                                 "content": m["content"]})

        req = urllib.request.Request(
            f"{self.base_url}/chat/completions",
            data=json.dumps({"model": self.model, "messages": messages,
                             "tools": TOOL_SCHEMAS}).encode(),
            headers={"Content-Type": "application/json",
                     "Authorization": f"Bearer {self.api_key}"})
        with urllib.request.urlopen(req, timeout=120) as r:
            msg = json.load(r)["choices"][0]["message"]

        if msg.get("tool_calls"):
            call = msg["tool_calls"][0]          # one call per turn keeps the loop simple
            self.calls.append(call)
            return {"type": "tool_call", "tool": call["function"]["name"],
                    "args": json.loads(call["function"]["arguments"] or "{}")}
        return {"type": "answer", "content": msg["content"]}

if __name__ == "__main__":
    model = OpenAICompatibleModel(
        base_url=os.environ.get("BASE_URL", "http://127.0.0.1:8080/v1"),
        model=os.environ.get("MODEL", "local"),
        api_key=os.environ.get("API_KEY", ""))
    print(run_agent(model, "What's the weather and the local time in Zurich right now?"))

Run it against a local model (no key needed). We used llama-server with gpt-oss-20b; see run agents locally to set one up:

BASE_URL=http://127.0.0.1:8080/v1 python real_model.py

[turn 1] get_weather({'city': 'Zurich'}) -> 18°C and clear in Zurich
[turn 2] get_time({'city': 'Zurich'}) -> 14:32 in Zurich
**Zurich – Current Status**
- **Weather:** 18 °C, clear skies
- **Local time:** 14:32 (2:32 pm)

Or point it at a hosted provider’s OpenAI-compatible endpoint. We also ran it against Anthropic’s (BASE_URL=https://api.anthropic.com/v1 MODEL=claude-haiku-5-5 API_KEY=$ANTHROPIC_API_KEY) and got the same two tool calls. The exact wording of the final answer will differ from run to run; the trajectory is what to check.

run_agent didn’t change, and neither did the three safety valves. The model now chooses the tools; your code still decides what runs. In production you would use the provider’s own SDK, validate the tool arguments against the schema, and retry malformed JSON (structured outputs covers that). From here, go two directions: give one agent better tools via MCP, or scale to several agents with the orchestrator pattern.

Where to go next

Before you trust a real model in this loop, give it a test suite: evaluate your agent builds golden-task checks on exactly this (answer, trajectory) shape. For the protocols that connect agents to tools, other agents and users, see the agent protocol stack.