The Free Encyclopedia
Log in
Article History

Tool Calling & Structured Outputs with Local Models

Revision as of Jun 25, 2026 23:03 by migration.

Two of Ollama's most useful features turn a local model from a chatbot into a component you can wire into real systems: tool calling lets the model invoke functions you define, and structured outputs force generation to match a JSON schema. At theLAB we lean on both — tool calling is exactly how ADI's MCP servers (todo, notes, memory, files, media, mariadb) get exposed to the model.

This walkthrough covers both paths against Ollama's native /api/chat endpoint, with the local-model caveats that frontier-model tutorials skip.

01Tool calling via /api/chat

You pass a tools array alongside your messages. Each tool is a function declaration — a name, a description, and a JSON schema for its arguments. The model decides whether to call one and, if so, returns the call in message.tool_calls rather than answering directly.

import requests

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name"},
                "unit": {"type": "string", "enum": ["c", "f"]},
            },
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What's the weather in Austin?"}]
r = requests.post("http://localhost:11434/api/chat", json={
    "model": "qwen3",
    "messages": messages,
    "tools": tools,
    "stream": False,
})
calls = r.json()["message"].get("tool_calls", [])
# calls[0] -> {"function": {"name": "get_weather",
#              "arguments": {"city": "Austin", "unit": "f"}}}
Which models can do this Tool calling only works on tool-capable models — recent families like qwen3 and llama3.1 and newer. Older or very small models silently ignore the tools array and just answer in prose, so check the model card before you build on it.

Closing the loop

A tool call is a request, not an answer. You execute the named function yourself, append the result as a {role:"tool"} message, and call /api/chat again so the model can fold the result into its reply. Repeat until the model stops asking for tools.

# loop: POST -> if tool_calls, run each + append results -> POST again
while True:
    msg = post(messages, tools)["message"]
    messages.append(msg)
    if not msg.get("tool_calls"):
        break                       # no more calls -> final answer
    for call in msg["tool_calls"]:
        fn   = call["function"]
        out  = dispatch(fn["name"], fn["arguments"])   # run it
        messages.append({"role": "tool", "content": str(out)})

print(messages[-1]["content"])      # the model's final answer
Small models are shaky tool-pickers Smaller local models are noticeably worse than frontier models at choosing the right tool and filling arguments correctly — expect wrong tools, missing required fields, and malformed values. Mitigate it: keep the tool set small, give each tool a tight, unambiguous description and a strict schema (enums, required), and validate and repair arguments before you execute anything. Never hand a local model's raw tool output straight to a destructive operation.

02Structured outputs via format

When you don't need a function call — just clean, parseable output — use the format parameter. Set it to a JSON schema and Ollama's runtime constrains generation so the response is always valid JSON matching that shape. This is the right tool for extraction and classification, where you want fields, not prose.

schema = {
    "type": "object",
    "properties": {
        "sentiment": {"type": "string", "enum": ["pos", "neg", "neutral"]},
        "topics":    {"type": "array", "items": {"type": "string"}},
        "urgent":    {"type": "boolean"},
    },
    "required": ["sentiment", "urgent"],
}

r = requests.post("http://localhost:11434/api/chat", json={
    "model": "qwen3",
    "messages": [{"role": "user", "content": f"Classify: {review}"}],
    "format": schema,              # constrain output to this schema
    "stream": False,
})
import json
result = json.loads(r.json()["message"]["content"])   # always valid JSON
Two tools, two jobs Reach for tools when the model needs to act — query a DB, hit an API, mutate state. Reach for format when you need a parseable result out of one call. They compose: a tool's own output can itself be schema-constrained before you append it back into the conversation.

How this powers ADI

Every ADI MCP server is surfaced to the model through exactly the loop in section 01: the MCP tool definitions become the tools array, a tool call is dispatched to the matching server, and the result is appended as a tool message before the next turn. The schema discipline above is what keeps a 4B local model from fumbling those calls in production — the wiring details are in Building MCP Tools.