Local LLM Field Guide
Search

Advanced14 min read

Give a local model a read-only tool

Learn the request–execute–reply loop with a local dictionary and explicit argument validation.

What happens during a tool call?

The app sends a description of an available function. A compatible chat model can return a tool name and arguments. The app validates them, performs the operation, and sends its result back so the model can finish its answer.

Use a small read-only exercise. Expose a single function called lookup_hours, which looks up hours for a fictional library branch. Use a model and chat template with tool-calling support. Smaller models may omit a call or return the wrong arguments, so inspect the response before dispatch.

Describe and validate the tool

Build on the Python request lesson. Save this as library_hours.py beside local_chat.py. The first request asks for a tool call, then the script checks the name and arguments against the local data.

from local_chat import chat

HOURS = {"central": "09:00–17:00", "west": "10:00–16:00"}
lookup_tool = {
    "type": "function",
    "function": {
        "name": "lookup_hours",
        "description": "Look up opening hours by fictional library branch.",
        "parameters": {
            "type": "object",
            "properties": {
                "branch": {"type": "string", "enum": ["central", "west"]},
            },
            "required": ["branch"],
            "additionalProperties": False,
        },
    },
}
messages = [{"role": "user", "content": "Use lookup_hours for the central branch."}]
payload = {
    "model": "qwen2.5:1.5b",
    "messages": messages,
    "options": {"temperature": 0},
    "stream": False,
}
assistant = chat({**payload, "tools": [lookup_tool]})["message"]
calls = assistant.get("tool_calls") or []
if len(calls) != 1:
    raise SystemExit("Expected one tool call. Check model support or try again.")

function = calls[0].get("function", {})
args = function.get("arguments")
if function.get("name") != "lookup_hours" or not isinstance(args, dict):
    raise SystemExit("Unexpected tool call.")
branch = args.get("branch")
if set(args) != {"branch"} or not isinstance(branch, str) or branch not in HOURS:
    raise SystemExit("Branch unavailable.")

messages.extend([
    assistant,
    {"role": "tool", "tool_name": "lookup_hours", "content": HOURS[branch]},
])
print(chat(payload)["message"]["content"])

The second request includes the assistant’s tool call and the app’s tool result in the message history. Run the file and check that the model reports 09:00–17:00. If it fails to call the tool, inspect assistant and try a larger tool-capable model that fits your machine.

Test unsupported inputs

Ask for “north,” a branch the tool cannot look up. A model might decline, send an unsupported branch, or guess a supported branch. Run each case and use HOURS as the reference. In an application where the branch comes from a form, compare the tool argument with the branch the user actually selected.

Keep execution in the app

Dispatch through a list of allowed functions with validated inputs. Give each tool only the access it needs and require user approval for actions that modify data or send messages.

Check your understanding

What actually reads the branch hours from the dictionary?

The application executes the allowed lookup after validating the model’s tool request.

Sources

Search the guide