Call a local model from Python

Check the server

Complete the Ollama quickstart first. Keep Ollama running and check that ollama list includes qwen2.5:1.5b. Python will send an HTTP request to 127.0.0.1:11434, the default local Ollama port. The example uses Python 3 and its standard library.

Make the request

Save the code as local_chat.py. Run python3 local_chat.py on macOS or Linux, or py -3 local_chat.py on Windows.

import json
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen

def chat(payload):
    request = Request(
        "http://127.0.0.1:11434/api/chat",
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urlopen(request, timeout=180) as response:
            return json.load(response)
    except HTTPError as error:
        raise SystemExit(f"Ollama returned {error.code}: {error.read().decode()}") from None
    except (URLError, TimeoutError) as error:
        raise SystemExit(f"Could not reach Ollama: {error}") from None


if __name__ == "__main__":
    result = chat({
        "model": "qwen2.5:1.5b",
        "messages": [
            {"role": "user", "content": "Give me two names for a plant journal."}
        ],
        "stream": False,
    })
    print(result["message"]["content"])

"stream": False asks for one JSON response. The generated text is inside message.content. Later lessons import the chat() function from this file to send their own requests.

Try it

Change the user message to request three names of at most three words each. Run the program again and compare. To try a multi-turn request, add the earlier user and assistant messages to the messages list before the follow-up question.

Check your understanding

Why can a standalone follow-up request lose the subject of a conversation?

The application must send the relevant prior messages as part of the new request.

Sources

• Ollama · Chat endpoint
• Ollama · Local FAQ
• Python · urllib.request