Local LLM Field Guide
Search

Beginner8 min read

How chat templates turn messages into tokens

Trace system, user, and assistant messages through a model’s chat template and diagnose a common formatting problem.

Look at the roles

A chat client sends an ordered list of messages. A system message can describe the task, user messages carry requests, and assistant messages can carry replies already in the conversation. The model still receives one token sequence. A chat template converts the list into the format the model saw during instruction tuning.

Templates may insert markers for roles and turn boundaries and open a new assistant turn before generation. The exact syntax depends on the model. A system message can guide behavior, while the application still controls what gets saved or executed.

Read a local model’s template

After the Ollama quickstart, run this in a terminal. It prints the stored template for the installed model.

ollama show --template qwen2.5:1.5b

Look for the loop over messages, the role field, and any text added around a message. This is the template source. Ollama applies it when you send the chat request.

You’re ready when

You can point to where the template uses a message’s role and identify the marker that starts a turn. For the example tag, look for im_start and im_end.

Try it with two turns

  1. Ask the model to remember the sample phrase “green notebook.”
  2. Send a second request containing that user message, the assistant’s reply, and “What phrase did I give you?”
  3. Now send only the last message in a new request. Compare the two answers and inspect which messages your client included.

If a raw completion endpoint echoes role markers or repeatedly writes both sides of the conversation, check which template the runtime applied. Start with the runtime’s normal chat endpoint and the template for the exact model variant.

Check your understanding

Should you type a model’s role markers into the user field when using a chat endpoint?

Send normal user text. Let the runtime insert the role markers using the model’s chat template.

Sources

Search the guide