Local LLM Field Guide
Search

Intermediate13 min read

Require a key for a local model server

Start a loopback llama.cpp server with authentication and verify that model requests require the key.

Generate a lab key

Use a fresh working folder on macOS or Linux for the commands below. Keep the GGUF file from the llama.cpp setup guide available. Generate a random key in a file that other accounts cannot read. This key is for the local lab. Keep it out of source control and chat transcripts.

umask 077
python3 -c 'import secrets; print(secrets.token_urlsafe(32))' > model.key
llama-server --host 127.0.0.1 --port 8080 --alias qwen-local -m "qwen2.5-1.5b-instruct-q4_k_m.gguf" --ctx-size 4096 --parallel 2 --metrics --no-slots --api-key-file model.key

If the model is in a different folder, put its full path after -m. On Windows, use the same server flags with a key file in a folder that only your account can read.

Check both client states

In a second terminal, run the first two commands with no key. The health endpoint should report readiness and the model listing should return HTTP 401. From the same lab folder, load the key and use compatible_chat.py from the local API compatibility lab.

curl --fail http://127.0.0.1:8080/health
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/v1/models
export LOCAL_MODEL_API_KEY="$(cat model.key)"
python3 compatible_chat.py --base http://127.0.0.1:8080/v1 --list
python3 compatible_chat.py --base http://127.0.0.1:8080/v1 --model qwen-local "Name one service a public library provides."

PowerShell can set the same variable with $env:LOCAL_MODEL_API_KEY = (Get-Content model.key -Raw).Trim(). Use curl.exe for the HTTP check and py -3 for the Python commands if needed.

You’re ready when

The model endpoint returns 401 without the key. With the key, the same client lists qwen-local and prints a reply. The server metrics endpoint uses the same key.

Connect an authorized client

For a second computer, use the SSH forwarding pattern and change the host-side destination port to 8080. Keep the model listener on loopback. Give the key only to the authorized client through your usual private secret-sharing method.

A shared key grants the same server access to every holder. For individual user permissions, TLS, and quotas, use a service in front of the model that enforces those controls. Stop the lab server when finished. On macOS or Linux use unset LOCAL_MODEL_API_KEY to clear the variable in the client shell.

Check your understanding

Why can the health request succeed while the model listing returns 401?

llama.cpp exposes readiness without a key and requires the key for the model API in this setup.

Sources

Search the guide