Glossary

A system’s decision to defer an answer or a label when it lacks enough evidence. An application can send the case to a person for review.

A small set of learned weights used with a base model to change its behavior. LoRA trains adapters by learning low-rank updates to selected layers.

A search method that aims to find close vectors without comparing every stored vector. Measure how many exact neighbors it recovers.

An operation that scores other positions against a query and mixes their value vectors. A mask can limit which positions contribute.

A model produced by pretraining, often optimized to continue text. An instruction-tuned variant has received additional training to follow requests.

A credential a client can present in an HTTP Authorization header. Anyone with the token can use the access it grants.

A pair of consecutive tokens. A bigram language model estimates the next token using only the previous token.

A restriction on attention that lets a position use the current and earlier input tokens and hides the later ones.

A model-specific way to turn chat roles and messages into one tokenizable input, usually with control markers for the start and end of each turn.

Assigning an input to one or more defined labels, such as sorting a library request into a return, renewal, or equipment reservation.

Having more than one request in progress at once. A server can queue them, process them in parallel, or combine some of their computation.

The amount of tokenized information the model can use for a request. Budget for instructions, conversation history, source text, and the reply.

A metric that accumulates events, such as generated tokens. Calculate a difference between samples from the same process to count events over that interval.

A model that reads two inputs together, such as a question and a passage, and scores the pair.

A loss that penalizes a model for assigning low probability to the target. For a single correct token, it is the negative log of the predicted probability for that token.

Finding passages written in a different language from the query. An embedding model can map related text across languages to nearby vectors.

A family of generative methods that learn to reverse a noising process. A text-to-image model can use a prompt to guide generation from noise.

Training a student model using outputs or other signals from a teacher model. The student keeps its own architecture and weights.

A value computed from a document’s bytes with a hash function such as SHA-256. Store it with a citation to identify the exact file version used.

The date when a policy or edition starts to apply. Use it with a question’s date to select the applicable source.

A vector that represents an input numerically. An embedding model can place related passages near one another so an application can retrieve them.

A repeatable way to measure model behavior on a set of tasks with defined expectations.

How many of the facts a question requires have supporting evidence in the retrieved passages or answer.

A summary made from selected sentences or excerpts of the source. Each selection can be traced back to the original text.

Additional training on examples that adjusts a model’s behavior. It can change the model’s weights or train a separate adapter.

Searching an index of terms extracted from documents. A search engine can rank matches and return the passages around matching terms.

A metric that records a value at the time it is measured, such as the number of active requests. It can rise or fall between samples.

A file format that packages model weights with metadata for inference engines such as llama.cpp. A GGUF file can hold weights in several different precisions.

Providing evidence to a model and checking that its answer follows from that evidence.

A generated claim that is unsupported or incorrect, even when the text sounds confident.

Examples reserved for the final comparison. Keep them out of training and out of the decisions used to tune the prompt or adapter.

Hierarchical navigable small world. A graph-based index that searches for nearby vectors by moving through connections between stored items.

Retrieval that combines two or more methods, such as a keyword index and a vector similarity search.

Using a trained model to produce an output from an input. Local inference performs that calculation on your machine.

Additional training that teaches a model to respond to requests in instruction or conversation formats.

Memory used to retain intermediate attention values for tokens the model has already processed. It typically grows with context and active requests.

The time you wait for a result. For chat, time to first token and time to finish describe different parts of the wait.

Short for large language model. It learns statistical patterns in training data and uses them to generate or process language.

A hostname that points back to the machine making the request. The IPv4 loopback address is 127.0.0.1.

Low-rank adaptation. A fine-tuning method that learns compact updates to selected model layers while keeping the original weights frozen.

Content an MCP server exposes at an identifier called a URI. A client can read the content and decide whether to give it to a model.

An architecture in which a router selects a subset of specialized networks for each input token. The total stored weights can be much larger than the active subset.

A framework for array computations and machine learning on Apple silicon. MLX LM uses it to run and fine-tune language models.

Documentation that describes a model’s intended uses, limitations, licensing information, and sometimes evaluation results.

MCP. A protocol clients can use to discover and access tools, resources, and prompts exposed by a server.

A model that accepts or generates more than one kind of data, such as text and images.

Newline-delimited JSON. Each line is a separate JSON value, which lets a client process a stream one event at a time.

Optical character recognition. It converts the visible text in an image or scan into machine-readable characters.

A text file stored beside a scan or PDF. It contains the recognized text for a known source page.

Assigning part of an inference workload, such as model layers, to a different processor or memory pool.

Weights made available for download under a license. That license determines what you may do with the model and its derivatives.

Learning patterns specific to the training examples that make predictions worse on other examples. Compare training metrics with held-out metrics to look for it.

A numerical value learned during training. A model label such as 7B indicates roughly seven billion parameters.

Machine-readable characters embedded in a PDF page. A scan may contain only an image, which needs OCR to recover text.

The exponential of mean negative log probability on observed tokens. Lower values mean the model assigns higher probability to those tokens on the same evaluation.

Representing weights or other values with lower precision to reduce memory use. Quality and speed depend on the method and hardware.

Splitting a question into separate parts so the application can retrieve and check the evidence each part requires.

Retrieval-augmented generation. An app finds relevant source passages and supplies them to the model with the question.

The share of a reference set recovered in the first k results. State which reference set you use, such as relevant documents or exact nearest neighbors.

A way to combine ranked lists by giving each result a contribution based on its position in each list.

A repeatable check for behavior that broke before. Keep the test after the fix so a later change can be checked against the same case.

A second stage that scores and reorders items returned by a retriever. It can only rank the candidates it receives.

Software that loads a model and carries out inference. A chat interface or application can sit on top of it.

Retrieval based on similarity in meaning. An embedding model can match related phrases even when the query and passage use different words.

A rule that names the source that governs a fact when multiple documents mention the same subject.

Turning spoken audio into text. A transcription model may also estimate the timing of words or segments.

Generating audio from text. A text-to-speech system uses a voice model and settings such as speaking rate.

An encrypted SSH connection that can forward a local port to a service the SSH host can reach, including a service bound to its loopback address.

A process can read a stream called standard input and write to a stream called standard output. In MCP’s stdio transport, a client starts a server and exchanges messages using those streams.

A response with a defined shape, such as a JSON object that conforms to a schema.

A sampling setting that changes how concentrated the next-token choices are. Lower values usually make output less varied.

Term frequency–inverse document frequency. A weighting method that gives more weight to terms that occur in a particular document and are less common across the collection.

The amount of work completed per unit of time, such as replies per second across all clients or generated tokens per second.

The elapsed time before the first audio chunk is ready. State the start point and distinguish chunk readiness from when the listener hears it.

A unit of text handled by a model’s tokenizer. It may represent a word, part of a word, punctuation, or whitespace.

A model-generated request for an application to run a named function with specific arguments. The application decides what to execute.

Finding which parts of an audio recording contain speech. A transcription pipeline can use it to skip long stretches of silence.

Memory used by a graphics processor. Dedicated GPUs typically have their own memory pool. Integrated systems may share system memory.

Learned numerical values stored in a model file. A runtime loads these values to perform inference.

The number of word insertions, deletions, and substitutions needed to match the reference, divided by the number of words in the reference.