Local LLM Field Guide
Search

Intermediate15 min read

Measure the speed and recall of a vector index

Compare exact search with HNSW on 30,000 local vectors and measure how search effort changes recall, latency, and index size.

On this page

Make an exact baseline

The PDF search lab compares a query with every stored vector. Faiss can do that with a flat index. HNSW builds a graph and visits a subset of vectors when looking for the closest ones. Compare both on identical inputs so you can measure the neighbors the graph missed.

Use a Python 3 virtual environment and a platform with a wheel for the package below. Save and run the script. It creates 30,000 random unit vectors and 100 separate queries from a fixed seed. It uses inner product, which equals cosine similarity for unit vectors.

Download compare_vector_indexes.py
View compare_vector_indexes.py
import argparse
from statistics import median
from time import perf_counter

import faiss
import numpy as np

DIMENSIONS = 96
TOP_K = 10


def unit_vectors(generator, count):
    vectors = generator.standard_normal((count, DIMENSIONS), dtype=np.float32)
    faiss.normalize_L2(vectors)
    return vectors


def build(index, vectors, label):
    start = perf_counter()
    index.add(vectors)
    elapsed = perf_counter() - start
    size = faiss.serialize_index(index).nbytes / 1024**2
    print(f"{label}: {elapsed:.2f}s to build, {size:.1f} MiB serialized")


def measure(index, questions):
    index.search(questions[:5], TOP_K)
    results, times = [], []
    for question in questions:
        start = perf_counter()
        _, ids = index.search(question.reshape(1, -1), TOP_K)
        times.append((perf_counter() - start) * 1000)
        results.append(ids[0])
    return results, median(times)


def compare(count, queries, seed):
    faiss.omp_set_num_threads(1)
    generator = np.random.default_rng(seed)
    vectors = unit_vectors(generator, count)
    questions = unit_vectors(generator, queries)

    exact = faiss.IndexFlatIP(DIMENSIONS)
    graph = faiss.IndexHNSWFlat(DIMENSIONS, 16, faiss.METRIC_INNER_PRODUCT)
    graph.hnsw.efConstruction = 80
    build(exact, vectors, "Exact")
    build(graph, vectors, "HNSW")

    baseline, exact_time = measure(exact, questions)
    print(f"Exact search: {exact_time:.3f} ms median per query")
    for width in (10, 32, 96):
        graph.hnsw.efSearch = width
        results, elapsed = measure(graph, questions)
        hits = sum(len(set(expected) & set(observed)) for expected, observed in zip(baseline, results))
        print(f"HNSW efSearch={width:2}: recall@{TOP_K} {hits / (queries * TOP_K):.1%}, "
              f"{elapsed:.3f} ms median per query")


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Compare exact and graph-based vector search on the same inputs.")
    parser.add_argument("--vectors", type=int, default=30_000)
    parser.add_argument("--queries", type=int, default=100)
    parser.add_argument("--seed", type=int, default=42)
    args = parser.parse_args()
    if args.vectors < TOP_K or args.queries < 1 or args.seed < 0:
        parser.error(f"Use at least {TOP_K} vectors, one query, and a nonnegative seed.")
    compare(args.vectors, args.queries, args.seed)
python -m pip install "faiss-cpu==1.14.2"
python compare_vector_indexes.py

Read the recall and latency

For each query, count how many of the exact top ten neighbors also occur in the graph’s top ten, then divide by ten. The script averages that value over all queries. It uses one Faiss thread and times one query at a time after warming up each search setting.

You’re ready when

You have build times and serialized sizes for both indexes, plus query times and recall for three efSearch settings. In one Apple silicon run, recall rose from 18% at 10 to 66% at 96, while median HNSW search time rose from about 0.01 to 0.05 ms. Those values describe the random data and machine used for this lab.

Compare a smaller index

python compare_vector_indexes.py --vectors 3000 --queries 100

Compare the cost of the graph and the query times with the flat index at both sizes. The serialized size measures the saved representation. Peak RAM and end-to-end latency also include other work and need separate measurement.

Before choosing an index for documents, run this comparison on embeddings from your actual corpus and queries. Exact neighbor recall measures agreement with the flat index. Use labeled source passages to measure whether retrieval found the answer.

Check your understanding

What does a recall of 80% against exact neighbors tell you?

On average the graph returned eight of the exact top ten neighbors. It still takes a labeled retrieval test to see whether those neighbors answer the questions.

Sources

Search the guide