Read dense, mixture-of-experts, and distilled model cards
Distinguish stored weights, active computation, and training methods before deciding whether a model will fit.
Read the architecture label
In a dense Transformer, each token goes through the same feed-forward network in a given layer. A mixture-of-experts (MoE) layer has several feed-forward networks called experts. A router selects a subset for each token. Other parts of the network, such as attention, can be shared.
The number of total parameters describes the stored weights. Activated parameters estimates how many participate in processing a token for a particular architecture. Different tokens can route to different experts. For local use, plan around the full model file and how the runtime places it in memory. Active parameters alone give neither the download size nor a speed guarantee.
Compare two model cards
| Variant | What the publisher reports |
|---|---|
| Qwen3-32B | Dense Transformer with 32.8 billion parameters. |
| Qwen3-30B-A3B | MoE with 30.5 billion total and 3.3 billion activated parameters. |
Using the rough weight calculation at four bits, the raw estimates are 16.4 GB for 32.8 billion and 15.25 GB for 30.5 billion. The quantized file listing and runtime memory report are better guides to a specific deployment.
Read the training label
Distillation trains a student model using behavior from a teacher. It can use teacher output probabilities or generated examples. The student has its own architecture, weights, model card, and license. A distilled model can have a different set of strengths from its teacher.
Distillation describes training. Quantization describes the precision used to represent values. A distributed model can use both techniques, so check the model card and the file variant separately.
Pick a downloadable dense model, an MoE model, and a distilled model from the linked cards or repositories. Record the base architecture, total and active parameters where reported, exact download size, and the tests that resemble your task. Use these facts to choose a small candidate you can run.
Check your understanding
Which number should go into the rough four-bit storage estimate for a model with 30.5 billion total and 3.3 billion activated parameters?
Use 30.5 billion for raw weight storage. Check the actual download and leave room for the runtime and context.
Sources
Progress is saved in this browser.