The letter in three claims
The open models letter, Open Weights and American AI Leadership (July 24, 2026), makes three claims. Each one depends on a technical term most readers skim past. This guide slows down on those terms.
You cannot judge a claim about "open" AI until you know what is actually open.
What a model actually is
A neural network is a function with billions of adjustable knobs. Each knob is a parameter, stored as a weight: a floating-point number inside a matrix. Training nudges those numbers until predictions match examples. The weights are what remains afterwards: the learned knowledge, stored as numbers.
A model is a recipe with 117 billion knobs. The weights are the knob settings.
So "releasing weights" means publishing the trained result itself. Not a product, not an API, and not a description of the result.
Open weights vs. open source
This is the most important distinction in the letter and the most misused phrase in AI. The Open Source Initiative (OSI) defines "open source" strictly. Many model releases fall short of that, which is why people say "open weights".
| Term | What you get | Example |
|---|---|---|
| Open weights | Downloadable trained parameters. Usage terms vary. | Llama, Qwen, DeepSeek-R1 distills, gpt-oss |
| Open source (OSI) | Code, and enough information to rebuild the work, under an approved licence. | Linux, PyTorch |
| Closed model | Access via API or app. No weights. | Most frontier models sold as a service |
Training vs. inference
Training creates the weights. It is expensive and done once or rarely. Inference uses the weights to generate answers. It happens billions of times and is what most organisations actually pay for.
When you run a model yourself, the first question is almost always: will it fit in memory? Every engineer should be able to do this arithmetic in their head: parameters × bits ÷ 8 bytes.
# Rough weight-only memory estimate (no KV cache, no activations) def weight_memory_gb(params_billion, bits_per_param): return params_billion * 1e9 * bits_per_param / 8 / 1e9 for bits in [16, 8, 4]: print(f"21B model @ {bits:>2}-bit: {weight_memory_gb(21, bits):5.1f} GB") 21B model @ 16-bit: 42.0 GB 21B model @ 8-bit: 21.0 GB 21B model @ 4-bit: 10.5 GB
This arithmetic explains why gpt-oss-20b, with 21 billion total parameters, is reported to need about 16 GB of memory. Its MoE design and compact format bring the footprint down to laptop range. Efficiency in the letter's sense is a memory and cost story as much as a quality story.
Inference is where AI becomes an operating cost. Memory decides which hardware can afford it.
Distillation: teaching with a big model's answers
Knowledge distillation trains a smaller "student" to imitate a larger "teacher". Hinton, Vinyals and Dean formalised it in 2015 (Distilling the Knowledge in a Neural Network). The key insight: a teacher's soft probabilities carry more information than its final answer. If a teacher says an image is 70% dog, 25% wolf and 5% cat, those ratios teach the student how dogs relate to wolves.
The distillation loss, in code
The classic loss combines two terms: match the teacher's softened distribution, and still learn the true labels. The temperature T softens both distributions. The T² factor keeps gradient sizes comparable.
import torch import torch.nn.functional as F def distillation_loss(student_logits, teacher_logits, labels, T=2.0, alpha=0.5): # 1. soft targets: match the teacher's softened distribution soft = F.kl_div(F.log_softmax(student_logits / T, dim=-1), F.softmax(teacher_logits / T, dim=-1), reduction="batchmean") * (T * T) # 2. hard targets: still learn the true labels hard = F.cross_entropy(student_logits, labels) return alpha * soft + (1 - alpha) * hard torch.manual_seed(0) loss = distillation_loss(torch.randn(4, 3), torch.randn(4, 3), torch.tensor([0, 2, 1, 0])) print("loss:", round(loss.item(), 4)) loss: (one scalar; exact value depends on the seed)
What the numbers say: DeepSeek-R1 and its distilled family
DeepSeek-R1 (January 2025) is the clearest recent example of distillation at scale. The authors released R1 plus six dense models distilled from it: 1.5B, 7B, 8B, 14B, 32B and 70B, built on Qwen and Llama bases. Their headline finding: distilling a large model's reasoning into a small model beat running reinforcement learning on that small model. For Qwen2.5-32B, RL alone reached about 47.0% on AIME 2024, while distillation from R1 reached 72.6%.
| Model | Params | AIME 2024 | MATH-500 | GPQA Diamond | LiveCodeBench |
|---|---|---|---|---|---|
| R1-Distill-Qwen-7B | 7B | 55.5% | 92.8% | 49.1% | 37.6% |
| R1-Distill-Qwen-32B | 32B | 72.6% | 94.3% | 62.1% | 57.2% |
| R1-Distill-Llama-70B | 70B | 70.0% | 94.5% | 65.2% | 57.5% |
Why this is in the letter
The letter calls distillation "a widely used technique for model improvement, evaluation, and validation". It also separates the technique from "unlawful efforts to extract value from closed models". These are different things. Distillation is a method. Whether a particular use is lawful depends on licences, terms of service and law. This guide covers the method only.
Mixture of Experts and quantization
A dense model uses every parameter for every token. A Mixture of Experts (MoE) model splits its feed-forward layers into many experts, and a small router picks only a few for each token. Total size can be large while the work per token stays small.
This is why the gpt-oss numbers look odd at first glance. Total parameters describe memory. Active parameters describe work per token. Always report both.
Quantization: trading precision for reach
Quantization stores weights with fewer bits. Figure 3 showed the size effect. Quality loss depends on the method and the task, so measure after quantizing rather than assuming it is free.
Safety, security and red teaming
Safety is about a model behaving as intended: refusing harmful requests, avoiding fabricated claims. Security is about attackers: stealing weights, extracting data, injecting malicious instructions. A model can be safe in conversation and still insecure once deployed.
Red teaming is structured adversarial testing: trying to make a model fail so the weaknesses get fixed before users find them. Benchmark contamination happens when test questions leak into training data and inflate scores.
The letter's core safety argument: open weights let many independent teams test the same model. Closed models can only be tested by their owner.
The counterargument, stated fairly
The same openness that enables testing also enables misuse. Once weights are public, the developer cannot retract them, and anyone can fine-tune them, including for harm. OpenAI's gpt-oss release addressed this directly: it reported testing deliberately harmful fine-tuning and said the models did not reach its high-risk thresholds in the areas it assessed. That is a specific, measured claim. It does not mean all open weights are safe.
Sovereignty, lock-in and the application layer
Lock-in happens when switching providers is too costly: prompts, fine-tunes, data formats and workflows are tied to one API. Portability is the opposite. Sovereign AI generally means an organisation or country can run, inspect and control the AI it depends on, without relying on one external provider.
The application layer is where products are built on top of models: retrieval systems, agents, domain tools. The model layer is the models themselves. The letter's point is that competition at both layers matters, and open weights let many companies compete at the application layer without first training a frontier model.
Lock-in is a cost you pay later. Portability is a choice you keep now.
Four case studies
Case 1 · GPT-2 and the staged release (2019)
OpenAI initially withheld the full GPT-2 model, citing misuse concerns, and released it in stages over the following months while studying its effects. The debate was about how much to release and when, not whether openness has value. OpenAI later described gpt-oss (2025) as its first open-weight language models since GPT-2.
Key terms: staged release, misuse risk.
Case 2 · DeepSeek-R1 and its distilled family (January 2025)
DeepSeek released R1 with open weights and six distilled dense models. According to the authors, the distilled 32B model outperformed o1-mini on several reasoning benchmarks, and the distilled 14B model beat QwQ-32B-Preview by a large margin. The practical lesson: a well-distilled 7B to 32B model can run on hardware many developers already own.
Key terms: distillation, active parameters, benchmark evaluation.
Case 3 · gpt-oss (August 5, 2025)
OpenAI released gpt-oss-120b and gpt-oss-20b under Apache 2.0, its first open-weight language models since GPT-2. The 120B model activates 5.1B of 117B parameters per token and reportedly runs on a single 80 GB GPU. The 20B model has 21B total parameters, activates 3.6B, and reportedly needs about 16 GB of memory. OpenAI said the release was reviewed by external experts, and that the models did not reach high-risk thresholds in its preparedness evaluations, even after adversarial fine-tuning.
Key terms: open weight vs open source, MoE, Apache 2.0, preparedness evaluation.
Case 4 · The open-source ancestor
The letter opens with open-source software, and the analogy is historically grounded: open-source code now runs much of the internet's infrastructure. The lesson most often drawn is that inspectability and many contributors raise quality. The comparison is imperfect, though. Model weights are the result of training, not human-readable source code, which makes independent auditing harder in practice.
Key terms: open source, auditability, community review.
Try it yourself
A ten-minute checklist that turns this guide into practice:
- Pick one open-weight model and read its licence before anything else.
- Compute its weight memory at 16-bit, 8-bit and 4-bit with the snippet above.
- Check whether it is dense or MoE. If MoE, record total and active parameters.
- Run the distillation loss on a toy dataset and watch how
Tchanges the student. - Take one benchmark claim and find its source. Note the sampling method.
Glossary
| Term | Plain-English meaning |
|---|---|
| Parameter / weight | An adjustable number inside a model. The learned knowledge lives here. |
| Open weights | Trained parameters anyone can download, inspect, modify and run. Licence terms vary. |
| Open source (OSI) | Code and enough information to rebuild the work, under an approved licence. |
| Frontier model | The most capable models available at a given time. |
| Closed model | Accessed only via API or app. Weights are not published. |
| Inference | Running a trained model to produce outputs. |
| Distillation | Training a student to imitate a teacher's outputs or probabilities. |
| Fine-tuning | Further training a pretrained model on task-specific data. |
| Quantization | Storing weights at lower numerical precision to cut memory and cost. |
| Mixture of Experts (MoE) | An architecture where a router activates only some expert sub-networks per token. |
| Active parameters | Parameters used for one token. In MoE models, this is smaller than the total. |
| Red teaming | Structured attempts to make a model fail so weaknesses can be fixed. |
| Benchmark contamination | Test data leaking into training data, inflating scores. |
| Lock-in | Being unable to switch providers without high cost. |
| Sovereign AI | Running and controlling AI without depending on one external provider. |
| Application layer | Products built on top of models, such as search, agents and domain tools. |
Sources
Figures were checked against the sources below at the time of writing. Benchmark scores change with evaluation settings, so re-check any number before you cite it in production work.
- Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv:1503.02531.
- DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. https://arxiv.org/abs/2501.12948
- DeepSeek-R1-Distill model cards, Hugging Face (benchmark tables for 7B to 70B variants). https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
- OpenAI (August 5, 2025). Introducing gpt-oss. https://openai.com/index/introducing-gpt-oss/
- InfoQ (August 2025). OpenAI Releases gpt-oss-120b and gpt-oss-20b, Open-Weight Language Models for Local Deployment.
- The Register (August 5, 2025). OpenAI makes good on its name, launches first open weights language models since GPT-2.
- Open Source Initiative. The Open Source AI Definition.
- Open Weights and American AI Leadership (July 24, 2026), the letter this guide is based on.
Send feedback
Found a wrong number, a confusing term, or a better example? Tell me. Every note goes straight to the author.