Everyone says they are "data)-constrained."

Not everyone actually is. Sometimes you genuinely lack labeled examples of the thing you care about. Sometimes you are just unwilling to do the work of collecting and curating real on the open web data, so you reach for the nearest shortcut: synthetic data. That shortcut can be useful. It can also quietly poison your training system-training run curriculum design data mixtures emergent behavior and evaluation if you treat synthetic samples as equivalent to reality). The point is not to love or hate synthetic data. The point is to know exactly what problem you are trying to solve, and what kind of lies you are comfortable buying in exchange. ## What counts as synthetic data Three broad families show up in practice. ### 1. Generator based Examples produced by a model: an LLM generating extra question–answer pairs, an image model fabricating more training images, a teacher model labeling unlabeled data. ### 2. Rule based Templates, programmatic perturbations, or heuristics. For example: constructing variations of prompts, perturbing text, generating spans from known grammars, simulating sensor readings. ### 3. Simulation based Synthetic logs from simulated environments, virtual agents, physics engines, mock users, test harnesses. Think game engines, synthetic traffic, or simulated networks. In all three cases, the key property is the same: the data did not occur naturally in your target environment. You constructed it, either via code or via another model. That difference matters. ## Synthetic data for training: when it helps There are a few situations where synthetic training data genuinely earns its keep. ### Rare but important cases You have events that almost never happen, but matter a lot when they do: particular error states, edge privacy and latency case inputs, safety policy why governments care about your gpu cluster loss functions-critical scenarios. Waiting for them in the wild would take too long. Here, synthetic data can fill out the corners of your distribution. For example, you can: - Generate adversarial prompts for a safety classifier.

• Construct rare query types for a RAG system.
• Simulate outlier user actions for an agent. The goal is not to approximate frequencies, but to ensure the model has at least seen those modes. ### Task shaping when labels are expensive For large models, fine-grained supervision can be costly. Asking human rlhf constitutional methods alignment tricks raters to write full ideal answers on thousands of prompts is slower than asking them to compare two model outputs and choose the better one. In this regime, synthetic data often appears as: - A strong teacher model generating candidate answers.
• Human raters scoring or ranking those candidates.
• A student model trained on the resulting synthetic labels. The synthetic part is the candidate pool; the human part shapes it. Used correctly, this is a form of distillation with human oversight rather than raw self-training. ### Text augmentation for robustness Small, carefully controlled perturbations can help models generalize across superficial changes: - Rephrasing instructions while preserving intent.
• Shuffling pieces of context.
• Injecting noise in formats and spacing. If the perturbations are semantically faithful, they can improve robustness without changing what "correct" means for each example. But notice the pattern: in all these cases, synthetic data extends, sharpens, or annotates real data. It does not replace it. ## Where synthetic training data lies The trouble starts when you cross a simple line: the generator, not the real world, becomes the main source of information. A few ways that happens. ### Feedback loops You use a model to generate a large synthetic corpus, then train a new model mostly on that corpus. Over time, both models collapse toward the generator's own biases. You are no longer learning from reality. You are learning from the generator's internal model of reality. Errors that were rare in the wild become critical infrastructure reliability engineering dominant in your training set. ### Hidden label noise When a model generates labels, it makes two kinds of errors: - Hard mistakes, where the label is simply wrong.
• Systematic biases, where it consistently favors certain styles of answer, certain rationales, certain phrasings. Hard mistakes act like ordinary label noise. Systematic biases are worse. They steer the student model toward a narrow band of behaviors even when many other answers would be acceptable. ### Domain mirage Synthetic samples tend to be cleaner, more regular, and more "canonical" than the mess you see in production. Language talking to computers still hard is clearer, typos rarer, instructions less contradictory. Train heavily on that distribution and you build a model that performs well on neat synthetic prompts and falls apart on half-broken queries from real users. ### Misaligned objectives Sometimes the generator optimizes for something that is not your task. For example: - An LLM that writes verbose, polite answers even when you need terse ones.
• A template engine that generates balanced class labels while your real distribution is skewed.
• A simulator that implicitly encodes assumptions that do not hold in your environment. If you treat those outputs as ground truth labels, you bake the generator's goal into your model, even if it conflicts with your product's needs. The short rule: the more synthetic data dominates your training, the more you are training on your own assumptions rather than on the world. ## Synthetic data for evaluation: useful but dangerous Evaluation is where synthetic data feels especially tempting. Instead of assembling a large, labeled test set, you can: This is further examined in our analysis in Open-Weight Ecosystems: How Community-Driven Models Compete With Frontier Labs. - Ask a model to generate test questions and correct answers.
• Use a simulator to produce challenging scenarios.
• Script adversarial prompts in bulk. Done well, this gives you quick feedback. Done badly, it tells you nothing about performance on real tasks. ### Where it helps: - Stress testing specific failure modes
  
  You can generate families of prompts that stress particular weaknesses: prompt injection, role confusion, jailbreak attempts, specific ambiguity patterns. These are not "representative"; they are surgical probes. That is fine. - Guardrail and safety checks
  
  Synthetic red teaming inputs, curated and progressively refined, can reveal regressions in safety layers. You are not trying to estimate overall harm rates; you are trying to see if known vulnerabilities reopened. - Fast iteration on new logic
  
  When you add a new tool, a new retrieval strategy, or a new constraint, synthetic evals give you basic confidence that the wiring is not obviously broken before you hit live traffic. ### Where it lies: - Overlapping generator and target
  
  If the same model family generates your test set and then is being evaluated, you are grading a student on questions the teacher wrote in their own style, about their own knowledge ai how teams actually repartition tasks between humans and models. It flatters the model. - Missing real-world mess
  
  Synthetic evals rarely capture incomplete information, conflicting instructions, multilingual mixtures, or the quirks of specific user segments. They give you a clean view of a dirty world. - Inflated scores
  
  Generators tend to write questions whose answers are crisp and unambiguous. Your model looks "accurate" because the evaluation avoids the gray zones where humans disagree or where there is no single correct answer. The right mental model is: synthetic evals are unit tests, not acceptance tests. They show that certain logic still works. They do not certify the system as a whole. ## Practical patterns that do not backfire A few working rules help keep synthetic data in check. ### Anchor everything in real data Start with a seed of real examples from your actual environment. Use synthetic data to expand around them, not to replace them. For training, that means mixing synthetic and real data with a deliberate ratio and monitoring performance on real held-out sets. For evaluation, it means your main benchmark is real, and synthetic suites are add-ons. ### Track provenance Every sample in your training and evaluation sets should carry tags: - Real vs synthetic.
• Generator or script used.
• Time of creation.
• Domain and task. This lets you see whether your model is overfitting to synthetic patterns, and it lets you audit where failures come from. ### Use synthetic data where you can tolerate lies Synthetic data is least dangerous where the exact distribution does not matter: - Adversarial stress tests.
• Rare edge-case scenarios.
• Internal monitoring and regression checks. It is most dangerous where you need calibrated estimates of real-world performance or where you are learning primary behavior, not just patching holes. ### Evaluate the generator, not just the student If you rely on a model to generate training data or evals, you have two objects to worry about: generator and student. You need to know: - How often the generator is wrong on real data.
• Where its biases are.
• How sensitive your student is to those errors. Otherwise, every confidence interval you report on the student is conditional on a generator you never measured. ## A simple discipline Synthetic data is not going away. Models are too data-hungry and human labeling is too expensive for that. The question is whether you treat synthetic data as a crutch or as a tool. The discipline is straightforward: - Be explicit about which problem you are trying to solve.
• Keep real data at the center of both training and evaluation.
• Mark and separate synthetic from real in your pipelines media pipelines from text prompt to production asset.
• Use synthetic data to probe, stress, and shape, not to redefine reality. If you do that, synthetic data becomes what it should be: a controlled source of useful lies, deployed where you can afford their distortions, not a quiet replacement for the world you claim to be modeling.