Before you approve the agent, ask for its training data disclosure

The pilot review has a slide for the model, a slide for the benchmark score, and a slide for the license. Somebody always asks about the weights: open or closed, which cloud, what it costs per thousand calls. Nobody in the room asks the one question that predicts whether this agent survives its first bad week: what data taught it what to do when the workflow breaks?

That gap is the useful part of "Data for Agents," a post NVIDIA and Hugging Face published on July 8. It is a first-party partner post about NVIDIA's Nemotron open-data work, so read it as an argument from a vendor, not as independent proof. But the question underneath the promotion belongs in every agent approval meeting: the model is not the whole decision. The data that shaped its behavior is a separate decision, and most approval rooms have no place to write it down.

The autocompleter test

NVIDIA's framing starts with a sentence worth repeating in the approval meeting: "Building AI agents is hard, because the real world does not behave like a benchmark." A clean eval can still miss the first customer who phrases a request sideways, the first tool call that times out, or the first workflow nobody put in the happy-path demo. The post sets the bar plainly: "An agent that can't recover from a broken API call, or a workflow it has never seen, is not really an agent. It is an autocompleter with tools."

That turns the review from a demo question into an evidence question. Not, does it work when the script is kind? What taught it to recover when the script disappears? Recovery is not only a property of the model architecture. It is also a property of the traces, failures, examples, and reviews the model was shown before it reached your workflow.

NVIDIA's own position is that weights do not answer this. "Open weights matter. But for agents, weights are only part of the story," the post argues. "Reproducibility also depends on the datasets, curation choices, training recipes, and evaluation methods behind the model." Then comes the sentence that should change how teams buy agents: "Agent behavior needs to be inspectable." Not just the output. The behavior: what the agent does when a system call fails, when a customer asks something messy, when the workflow takes a turn the benchmark never saw.

Where synthetic data enters

The honest complication is that a lot of data teaching an agent to recover is not purely real. Simulated users, generated tool-failure scenarios, synthetic labels, and model-written traces are how teams get coverage without waiting for every failure mode to happen to a real customer first. NVIDIA does not pretend otherwise, and it does not make synthetic data a cure-all. The post says synthetic data "can reduce risk, but it does not remove the need for grounding, lineage, curation, evaluation, and human judgment."

The phrase to borrow is "synthetic thresholds": the points where data can no longer be treated as purely real. Real workflow logs, human feedback, model-generated traces, simulated users, and synthetic labels can all get blended together. From the outside, the row may look like training data either way. NVIDIA's proposed fix is not to avoid synthetic data. It is to document "what was generated, what was grounded, what was reviewed, and what the data is meant to test."

NVIDIA's example is Nemotron-Personas, described as synthetic personas built to mirror official regional demographic and geographic statistics. The Data for Agents post says the collection had reached its tenth country and represented more than 2.4 billion people after a prior VivaTech launch. That is a concrete design claim: ground the synthetic slice against public statistics, then let local reviewers inspect whether it resembles real people well enough to test against. It is also still NVIDIA describing NVIDIA's own data, which is why the disclosure matters. The grounding source should be inspectable, not absorbed as a reassurance.

The broader ecosystem signal is real but should not be overread. NVIDIA says it had 74 papers accepted at ICML 2026, and that roughly 2,000 accepted papers cite NVIDIA GPUs while 145 cite Nemotron models and datasets. The tooling is public too: NeMo Data Designer, the Apache-2.0 licensed project NVIDIA describes as a way to generate high-quality synthetic data from scratch or seed data, had 2,082 GitHub stars when checked on July 9. That shows the machinery exists and is active. It does not prove the agent in front of you was built with the same care.

The sheet nobody hands you at approval time

Put those threads together and a gap opens up that belongs to buyers and operators, not only model builders. A model card may tell you the model's capabilities and limitations. A license tells you what you can do with the software. Neither tells you what data shaped this agent's judgment: which real workflows it learned from, which parts were synthetic, who reviewed the mix, what was left out, and which customer promise the data actually supports.

That's upstream of three questions BaristaLabs readers may already know to ask. An Agent BOM tells you what code and tool dependencies an agent pulls in. A capability shelf tells you which tools an agent can discover and call at runtime. Workflow-receipt evals test whether an agent's actions match what it claims to have done. All three assume the agent already exists and ask what it does or touches. The training data disclosure sheet asks what taught it to behave that way in the first place.

Scroll sideways to see all 3 columns.

Field	What it pins down	Why the model card cannot answer this for you
Real workflow source	The logs, tickets, transcripts, tasks, or traces used, with a date range and owning system	Recovery behavior only reflects situations the agent actually saw or was asked to practice
Synthetic data share and generator	Which slices were generated, by what process, and roughly how much of the training or eval set they represent	Synthetic data is not automatically bad; invisible synthetic data is the problem
Synthetic threshold	The point where real records, human feedback, model-generated traces, simulated users, or labels begin to blend	The reviewer needs to know when a row stops being a record and starts being a construction
Grounding source	The public, internal, demographic, domain, or customer data used to anchor a synthetic slice	A synthetic persona grounded in nothing is a guess with a friendly face
Failure traces	Broken API calls, timeouts, confusing handoffs, malformed inputs, and unfamiliar workflows the agent practiced	Edge-case confidence should point to examples, not adjectives
Persona assumptions	Which user groups the data claims to represent, and who checked that representation	A recovery path trained on the wrong user can still fail politely
Human reviewer	The named person or role that reviewed the data mix, curation choices, and documented exclusions	A disclosure nobody owns will not survive the first incident review
Exclusion log	Languages, regions, workflows, systems, or customer cases deliberately left out	What the agent was never shown is often where the rollout breaks first
Eval link	The specific test or benchmark this data slice supports, and whether it resembles the workflow being approved	A score answers one question. The sheet tells you whether it was your question
Customer promise link	The SLA, support commitment, compliance boundary, or internal promise this data is supposed to support	The agent can be well tested and still untested against the promise your business is about to make
Refresh date	When the data mix gets reviewed again as workflows, tools, users, or policies change	A launch disclosure is a snapshot, not a standing guarantee

Two of these a demo cannot answer for you, no matter how good it looks: the exclusion log and the customer promise link. A vendor demo shows what the agent does well. It does not show what the training or eval data left out, and it rarely checks that omission against the promise you are about to make to your own customers. If those two rows are blank, you do not have an approved agent. You have a well-rehearsed one.

What this post proves, and what it does not

It is worth being clear about the source. "Data for Agents" is NVIDIA and Hugging Face advocating for NVIDIA's open-data approach, published the same week as an NVIDIA research recap citing NVIDIA paper counts. Independent coverage of the specific post was thin when this article was written. That does not make the recovery-behavior test or synthetic-threshold framing wrong. It means the right way to use the post is not as proof that agent data governance has arrived. Use it as a reason to ask a sharper approval question.

That question is especially useful because it travels beyond NVIDIA. A vendor selling a support agent, an internal team building a finance agent, and a developer tooling group testing an agentic workflow all owe the same answer in different language: what real traces did this system learn from, what synthetic traces filled the gaps, what was reviewed, what was excluded, and which promise does the evidence actually support?

Filling in the disclosure sheet on one agent

You do not need NVIDIA's dataset scale to use this. You need one agent, one workflow, and a willingness to ask for an answer the vendor or internal team may not have prepared. Start with the failure-trace row. Ask for one specific broken API call, timeout, malformed input, or unfamiliar workflow the agent was trained or evaluated against. If the answer is a general assurance that it handles edge cases, the row is still empty.

Then ask for the exclusion log. Which language, region, workflow, customer segment, system, or failure mode was deliberately left out? Does that gap overlap with the work you are about to approve? Finally, ask for the customer promise link. If the agent is going to answer customers faster, reduce escalations, draft invoices, update tickets, or move data between systems, which part of the data supports that promise?

This is the same discipline behind keeping AI workflows inside a defined boundary before they touch production systems and customer promises. The model is one decision. The runtime is another. The data that shaped the agent's behavior is a third, and it is often the one nobody in the room has asked to see.

If you are about to approve an agent for real work, bring one agent and one workflow to the session. We'll fill in the disclosure sheet with you: the real traces, the synthetic share, the exclusions, the named reviewer, and the customer promise. That is the same process automation work we bring to any AI system before it earns a place in production.