LettuceDetect v2 checks grounded answers one unsupported span at a time

LettuceDetect v2 gives Semantic Router a precise verification job: compare supplied evidence with a generated answer and return the exact answer spans that the evidence does not support. That output matters when one fabricated method name, field, number, or clause can make an otherwise useful answer costly to deliver.

This explainer shows how the served detector path works, what the project’s measurements cover, and where its authority ends. It also gives technical leaders a decision rule: use a separate verifier when the cost of one missed unsupported span exceeds the latency, review work, and operating burden of another model call.

The output is evidence, an answer, and an unsupported span

The detector receives grounding evidence and the answer to check. The evidence can include source code, tool output, retrieved passages from retrieval-augmented generation (RAG), documents, Markdown, or tables. RAG supplies selected source material to a model before generation; the detector then checks whether the resulting answer stays within that material.

Consider this constructed API example. The supplied code contains client.fetch(id), while the answer tells a developer to call client.fetch_async(id). A useful verifier should identify fetch_async as the unsupported substring, classify the type of problem, and preserve the rest of the answer instead of reducing the result to one pass-or-fail score.

LettuceDetect v2 is designed for that narrow comparison. Its output can include a verbatim span, a category and subcategory, and an optional explanation. Semantic Router maps the returned text to character offsets in the answer, which lets another system display or process the exact part that needs attention.

The detector’s scope ends at the supplied evidence. A detector can mark an answer as grounded when that evidence is wrong, stale, or incomplete. It cannot confirm whether a claim is true in the wider world when the fact needed to check it is absent from the supplied evidence. This is the same practical boundary that appears in document-grounded answer evaluation: better checking does not repair a bad source set.

The endpoint path runs after generation and before delivery

Semantic Router added the endpoint detector backend in pull request 2526, which merged on August 3, 2026. The project post published on August 5 describes the endpoint as an optional path. In its published configuration, the in-process candle backend remains the default.

The demonstrated endpoint serves KRLabsOrg/lettucedect-v2-qwen-2b, a fine-tuned Qwen3.5-2B generative detector, through an OpenAI-compatible server such as vLLM. “OpenAI-compatible” describes the request and response interface used here. It does not show that every server or API feature will behave the same way.

The dependencies run in a fixed order. First, the router captures grounding evidence from tool messages or RAG passages. The main model generates an answer from that material. The router then sends the evidence, request, and answer to the detector endpoint in one structured call.

The detector returns typed spans as strict JSON. Semantic Router matches each returned string to the answer and recovers its character offsets. The configured response path can then add the x-vsr-response-warnings: hallucination header or return an annotated answer that exposes the flagged text.

The endpoint also depends on exact detector inputs. The detector was fine-tuned with a frozen system prompt and a specific evidence serialization format. Those inputs are compatibility requirements. Prompt or formatting drift can reduce detection quality without causing a transport or parsing error, so teams must version and test them with the model and router.

The reported measurements cover two different evaluations

The paper reports the first four rows for the fine-tuned Qwen3.5-2B detector served as KRLabsOrg/lettucedect-v2-qwen-2b. These rows use character-overlap span-F1 on the paper’s span-labeled test data. Semantic Router reports the last row from a separate 10,000-example HaluEval QA harness. Its precision, recall, and F1 are example-level metrics, and its p50 measures the reported endpoint serving path.

Scroll sideways to see all 3 columns.

Evidence set	Project-reported measurement	Evidence boundary
Unified span test	Span-F1 0.689	Combined span-labeled test sources
Code-agent test	Span-F1 0.602	Answers grounded in repository evidence and coding-agent traces
Tool-output test	Span-F1 0.719	Answers checked against tool observations
README test	Span-F1 0.866	Answers grounded in project documentation
HaluEval QA harness, 10,000 examples	Precision 0.962; recall 0.769; F1 0.855; p50 about 116 ms	Semantic Router’s separate example-level harness; p50 is limited to its reported endpoint serving path

The paper’s span metrics use character overlap. Span precision divides the predicted unsupported characters that overlap labeled spans by all characters predicted as unsupported. Span recall divides that same overlap by all characters labeled as unsupported. Span-F1 is the harmonic mean of span precision and span recall. The HaluEval harness uses whole answers instead: an answer is predicted positive when the detector returns at least one span. Its precision, recall, and F1 therefore measure example-level detection. The HaluEval F1 and p50 are not extensions of the paper’s span test.

Most labels in the paper’s benchmark come from synthetic injection. The authors start with grounded correct answers and use a model to insert small, localized hallucinations. The code test split adds model-assisted evidence-based review, but it was not independently annotated by multiple people. The cited sources do not report an independent production replication. The reported accuracy and the roughly 116 ms p50 apply only to the project’s evaluations and reported serving path, not to a different server, prompt, evidence mix, hardware configuration, or workload.

A detector signal does not choose the workflow action

A returned span says that the detector found text unsupported by the supplied evidence. It does not decide whether the application should warn, annotate, regenerate, request review, block delivery, or continue. That consequence belongs to the application because the cost of a false alarm and the cost of a missed span depend on the work.

For a low-consequence drafting tool, an annotation may give the user enough information to check the source. For code changes, financial fields, policy clauses, or other costly details, a hit should stop unqualified delivery of the affected content. The application can retry generation with the same evidence, send the span and evidence to a reviewer, or block the response until the issue is resolved. The chosen behavior should be explicit in the AI workflow controls around the detector.

Detector failure needs its own branch. The published implementation treats transport failures, non-2xx responses, and malformed output as detector errors instead of clean verdicts. Semantic Router can degrade without blocking traffic when the endpoint is unavailable, but that router behavior does not settle the delivery decision for the surrounding application.

If a workflow justified the verifier because one unsupported detail is costly, silently delivering an unverified answer during an outage defeats that reason. A suitable high-consequence policy is to retry within a fixed limit and then queue, route to review, or block. Continuing can be reasonable for a lower-consequence path, provided the response carries a visible unverified state and the application records the detector failure.

Local evaluation must measure errors, delay, and availability together

A production decision should use labeled traces from the actual workflow, including clean answers and answers with unsupported spans. Labels need exact spans where possible because example-level labels cannot show whether the detector found the costly field or only a nearby sentence. Production model evaluation should also preserve the evidence that was available at generation time, since later or repaired evidence changes the task.

For a local span-level comparison, calculate character-overlap precision and recall with the same units and denominators used above. Report their harmonic mean as span-F1. If the operational decision applies to a whole answer, report example-level precision, recall, and F1 separately. Add the review time caused by false positives and the expected cost of missed spans. A detector with attractive F1 can still be a poor fit if its false alarms consume scarce reviewers or its misses cluster around the fields with the highest consequence.

Track end-to-end latency, including queue time, network time, detector inference, and retries. Report the median and a tail percentile across the same fixed comparison window. Also publish detector availability as valid verdicts divided by all answers selected for verification. Transport errors, non-2xx responses, timeouts, and malformed output belong in that denominator as unavailable verdicts; none should count as a clean answer.

Use one fixed trace set or one fixed calendar window for the comparison. Hold the generator, grounding evidence, frozen detector prompt, evidence serialization, and application action policy constant while detector configurations change. Otherwise, a better score could come from different evidence or a different generator instead of the detector. If the work must connect retries, review queues, and delivery systems, BaristaLabs can help implement that bounded path through process automation.

The architecture is justified only when the separate decision is useful

LettuceDetect v2 makes span-level verification available as a separate served model in Semantic Router. That separation is useful when exact unsupported text is more actionable than a general score and when the application has a defined response to a hit. It also adds endpoint latency, model and prompt versioning, monitoring, review cost, and an outage policy.

Adopt the endpoint path when a real trace evaluation shows that it catches costly unsupported spans at an acceptable operating cost. Keep it out of the delivery path when annotations do not change a decision, when the evidence is too weak to support the check, or when the application cannot handle detector failure safely. The model can expose a grounding problem within supplied evidence. The application still owns what happens next, and neither component can prove that the evidence itself is true or complete.

Sources

• LettuceDetect v2 in Semantic Router: Generative Hallucination Detection as a vLLM Endpoint, August 5, 2026
• Semantic Router pull request 2526, merged August 3, 2026
• Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents, submitted July 1, 2026
• KRLabsOrg/lettucedect-v2-qwen-2b model