MCP Apps put incident evidence beside the AI answer

Amazon OpenSearch Service now supports MCP Apps that can place an interactive trace, service map, alert view, log analysis, or metric chart inside a compatible AI conversation. One tool call returns a compact text summary for the agent and a visual result for the operator.

That changes a real operations bottleneck: the handoff from an AI-generated incident hypothesis to evidence a person can inspect. It does not make the hypothesis true. Teams should pilot the feature as a verification surface, measure whether it improves the review, and keep the limits of that evidence visible.

What does the OpenSearch MCP App return?

AWS calls the pattern a dual response. A local MCP server receives a tool call from a compatible AI client, queries an OpenSearch UI application, and returns two outputs in the same thread. The agent receives structured text it can use for its next step; the person receives an interactive visualization.

The OpenSearch documentation lists views for firing alerts, aggregated observability reports, distributed traces, service dependencies, logs, and PromQL metrics. An operator can inspect a span hierarchy, move through a service map, or review a metric series without first finding the equivalent dashboard in another tab.

This is more than formatting. Incident review often loses time between the summary and the source view: opening the correct system, rebuilding the query, finding the relevant trace, and confirming that the chart covers the same time window and service. Keeping the visual result beside the conversation can remove parts of that navigation.

AWS does not provide a controlled study showing how much time it saves or whether reviewers make fewer mistakes. Those are outcomes for a local pilot to measure.

Why is the visualization useful but not independent proof?

AWS says the OpenSearch visualization is generated by executing code against the same data sources that power the dashboards, and describes that result as deterministic. The important distinction is that the chart is an actual query result rather than a picture invented by the language model.

An operator can therefore check whether the prose matches the returned spans, logs, topology, or metric series. If the agent says the checkout service began failing after a database call slowed, the trace waterfall should show the relevant timing and dependency. If it does not, the summary has not survived its first check.

The chart and summary still travel through one investigation path. They can share a bad time range, incomplete telemetry, overbroad permissions, a misleading aggregation, or an incorrect query. Agreement between them does not independently confirm the root cause.

The OpenTelemetry gateway test addresses an earlier dependency: whether a complete trace reached the backend through the intended route. MCP Apps operate after that point. They make stored evidence easier to inspect; they do not establish that the stored evidence is complete.

What must be in place before a pilot?

The current AWS setup requires an OpenSearch UI application with an Observability workspace connected to at least one supported data source. The documentation lists OpenSearch domains, OpenSearch Serverless collections, and Amazon Managed Service for Prometheus among the data paths.

The operator also needs a client with MCP Apps support. AWS currently lists Claude Desktop, VS Code with GitHub Copilot, Goose, ChatGPT, and Cursor. Host support is part of the deployment boundary, not a minor display choice. The MCP Apps documentation says support varies by client, so confirm the exact client and version your on-call team will use.

The local server requires Node.js 22 or later and AWS credentials that permit es:ESHttpGet and es:ESHttpPost against the OpenSearch UI application. AWS publishes a downloadable package and an optional signature file. Verify the package before installation, pin the reviewed artifact, and grant the test identity only the data access the pilot requires.

MCP Apps typically render HTML in a sandboxed iframe. That isolation prevents the app from reading the host page or its cookies, but it is not the whole security decision. An app can declare content-security-policy origins and request capabilities; the local server can make authenticated queries; and the host mediates tool calls. Review all three surfaces: UI resource, local server, and AWS identity.

How should the comparison be run?

Choose one sanitized incident with known evidence and a reviewed explanation. It should contain enough depth to require several steps: an alert, a relevant service path, a trace, and supporting logs or metrics. Keep the same OpenSearch data, time window, questions, and reviewers for both paths.

Run the first investigation through the team’s existing browser workflow. Run the second through the MCP App in the intended AI client. Do not tell the second reviewer which conclusion to find.

Capture these observations:

• time from the initial question to the first relevant source view;
• number of context switches and queries needed;
• whether the reviewer found the correct trace, service, and time range;
• differences between the agent summary and the returned visualization;
• claims that required another query or an independent source;
• failed, partial, or stale UI renders;
• every AWS action used by the pilot identity; and
• whether another operator could reproduce the investigation from the saved query details.

This is not a benchmark of the language model. It is a comparison of two review paths. The useful result may be narrower than “faster incident response.” For example, inline views might reduce time to the first trace while making no difference to final root-cause accuracy.

What should stop the rollout?

Stop if the inline path hides the query, time range, source, or trace identifier needed to reproduce a finding. A persuasive chart without those details makes review feel easier while weakening the evidence record.

Stop if the MCP identity can reach more observability data than the on-call role needs. Also stop if a missing visualization silently leaves only the agent’s prose, if reviewers cannot distinguish stale output from current data, or if the supported client handles the UI inconsistently across the team’s managed environment.

Treat actions separately from investigation. The ability to inspect a trace does not justify permission to restart a service, acknowledge an alert, change a monitor, or modify a dashboard. If later versions add operational actions to the same conversation, give them a separate identity, confirmation rule, and audit path.

What decision can the pilot support?

A successful pilot shows that operators reach inspectable evidence with less navigation, preserve the query context, catch disagreements between prose and data, and reproduce the result under an appropriately limited identity. It does not prove the agent found the root cause, the telemetry was complete, or the incident process improved end to end.

That boundary is the practical value of OpenSearch MCP Apps. The feature can move evidence closer to the AI answer without collapsing evidence into the answer. Adopt it when the measured review path improves and remains reproducible; otherwise keep the existing dashboard workflow until the client, access, or evidence gaps are fixed.

BaristaLabs can help review one investigation path with a controlled incident sample, limited identity, and explicit verification measures.

Sources

• AWS: Agentic observability with Amazon OpenSearch Service MCP Apps, August 25, 2026.
• AWS: Agentic Observability with MCP Apps, accessed September 7, 2026.
• Model Context Protocol: MCP Apps, accessed September 7, 2026.