Skip to main content
AI Development

Gemini Computer Use is GA: injection checks stay opt-in

Gemini API Computer Use reached general availability on Oct. 30. The safety system stays on, but screenshot prompt injection checks are opt-in.

Sean McLellan profile photo

Sean McLellan

Lead Architect & Founder

6 min read
Brass mechanical arm with a stylus over a walnut board of cream keys, one column of keys under a closed copper guard, and a brass desk bell beside it
Constructed diagramConstructed illustration: an agent can press most keys on its own, but guarded actions wait for a person to confirm.

Google made the Computer Use tool in the Gemini API generally available on Oct. 30, 2026, across both the Interactions API and the GenerateContent API. The tool lets a Gemini model look at a screenshot of a browser, an Android device or a desktop and return the next click, keystroke or scroll for your code to run. If you held a computer-use agent back because it was a preview feature, the decision now is which safety settings to change before it touches real accounts.

The catch is one default. Gemini's built-in safety system flags risky actions on every request, but its check for prompt injection hidden in screenshots is opt-in, and the documentation says "The default is false." An agent that reads web pages, inboxes or tickets should have it turned on before launch.

What changed in Gemini Computer Use?

Computer Use moved from public preview to general availability. The June 24, 2026 preview, launched with Gemini 3.5 Flash, already included the parts that matter here: actions with an intent field, browser, mobile and desktop environments, configurable safety policies and prompt injection detection. The Oct. 30 release note changes the support status and confirms the tool works on both API surfaces.

That second point matters if your Gemini code still uses generateContent. You can add Computer Use there without moving to the Interactions API first.

Google lists four models for the tool:

  • gemini-3.8-flash, which Google calls "the recommended model for computer use."
  • gemini-4-argon, its most capable model, for complex reasoning.
  • gemini-3.5-flash-lite, described as low-latency and cost-effective.
  • gemini-3-flash-preview, still a preview model.

For most business tasks, BaristaLabs would start on gemini-3.8-flash and only try Argon if a specific task keeps failing. Google also notes that lower thinking levels "generally achieve a good balance for standard automation tasks."

How does a Gemini computer-use agent work?

Gemini suggests actions, and your application carries them out. Each turn of the loop has four steps.

Four-step loop: your app sends a screenshot, Gemini returns a function call, your code handles allowed, require_confirmation or blocked, and your app replies with a new screenshot
Constructed diagramConstructed diagram from the Gemini API Computer Use documentation, Oct. 30, 2026. Not Google product UI.
  1. Your app sends the prompt, a screenshot and the computer_use tool settings, including the environment.
  2. Gemini returns a function_call for one UI action, with an intent that explains why. Coordinates come back on a 0 to 999 grid that your code scales to the real screen.
  3. The response may include a safety_decision. Google's safety system classes each action as allowed, require_confirmation or blocked.
  4. Your app runs the action, takes a new screenshot and sends it back in a function_result.

Because your code runs every action, your code also owns the limits. Google's setup guide asks for a sandboxed VM or container, and its reference implementation ships a Docker-based sandbox to start from. The same split between model and executor shows up in Anthropic's browser use tool, and the design lessons carry over.

Which Gemini safety policies should you keep?

Keep all seven on unless you have a narrow, tested reason. Each category decides when the safety system asks for confirmation or blocks an action, and the disabled_safety_policies setting turns one off.

Scroll sideways to see all 3 columns.

PolicyWhat Google says it coversBaristaLabs recommendation
FINANCIAL_TRANSACTIONSPayments, retail checkout, regulated goodsKeep on
SENSITIVE_DATA_MODIFICATIONHealth, financial and government recordsKeep on
COMMUNICATION_TOOLSending emails, chat messages or draftsKeep on; let a person send
ACCOUNT_CREATIONRegistering new accounts on websitesKeep on
DATA_MODIFICATIONFile changes, data sharing, storage deletionOnly consider turning off inside a disposable sandbox with nothing to lose
USER_CONSENT_MANAGEMENTCookie banners and privacy promptsKeep on
LEGAL_TERMS_AND_AGREEMENTSAccepting terms of service or contractsKeep on

Turning a policy off is a request, not a guarantee. Google says overrides "indicate your preference, but the model may still return a safety_decision with require_confirmation in some cases," so your client must handle confirmations whatever you configure.

How should your code handle require_confirmation?

Show the user the action and its explanation, and wait. If the user confirms, send safety_acknowledgement set to true in the function_result. If not, stop the task. When the decision is blocked, Google says your client "should halt the execution or handle the interruption."

For tasks where a person must step in, such as a second-factor code, Google documents another pattern. You can drop a predefined action with excluded_predefined_functions and register your own function, such as yield_to_user, so the agent hands control back instead of guessing. Agents also get stuck on bot checks, and an earlier post covers why CAPTCHAs still need a fallback path.

Why prompt injection detection matters for business agents

A computer-use agent reads whatever is on the screen, including text an attacker placed there. With enable_prompt_injection_detection set to true, Gemini checks screenshots in the request for hidden instructions, such as "Ignore previous commands," and blocks execution when it finds them. The feature works on Gemini 3.5 Flash-Lite and later.

BaristaLabs interpretation: the cost of turning it on is an occasional blocked step, which your confirmation path already has to handle. The cost of leaving it off is an agent that may follow instructions from a supplier's web page or a customer's email. For any agent that browses outside your own systems, turn it on.

Google's other advice is worth following too. Keep an allowlist of sites the agent may visit, which Google calls "even more secure" than a blocklist. Start each task from a clean state. Log prompts, screenshots, suggested actions, safety responses and the actions your client actually ran. That log is the only record of what the agent did on a screen. The same app-permission thinking applies to Copilot's computer use preview.

Pick one repetitive browser task that ends before anything is sent or paid, such as collecting order statuses from a supplier portal.

  1. Run it in a sandboxed browser on gemini-3.8-flash, with an allowlist of the portal's domain.
  2. Set enable_prompt_injection_detection to true and leave all seven safety policies on.
  3. Build the require_confirmation and blocked paths before the happy path, and log every turn.
  4. Review a week of logs, then decide whether the task earns more access.

If you want help choosing the first task and its limits, see process automation.

Sources and scope

This post is based on Google's documentation. BaristaLabs has not tested Computer Use for this post, and the sources do not say how often prompt injection detection catches or misses attacks.

Gemini Computer Use

Scope your first computer-use agent

Bring one browser task you want an agent to handle. BaristaLabs can help you set the sandbox, the allowlist, the safety settings and the confirmation step, and decide what the logs need to show.

One task in one sandboxed browser is enough to start.

Turn this idea into a pilot

Which workflow should go first?

Use the readiness check to compare impact, effort, risk, owner, and next step before requesting a review.

  • 3-5 minutes
  • Deterministic score
  • No sensitive data
Check workflow readiness

Practical AI Workflow Notes

Want more practical AI operations ideas?

Get short notes on applying AI inside real small-business workflows — from document handling and customer follow-up to internal reporting, compliance, and automation guardrails.

A useful next step if you’re still exploring and not ready to request a 20-minute workflow assessment.

Occasional emails. Practical workflow guidance only. Unsubscribe anytime.