Desktop automation needs a runtime, not just a prompt

Desktop automation needs a runtime because the hard part is not only deciding what to click. The hard part is running the same task reliably on real software, with state, credentials, failures, and outputs that downstream systems can trust.

Cyberdesk is built around that runtime layer. It connects AI agents to Windows desktops and virtual machines, keeps the execution observable, and turns successful work into reusable workflows.

The prompt is only the beginning

A prompt can describe the job: open an application, search for a record, extract a field, upload a document, or submit a form. That instruction is useful, but it is not the whole system.

Production desktop automation needs a place to run. It needs connected machines, browser and desktop state, screenshots, logs, cancellation controls, retries, and a way to return structured data.

This is especially important for tools that were never designed for APIs. EHRs, ERPs, claims portals, thick clients, and remote desktops often require visual navigation and stateful sessions.

The mistake is treating the prompt as the deployment unit. In a demo, the model can be handed a screen and asked to figure it out. In production, the deployment unit is the whole execution path: which machine is used, what credentials are available, what happens if the app opens on the wrong screen, what evidence is retained, and how the result is checked.

That execution path needs to be designed explicitly. Otherwise every run is a fresh experiment.

A runtime keeps the desktop connected

Cyberdesk uses connected machines as the execution surface for workflow runs. A workflow can run on a Windows desktop or virtual machine where the target software is already installed and configured.

The runtime keeps track of the run, the machine, the screen, and the action history. That lets teams debug what happened instead of treating every agent run as a black box.

For desktop-heavy operations, that observability is the difference between a demo and an operating system.

Connected machines also make the environment repeatable. A finance workflow may require a specific ERP client, a mapped network drive, Excel add-ins, printer settings, or a VPN profile. A healthcare workflow may require a browser certificate, a remote desktop connection, or a preconfigured workspace. These details are not incidental. They are part of the automation.

When the runtime owns the machine connection, teams can answer basic operational questions:

Which machine ran the workflow?
Was the desktop already logged in?
What was visible before the agent acted?
Which screenshots and actions led to the final output?
Was the run cancelled, retried, or completed?

Without that layer, failures become hard to reproduce. With it, the run has a trail.

Sessions preserve state across steps

Many workflows are not isolated one-shot tasks. A team may need to log in once, navigate through a queue, export a file, then run another workflow on the same desktop.

Cyberdesk sessions reserve a machine so multiple workflow runs can happen in sequence with shared desktop state. This is useful when an application requires persistent login context, temporary files, or strict ordering.

Sessions make the desktop feel less like a stateless API call and more like an automation workspace.

That matters because many desktop systems punish stateless automation. Login flows can be slow or rate limited. Some applications keep context in local windows instead of URLs. Export files may land in a downloads folder and need to be picked up by the next step. A modal left open by one action can change what the next action sees.

A session gives teams a controlled way to say: keep this machine reserved until this chain of work is done. It also gives the platform a natural place to enforce cleanup. After the session, the workflow can close windows, clear temporary files, release the machine, or hand the desktop to a human reviewer.

Sensitive inputs need separate handling

Desktop automation often touches credentials, patient identifiers, financial records, or other sensitive values. Those values should not be treated like normal prompt text.

Cyberdesk separates sensitive inputs from ordinary workflow data. Sensitive variables use a dedicated syntax and are resolved only when needed for real computer actions such as typing into an application.

That separation helps keep secrets out of logs, model traffic, and long-lived workflow records.

The important design point is that the agent can know what to do with a value without being allowed to treat the value like ordinary context. A workflow can instruct the agent to click a password field and type a sensitive input, but the secret itself does not need to appear in the human-readable instruction, action history, or output payload.

This distinction becomes more important as workflows grow. A single prompt convention like "do not reveal this password" is not enough. The runtime should know which values are sensitive and enforce different handling for them.

Structured outputs make automation useful

Automation is only useful if the result can be consumed. A screenshot or transcript is not enough when the next system expects JSON, a webhook payload, or a typed object.

Cyberdesk workflows can define output schemas so unstructured desktop observations become structured output_data.

{
  "patient_status": "ready_for_review",
  "claim_number": "CLM-10492",
  "missing_fields": ["prior_authorization_id"]
}

Structured outputs make desktop automation composable with backend services, analytics pipelines, and human review queues.

This is where desktop automation starts to look like a real integration. The agent may have read the result from a screen, but the rest of the business should not have to parse a screen recording. It should receive a stable object with the fields it needs.

Good output schemas are small and operational. They include the answer, the confidence or review flag, and enough context for a downstream system to route the case.

{
  "status": "needs_review",
  "reason": "missing_prior_authorization",
  "source_system": "billing_portal",
  "review_queue": "claims_ops"
}

The goal is not to capture everything the agent saw. The goal is to return the decision points the business needs.

Failures need first-class treatment

Desktop workflows fail in boring ways: a login expires, a popup appears, a row is missing, a button is disabled, or a file download takes longer than usual. A runtime should make those failures visible and actionable.

That means a workflow should be able to stop with a useful state instead of pretending every exception is the same. "Could not find patient" is different from "application unavailable." "Export completed but total did not match" is different from "agent lost screen context."

When failure states are explicit, teams can route them correctly. Some failures should trigger a retry. Some should create a human review task. Some should page an operator because the target system changed.

The durable path is prompt plus runtime

The best desktop automation systems combine natural-language instructions with infrastructure that understands state, machines, outputs, and controls.

That is the shape Cyberdesk is built around: prompt-based workflow authoring, connected desktops, sessions, sensitive input controls, structured outputs, and reusable trajectories.

For teams automating legacy software, the runtime is not a detail. It is the product surface that makes agentic desktop work reliable enough to run repeatedly.

The prompt explains intent. The runtime makes that intent operational. Production desktop automation needs both.