Test an application surface

Create a repeatable smoke test for an application's own interface, then inspect both its assertions and evidence. Test Runner drives a registered app surface through a virtual cursor; it is not a general desktop automation tool.

Before you begin

Use a Windows or Android application package that includes Test Runner 0.4.2 or later. Select it in Library, build the package and install that package. Merely viewing the public documentation or selecting the next build's Sidecar does not put a runner into an already installed app.

Use a test account or harmless local fixture. Scripts can activate real application controls: a bounded UI surface is not the same as a guarantee that every button is harmless. Keep purchases, publishing, deletion and wallet operations out of a general smoke test unless you have deliberately configured and approved that environment.

Select Test Runner for the installed app

Open the floating Tests chip, or use Ctrl+Shift+T where a keyboard is available. Check the header's agent/bridge state and registered target surface before starting.

The workspace offers Template, Import, Export, Run test and Stop. cApp self-test loads the supplied test for cApp's own interface; it is not a universal test for an unrelated application. Use Template or your own script for your app.

When a run begins, the workspace collapses so the app remains visible. The virtual cursor and live step log show progress, and Stop remains available. Windows packages can have an outer shell and inner app frame; choose the registered application surface rather than assuming the shell's document contains the app controls.

Make targets stable

Give important controls stable data-capp-test-id attributes. Prefer them to generated CSS classes, layout coordinates or labels that change with localisation. The identifiers below are illustrative application code, not built-in cApp selectors.

Save this complete local fixture as index.html, create a project around its folder, and package it with Test Runner to try the script that follows. It changes only visible in-memory text; reloading the page resets it.

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Settings test fixture</title>
</head>
<body>
  <h1>Settings test fixture</h1>
  <button type="button" data-capp-test-id="open-settings">Settings</button>
  <section data-capp-test-id="settings-panel" hidden>
    <h2>Settings</h2>
    <form data-capp-test-id="settings-form">
      <label for="display-name">Display name</label>
      <input id="display-name" data-capp-test-id="display-name" required>
      <button type="submit" data-capp-test-id="save-settings">Save</button>
    </form>
    <p role="status" data-capp-test-id="save-result"></p>
  </section>
  <script>
    document.querySelector('[data-capp-test-id="open-settings"]')
      .addEventListener('click', () => {
        document.querySelector('[data-capp-test-id="settings-panel"]').hidden = false;
      });
    document.querySelector('[data-capp-test-id="settings-form"]')
      .addEventListener('submit', event => {
        event.preventDefault();
        const name = document.querySelector('#display-name').value.trim();
        document.querySelector('[data-capp-test-id="save-result"]').textContent =
          name ? `Saved for ${name}` : 'Enter a display name';
      });
  </script>
</body>
</html>

The test should wait for the application state it needs. A long sleep can hide a readiness problem; waitFor with an explicit timeout makes the intended condition clear.

Import and run your first script

Save this JSON as settings-smoke.capp-test.json. In Tests, choose Import, select the file and review the parsed script. Its protocol is capp.test-script.v1; surface: "app" targets the registered app surface.

{
  "protocol": "capp.test-script.v1",
  "name": "Settings smoke",
  "surface": "app",
  "steps": [
    {"action": "waitFor", "selector": "[data-capp-test-id='open-settings']", "visible": true, "timeoutMs": 8000},
    {"action": "click", "selector": "[data-capp-test-id='open-settings']"},
    {"action": "waitFor", "selector": "[data-capp-test-id='settings-panel']", "visible": true, "timeoutMs": 5000},
    {"action": "clear", "selector": "[data-capp-test-id='display-name']"},
    {"action": "type", "selector": "[data-capp-test-id='display-name']", "text": "Ada"},
    {"action": "click", "selector": "[data-capp-test-id='save-settings']"},
    {"action": "waitFor", "selector": "[data-capp-test-id='save-result']", "textIncludes": "Saved for Ada", "timeoutMs": 5000},
    {"action": "assert", "selector": "[data-capp-test-id='save-result']", "textEquals": "Saved for Ada"},
    {"action": "screenshot", "name": "settings-saved"}
  ]
}

Choose Run test once and follow the recorded steps. The first wait checks readiness, the actions open and fill the form, and the final assertion checks the application's result. Export saves the currently validated script; it is not an export of all app data or proof that the test passed.

Imports are limited to 1 MiB and 500 steps. Use valid JSON with no comments or trailing commas. The script schema describes accepted fields and bounds; runtime target restrictions apply in addition to schema validation.

Keep assertions meaningful

Assert a visible outcome after an action: a panel, validation message, recorded operation or receipt. A clicked button is not proof that asynchronous work completed. waitFor checks the requested text/value/checked conditions as well as visibility. It normally waits for a visible element; visible: false waits for an existing hidden element, and exists: false waits for absence. Use assert with exists: false to assert absence; a missing target is otherwise a failure. For a remote operation, wait for the app's actual reported state and inspect its operation/receipt separately; do not infer backend correctness solely from a success-looking animation.

clickAt, scrolling and fixed waits also exist, but stable selectors and observable conditions are usually easier to maintain. Prefer a narrow failure message and a relevant assertion to a long script that clicks every possible control.

Boundaries and safe stopping

The runner operates only on registered same-origin application surfaces. It does not move your OS pointer, run arbitrary JavaScript, use global input hooks, click other windows or grant network/filesystem access. Password and file inputs, protected Sidecar UI and unrelated surfaces are blocked.

Stop prevents further test progression; it cannot undo a form submission or external effect already performed. Inspect the completed steps before rerunning after interruption. Design smoke tests to be repeatable against known fixture state, and reset that state deliberately when a test changes it.

A missing app agent, navigation to a different surface or a disabled control needs diagnosis, not a broader selector or permission bypass. Agent invocations of surface tools have their own grants; installing Test Runner does not authorise every agent to drive it.

Interpret screenshots correctly

Screenshot steps are best-effort evidence of the app surface. Renderer limitations produce warnings and do not automatically abort otherwise functional steps. Check assertion results and screenshot warnings separately. A blank or missing screenshot should not be presented as visual proof that the expected state appeared.

For a failed run, retain the script, app/release identity, platform, first failing step and relevant log/evidence. Review screenshots and diagnostics for private app content before sharing them. A test against a development preview should be labelled as such, not described as an installed Android test.

Troubleshooting

What success looks like

The script passes its observable assertions on the intended application and platform, with a run record and honestly described screenshot evidence. Repeat it after relevant UI changes, and supplement it with native, backend and recovery checks that an app-surface runner cannot prove. See the Test Runner API/reference and release checklist.