Test Runner

Run a declared test

Test Runner executes portable, bounded UI scripts against the selected application's registered same-origin surface. A visible virtual cursor and step log make the run observable. It does not operate the real OS mouse or turn a script into arbitrary JavaScript, filesystem or network authority.

Select uk.capp.sidecar.test-runner for Windows or Android and install the updated package. Open the persistent Tests chip or use Ctrl+Shift+T where a keyboard is available. The default workspace starts open, but test execution is not enabled automatically by default.

Use Template, Import or the script editor to prepare a capp.test-script.v1 document. cApp self-test is specifically for the cApp owner's interface, not a universal test for any application. Importing that script into a different product does not create matching routes or controls.

First passing test

1. Open the app route to test and confirm the runner identifies a registered target surface/agent.
2. Load a small script with stable data-capp-test-id selectors. Set up the starting state in the script instead of depending on what the previous run left behind.
3. Select Run test. The workspace collapses while the virtual cursor drives the visible app; step progress remains available from the Tests chip.
4. Check the first failed step, its expected state and any evidence warning. Do not treat a missing screenshot as the same failure as a wrong application value.
5. Run the same script again. A useful smoke test should explain or reset its own state and should not publish, charge, delete or trigger an unintended external effect.

Use Test an application surface for the larger Settings example and workflow. This reference supplies the script/action semantics.

A repeatable counter fixture

Save this as a small application HTML file. It has no provider, backend or persistent data. The buttons and output remain meaningful without Test Runner.

<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><title>Counter test</title></head>
<body>
  <main>
    <h1>Counter</h1>
    <button data-capp-test-id="counter-reset" type="button">Reset</button>
    <button data-capp-test-id="counter-add" type="button">Add one</button>
    <output data-capp-test-id="counter-value" aria-live="polite">0</output>
  </main>
  <script>
    const counter = document.querySelector('[data-capp-test-id="counter-value"]');
    let count = 0;
    document.querySelector('[data-capp-test-id="counter-reset"]').addEventListener('click', () => { count = 0; counter.textContent = '0'; });
    document.querySelector('[data-capp-test-id="counter-add"]').addEventListener('click', () => { count += 1; counter.textContent = String(count); });
  </script>
</body>
</html>

Import this document as JSON into the runner after packaging the fixture with the Sidecar. It deliberately resets the counter so repeated runs have the same outcome.

{
  "protocol": "capp.test-script.v1",
  "name": "Counter increments once",
  "surface": "app",
  "steps": [
    { "action": "waitFor", "selector": "[data-capp-test-id='counter-reset']", "visible": true, "timeoutMs": 5000 },
    { "action": "click", "selector": "[data-capp-test-id='counter-reset']" },
    { "action": "assert", "selector": "[data-capp-test-id='counter-value']", "textEquals": "0" },
    { "action": "click", "selector": "[data-capp-test-id='counter-add']" },
    { "action": "waitFor", "selector": "[data-capp-test-id='counter-value']", "textEquals": "1", "timeoutMs": 3000 },
    { "action": "assert", "selector": "[data-capp-test-id='counter-value']", "exists": true, "textEquals": "1" },
    { "action": "screenshot", "name": "counter-one" }
  ]
}

The functional outcome is a visible value of 1, not merely a click event. Screenshot capture is best-effort: a renderer failure is an evidence warning, while the counter assertions still decide the functional result.

Script contract and limits

The portable schema accepts protocol, optional name/surface, and a steps array. There is no script field for an executable JavaScript body. The workspace bounds imports to 1 MiB and 500 steps; smaller focused scripts are easier to diagnose.

Step timeoutMs ranges from 100 to 60,000 ms. Timed waits/movement durations can be bounded up to 60,000 ms; path input has at most 200 points. Runtime checks additionally enforce registered surfaces, targets and protected-element restrictions. Passing JSON Schema is not a grant to automate any document.

Actions and arguments

Selectors are authored application test input, not a model permission to target arbitrary windows. Prefer stable attributes over generated class names, text position or viewport coordinates. Native dialogs, system keyboards and remote Links pages need their own acceptance testing.

Wait for state rather than sleeping

wait delays for a bounded time but proves nothing. Use waitFor to wait for the actual postcondition, then assert where a final state check helps.

When supplied together, text/value/checked/visibility conditions must all match. An assertion requires its target unless absence is explicitly requested. A missing target cannot satisfy a positive text assertion. Protected targets remain refused even for an absence wait; the runner is not a probe for private Sidecar surfaces.

Test Runner 0.4.2 carries these observable-state semantics. An older installed runtime does not acquire them from reading this page; rebuild/install the matching package when qualifying a script.

Scope

The runner stays within registered same-origin application documents. It does not move the operating-system pointer, install global input hooks, click unrelated windows or execute arbitrary script source. Password/file inputs and protected Sidecar surfaces are blocked.

Synthetic events are not trusted human input. They cannot satisfy a Guide action that requires a trusted confirmation click, a wallet's approval boundary or an OS permission prompt. Do not weaken those product controls to make a test appear green.

An app event handler can still perform a real business effect when a test clicks it. Use synthetic data and a safe fixture; the absence of an arbitrary network tool does not make every possible button click harmless. Keep payment, publication, deletion and account actions out of a general smoke test unless a separately authorised test environment explicitly requires them.

Agent tools for test surfaces require their own registration, grants and approval policy. The supported public entry point here is the runner UI and portable JSON contract, not private app-agent globals or an unrestricted model driver.

Results, Stop and recovery

Read run status and individual step records separately. passed means the declared functional steps passed on that surface; it does not independently verify a backend transaction or every possible app workflow. failed identifies a failing step; cancelled identifies an interrupted run. Screenshot warnings are separate evidence outcomes.

Use Stop during a pending run. Condition waits observe cancellation, but cancelling a run cannot undo an application handler that already performed an effect. Reconcile that application state before retrying, and keep retryable smoke fixtures intentionally resettable.

Export the current validated script for reuse. Review screenshots, values and diagnostics before sharing: even a permitted app surface may display personal data. A finite screenshot/step record is not an unrestricted screen recorder.

Troubleshooting

Related data contracts

The schema describes script shape and bounds, not approval to automate sensitive controls or evidence that a business operation succeeded.

• capp.test-script.v1.schema.json

Configuration reference

These are the selected package's configuration fields. String credential values and installation-specific defaults are never published here. A field's presence does not bypass its runtime policy.

Agent interfaces

These names are declared by the package for host/Agent integration, not JavaScript methods to invoke directly. The host must register a tool, grant its scope and enforce its approval policy. A manifest declaration alone does not prove availability in every Agent catalogue. Prefer the application API above when integrating your own app.