Run a local subset of the AP e2e suite without instance data

Description

Summary

Developers can run a self-contained subset of the Approval Path e2e suite against their local backend and the local /sandbox, without credentials and without data prepared on an Atlassian site, to check changes to the app and to new or existing tests before pushing. It follows the macro-pack model. Every other test keeps running only on qa and prod.

Background

  • Since AP-2455 most tests run on /sandbox, and on the dev profile the sandbox already needs no setup: the backend creates ~/.approval-path/sandbox-secret and a local run reads it.

  • A full local run is not practical:

    • e2e/local.json has to hold the whole GlobalData: Atlassian users with passwords and API tokens, a WebApi key, the app's external user registered in the local backend, a second user, an inactive user, a group, and OVH mailboxes that tests read for e-mail checks and Atlassian 2FA codes.

    • Many tests depend on configuration that exists only on the qa and prod sites: workflow conditions, validators and post functions applied by hand, JSM projects and portal, Slack workspaces, the group picker custom field, the Test Manager app, existing instance errors.

    • About 25 values are hard-coded per qa/prod (Forge app and environment ids, (QA) labels, the issue propertyapprovalPathQa, workflow module ids -qa), while the local apps use (local), approvalPathlocal, -local.

    • About 45 checks encode "qa build vs prod build" (profile === 'qa', isProdEnv, hasPriorityColumn) and do not map to a developer's branch; some are already stale against the current code.

    • The local Playwright target does not split Jira and Confluence (no @jira_only / @confluence_only filter), and on real Atlassian ApprovalsListIframe.buildUrl falls back to Connect URLs for local.

Model to follow: macro-pack (MP-812, MP-909)

  • The local profile holds no credentials, only the backend address.

  • The target host and acting user come from the local DB through sandbox endpoints (/sandbox/api/e2e/defaults,/sandbox/api/hosts, /sandbox/api/hosts/{id}/users). The DEV TUI "E2E tests" tab picks them and saves e2e/.e2e-target.json; node run-headless.mjs runs the suite with TEST_DATA=local.

  • Tests provision their own data through sandbox endpoints that use the app installation (create or reset a page in a default space, upload attachments as the acting user) and track it for cleanup.

  • Tests that need third-party integrations or qa-only data skip on local.

Scope

  1. Local target from the local DB. Sandbox endpoints that list the installed AP hosts (Jira and Confluence) and their users, plus defaults; run-headless.mjs and .e2e-target.json; an entry for apfj / apfc in the DEV TUI "E2E tests" tab (extra/forge-dev/dev.service.js, APP_E2E). One local Playwright project per product with the @jira_only / @confluence_only filter, sandbox host only, so no Atlassian login and no 2FA mailbox.

  2. Test data without credentials.

    • Work items and pages, and project or space lookups, created through the sandbox as the acting user. The sandbox product relay already does this; alternatively add provisioning endpoints as in macro-pack.

    • A WebApi key created for the run through the connector API with the sandbox-minted app token, instead of a key kept in a data file.

    • Cleanup of everything the run created, the same way.

  3. The local subset, marked explicitly, for example with a @local tag selected by the local projects.

    • In: tests that need only the host, the acting user and the data they create. That covers global settings and the project view (definitions, approvals, conditions, delegations, activity, settings that need no e-mail or Slack), and the approval-path menu (issue panel, byline) without e-mail steps.

    • Out, qa and prod only: workflow conditions, validators and post functions; JQL search; JSM; Test Manager; Slack; anything that reads a mailbox; the external user (dashboard, sign links, reset password); the non-admin user; instance errors; the group custom field.

  4. Environment values for the subset taken from the target and the app identity instead of the profile name, for example the issue property key and the app label. Every build-level check that the subset uses is verified against the current code.

  5. Housekeeping.

    • .gitignore for the local artefacts (app-jwt-token-local.json, app-user-state-local.json, .e2e-target.json).

    • The Jira integration reporter does nothing for local; with CI set it would otherwise close shared ET issues.

    • A test-local script and a "Running locally" section in e2e/Readme.md.

    • A clear error when the sandbox serves a missing or stale bundle. The local sandbox serves the last build:forge output from target/classes/forge.

Out of scope

  • Running the full suite locally, or reproducing the qa site configuration on developer sites.

  • Moving the Test Manager tests out of the AP suite (separate ticket).

Acceptance criteria

  • With the backend on the dev profile and the local apps installed on a site, node run-headless.mjs (or the TUI "E2E tests" tab) runs the local subset for Jira and for Confluence, with no data file and no credentials, and it passes on the current dev.

  • Tests outside the subset are not run locally, and the reason is visible: not selected, or a skip reason.

  • qa and prod are unchanged: the same tests per project in playwright test --list, and a green qa run.

Risks

  • Creating work items and pages as the acting user needs the impersonation scopes on the local app; if a scope is missing, provisioning has to go through the app installation, as in macro-pack.

  • Build-level checks in the subset can drift as features move between qa and prod; the subset should avoid them where possible.

Change

Backend: sandbox endpoints for the local target. They sit behind the same /sandbox/** token and ap.sandbox.allowed-sites guard as the rest of the sandbox, on every profile.

  • GET /sandbox/api/hosts?product=jira|confluence: the active Approval Path installations on allowed sites. qa and prod look up each allowed site; only dev (*) lists the table. A site whose tenant lookup fails with anything other than "site gone" now fails the target endpoints instead of reading as a site without a token.

  • GET /sandbox/api/hosts/{id}: one installation and whether a Forge system token is stored. Any id the sandbox may not use gets the same 404, which names no site. A host id saved in .e2e-target.json that is no longer the site's current installation is refused with 404; pick the site again in the TUI.

  • GET /sandbox/api/hosts/{id}/users: the accounts that have a stored Forge user token for that site, newest first (at most 50), with display names from one bulk lookup.

  • GET /sandbox/api/e2e/defaults?product=: the default installation and acting user, plus the backend's app.url and public addon.base-url.

E2E suite: TEST_DATA=local, no data file.

  • npm run test-local (node run-headless.mjs) and the forge-dev TUI "E2E tests" tab (apfj, apfc) run the local subset. Playwright options pass through.

  • The target comes from the local DB:

    • site, acting user and app address from the endpoints above;

    • the acting user's name and the project or space through the sandbox relay;

    • .e2e-target.json, saved by the TUI, and AP_E2E_* env vars override.

  • A run covers both products, or only those named in AP_E2E_PRODUCTS; the TUI runs its own. A product that cannot be set up is left out with a notice.

  • Projects local_<product>_sandbox, _sandbox_settings and _sandbox_permissions run on the sandbox host only, with the @jira_only / @confluence_only filter. They run only tests tagged @local.

  • No credentials:

    • work items and pages go through /sandbox/api/product as the acting user;

    • the run creates its own WebApi key, "E2E local run", and archives it at the end;

    • the external-user login is skipped.

  • Global setup stops with a clear message when the sandbox's frontend bundle is missing or older than the frontend sources.

  • The Jira integration reporter does nothing for a profile without an env label.

  • .gitignore covers the local artefacts, and e2e/Readme.md has a "Running locally" section.

Local subset. 177 tests over both products. They passed on the current dev (Jira mblaszczykowski, Confluence michalb2023) in 12 minutes, with 2 skipped by design.

  • In: global settings and project view (approvals, conditions, definitions), API keys, documentation links, and the settings pages that need no mailbox, second user or group. For conditions, filter, edit and archive run only in the global view. Also the approval-path menu: manual initiation, automation (Confluence), triggers, notification/archive/delete, webhook steps and bulk actions.

  • Out (qa and prod only, not selected): workflow conditions, validators and post functions; JQL search; JSM; Slack; e-mail and mailbox checks; the external user (dashboard, sign links, reset password, activity, webhooks settings); the non-admin user; delegations; instance errors and the overview, which need existing errors; the group field; work item type mapping; the work-calendar tests that need a site without calendars; steps updates, which need a distinct second user. Also tests waiting for Jira events, the search index or the expiry scheduler (dynamic user field, JQL filter, validity expiration), and validity creation, which expects a backend on UTC. The project-view edit and archive condition tests are also out: they pick up conditions left by other condition tests in a full run, and the archive test fails on qa too (ET-22705).

  • Skipped with a reason: Jira automation steps, which need the Withdrawn transition of the qa workflow.

Decisions

  • The acting user stands in for the second user where a test only needs a step target.

  • The sandbox offers as acting users only accounts with a stored Forge user token, so impersonation works.

  • Project and space are picked automatically: the first company-managed project with a Bug type, and the first non-personal space. AP_E2E_PROJECT_KEY / AP_E2E_SPACE_KEY override.

  • A local run cleans up like qa: AQA data, settings it changed, and the acting user's delegations. So act as a test account. The acting user's Atlassian language must be English; the run prints a notice otherwise.

  • The subset needs no profile-specific app values (issue property key, app label, Forge ids): they appear only in Atlassian-host and JQL tests, which are out.

  • A qa-vs-prod check that dev passes like qa uses isPreReleaseBuild (qa or local).

Fixes included

Stale e2e checks, fixed for qa as well:

  • the Message templates option order (ET-22574);

  • the Actions settings toggles are clicked through their label (ET-22007);

  • the Expiration settings test waits for the visible spinner, not two of them (ET-22579);

  • the archived condition status does not depend on the hover state;

  • the Jira Details View compares the work item's own status and priority;

  • hasPriorityColumn is gone (the column is always there);

  • the page versioning status dropdown has a single locator.

Manual tester notes

No change in the app's UI. On QA: the qa e2e run keeps the same tests per project (playwright test --list is unchanged) and stays green.