Context

Approval Path for Jira and Approval Path for Confluence share one backend and one database. Background work runs through a job queue table in that database:

• product events: Confluence page, space and user events, Jira work item, project and user events;

• approval start, archive and delete from Jira workflow post functions and Automation;

• AI prompt steps.

The queue is the shared queue library that External Share also uses, but Approval Path runs its older mode:

• A job is deleted when it is picked up, in a database transaction that stays open until the job finishes. A crash rolls the deletion back and the job runs again. But a job that hangs keeps its row, its worker and a database connection until the app restarts. The open transaction is also held through every Jira, Confluence, Slack and AI call the job makes.

• A failed job is retried immediately, which does not help when the failure was a rate limit.

• A job cannot wait before it runs, so a burst of events cannot be grouped into one job.

External Share for Jira and External Share for Confluence already run the newer mode (ESFJ-2510, ESFC-2281). Automatic approval start (AP-2062) needs event grouping to limit Confluence API calls and stay within rate limits.

Scope

• One database migration adds every column the newer mode needs: delayed start, group key, maximum hold and lease token, plus the same two indexes External Share has. In External Share for Confluence a column added in a later step was missed and the queue failed in production (ESFC-2281).

• Leasing for all three job types: product events, workflow and Automation actions, and AI prompt steps.

  • A picked-up job is leased for 10 minutes instead of deleted, and removed when it finishes.

  • If the app stops while a job runs, the job runs again when its lease expires. The interruption uses up one retry.

• Delayed retries: a failed job is retried 1 minute later.

  • Retry counts stay as they are: 1 for product events and workflow actions, 3 for AI prompt steps.

  • After the last attempt the job is removed and logged as "[IMPORTANT] Failed on job", as today.

• Jobs already in the queue when the deployment starts become due at migration time and run after deployment.

• Grouping becomes available. Events with the same group key that arrive within the time window join one job, up to 50 events and the maximum hold. No existing job type switches to grouping; AP-2062 does that.

• Queue monitoring: once a minute the backend logs how many jobs are due, in flight and in the queue in total.

  • Due means ready to run, including expired leases.

  • In flight means running under a lease or waiting for a retry.

  • It logs an error when more than 10,000 jobs are due, the same threshold as External Share. Errors reach the errors stream.

• Integration test on a throwaway PostgreSQL migrated with the app's own changelog. It runs locally and is skipped where Docker is not available, including the Bitbucket build.

Behaviour changes to be aware of

• A job running when the backend stops resumes up to 10 minutes later. This happens on every deployment and uses up one retry. A job already on its last attempt is dropped and logged. Today such a job reruns right after the restart. External Share behaves the same way.

• A later job can now run before an earlier job's retry. A failed job waits 1 minute, so a later job for the same work item or page can run first. Example: an archive from a workflow fails on a transient Jira error, a start for the same work item runs, and the archive's retry then archives the new approval. Today this can already happen, but only for jobs queued before the failure.

• A job running longer than 10 minutes can be picked up a second time. No job type is expected to take that long. Deleting a project or space with thousands of approvals and Slack messages is the only plausible case. The AI prompt step's worst case is about 3 minutes.

• First deployment only: the deployer starts the new backend before it stops the old one. For those seconds the old backend ignores leases, so a job can run twice. A second run is harmless for every job type, with two exceptions: an AI prompt step can call the AI provider twice, and Slack message edits can repeat.

Acceptance criteria

• No job is lost in the deployment. Jobs waiting in the queue before deployment run after it. This covers Jira and Confluence jobs, which share the queue.

• An interrupted job runs again. A job interrupted by an app restart runs again after its 10-minute lease expires, as long as it has a retry left.

• Failed jobs are retried after a delay. A failed job is retried after 1 minute. After the last attempt it is removed from the queue and logged as "[IMPORTANT] Failed on job".

• Grouping works (integration test). A series of events with the same group key within the time window creates one job containing all the events, up to 50.

• Results are unchanged. These produce the same results as before the change:

  • Confluence page, space and user events;

  • Jira work item, project and user events;

  • approval start, archive and delete from Jira workflows and Automation;

  • AI prompt steps.

• Monitoring shows more than ready jobs. The once-a-minute queue log line shows jobs in flight (running or waiting for a retry), not only jobs ready to run. It logs an error above 10,000 due jobs.

• Stuck prompt step cleanup is unaffected. The cleanup of stuck AI prompt steps treats running and retry-waiting jobs as alive and does not fail their steps.

Manual tester notes

• Workflow actions:

  • A workflow post function starts an approval, and an Automation rule starts one.

  • Archive all and delete all from post functions work.

  • The approval appears within the usual time, about 10 seconds.

• Product events:

  • Editing a Confluence page with "rerun on page update" creates the new approval version.

  • Changing a work item user field updates a dynamic user step.

  • Deleting a page or work item deletes its approvals.

• AI prompt step: an approval with an AI prompt step completes the step.

Out of scope

• Running jobs to completion before the backend stops. Jobs interrupted by a deployment resume after their lease expires, as in External Share.

• Keeping jobs for one work item or page in order.

• Switching job types to grouping. AP-2062 does that.

• Changing the queue library's lease (10 minutes), retry delay (1 minute) or group size (50 events). They are shared with External Share.

• Running the integration test in the Bitbucket build. That needs Docker in the pipeline.