[@layout]: ../docs.layout.md # Durable jobs Plugins register `jobs` to handle trusted Events after the originating operation commits. Bun server projects with jobs automatically open durable storage, recover unfinished operations before indexing Sources, and run due work after startup. An Action can return once its Source changes and notification work are durably admitted; it does not wait for a provider request. --- ## Define a job Use `defineJob` with a registered, non-browser Event. Job identity is its Plugin id and local contribution name. Keep that name stable and change `version` when a saved input or step history is no longer compatible. ```ts import { defineEvent, defineJob, definePlugin } from "@pathmx/core" import { z } from "zod" const ReportRequested = defineEvent({ data: z.strictObject({ report: z.string() }), }) export function Reports(readReport: (id: string) => Promise) { return definePlugin({ id: "reports", events: { requested: ReportRequested }, jobs: { prepare: defineJob({ on: ReportRequested, version: "1", async run(event, { step }) { await step.run( "read-report", async () => ({ status: "completed", value: await readReport(event.data.report), }), { recover: "retry", decode: (value) => z.string().parse(value) }, ) }, }), }, }) } ``` An Action stages the Event with `events.record`; trusted server or Plugin CLI code can await `ctx.events.emit`. Matching handlers are selected at admission. Installing a new handler later does not replay historical Events into it. Ordinary `events.subscribe` listeners remain process-local observers; use jobs for work that must survive restart. --- ## Save steps and retry safely Call `step.run(name, activity, { recover, decode })` sequentially. Each activity receives `{ key, interrupted }`: a stable identity for provider deduplication and whether execution was interrupted. Completed JSON results are saved and decoded on replay, so a completed activity is not repeated. Names and order must remain compatible. Parallel or nested steps are rejected. Activities return one of these outcomes: | Outcome | Behavior | | --- | --- | | `{ status: "completed", value }` | Persist JSON data and continue. | | `{ status: "retry", at, reason }` | Retry at an absolute Unix time in milliseconds, with a bounded attempt count. | | `{ status: "failed", reason }` | Stop failed work for inspection. | | `{ status: "blocked", reason }` | Stop until its execution prerequisite is restored. | | `{ status: "unknown", reason }` | Stop an uncertain effect for reconciliation. | `recover: "retry"` permits rerunning an interrupted activity; choose it only for safe reads or effects with a suitable idempotency protocol. `recover: "stop"` marks an interrupted activity unknown. Unexpected exceptions also stop uncertain work. Do not wrap a mutating Action in `step.run`; use `step.action` so its commit and saved receipt share the same durable boundary. Core preserves identity and history; each integration owns its provider's retry safety. [Email](../email.reference) supplies that protocol for messages. When a step throws, the host logs `job.step.error` through the owning Plugin's logger, with Plugin, handler, run and step identifiers. The error message is capped at 2,000 characters and JSON-escaped for the log. It is not automatically redacted; provider adapters should sanitize messages that may contain secrets. Saved step and run reasons remain generic, and diagnostics do not make unknown work retryable. For an expected failure, return an explicit outcome with a suitable public reason. Each job receives `{ app, state, step }`. `app` is the receiving Plugin's normal `AppContext`, including `app.view`, Events, database and scoped job operations. It is supplied at execution, not serialized into the run. The trusted job must choose an explicitly admitted Actor/session; an Event payload alone does not authorize impersonating an Actor. An optional synchronous `capture()` on the job returns JSON configuration saved at admission, exposed as `state` during execution. Use it for policy that must survive a deployment change; do not capture credentials. Missing handlers, incompatible versions, or invalid persisted inputs stop execution. --- ## Save an Action as a step Use `step.action(name, view, action, input, { target? })` for an existing Action. It returns the normal Action receipt and shares the same sequential history as `step.run`. The View must come from the job's owning host, with an admitted Actor and no read-only subject override. Options match `view.actions.run`; the host owns operation identity, so callers do not supply a retry key. For example, a trusted report job can infer a summary and then save it through a normal Action. The Action remains responsible for permission checks and finite Source effects: ```ts import { defineAction, defineEvent, defineJob, definePlugin } from "@pathmx/core" import { z } from "zod" const SummaryRequested = defineEvent({ data: z.object({ actor: z.string(), report: z.string() }), }) export function Summaries(summarize: (body: string) => Promise) { return definePlugin({ id: "summaries", events: { requested: SummaryRequested }, actions: { save: defineAction({ target: "report", input: z.object({ summary: z.string() }), available: ({ can }) => can("write"), prepare({ target, input }) { return { effects: { update: [target.id] }, run({ tx }) { tx.source(target.id).data({ summary: input.summary }) return { report: target.id } }, } }, }), }, jobs: { summarize: defineJob({ on: SummaryRequested, version: "1", async run(event, { app, step }) { // The trusted Event producer has authorized acting for this Actor. const view = app.view({ type: "local", actor: event.data.actor }) const summary = await step.run("summarize", async () => { const report = view.readSource(event.data.report) if (!report) return { status: "blocked", reason: "Report unavailable" } return { status: "completed", value: await summarize(report.body) } }, { recover: "stop", decode: (value) => z.string().parse(value) }) await step.action("save", view, "summaries.save", { summary }, { target: event.data.report, }) }, }), }, }) } ``` No Source transaction stays open during `summarize`. Before an uncommitted Action runs, Core checks current Actor, View and Action admission, including changes made while external work was in flight. Permission denial blocks the step; restore the prerequisite and use the existing job retry operation. Rejected preparation or validation fails the step. Uncertain outcomes stop for reconciliation unless the journal already holds a prepared operation that can be recovered. The Action's receipt, Events, downstream job admission and committed operation marker are saved together. After a crash, a committed step returns that receipt without invoking the Action or admitting its Events again. A prepared operation is recovered before job execution resumes. Source-file recovery retains the [storage boundary](#storage-boundary) described below. Saved invocation identity includes the Actor/session authority, View context, Action name, explicit target and JSON input. Changing any of those for the same step fails; object property order is immaterial. Inputs are captured before work begins. Output must be JSON or `undefined` for a void Action; unsupported outputs fail before commit. Replay returns the original receipt, including its original operation id and time. Incompatible handler/output changes require a job version change, just like other saved steps. Completed receipts are private execution history for the same trusted job and matching View; replay does not rerun current Action availability checks. Creating a new contextual View still requires normal admission. Do not expose saved results to a browser without its own current access checks. This protocol covers Source mutations and Event/job admission, not arbitrary network or provider effects inside an Action; keep those in explicit `step.run` integrations. --- ## Inspect and retry from a live Plugin The trusted `AppContext` supplied to Plugin hooks includes `app.jobs`. A Plugin can list and retry runs belonging to its currently registered jobs without stopping the server: ```ts // Inside server code that has already authorized the operator: const runs = await app.jobs.list({ limit: 50 }) const failed = runs.find((run) => run.status === "failed") if (failed) await app.jobs.retry(failed.id) ``` List returns newest-admitted runs first, filtering to the receiving Plugin before applying the limit (default 100, range 1–1000). Each summary includes `id`, `event`, `handler`, `version`, `status`, `due`, `attempts`, and `reason`. `event` is the Event id; `due` is a Unix time in milliseconds. Captured configuration and step payloads are not included. A Plugin with no registered jobs sees an empty list when the host supports these operations. Missing and other-Plugin run ids receive the same not-found error on retry. `retry(id)` requeues compatible failed or blocked work and wakes the owning host. Completed step results and the original run/Event identities are preserved. A pending run is unchanged, including its scheduled due time. Running, completed, and unknown runs cannot be retried; unknown or interrupted step outcomes require reconciliation. Unrelated executing jobs do not block requeueing. The worker still executes jobs sequentially, and retry returning does not mean the job finished. These are trusted server capabilities, not browser permissions. The Plugin must check operator access before displaying results or invoking retry from an Action, query, or route. Keep retry in a retry-only Action: it is an immediate durable operation, not staged with Source writes, and a later Action failure cannot undo it. Hosts without live job operations, including static execution and inspection hosts, report that the operation is unavailable. `JobDefinition`, `Json`, `JobRun`, and `PluginJobs` are exported from `@pathmx/core` for reusable job definitions, step values, and typed operation helpers. The runner and storage records remain internal. --- ## Operate the queue ```sh bun run pmx jobs list --limit 100 bun run pmx jobs inspect '' bun run pmx jobs operation '' bun run pmx jobs retry '' bun run pmx jobs drain --limit 32 ``` `jobs operation` shows the operation's outcome, original request receipt when present, and a summary of changed Source paths, Event identities, asset moves and Action steps. Only unresolved operations include full recovery intent, with saved before/after Source content. Completed operations are not a Source-content archive. List and inspection read saved state directly. With `pmx dev` or `pmx serve` running, `retry` and `drain` go to that server's worker, so no second worker starts. One native owner holds the repository for its lifetime, and process death releases that lock. Without a server, the command owns the repository while it runs. If a server exists but cannot be reached, the command fails instead of taking over its work. Drains run one batch at a time: a drain requested while the worker is busy waits for that batch, then processes up to its own `--limit` and reports only the jobs it ran. `retry` requeues compatible failed or blocked work while preserving completed steps; pending work is unchanged, and unknown outcomes cannot be cleared this way. Live Plugins use `app.jobs.retry` through the already-owning host instead of opening a second owner. `drain --production` uses the deployment's production effect policy; against a running server it must match that server's mode. Action failures can include an `operation` id. Inspect it when commit outcome is uncertain. An unresolved prepared operation fences normal repository access and job execution until its saved Source intent can be recovered; do not bypass this fence by deleting journal records. A commit already running in the current host is not abandoned recovery work. Readiness checks do not scan committed history or wait in the durable commit queue; read-freshness checks still protect overlapping repository updates. For an independent test instance, use a separate copy of the Sources and set `PATHMX_DB_DIR` to its own state directory outside the Source root. Changing the port alone does not separate ownership. A separate state directory has its own journal, caches and development sign-in state; it does not coordinate writes to Sources shared with another instance. --- ## Storage boundary Bun saves operation intent before replacing Source files and admits Events, run identities, Action-step receipts, and the committed marker in one SQL transaction. Recovery accepts files matching their saved before or after content and stops on conflicting external edits. Individual asset renames recover alongside their reference edits; mismatched asset bytes stop recovery. Durable folder moves are unsupported. An unrelated editor can still race filesystem writes; this protocol does not claim atomic coordination with every external writer or tested guarantees for every power-loss scenario. When a commit succeeds, the same SQL transaction replaces its full recovery payload with a content fingerprint, request receipt and summary, and releases copied job-admission data. Event records, admitted runs and step outcomes remain with their respective owners. Unresolved intent stays complete until recovery succeeds. Opening an existing journal compacts completed history in restartable batches without changing operation identities or repeating work. Job history lives in private project storage outside Sources. Request receipts, Events, runs and steps have no automatic time-based expiry. Email separately prunes its message data. SQLite can reuse space freed by compaction; the physical database file does not necessarily shrink immediately. Static builds do not execute jobs. The Cloudflare live host uses a repository Durable Object with the shared journal and runner, transactional storage, and alarms to wake due work. This does not add authored recurring schedules or parallel jobs.