[@layout]: ../docs.layout.md

# AI

`@pathmx/ai` gives Plugins named AI models, called through an admitted View.

Requests stream partial results, validate structured output, can call your tools
and can end a [durable job](./jobs.reference) step. Image models draw pictures
through the same service. For agents authored as Markdown Actors, see
[Agents](./agents.reference).

It is built on the [AI SDK](https://ai-sdk.dev). Requests run on the server; do not
include this package in a browser publication.

---

## Name your models

Add `@pathmx/ai` and `ai` to your project's dependencies, aligned with your other
PathMX packages. Create one service and pass it to the Plugins that use it.
Create `plugins/course.plugin.ts`:

```ts
import { createGateway } from "ai"
import { AI } from "@pathmx/ai"
import { Tutor } from "./tutor.ts"

const gateway = createGateway({ apiKey: process.env.AI_GATEWAY_API_KEY })

const ai = AI({
  models: {
    tutor: gateway("provider/model-name"),
    reviewer: {
      model: gateway("provider/model-name"),
      maxOutputTokens: 2000,
      providerOptions: { gateway: { caching: "auto" } },
    },
    painter: gateway.imageModel("provider/image-model-name"),
  },
})

export default function coursePlugins() {
  return [Tutor(ai)]
}
```

Only files named `*.plugin.ts` are loaded as Plugins, so `tutor.ts` is an ordinary
module. Replace the model names with models your provider offers. Sources and code
refer only to the names you choose, here `tutor`, `reviewer` and `painter`. A request
for any other name stops before anything is sent. Keep API keys in the server
environment, not in Sources or package configuration.

Each name accepts any AI SDK language or image model: a gateway, a direct provider,
a local model, or a deterministic model for tests. Settings beside a model apply to
every request to it:

- `maxOutputTokens`: output cap for each model call. Defaults to 4096.
- `maxRetries`: retries of one model call after a transient provider error, such as
  a 5xx or a rate limit. Defaults to 0, so nothing is spent again unless you choose.
- `providerOptions`: AI SDK provider options, such as caching or reasoning effort.
  A request's own `providerOptions` merge over them, provider by provider.

Configured models are called whenever a request reaches them. Use deterministic or
local models where no provider call should happen, such as in tests.

---

## Ask a model from a route

Pass the caller's View, a model name, instructions and input. Create
`plugins/tutor.ts`, which `course.plugin.ts` imports:

```ts
import { definePlugin, route } from "@pathmx/core"
import type { AI } from "@pathmx/ai"
import { z } from "zod"

const Question = z.object({ lesson: z.string(), question: z.string().max(500) })
const Answer = z.object({ explanation: z.string(), nextQuestion: z.string() })

export const Tutor = (ai: AI) => definePlugin({
  id: "tutor",
  routes: [
    route.post("ask", async (ctx) => {
      const session = await ctx.session()
      if (session.type !== "authenticated") return new Response(null, { status: 401 })
      const view = ctx.view(session)
      const { lesson: id, question } = Question.parse(await ctx.request.json())
      const lesson = view.readSource(id)
      const guide = view.readSource("/tutors/socratic.tutor")
      if (!lesson || !guide) return new Response(null, { status: 404 })
      const response = ai.generate({
        view,
        model: "tutor",
        instructions: guide,
        input: [lesson, question],
        schema: Answer,
        signal: ctx.request.signal,
      })
      return Response.json(await response.output)
    }),
  ],
})
```

Create `paths/tutors/socratic.tutor.md`. Its body is the instructions:

```md
# Socratic tutor

Help the learner reason about their own evidence. Ask what they observed before
explaining anything. End with one useful question.
```

A signed-in reader can `POST /api/tutor/ask` with
`{ "lesson": "/lessons/bridge.page", "question": "How should I test it?" }` and
receives `{ "explanation": "…", "nextQuestion": "…" }`. The route decides who may
make paid requests; here, only signed-in readers.

---

## Requests

`ai.generate(request)` accepts:

- `view`: the admitted View making the request.
- `model`: a configured language model name.
- `instructions`: optional Source whose body is the instructions, or instruction text.
- `input`: Sources, strings and files, in order: the request's message.
- `history`: optional earlier turns, replayed before `input` as real messages:
  `{ role: "user", content: [...] }` with the same items as `input`, or
  `{ role: "assistant", content: "…" }`. Sources in them must be readable too.
- `schema`: optional Zod schema for structured output.
- `tools`: optional AI SDK tools the model may call. See
  [Give the model tools](#give-the-model-tools).
- `maxSteps`: model calls allowed when tools are given. Defaults to 20.
- `providerOptions`: optional provider options for this request, merged over the
  model's.
- `signal`: optional `AbortSignal` that cancels the request.

Each Source in `input` is sent as its Markdown body inside
`<source id="/lessons/bridge.page">…</source>`, so a model can refer to it by id.
Frontmatter and attachments are not sent; pass the values you intend to share as
strings. Every Source in `input` or `instructions` must be readable through `view`.

A file is `{ data, mediaType, name? }`, with the bytes as a `Uint8Array`: a photo,
a screenshot, a PDF or another file the provider accepts. Like a string, it is sent
as given: read it with the access checks your domain needs, and keep it within the
provider's size limit.

```ts
const response = ai.generate({
  view,
  input: [problem, key, "Check the student's work.", { data: bytes, mediaType: "image/jpeg" }],
  schema: Reply,
})
```

---

## Give the model tools

Tools are ordinary [AI SDK tools](https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling).
The model calls them, sees their results and continues, up to `maxSteps` model
calls. Choose the tools for each request: a tool you leave out cannot be called,
and a tool can only do what its own code and the request's View allow.

```ts
import { tool } from "ai"

const files = new Map(draft)
const response = ai.generate({
  view,
  model: "builder",
  instructions: guide,
  input: [overview, request],
  tools: {
    read_file: tool({
      description: "Read a file from the draft.",
      inputSchema: z.object({ path: z.string() }),
      execute: async ({ path }) => files.get(path) ?? "No such file.",
    }),
    look: tool({
      description: "Screenshot the draft on a phone.",
      inputSchema: z.object({}),
      execute: async () => [{ data: await screenshot(files), mediaType: "image/png" }],
    }),
  },
  maxSteps: 40,
  schema: z.object({ message: z.string() }),
  signal,
})

for await (const step of response.steps)
  progress(step.toolCalls.map((call) => call.toolName))
const reply = await response.output
```

A tool may return strings, Sources and files, alone or in an array; they reach the
model as request input does. Each Source, including its instance when present,
must be readable through the request's View, or the model receives an error
instead. This check applies to every async-generator yield and runs before a
custom `toModelOutput` formatter. Other results are sent as JSON unless the tool
provides its own formatter.

With a `schema`, the model finishes by calling a tool named `answer` with its
answer. An invalid answer goes back to the model as an error, so the request
continues rather than ending with nothing. A model that ends with text instead is
reminded once to call `answer`. Without a schema, the answer is the text of the
model's last call. A request that reaches `maxSteps`, or ignores the reminder,
without an answer fails.

A request with tools runs within one server request. A tool loop inside a durable
`step.run` is one step: if the server stops during the loop, the step's outcome is
unknown, and its tools may already have had effects.

---

## Responses

The response is returned immediately:

- `partials`: async iterable of the answer so far: an incomplete object, or the
  text of the current model call without a schema. For display only.
- `steps`: async iterable of each model call as it finishes, with its tool calls
  and results.
- `output`: promise of the validated answer, or text without a schema.
- `usage`: promise of provider-reported token usage across the request's model
  calls, or `undefined` when none was reported.
- `outcome`: promise of the result as a durable step outcome. It never rejects.

Stream `partials` or `steps` to show progress, then use `output`. Awaiting `output`
alone also completes the request. `output` rejects with the provider's error, a
schema validation error, or a cancellation. Nothing is retried unless the model's
`maxRetries` allows it.

---

## Generate an image

`ai.image(request)` draws one image with a configured image model:

```ts
const response = ai.image({
  view,
  model: "painter",
  prompt: "A brass telescope on a desk, flat colours.",
  references: [{ data: sketch, mediaType: "image/png" }],
  size: "1024x1024",
  providerOptions: { openai: { quality: "medium" } },
  signal,
})
const picture = await response.output // { data, mediaType }
```

`references` are optional images to draw from or edit, sent as given. `size` or
`aspectRatio` is optional, in the forms your model accepts. The response
has `output`, `usage` and `outcome`, with the outcomes below. The image is returned
to your code; nothing is saved until you save it.

---

## Record usage

Pass `meter` to record every request that reached a provider, including requests
made inside tools:

```ts
const ai = AI({
  models,
  meter: ({ view, model, modelId, usage, status }) => recordUsage(view, model, modelId, usage, status),
})
```

`meter` runs once per request, after it settles, with the request's View, the
configured model name, the provider's model id, provider-reported usage and the
outcome status. A request stopped before sending is not metered. A response's
`output` and `outcome` resolve after `meter` returns; an error thrown by `meter` is
logged and does not change the result.

Name models by role when requests need different limits or provider options: two
names, such as `builder` and `drafter`, may use the same provider model. Usage then
arrives labelled by role.

---

## Use a request in a durable job

Return `outcome` from a [`step.run`](./jobs.reference#save-steps-and-retry-safely)
activity. The job saves the validated result once, so replay never calls the model
again. Then save it through an Action with `step.action`:

```ts
const feedback = await step.run("review", async () => {
  const submission = view.readSource(event.data.submission)
  if (!submission) return { status: "blocked" as const, reason: "Submission unavailable" }
  return ai.generate({ view, model: "reviewer", instructions: guide, input: [submission], schema: Feedback }).outcome
}, { recover: "stop", decode: (value) => Feedback.parse(value) })
await step.action("save", view, "reviews.save", feedback, { target: event.data.submission })
```

`view` is the admitted View the job acts through, `Feedback` a Zod schema and
`reviews.save` an Action. An [agent](./agents.reference#use-an-agent-in-a-durable-job)
is a natural Actor for such a job. Save results with `step.action` after the
request rather than inside a tool, so a recovered job finds the saved receipt
instead of repeating the effect. Outcomes follow the provider's result:

| Result | Outcome |
| --- | --- |
| Validated output | `completed` |
| Unconfigured model or unreadable Source | `blocked`; nothing was sent |
| A language model passed to `ai.image` or an image model to `ai.generate`, or a tool named `answer` beside a schema | `failed`; nothing was sent |
| Output that fails the schema, no image, or no answer within `maxSteps` | `failed`; the call may have been charged, so it is not repeated |
| Provider rate limit (429) before any caller tool starts, after any `maxRetries` | `retry` after the provider's `Retry-After`, or 60 seconds |
| Provider rate limit (429) after a caller tool starts, even if that tool throws | `unknown`; reconcile possible tool effects before retrying the request |
| Provider refusal (401, 403 or 404) | `blocked` until access is restored |
| Any other error, including cancellation | `unknown`; the provider may have charged |

Retry a blocked or stopped job with the existing job operations once its cause is
fixed. Model output that you write into a Source is authored content: validate it,
and escape text so it cannot introduce Blocks, links or components.