AI

@pathmx/ai gives Plugins named AI models, called through an admitted View.

Requests stream partial results, validate structured output, can call your tools and can end a durable job step. Image models draw pictures through the same service. For agents authored as Markdown Actors, see Agents.

It is built on the AI SDK. Requests run on the server; do not include this package in a browser publication.

Add @pathmx/ai and ai to your project's dependencies, aligned with your other PathMX packages. Create one service and pass it to the Plugins that use it. Create plugins/course.plugin.ts:

import { createGateway } from "ai"
import { AI } from "@pathmx/ai"
import { Tutor } from "./tutor.ts"

const gateway = createGateway({ apiKey: process.env.AI_GATEWAY_API_KEY })

const ai = AI({
  models: {
    tutor: gateway("provider/model-name"),
    reviewer: {
      model: gateway("provider/model-name"),
      maxOutputTokens: 2000,
      providerOptions: { gateway: { caching: "auto" } },
    },
    painter: gateway.imageModel("provider/image-model-name"),
  },
})

export default function coursePlugins() {
  return [Tutor(ai)]
}

Only files named *.plugin.ts are loaded as Plugins, so tutor.ts is an ordinary module. Replace the model names with models your provider offers. Sources and code refer only to the names you choose, here tutor, reviewer and painter. A request for any other name stops before anything is sent. Keep API keys in the server environment, not in Sources or package configuration.

Each name accepts any AI SDK language or image model: a gateway, a direct provider, a local model, or a deterministic model for tests. Settings beside a model apply to every request to it:

• maxOutputTokens: output cap for each model call. Defaults to 4096.
• maxRetries: retries of one model call after a transient provider error, such as a 5xx or a rate limit. Defaults to 0, so nothing is spent again unless you choose.
• providerOptions: AI SDK provider options, such as caching or reasoning effort. A request's own providerOptions merge over them, provider by provider.

Configured models are called whenever a request reaches them. Use deterministic or local models where no provider call should happen, such as in tests.

Pass the caller's View, a model name, instructions and input. Create plugins/tutor.ts, which course.plugin.ts imports:

import { definePlugin, route } from "@pathmx/core"
import type { AI } from "@pathmx/ai"
import { z } from "zod"

const Question = z.object({ lesson: z.string(), question: z.string().max(500) })
const Answer = z.object({ explanation: z.string(), nextQuestion: z.string() })

export const Tutor = (ai: AI) => definePlugin({
  id: "tutor",
  routes: [
    route.post("ask", async (ctx) => {
      const session = await ctx.session()
      if (session.type !== "authenticated") return new Response(null, { status: 401 })
      const view = ctx.view(session)
      const { lesson: id, question } = Question.parse(await ctx.request.json())
      const lesson = view.readSource(id)
      const guide = view.readSource("/tutors/socratic.tutor")
      if (!lesson || !guide) return new Response(null, { status: 404 })
      const response = ai.generate({
        view,
        model: "tutor",
        instructions: guide,
        input: [lesson, question],
        schema: Answer,
        signal: ctx.request.signal,
      })
      return Response.json(await response.output)
    }),
  ],
})

Create paths/tutors/socratic.tutor.md. Its body is the instructions:

# Socratic tutor

Help the learner reason about their own evidence. Ask what they observed before
explaining anything. End with one useful question.

A signed-in reader can POST /api/tutor/ask with { "lesson": "/lessons/bridge.page", "question": "How should I test it?" } and receives { "explanation": "…", "nextQuestion": "…" }. The route decides who may make paid requests; here, only signed-in readers.

ai.generate(request) accepts:

• view: the admitted View making the request.
• model: a configured language model name.
• instructions: optional Source whose body is the instructions, or instruction text.
• input: Sources, strings and files, in order: the request's message.
• history: optional earlier turns, replayed before input as real messages: { role: "user", content: [...] } with the same items as input, or { role: "assistant", content: "…" }. Sources in them must be readable too.
• schema: optional Zod schema for structured output.
• tools: optional AI SDK tools the model may call. See Give the model tools.
• maxSteps: model calls allowed when tools are given. Defaults to 20.
• providerOptions: optional provider options for this request, merged over the model's.
• signal: optional AbortSignal that cancels the request.

Each Source in input is sent as its Markdown body inside <source id="/lessons/bridge.page">…</source>, so a model can refer to it by id. Frontmatter and attachments are not sent; pass the values you intend to share as strings. Every Source in input or instructions must be readable through view.

A file is { data, mediaType, name? }, with the bytes as a Uint8Array: a photo, a screenshot, a PDF or another file the provider accepts. Like a string, it is sent as given: read it with the access checks your domain needs, and keep it within the provider's size limit.

const response = ai.generate({
  view,
  input: [problem, key, "Check the student's work.", { data: bytes, mediaType: "image/jpeg" }],
  schema: Reply,
})

Tools are ordinary AI SDK tools. The model calls them, sees their results and continues, up to maxSteps model calls. Choose the tools for each request: a tool you leave out cannot be called, and a tool can only do what its own code and the request's View allow.

import { tool } from "ai"

const files = new Map(draft)
const response = ai.generate({
  view,
  model: "builder",
  instructions: guide,
  input: [overview, request],
  tools: {
    read_file: tool({
      description: "Read a file from the draft.",
      inputSchema: z.object({ path: z.string() }),
      execute: async ({ path }) => files.get(path) ?? "No such file.",
    }),
    look: tool({
      description: "Screenshot the draft on a phone.",
      inputSchema: z.object({}),
      execute: async () => [{ data: await screenshot(files), mediaType: "image/png" }],
    }),
  },
  maxSteps: 40,
  schema: z.object({ message: z.string() }),
  signal,
})

for await (const step of response.steps)
  progress(step.toolCalls.map((call) => call.toolName))
const reply = await response.output

A tool may return strings, Sources and files, alone or in an array; they reach the model as request input does. Each Source, including its instance when present, must be readable through the request's View, or the model receives an error instead. This check applies to every async-generator yield and runs before a custom toModelOutput formatter. Other results are sent as JSON unless the tool provides its own formatter.

With a schema, the model finishes by calling a tool named answer with its answer. An invalid answer goes back to the model as an error, so the request continues rather than ending with nothing. A model that ends with text instead is reminded once to call answer. Without a schema, the answer is the text of the model's last call. A request that reaches maxSteps, or ignores the reminder, without an answer fails.

A request with tools runs within one server request. A tool loop inside a durable step.run is one step: if the server stops during the loop, the step's outcome is unknown, and its tools may already have had effects.

The response is returned immediately:

• partials: async iterable of the answer so far: an incomplete object, or the text of the current model call without a schema. For display only.
• steps: async iterable of each model call as it finishes, with its tool calls and results.
• output: promise of the validated answer, or text without a schema.
• usage: promise of provider-reported token usage across the request's model calls, or undefined when none was reported.
• outcome: promise of the result as a durable step outcome. It never rejects.

Stream partials or steps to show progress, then use output. Awaiting output alone also completes the request. output rejects with the provider's error, a schema validation error, or a cancellation. Nothing is retried unless the model's maxRetries allows it.

ai.image(request) draws one image with a configured image model:

const response = ai.image({
  view,
  model: "painter",
  prompt: "A brass telescope on a desk, flat colours.",
  references: [{ data: sketch, mediaType: "image/png" }],
  size: "1024x1024",
  providerOptions: { openai: { quality: "medium" } },
  signal,
})
const picture = await response.output // { data, mediaType }

references are optional images to draw from or edit, sent as given. size or aspectRatio is optional, in the forms your model accepts. The response has output, usage and outcome, with the outcomes below. The image is returned to your code; nothing is saved until you save it.

Pass meter to record every request that reached a provider, including requests made inside tools:

const ai = AI({
  models,
  meter: ({ view, model, modelId, usage, status }) => recordUsage(view, model, modelId, usage, status),
})

meter runs once per request, after it settles, with the request's View, the configured model name, the provider's model id, provider-reported usage and the outcome status. A request stopped before sending is not metered. A response's output and outcome resolve after meter returns; an error thrown by meter is logged and does not change the result.

Name models by role when requests need different limits or provider options: two names, such as builder and drafter, may use the same provider model. Usage then arrives labelled by role.

Return outcome from a step.run activity. The job saves the validated result once, so replay never calls the model again. Then save it through an Action with step.action:

const feedback = await step.run("review", async () => {
  const submission = view.readSource(event.data.submission)
  if (!submission) return { status: "blocked" as const, reason: "Submission unavailable" }
  return ai.generate({ view, model: "reviewer", instructions: guide, input: [submission], schema: Feedback }).outcome
}, { recover: "stop", decode: (value) => Feedback.parse(value) })
await step.action("save", view, "reviews.save", feedback, { target: event.data.submission })

view is the admitted View the job acts through, Feedback a Zod schema and reviews.save an Action. An agent is a natural Actor for such a job. Save results with step.action after the request rather than inside a tool, so a recovered job finds the saved receipt instead of repeating the effect. Outcomes follow the provider's result:

Retry a blocked or stopped job with the existing job operations once its cause is fixed. Model output that you write into a Source is authored content: validate it, and escape text so it cannot introduce Blocks, links or components.