AI

All AI access goes through the Vercel AI Gateway via getAIClient(product) from @/lib/ai. No provider SDKs, no per-provider keys, one bill. Planer is the working reference: a streaming document refiner with tiers, quota, rate limiting, and full cost logging.

Setup

On Vercel, gateway auth is automatic (OIDC). Anywhere else, set one key:

AI_GATEWAY_API_KEY=...

Without it, AI features return a friendly not-configured error instead of crashing.

Tiers, not model names

Product code asks for a tier, never a model:

const ai = getAIClient("planer");
const result = await generateText({
  model: ai.model("fast"),
  providerOptions: ai.options("suggest-title", user.id),
  prompt,
  abortSignal: aiDeadline("quick"),
});

AITier is "default" (quality work: writing, reasoning, reliable structured output) or "fast" (simple, high-volume, latency-sensitive calls). A tier's gateway slug resolves in order:

1. the product manifest's aiModels[tier] in src/lib/products.ts (swapping a product's model is a one-field edit)
2. env AI_MODEL_DEFAULT / AI_MODEL_FAST (a fleet-wide swap without a code change)
3. the chassis defaults

ai.slug(tier) returns the resolved slug, useful for logging and for honest UI (planer's pass toggle shows the real model name). ai.options(feature, userId) tags the call for gateway cost attribution.

Every call has a deadline

A model call with no limit waits as long as the gateway does. If the gateway hangs, the function is killed at its maxDuration, and a killed function runs no catch: no failure row, no words for the user, and in a cron every later step starves. So every generateText / generateObject / streamText / streamObject call passes abortSignal: aiDeadline(kind) from @/lib/ai/deadline:

Kind	Limit	For
quick	30 s	a short read on the fast tier: a title, a caption, a classification
standard	45 s	a judgement or a short generation; fits a 60 s route with room to fail
long	100 s	a large structured generation; needs a 120 s+ host
extended	240 s	one pass of a multi-minute write; only for a host with minutes to spare

Pick by what the call does AND what hosts it: the limit has to leave the function time to run its failure path. A call past its deadline rejects from the same await a provider error would, so your existing catch handles it. A stream is the exception: an aborted stream ends its iterator instead of throwing, so a streaming route merges the client's signal in (aiDeadline("standard", request.signal)) and reads the merged signal's aborted after its loop; planer's refine route shows both. src/app/_products/ai-deadlines.test.ts fails any model call without one.

Gate before you spend

Order matters, and the planer refine route (src/app/(planer)/api/planer/refine/route.ts) shows the canonical sequence: validate input, authenticate, resolve the org, rateLimit(..., "ai"), consume quota, and only then call the model.

Quota is claimed atomically up front through a database RPC (planer.claim_pass), not a read-then-write check that two concurrent requests can race past. One transaction counts the month and inserts the running row, so the counted set is the inserted set: a crash between the two cannot burn an invisible slot. It counts all statuses, so an aborted call still consumed its slot; a NULL cap means unlimited.

The ledger: logUsage

Every call ends in exactly one logUsage(product, record):

await logUsage("planer", {
  orgId,
  userId: user.id,
  provider: providerOf(ai.slug(tier)),
  model: ai.slug(tier),
  feature: "refine",
  inputTokens: usage.inputTokens,
  outputTokens: usage.outputTokens,
  durationMs,
  success: true,
});

It does two things at once: writes a row to core.ai_usage_events (your own queryable ledger, priced via the local table or the gateway-reported cost) and emits the PostHog $ai_generation span, so spend dashboards cover every product with no extra wiring. Because logUsage is the single emitter, never emit $ai_generation yourself; a second emitter double-counts every spend tile.

Failed calls are not free

A call that fails after the model produced output (schema mismatch, parse failure, aborted stream) was still billed, but there is no result object to read token usage from. On every failure path, pass the thrown error through:

await logUsage("planer", {
  orgId,
  provider, model, feature,
  success: false,
  errorMessage: message,
  error: err, // logUsage recovers the billed tokens from the error body
});

extractErrorUsage pulls the provider-billed tokens out of the error and prices them, so the ledger stays honest. Never hardcode costUsd: 0 on a failure; genuinely un-billed failures (the request never reached the model) resolve to zero on their own. Planer logs both the success and the failure path of every streaming pass; copy that shape.

When the gateway runs out of credit

The AI Gateway runs on a prepaid balance. When it reaches zero, every text call on every product fails until someone tops it up, and to the code that made the call it looks like any other failure. isAIGatewayOutOfCredit(err) (from @/lib/ai) tells this one apart: it reads the gateway's own sentence ("A positive credit balance is required…") or a gateway 402, on a thrown error or a stored message, and leaves every other failure alone.

You do not wire the alarm yourself. logUsage checks every failed call it records, and the first one that is the gateway out of credit emails SUPER_ADMIN_EMAILS (an owner alert of kind ai_gateway_dry, with the gateway's words and a top-up link). After that it stays quiet for an hour across every instance, then sounds again if the gateway is still dry. The throttle is claimOncePer from @/lib/rate-limit, an atomic Redis claim. The alarm never throws, so it cannot break the failure path it reports on. This is one more reason to call logUsage on every failure path.

What your users see is handled for you at the action wrappers. When a handler wrapped in action(), formAction() or actionRoute throws a dry-gateway error, the wrapper answers with aiDownError instead of the fallback and files no error report (the alarm already rang). The default, AI_DOWN_ERROR, says the feature is down for a while and to try again later; pass aiDownError on the config, once on a shared base, to say it in your product's voice. The route door answers it with HTTP 503. A manual action that ends its catch with failure(name, err, fallback) gets the same: AI_DOWN_ERROR instead of the fallback, logged, never reported. A failure path outside the wrappers (a cron, a webhook, a stored issue's error message) checks isAIGatewayOutOfCredit(err) itself and says the same, rather than "try again", which cannot help.

Budgets: there are none

The chassis measures AI spend and does not cap it. logUsage writes cost_usd into core.ai_usage_events on every call, success and failure, so you can see what you spent. Nothing reads that back and nothing refuses a call because the month got expensive. There is no budget env var.

If you need a ceiling, it goes in the same place as the quota gate (step 2 of the recipe at the end of this page): sum cost_usd for the product and the current month, compare it to your own limit, and refuse BEFORE you claim quota, the way "Gate before you spend" refuses on the unconfigured-gateway check. Cache the sum, or you buy one query per call.

Until then the real bound on a public deployment is whatever per-org quota the product enforces for itself, multiplied by the number of orgs, and an org is created per signup. That makes SIGNUPS_CLOSED the spend control. See "Running the worked products in public" in deploying.

Streaming

Planer streams with streamText over a hand-rolled SSE envelope ({passId}, {content}, {done}, {error, message}) and a small client reader hook (use-refine-stream.ts). It persists the row before streaming starts (so an aborted pass survives), then updates it and logs usage on settle. If you need streaming, start from that route; it is the documented reference.

What to copy for a new AI feature

1. Pick a tier and a feature name (the ledger and gateway tags key on it).
2. Gate: auth, entitlement or quota (atomic), rateLimit(..., "ai").
3. Call through ai.model(tier) with ai.options(feature, userId) and abortSignal: aiDeadline(kind).
4. logUsage on success and on failure, with error: err on the failure path.

The recipe (with the sharp edges: generateObject schema bounds, cron retry caps) lives in .claude/rules/recipes.md, and docs/prompts/add-an-ai-feature.md is a ready-to-run prompt for an agent.