AI

All AI access goes through the Vercel AI Gateway via getAIClient(product) from @/lib/ai. No provider SDKs, no per-provider keys, one bill. Planer is the working reference: a streaming document refiner with tiers, quota, rate limiting, and full cost logging.

Setup

On Vercel, gateway auth is automatic (OIDC). Anywhere else, set one key:

AI_GATEWAY_API_KEY=...

Without it, AI features return a friendly not-configured error instead of crashing.

Tiers, not model names

Product code asks for a tier, never a model:

const ai = getAIClient("planer");
const result = await generateText({
  model: ai.model("fast"),
  providerOptions: ai.options("suggest-title", user.id),
  prompt,
});

AITier is "default" (quality work: writing, reasoning, reliable structured output) or "fast" (simple, high-volume, latency-sensitive calls). A tier's gateway slug resolves in order:

1. the product manifest's aiModels[tier] in src/lib/products.ts (swapping a product's model is a one-field edit)
2. env AI_MODEL_DEFAULT / AI_MODEL_FAST (a fleet-wide swap without a code change)
3. the chassis defaults

ai.slug(tier) returns the resolved slug, useful for logging and for honest UI (planer's pass toggle shows the real model name). ai.options(feature, userId) tags the call for gateway cost attribution.

Gate before you spend

Order matters, and the planer refine route (src/app/(planer)/api/planer/refine/route.ts) shows the canonical sequence: validate input, authenticate, resolve the org, rateLimit(..., "ai"), consume quota, and only then call the model.

Quota is claimed atomically up front through a database RPC (planer.claim_pass), not a read-then-write check that two concurrent requests can race past. One transaction counts the month and inserts the running row, so the counted set is the inserted set: a crash between the two cannot burn an invisible slot. It counts all statuses, so an aborted call still consumed its slot; a NULL cap means unlimited.

The ledger: logUsage

Every call ends in exactly one logUsage(product, record):

await logUsage("planer", {
  orgId,
  userId: user.id,
  provider: providerOf(ai.slug(tier)),
  model: ai.slug(tier),
  feature: "refine",
  inputTokens: usage.inputTokens,
  outputTokens: usage.outputTokens,
  durationMs,
  success: true,
});

It does two things at once: writes a row to core.ai_usage_events (your own queryable ledger, priced via the local table or the gateway-reported cost) and emits the PostHog $ai_generation span, so spend dashboards cover every product with no extra wiring. Because logUsage is the single emitter, never emit $ai_generation yourself; a second emitter double-counts every spend tile.

Failed calls are not free

A call that fails after the model produced output (schema mismatch, parse failure, aborted stream) was still billed, but there is no result object to read token usage from. On every failure path, pass the thrown error through:

await logUsage("planer", {
  orgId,
  provider, model, feature,
  success: false,
  errorMessage: message,
  error: err, // logUsage recovers the billed tokens from the error body
});

extractErrorUsage pulls the provider-billed tokens out of the error and prices them, so the ledger stays honest. Never hardcode costUsd: 0 on a failure; genuinely un-billed failures (the request never reached the model) resolve to zero on their own. Planer logs both the success and the failure path of every streaming pass; copy that shape.

Budgets: there are none

The chassis measures AI spend and does not cap it. logUsage writes cost_usd into core.ai_usage_events on every call, success and failure, so you can see what you spent. Nothing reads that back and nothing refuses a call because the month got expensive. There is no budget env var.

If you need a ceiling, it goes in the same place as the quota gate (step 2 of the recipe at the end of this page): sum cost_usd for the product and the current month, compare it to your own limit, and refuse BEFORE you claim quota, the way "Gate before you spend" refuses on the unconfigured-gateway check. Cache the sum, or you buy one query per call.

Until then the real bound on a public deployment is whatever per-org quota the product enforces for itself, multiplied by the number of orgs, and an org is created per signup. That makes SIGNUPS_CLOSED the spend control. See "Running the worked products in public" in deploying.

Streaming

Planer streams with streamText over a hand-rolled SSE envelope ({passId}, {content}, {done}, {error, message}) and a small client reader hook (use-refine-stream.ts). It persists the row before streaming starts (so an aborted pass survives), then updates it and logs usage on settle. If you need streaming, start from that route; it is the documented reference.

What to copy for a new AI feature

1. Pick a tier and a feature name (the ledger and gateway tags key on it).
2. Gate: auth, entitlement or quota (atomic), rateLimit(..., "ai").
3. Call through ai.model(tier) with ai.options(feature, userId).
4. logUsage on success and on failure, with error: err on the failure path.

The recipe (with the sharp edges: generateObject schema bounds, cron retry caps) lives in .claude/rules/recipes.md, and docs/prompts/add-an-ai-feature.md is a ready-to-run prompt for an agent.