AI

The AI add-on wires a single Convex action: api.ai.complete. It wraps the Vercel AI SDK with Anthropic and OpenAI providers, runs on the Convex side so your provider key never ships to the device, and is rate-limited per user.

Your app calls it through the useAi() hook — one seam, whatever the sensors or prompt. Pair it with GPS, orientation, motion, or Neural Band input as your product needs.

Enable

pnpm run addon enable ai

This single command:

• renames .disabled files for convex/ai.ts, app/src/lib/useAi.ts, and app/src/lib/AiResponse.tsx
• inserts the AI env schema (AI_PROVIDER, ANTHROPIC_API_KEY or OPENAI_API_KEY, optional AI_MODEL) in env.ts

Then pick a provider and push its key to Convex:

cd packages/backend
pnpm exec convex env set AI_PROVIDER anthropic    # or `openai`
pnpm exec convex env set ANTHROPIC_API_KEY sk-ant-...    # or OPENAI_API_KEY=sk-...
pnpm exec convex dev --once

pnpm run addon disable ai reverses everything (renames the files back to .disabled). Drop the env keys with pnpm exec convex env remove ….

What ships in the box

Why a server-side action

This also lets Convex enforce rate limits per user (via the @convex-dev/rate-limiter component) before paying a provider for the call. See rateLimiter.ts's aiPrompt token bucket.

Configuring the provider

Set in the Convex dashboard (Settings → Environment Variables). Convex functions run in Convex's cloud, not from a local .env:

AI_PROVIDER=anthropic        # or "openai"

# Pick one — the action chooses based on AI_PROVIDER above:
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...

# Optional — override the specific model. Defaults to a cheap fast one:
#   anthropic → claude-3-5-haiku-latest
#   openai    → gpt-4o-mini
AI_MODEL=

Swap providers by changing AI_PROVIDER. No code changes needed.

Calling the seam

import { useAi } from "../lib/useAi";
import { AiResponse } from "../lib/AiResponse";

function MyApp() {
  const ai = useAi();
  return (
    <>
      <button
        className="focusable"
        onClick={() => ai.ask("Write a haiku about glasses.")}
        disabled={ai.loading}
      >
        {ai.loading ? "Thinking…" : "Generate"}
      </button>
      <AiResponse ai={ai} placeholder="Press Generate." />
    </>
  );
}

ai.ask(prompt, {
  system: "You are a concise navigation assistant.",
  maxTokens: 80,
});

"use client";
import { useAction } from "convex/react";
import { api } from "@glasskit/backend/convex/_generated/api";

export function Generate() {
  const complete = useAction(api.ai.complete);
  // …
}

Extending: streaming

The shipped action uses generateText (one-shot). For long responses you'll want streaming. Replace generateText with streamText and return a stream from an HTTP route on the Convex side, then consume it on the client with the AI SDK's useChat hook.

A streaming complete action in convex/ai.ts:

import { streamText } from "ai";
// in an httpAction:
const result = streamText({ model: model(), prompt, system });
return result.toDataStreamResponse();

The companion side then uses useChat({ api: "/your-route" }) to consume the stream. See the Vercel AI SDK docs.

Extending: structured output

import { generateObject } from "ai";
import { z } from "zod";

const { object } = await generateObject({
  model: model(),
  schema: z.object({
    title: z.string(),
    tags: z.array(z.string()).max(5),
  }),
  prompt,
});

Extending: tool use

import { generateText, tool } from "ai";
import { z } from "zod";

const { text } = await generateText({
  model: model(),
  prompt,
  tools: {
    getWeather: tool({
      description: "Get the current weather for a city",
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => ({ city, tempC: 22 }),
    }),
  },
  // Allow the model to call tools then synthesize:
  maxSteps: 5,
});

If you need to call a Convex query/mutation from a tool, do it via ctx.runQuery / ctx.runMutation inside the tool's execute.

Cost discipline

Provider-specific features

If you need a provider-specific feature the AI SDK hasn't surfaced yet (Anthropic extended thinking, OpenAI o1 reasoning budgets, etc.), drop down to the native SDK. Both @ai-sdk/anthropic and @ai-sdk/openai are already installed; you can also pnpm add @anthropic-ai/sdk (or openai) directly and call them from a Convex action exactly the same way.