Models and the budget guards

Connecting OpenRouter

Novaterra speaks to exactly one LLM provider directly: OpenRouter, using its OpenAI-compatible API. Get a key from your OpenRouter dashboard and set it in .env:

OPENROUTER_API_KEY=sk-or-v1-...

That’s the entire integration step — no SDK config, no per-model keys. Restart the API and GET /api/health will report the LLM as configured. Without a key, the app still boots; every LLM-backed route (Muse chat, Studio project runs, memory extraction, image generation) returns a clear, typed error instead of hanging or crashing.

packages/llm is deliberately provider-agnostic underneath: it only speaks the OpenAI-compatible protocol, which is what makes local-first inference possible per tier without a rewrite — move cheap and standard onto your own GPU and leave strong on OpenRouter, all from .env.

The three tiers (and two more)

Every LLM call in Novaterra names a tier, never a raw model id. ModelTier is 'cheap' | 'standard' | 'strong' | 'image' | 'embed', and the mapping from tier to actual model lives in exactly one file, packages/llm/src/models.ts, so the whole system can be re-pointed at different models without touching call sites:

Tier	Default model	Fallback	Used for
`cheap`	`google/gemini-2.5-flash-lite`	`openai/gpt-4.1-nano`	Workers, summaries, classification, memory extraction — the default for almost everything
`standard`	`google/gemini-2.5-flash`	`openai/gpt-4.1-mini`	Muse chat, agent reasoning inside a Studio crew
`strong`	`anthropic/claude-sonnet-4`	`google/gemini-2.5-pro`	The Studio planner (breaking a brief into a task graph) and the QA reviewer — nowhere else
`image`	`google/gemini-2.5-flash-image`	`google/gemini-3.1-flash-lite-image`	`image.generate`, with an SVG placeholder fallback if the model is unavailable
`embed`	`openai/text-embedding-3-small`	(same)	Memory and file embeddings for semantic search

The fallback model is used automatically on a 5xx, a 429, or a timeout from the primary — you don’t see the failure unless both fail.

Override any of these without touching code, via .env:

NOVA_MODEL_STANDARD=openai/gpt-4.1-mini
NOVA_MODEL_STANDARD_FALLBACK=google/gemini-2.5-flash

GET /api/llm/models returns the fully resolved table (env overrides applied) if you want to confirm what’s actually active.

Operating rule: cheap by default. strong is reserved for the planner and QA steps only — no seed generator step uses it anywhere else, and this is enforced structurally: the generator validator (validateSeedGenerators in apps/api/src/studio/generators/index.ts) rejects any step that requests strong.

The budget guards

Because the OpenRouter key funding a Novaterra install is a real, finite balance, every call is checked against three hard ceilings before it’s made — not after, and not as a soft warning.

LLM_BUDGET_TOTAL_USD=18        # lifetime ceiling for this install
LLM_BUDGET_DAILY_USD=18        # ceiling per calendar day
LLM_BUDGET_PER_PROJECT_USD=1.5 # ceiling per Studio project

The mechanism (apps/api/src/core/budget.ts):

Every completed call — success or failure — is written to the llm_calls table with its real cost (costUsd, derived from OpenRouter’s own usage.cost when present, or estimated from the per-tier price table otherwise).
Before a new call goes out, assertWithinBudget() sums relevant prior spend (total, today, and — if the call carries a projectId — that project alone) and compares against the three ceilings.
If any applicable ceiling is already met or exceeded, the call is refused before it reaches OpenRouter, by throwing LlmBudgetExceededError with the exhausted scope ('total' | 'daily' | 'project') and the limit that was hit.
The first refusal in a scope also raises a decision-kind Signal on the desktop, so a human sees “the daily budget is spent” rather than a silent wall of errors.

0 means zero dollars, not unlimited — set that way, every call is refused, guard working exactly as designed. To actually disable one of the three guards, set it to off, none, or unlimited instead — useful for a throwaway local sandbox where cost genuinely doesn’t matter, but not recommended once a real key is attached.

Local tiers are free and don’t draw down the budget

Since local-first inference (PLAN §7b), any tier can be moved from OpenRouter onto your own GPU with LLM_PROVIDER_<TIER>=local. A call answered locally costs $0 — there’s no OpenRouter usage to bill — so it never counts against LLM_BUDGET_TOTAL_USD, LLM_BUDGET_DAILY_USD, or LLM_BUDGET_PER_PROJECT_USD above. Move cheap and standard local and only strong (say) still draws down the budget guards.

That does not mean local calls go unrecorded: every call, local or remote, is still written to llm_calls (with costUsd: 0 for local ones), so GET /api/llm/usage and the wallet widget keep one complete history no matter where each tier is actually answered.

Checking usage

GET /api/llm/usage

Returns LlmUsage: today’s spend, total spend, call count, a breakdown by model, and the most recent calls — plus a budget snapshot with each ceiling and how much headroom remains. The wallet widget on the Living Desktop surfaces a live version of this so spend is always visible, never hidden — which is also Article IX-adjacent in spirit: a world that hides its costs will eventually lie about them.

A note for anyone testing against a real key

If you’re running smoke tests against a live OPENROUTER_API_KEY, treat it as scarce: use the cheap tier, keep prompts short, cap yourself at a handful of real calls per test, never loop, and avoid running a full Studio project on strong more than once. Prefer mocking the LLM client for anything that doesn’t need to prove real model behaviour, and reserve the real key for one end-to-end proof per feature.