Models and the budget guards

Connecting OpenRouter

Novaterra speaks to exactly one LLM provider directly: OpenRouter, using its OpenAI-compatible API. Get a key from your OpenRouter dashboard. There are two places it can go, and one of them is not a file.

In the app, which is what an installed node uses. Settings → OpenRouter, owner only (POST /api/llm/key is behind requireOwner). The key is encrypted at rest with the node’s own ENCRYPTION_KEY and applies immediately: storing it rebuilds the model client in-process, so nothing needs restarting. Anyone who is not the owner sees the panel report whether the node has a key, and no box to type one into — the key belongs to the node, and there is no per-being key by design.

In .env, which is the developer path:

OPENROUTER_API_KEY=sk-or-v1-...

GET /api/health reports the LLM as configured either way.

⚠️ OPENROUTER_API_KEY in the environment SHADOWS a key stored in Settings. The environment wins, deliberately — a key on the machine is the operator’s and must not be overridden from a browser — and keyStatus() reports shadowed: true so the panel can say so. If you paste a key into Settings on a box that also has one in .env and nothing appears to change, that is why.

Without a key, the app still boots; every LLM-backed route (Muse chat, Studio project runs, memory extraction, image generation) returns a clear, typed error instead of hanging or crashing.

packages/llm is deliberately provider-agnostic underneath: it only speaks the OpenAI-compatible protocol, which is what makes local-first inference possible per tier without a rewrite — move cheap and standard onto your own GPU and leave strong on OpenRouter, all from .env.

The three tiers (and two more)

Every LLM call in Novaterra names a tier, never a raw model id. ModelTier is 'cheap' | 'standard' | 'strong' | 'image' | 'embed', and the mapping from tier to actual model lives in exactly one file, packages/llm/src/models.ts, so the whole system can be re-pointed at different models without touching call sites:

Tier	Default model	Fallback	Used for
`cheap`	`google/gemini-2.5-flash-lite`	`openai/gpt-4.1-nano`	Workers, summaries, classification, memory extraction — the default for almost everything
`standard`	`google/gemini-2.5-flash`	`openai/gpt-4.1-mini`	Muse chat, agent reasoning inside a Studio crew
`strong`	`anthropic/claude-sonnet-5`	`google/gemini-2.5-pro`	The Studio planner (breaking a brief into a task graph) and the QA reviewer — nowhere else
`image`	`google/gemini-2.5-flash-image`	`google/gemini-3.1-flash-lite-image`	`image.generate`, with an SVG placeholder fallback if the model is unavailable
`embed`	`openai/text-embedding-3-small`	(same)	Memory and file embeddings for semantic search

The fallback model is used automatically on a 5xx, a 429, or a timeout from the primary — you don’t see the failure unless both fail.

The strong tier moved on 2026-09-20, and two things about it are worth knowing

This row said anthropic/claude-sonnet-4 at $3 / $15 per million tokens. Both halves were stale: Sonnet 4 is retired from Anthropic’s first-party API, and claude-sonnet-5 is $2 / $10.

That is not a 33% saving, and you should not plan as though it were. Claude 4.7-generation tokenizers produce roughly 30% more tokens for the same text, so a third more tokens against a third less per token is close to flat. That number is an estimate from Anthropic’s migration guidance, not a measurement we have taken — nobody has run countTokens over a real Studio brief on both tokenizers. Treat this tier as roughly flat in real cost until somebody does, and do not spend the headline drop twice.

No model on this tier has been benched. The measurements in design/MODEL-ESTATE.md are of claude-sonnet-4, which is evidence about that model and not about this tier. Sonnet 5’s grades are absent, which is a different thing from zero, and a step declaring requireMeasured at this tier will be refused rather than quietly downgraded.

Override any of these without touching code, via .env:

NOVA_MODEL_STANDARD=openai/gpt-4.1-mini
NOVA_MODEL_STANDARD_FALLBACK=google/gemini-2.5-flash

GET /api/llm/models returns the fully resolved table (env overrides applied) if you want to confirm what’s actually active.

Operating rule: cheap by default. strong is reserved for the planner and QA steps only — no seed generator step uses it anywhere else, and this is enforced structurally: the generator validator (validateSeedGenerators in apps/api/src/studio/generators/index.ts) rejects any step that requests strong.

The budget guards

Because the OpenRouter key funding a Novaterra install is a real, finite balance, every call is checked against three ceilings before it’s made — not after, and not as a soft warning.

LLM_BUDGET_TOTAL_USD=40        # the operator's exposure, over a moving window (default 40)
LLM_BUDGET_DAILY_USD=6         # the same exposure, per UTC day (default 6)
LLM_BUDGET_PER_PROJECT_USD=1.5 # ceiling per Studio project
LLM_BUDGET_WINDOW_DAYS=30      # how far back the first one looks (default 30)

This is the operator’s exposure ceiling. It is not a purse, and it is not a per-member limit. That distinction is the whole of this section and it used to be wrong here: LLM_BUDGET_TOTAL_USD was described as a lifetime ceiling on every call, and on a node selling memberships that made the sixth paying member able to stop the node thinking for everybody else. Since 2026-09-20 there are two separate controls:

who is exposed	what limits them	where
a member	the credits they bought	`credit_entries`, per being, append-only
the operator	`LLM_BUDGET_*_USD`	`apps/api/src/core/budget.ts`

One rule: the ceiling counts, and refuses, only the spend nobody is billed for. A member’s call is billed to them (llm_calls.billed_to, resolved from the session), and a billed call can neither be refused by the ceiling nor move it. What the ceiling counts is the node’s own background work, the owner’s own use, and calls on a node whose owner has opened its key to everybody.

The mechanism (apps/api/src/core/budget.ts):

Every completed call — success or failure — is written to the llm_calls table with its real cost (costUsd, derived from OpenRouter’s own usage.cost when present, or estimated from the per-tier price table otherwise) and with billed_to recording who pays, or NULL for nobody.
Before a new call goes out, budgetGuard() returns immediately if somebody is being billed for it. Otherwise it sums unbilled prior spend — the last LLM_BUDGET_WINDOW_DAYS days, today, and, if the call carries a projectId, that project alone — and compares against the ceilings.
If any applicable ceiling is already met or exceeded, the call is refused before it reaches OpenRouter, by throwing LlmBudgetExceededError with the exhausted scope ('total' | 'daily' | 'project') and the limit that was hit.
The first refusal in a scope also raises an alert-kind Signal for the owner, so a human sees that the node’s own ceiling is spent rather than a silent wall of errors. Its button opens /admin#money — the Money panel in the operator console, which is where the caps can be raised at runtime. Settings has no budget panel, and /admin#budget is not the right link either: the admin console reads its visible area from the hash and only knows ten area ids, so #budget silently lands on Processes with the caps not even mounted — the budget caps panel lives inside the Money area, which is why the area, not a guess at the panel’s own name, is what the link has to name.

LLM_BUDGET_TOTAL_USD counts a moving window, not a lifetime: set LLM_BUDGET_WINDOW_DAYS to off and it counts every call the node has ever made, which is what it used to do unconditionally. A limit that can only be approached and never recede is a limit that will be hit, by arithmetic, on a long enough timeline.

0 means zero dollars, not unlimited — set that way, every call is refused, guard working exactly as designed. To actually disable one of the three guards, set it to off, none, or unlimited instead — useful for a throwaway local sandbox where cost genuinely doesn’t matter, but not recommended once a real key is attached.

Local tiers are free and don’t draw down the budget

Since local-first inference (PLAN §7b), any tier can be moved from OpenRouter onto your own GPU with LLM_PROVIDER_<TIER>=local. A call answered locally costs $0 — there’s no OpenRouter usage to bill — so it never counts against LLM_BUDGET_TOTAL_USD, LLM_BUDGET_DAILY_USD, or LLM_BUDGET_PER_PROJECT_USD above. Move cheap and standard local and only strong (say) still draws down the budget guards.

That does not mean local calls go unrecorded: every call, local or remote, is still written to llm_calls (with costUsd: 0 for local ones), so GET /api/llm/usage and the wallet widget keep one complete history no matter where each tier is actually answered.

Checking usage

GET /api/llm/usage

Returns LlmUsage: today’s spend, total spend, call count, a breakdown by model, and the most recent calls — plus a budget snapshot with each ceiling and how much headroom remains. The wallet widget on the Living Desktop surfaces a live version of this so spend is always visible, never hidden — which is also Article IX-adjacent in spirit: a world that hides its costs will eventually lie about them.

A note for anyone testing against a real key

If you’re running smoke tests against a live OPENROUTER_API_KEY, treat it as scarce: use the cheap tier, keep prompts short, cap yourself at a handful of real calls per test, never loop, and avoid running a full Studio project on strong more than once. Prefer mocking the LLM client for anything that doesn’t need to prove real model behaviour, and reserve the real key for one end-to-end proof per feature.