AI Provider Settings

AI Provider Settings — Per-User LLM Chain

Every authenticated dashboard user can pick their preferred LLM provider and an ordered failover chain at /portfolio/ai-providers. The selected chain is consulted by dashboard endpoints that have an authenticated principal (currently /api/research/analytics/ai-insight); the trading bot's main loop, the AI research worker, and Telegram commands keep using the system-wide Codex CLI default.

How it fits together

• db/074_user_provider_settings.sql — single-row-per-user table in public; API keys are Fernet-encrypted at rest using BROKER_SECRET_KEY / PORTFOLIO_SECRET_KEY.
• arb_bot/portfolio_app/db/provider_settings_repository.py — per-user CRUD; the canonical tenant-scoped read path. RLS policy is installed by the existing bootstrap via the TENANT_TABLES list in arb_bot/auth/tenancy.py.
• arb_bot/portfolio_app/llm/providers.py — LLMProvider Protocol plus four implementations: CodexCLIProvider (wraps the existing CodexRunner, default, no key required), MiniMaxProvider (HTTP, MiniMax OpenAI-compatible), OpenRouterProvider (HTTP, OpenRouter), and OpenCodeGoProvider (HTTP, OpenCode Go's OpenAI-compatible gateway, multi-format transport). An LLMProviderChain wrapper tries each in order and returns the first success.
• arb_bot/portfolio_app/llm/resolver.py — resolve_user_provider(user_id) reads the row, decrypts keys, and builds the chain. resolve_default_provider(feature=...) returns a codex-only chain for system paths without a tenant, unless the feature maps to the fast tier (then opencode-go + codex failover, using the system key).
• arb_bot/portfolio_app/api/routes/ai_providers.py — GET /api/portfolio/ai-providers/catalog, GET/PUT/DELETE /api/portfolio/ai-providers/settings. Plaintext keys are never returned; responses carry only a presence flag + last 4 chars per provider.
• frontend/src/portfolio/pages/AIProviderSettings.tsx — dashboard UI: provider select, drag-reorder failover chain, masked API-key inputs, save / reset actions.

Default behaviour

A fresh user with no row sees preferred_provider = codex and a single-element failover list — Codex CLI uses the machine's codex login, so no API key is required and the dashboard works out of the box. HTTP providers only enter the chain when the user adds an API key.

Configuration env vars (system defaults)

• LLM_PROVIDER_DEFAULT — fallback when a user has no row.
• LLM_FAILOVER_DEFAULT — comma-separated default failover order.
• LLM_MINIMAX_BASE_URL, LLM_MINIMAX_DEFAULT_MODEL, LLM_MINIMAX_TIMEOUT_SEC.
• LLM_OPENROUTER_BASE_URL, LLM_OPENROUTER_DEFAULT_MODEL, LLM_OPENROUTER_TIMEOUT_SEC.
• LLM_OPENCODE_GO_BASE_URL, LLM_OPENCODE_GO_DEFAULT_MODEL, LLM_OPENCODE_GO_TIMEOUT_SEC. The default model is deepseek-v4-flash — the opencode-go relay serves the official DeepSeek V4 Flash 0731 build under that ID (the deepseek-v4-flash-0731 ID is only for the DeepSeek-direct / OpenRouter endpoints).
• LLM_OPENCODE_GO_API_KEY — system-wide OpenCode Go key for the fast tier; empty by default, never logged or returned by any API.
• LLM_HTTP_MAX_RETRIES, LLM_HTTP_RETRY_BACKOFF_SEC — shared by the HTTP providers.

Operator notes

• RLS keeps public.user_provider_settings strictly tenant-scoped.
• Plaintext API keys are Fernet-encrypted at rest; only a presence flag + last 4 chars are exposed by the API.
• The first read for a new user seeds a default codex-only row, so the UI never 404s.
• Migration 074_user_provider_settings is additive and applies via the standard db/migrate.py flow.

OpenCode Go model catalog & multi-format transport

arb_bot/portfolio_app/llm/opencode_go_catalog.py holds a static catalog of ~14 curated OpenCode Go models, each carrying its model_id, a display label, its required transport api_format (chat_completions, responses, or messages), and an advisory tier_hint. OpenCodeGoProvider now speaks all three transports, dispatching each call to /chat/completions, /responses, or /messages on the base URL according to the catalog entry for the requested model (unknown models default to chat_completions). The catalog is exposed in GET /api/portfolio/ai-providers/catalog as the models field on the opencode-go entry, which powers the per-user model picker in the dashboard.

Streaming transport

The three HTTP providers (OpenCodeGoProvider, MiniMaxProvider, OpenRouterProvider) implement an optional complete_stream(...) method that yields response tokens incrementally; CodexCLIProvider omits it. LLMProviderChain.complete_stream streams from the first provider that supports it and otherwise falls back to the first successful batch complete(...) call, returned as a single-chunk iterator. arb_bot/portfolio_app/llm/shim.py mirrors this on CodexRunnerShim.run_stream(), which the streaming AI assistant endpoint (POST /api/ai/ask/stream) consumes — codex-only chains therefore degrade to a one-shot batch answer.

Two-tier routing

resolve_default_provider(feature=...) maps each AI feature to a fast or premium tier via Config.AI_FEATURE_TIERS. fast resolves to opencode-go (deepseek-v4-flash) backed by the system-wide LLM_OPENCODE_GO_API_KEY, falling back to codex; premium (and any unknown feature) stays the codex-only chain (gpt-5.6-terra). When no system key is configured, fast degrades to codex.

LLM usage & cost metering

Every successful metered provider call is recorded to portfolio.llm_usage and surfaced on the dashboard LLM usage page at /portfolio/llm-usage, backed by GET /api/portfolio/llm-usage/summary. Cost is an estimate derived from a static per-model price table; metering is best-effort and tenant-scoped, so a failed insert can never take down a live provider call.

The llm_usage table

db/098_llm_usage.sql adds portfolio.llm_usage with id, owner_user_id, feature, provider, model, input_tokens, output_tokens, cached_tokens, cost_usd, and created_at, plus an index on owner_user_id. Rows are tenant-owned; the summary read is always filtered by the authenticated user's id, so one user can never read another user's metering rows.

Price table

arb_bot/portfolio_app/llm/usage.py holds a notional PRICE_TABLE — input and output USD per one million tokens. estimate_cost_usd(model, input_tokens, output_tokens) scales those prices by the reported token counts; an unknown model falls back to 0.0 so metering never raises before a model is catalogued. cached_tokens is recorded but not yet priced.

These are notional list prices for cost visibility, not invoices. Keep PRICE_TABLE in sync with this table if pricing changes.

Recorder & opt-in

LLMUsageRecorder (same module) opens its own connection and swallows failures, so a down database or a bad row never surfaces to the call site. Metering is opt-in: LLMProviderChain.complete only records when the caller has set the usage_context ContextVar to a (feature, user_id) tuple around the call — this lets the stock-task worker, the AI research worker, and dashboard endpoints attribute spend to the right feature and tenant without coupling the chain to any one feature.

Summary endpoint & dashboard

arb_bot/portfolio_app/api/routes/llm_usage.py exposes GET /api/portfolio/llm-usage/summary, returning per-(provider, model) aggregates of tokens, cost, and call count via LLMUsageRecorder.summary. Auth mirrors the sibling ai-providers routes (Depends(get_current_user), 401 when no principal). The dashboard page frontend/src/portfolio/pages/LLMUsage.tsx renders those rows as a responsive table (Provider, Model, Input tokens, Output tokens, Cost USD, Calls) with an empty state until the first metered request.