Documentation

AI Portfolio Research

title: "AI Portfolio Research" section: "portfolio-ai-research" slug: "portfolio-ai-research"

AI Portfolio Research and Thesis Intelligence

A first-party, advisory-only subsystem inspired by the multi-agent research workflow in TauricResearch/TradingAgents. Portfolio Planner keeps authoritative ownership of accounting, valuation, allocation, reconciliation, policy approval, and broker execution.

Capability

Disabled by default. Enable in three layers:

| Flag | Effect | |---|---| | PORTFOLIO_AI_RESEARCH_ENABLED=1 | capability on; API live | | PORTFOLIO_AI_RESEARCH_WORKER_ENABLED=1 | background worker polls for queued runs | | PORTFOLIO_AI_RESEARCH_SCHEDULE_ENABLED=1 | thesis rechecks at 09:30 IST | | PORTFOLIO_AI_RESEARCH_OUTCOME_ENABLED=1 | 5/20/60-day outcome evaluation at 16:00 IST |

The app must start normally with all flags off. No AI provider failure may break Watchlist Intelligence, holdings, reconciliation, policies, research backtests, or the dashboard.

Cost & limits

PORTFOLIO_AI_RESEARCH_MAX_COST_USD_PER_RUN=1.0, PORTFOLIO_AI_RESEARCH_DAILY_RUN_LIMIT=25, PORTFOLIO_AI_RESEARCH_MAX_INPUT_TOKENS=48000, PORTFOLIO_AI_RESEARCH_MAX_OUTPUT_TOKENS=8000, PORTFOLIO_AI_RESEARCH_TIMEOUT_SEC=60, PORTFOLIO_AI_RESEARCH_MAX_RETRIES=1.

Workflow

The compatibility lifecycle runs 13 sequential stages: CONTEXT → MARKET → FUNDAMENTALS → NEWS → SENTIMENT → BULL_CASE → BEAR_CASE → RESEARCH_SYNTHESIS → AGGRESSIVE_RISK → NEUTRAL_RISK → CONSERVATIVE_RISK → FINAL_SYNTHESIS → THESIS_COMPARISON. Stages checkpoint after every completion; the worker resumes from the first unfinished stage on restart. New runs then execute the 19-stage institutional-publication graph, including specialist, adversarial, reconciliation, committee, editor, and QA stages. A run reaches COMPLETED only after publication QA explicitly passes; a degraded pipeline run (for example a specialist stage whose dependencies were unresolved) persists a structurally complete publication with QA=UNAVAILABLE and its blocking defects, and still reports COMPLETED.

Institutional publication v2

New publication artifacts use a strict, versioned InstitutionalResearchPublicationV2 payload. The fact ledger performs deterministic calculations and preserves source conflicts; publication quality checks block unsupported claims and missing core modules. The PDF route renders the persisted payload with a deterministic A4 renderer and never invokes a model during download. Existing v1 reports remain readable and use the legacy renderer. A specialist may mark an unavailable evidence item as a disclosed data gap and continue; it must not invent a replacement value. Fact-check, committee, editor, and QA failures remain blocking for publication quality (the pipeline marks the run COMPLETED with a degraded publication rather than failing it). PDF download requires only structural completeness: a publication whose QA is not PASS still renders, and the PDF carries a "Not QA approved" banner listing the QA status and blocking defects so a degraded report is never mistaken for a QA-approved one.

Evidence & provenance

Every source is classified as VERIFIED_INTERNAL, VERIFIED_FETCHED, DISCOVERED, or UNAVAILABLE. Material claims carry one of VERIFIED_FACT, CALCULATED_FACT, SOURCE_REPORTED, AI_INFERENCE, UNSUPPORTED. Claim text is stored separately from prose; unsupported factual claims are removed or downgraded.

Historical mode

Historical runs (mode=HISTORICAL) require an explicit cutoff timestamp. The cutoff filter runs in provider-side code before prompts are built; future-dated evidence is rejected and unknown-publication-time evidence is downgraded from VERIFIED_* to DISCOVERED.

Trust Center

Ten event kinds are appended to the existing evidence.events table under pillar=research:

AI_RESEARCH_REQUESTED, AI_RESEARCH_COMPLETED, AI_RESEARCH_PARTIAL, AI_RESEARCH_FAILED, AI_RESEARCH_CANCELLED, AI_RESEARCH_BASELINE_SET, AI_RESEARCH_BASELINE_REPLACED, AI_RESEARCH_DRIFT_DETECTED, AI_RESEARCH_OUTCOME_EVALUATED, AI_RESEARCH_PROMOTION_EVIDENCE_ATTACHED.

PAPER evidence

PromotionRequest accepts an explicit ai_research_dossier_ids list. The selected IDs contribute to the request fingerprint and are recorded with the promotion row. AI evidence cannot change validation, eligibility, or initial capital.

Safety boundary

The AI Research package never imports arb_bot.bot, arb_bot.execution, or arb_bot.command_handler (enforced by a static AST scan in tests).
The orchestrator is exercised end-to-end with patched exploding services for transactions, holdings, broker sync, snapshots, reconciliation, approvals, policy go-live, and orders; none are invoked.
Missing financial values stay unavailable — never coerced to zero (enforced by a tokenize-based code scan).
POST /api/portfolio/ai-research/runs and POST /api/portfolio/ai-research/v3/runs resolve the caller's symbol through the instrument registry (exact canonical_symbol, then a confirmed InstrumentAlias — a REJECTED alias is never a match) before the run is queued. Holding symbols backfilled as aliases (the documented fund case where two symbols share one ISIN+exchange) therefore reach the eligibility gate, which refuses bonds, mutual funds, commodities, and unlisted holdings up front with an actionable message instead of spending a run to emit an empty equity report. The route resolver is read-only and never queues an InstrumentReviewQueue row on a miss.
When the V3 registry lookup misses, POST /api/portfolio/ai-research/v3/runs falls back to the same GlobalResearchInstrumentService the V2 global run uses (_resolve_global_instrument): a ticker yfinance can verify as an equity is resolved and persisted (a yfinance:* instrument row), so a raw symbol like SBIFUNDS that is not yet in the registry queues a run instead of 422-ing with unknown symbol. A symbol yfinance cannot verify, or a non-equity quote (ETF/fund), surfaces the resolver's own actionable 422. The fallback only runs on a registry miss — a registry hit never calls yfinance.

Deployment

Apply migration 051 (ai_research_* + ai_thesis_* tables) and 052 (Trust Center event kinds).
Deploy with all AI Research flags off.
Verify normal Portfolio Planner smoke tests.
Enable manual AI Research capability.
Enable worker only after capability health succeeds. In the production Docker stack the dashboard service is the single durable queue owner; arb-bot explicitly disables its duplicate worker loop.
Run controlled dossiers for a small symbol set; inspect evidence, cost, source quality, historical cutoffs.
Enable thesis schedule and outcome evaluation separately.
Keep all broker and policy execution flags unchanged.

Run leases use the latest durable stage progress rather than the original run start time, so a healthy long publication is not reclaimed after five minutes. Evidence-source persistence is an idempotent PostgreSQL upsert, making resume and crash recovery safe under the (run_id, source_key) uniqueness rule.

Rollback

Feature disablement, not destructive migration rollback. Existing AI Research rows remain available for audit while new execution is disabled.

Live progress feed

While a research run (V2/V3) or a watchlist narrator job generates, the worker publishes research_progress envelopes over the existing /ws/live WebSocket, so the dashboard can render stage transitions and incremental model output instead of only a terminal status. This is observability only — it never changes results, authority, or persistence.

The envelope is

{"type": "research_progress", "run_id", "stage", "event", "text", "ts"}

where event is one of started, token, done, or failed. token frames are emitted only by LLM stages that stream through an HTTP provider (CodexRunnerShim.run_stream); batch (codex) stages publish a single done. The V3 worker publishes stage="run" started/done/failed frames around orchestrator.execute(...); the watchlist narrator publishes stage="narration" frames around its single model call.

Publication is best-effort and tenant-scoped. ProgressPublisher.publish (arb_bot/portfolio_app/ai_research/progress.py) composes the envelope and invokes an injected send callable — in production ConnectionManager. send_to_user from dashboard_server — and never raises. The send callable is injected so the module never imports dashboard_server and stays import-safe for workers. publish_sync drives the async publish from synchronous call sites, and a None publisher is a no-op, so the worker and narrator stay hermetic without one. ConnectionManager stores each socket as a (ws, user_id) tuple; send_to_user(user_id, message) fans a message out only to the sockets owned by the authenticated principal and skips user_id=None conservatively, so a tenant-scoped event is never leaked to a socket whose owner is unknown. A WebSocket hiccup never fails the run, and progress text carries no secrets — only stage names and model output the user is entitled to see.

Pinning a thesis baseline

A thesis baseline is the dossier snapshot that subsequent runs are measured against. Pinning is a deliberate, two-step operator action — it overwrites whatever baseline was previously active for the same instrument (and optional portfolio) and is logged in the Trust Center as AI_RESEARCH_BASELINE_SET / AI_RESEARCH_BASELINE_REPLACED.

Two-click confirmation flow

Inside a dossier on the Holding Detail page, the Pin as thesis baseline button does not place the pin immediately. The first click arms the action — the button label switches to Confirm pin and adopts the accent-3 hover style. The second click posts to POST /api/portfolio/ai-research/runs/{run_id}/set-baseline and, on success, the dossier is refetched so the new baseline row appears in thesis_baselines. The ActiveBaselineBanner then renders and the pin button is hidden (the active-baseline indicator replaces it). On a non-2xx response the button returns to its un-armed state and the operator surfaces the failure through the existing onPinError callback. A click outside the armed button (or any document-level mousedown) cancels the confirm state so a stray click cannot pin by accident. While the pin is in flight the "Run new research" button is also disabled, so the two actions cannot race.

Partial dossiers cannot be pinned

POST /runs/{run_id}/set-baseline is strict about run state:

PARTIAL dossiers (any unfinished stage, or stages that did not produce a persisted AIResearchReport) are rejected with 409 Conflict.
The response body starts with "Run is …" — for example Run is PARTIAL; only COMPLETED runs can be pinned as a baseline. or Run is FAILED; only COMPLETED runs can be pinned as a baseline.
The frontend types this as PinBaselineUnavailableError and surfaces the message via a destructive toast through the onPinError callback wired in HoldingDetail. There is no retry path — finish or replace the dossier first, then pin.

Drift comparisons only become meaningful with a baseline

compare_to_active_baseline returns INSUFFICIENT_EVIDENCE whenever there is no active AIThesisBaseline row or the candidate run has no persisted report. In practice this means:

With zero pinned baselines, every comparison is INSUFFICIENT_EVIDENCE.
Once an operator pins their first COMPLETED dossier, subsequent runs can be classified against it as STRENGTHENING, UNCHANGED, or WEAKENING, and the comparison is persisted to ai_thesis_comparisons plus emitted as AI_RESEARCH_DRIFT_DETECTED.
Each new pin replaces the previous active baseline for the same instrument context (one active baseline per instrument, optionally scoped by portfolio_id); old comparisons remain for audit.

Reading a dossier

A COMPLETED dossier renders a layered view: a decision dashboard at the top, the pinned-baseline banner (when present), the bull/bear/catalyst-and-risk column block, and three optional sections (segment breakdown, peer comparison, valuation-basis chip). Every component is rendered from the same ResearchReport payload and degrades gracefully when the optional fields are absent — legacy dossiers that pre-date the new fields continue to display the rating, action, confidence, thesis, bull/bear, catalysts, risks, and invalidation conditions exactly as before.

Decision dashboard

DossierDashboard is the top card. It shows the rating pill, the portfolio action pill, and the confidence meter in a single flex row, followed by a "Why this rating" header and one-or-two sentence summary. The summary prefers report.why_this_rating when the model client populates it (typically a curated ≤ 60-word paragraph that complements, rather than repeats, the thesis). When why_this_rating is null — for example on a dossier generated before the field existed — the dashboard falls back to the first sentence of report.thesis, so the "Why this rating" UX is preserved for legacy reports. The card uses the shared AIRatingBadge / AIPortfolioActionBadge / AIConfidenceBar from AIResearchBadges, so colour, icon, and tooltip treatment are identical to the inline badges that appeared before.

Active baseline banner

When the dossier's thesis_baselines list contains an active baseline (is_active === true), an ActiveBaselineBanner is rendered immediately beneath the dashboard. The banner shows the pinned run id in a monospace font and the timestamp the baseline was pinned (created_at, em dash if absent). The banner and the DossierActions "Pin as thesis baseline" button are mutually exclusive — when an active baseline exists the button is hidden (DossierActions returns null), and the banner is the single source of truth for "this dossier is the active baseline". When no active baseline exists the banner is hidden and the pin button is the action surface. This pairing avoids ambiguity when the dossier body was auto-loaded from an older run while a fresh run is still in progress.

Catalyst timeline

Inside the catalysts/risks/invalidations column block, CatalystTimeline renders catalysts with a best-effort date column. The component scans each catalyst string for a quarter+fiscal-year notation (Q1 FY27, Q3FY2026) or a month-name + 2- or 4-digit year (Mar 2027, Jul-2026). When at least one catalyst mentions a parseable date, the catalysts render as a vertical <ol> with the matched date fragment in a fixed-width left column (w-28) and the catalyst text on the right; entries without a parsed date show TBD. When no catalyst contains a parseable date, the component falls back to a flat bullet list (the timeline column is hidden in that case so the UI does not show an empty TBD stack). An empty catalysts array is rendered as null and the column shows the No catalysts recorded. fallback. Risks and invalidation conditions keep their original bullet lists — the timeline treatment only applies to catalysts.

Segment breakdown

When report.segment_views is a non-empty array, SegmentBreakdown renders a two-column grid (one column on mobile) of segment cards. Each card shows the segment name, an optional N% revenue chip when revenue_share_pct is present, the segment thesis, and (when populated) catalysts and risks sub-lists with accent2 / danger section headers respectively. The card uses the standard rounded-border / surface treatment that the rest of the dossier uses. The component returns null when the array is empty or null, so legacy dossiers without the field display nothing in its place.

Peer comparison

When report.peer_set is a non-empty array, PeerTable renders a five-column table: Symbol (monospace), Name, Rating (an AIRatingBadge so the colour treatment matches the dashboard), 1Y return (em dash when the field is null), and Rationale. The table is horizontally scrollable on narrow viewports, the peer-table data-testid anchors it for selector queries, and each row uses the same border treatment as the segment cards. The candidate universe is bounded by arb_bot/portfolio_app/ai_research/peer_set.py — a JSON-backed tier map (RELIANCE → {IOC, BPCL, HINDPETRO, BHARTIARTL, AVENUESUPER} plus comparable maps for HDFCBANK and INFY) — which the orchestrator passes into the FINAL_SYNTHESIS prompt as the peer_symbols input. The model emits only those peers that are supported by the supplied evidence. Unknown symbols return an empty peer list from peer_set.lookup, the prompt passes nothing, the model returns nothing, and the dossier simply omits the table.

Valuation basis chip

When report.valuation_basis is a non-empty string, ValuationBasisChip renders an accent-tinted pill with the label Valuation basis: <basis>. The basis is one of a constrained set emitted by the FINAL_SYNTHESIS prompt scaffold: PEG vs sector, DCF stub, SOTP via segment EBITDA, Cyclical P/E band, or Structural re-rating thesis. The chip is hidden when the field is null or empty, so legacy dossiers without a valuation basis show nothing in its place. The chip sits below the peer table and above the claims section, so a single glance communicates why the operator is being asked to look at this number.

Field provenance

The four new dossier fields are all additive and optional on FinalResearchReport (why_this_rating, valuation_basis, segment_views, peer_set). They are produced by the FINAL_SYNTHESIS prompt scaffold and wired through AIResearchOrchestrator._to_final_report into the persisted FinalResearchReport, then carried in AIResearchReport.payload_json. The dossier route promotes them from payload_json to top-level keys in the response so the frontend parser sees the same shape it has always validated against. SegmentView and PeerRef Pydantic models live in arb_bot/portfolio_app/ai_research/contracts.py and validate extra fields as forbid, so malformed payloads surface as a ValidationError upstream of the dashboard rather than rendering partial UI. The frontend parser in frontend/src/portfolio/api/aiResearch.ts mirrors the same shapes — parseSegmentView, parsePeerRef, and the four new parseReport branches — so a backend regression surfaces as an AiResearchValidationError and a destructive state instead of a silent fallback. parseSegmentView enforces the same 0..100 bound on revenue_share_pct that the backend enforces, and rejects present-but- malformed catalysts / risks arrays rather than silently coercing them to [].

V3 Report Quality Evaluation

The V3 quality evaluation subsystem provides a deterministic, blinded, multi-LLM comparison harness for benchmarking AI-generated research reports before V3 generation changes are deployed. It runs offline — never during ordinary generation — and never mutates existing publications.

Architecture

The evaluation pipeline is in arb_bot/portfolio_app/ai_research/v3/evaluation/ and is fully isolated from the V1/V2 generation, API, and PDF paths:

| Step | Module | What it does | |---|---|---| | Adapt | adapters.py | Converts any persisted V2 publication into a neutral, model-agnostic EvaluationDocument | | Gate | hard_gates.py | Runs nine deterministic quality checks (unknown sources, missing invalidation, non-finite numbers, etc.) — all must pass before judges are invoked | | Blind | anonymize.py | Removes generator identity, tier labels, and metadata; creates forward/reverse JudgeRequest pairs keyed on a seed | | Judge | judges.py | Structured, evidence-requiring ReportJudge protocol and a CodexReportJudge adapter; three or more independent judges required | | Consensus | consensus.py | Reconciles forward/reverse scorecards, computes median dimension scores, applies tier thresholds, and returns a PASS/FAIL/REVIEW_REQUIRED verdict | | Harness | harness.py | End-to-end coordinator: load files, gate, blind, run judges, calculate consensus | | CLI | cli.py | Offline benchmark CLI: python -m arb_bot.portfolio_app.ai_research.v3.evaluation.cli --left left.json --right right.json --tier STANDARD --seed 42 |

Key invariants

No judge call without passing hard gates. A report with blocking defects (dangling source references, non-finite numbers, failed QA status) short-circuits before any judge is invoked.
Both orderings always run. Every judge scores the forward pair (seed-determined) and the reverse pair to neutralise position bias.
Minimum three valid judges. Fewer than three valid (complete, non-position-sensitive) judges always yields REVIEW_REQUIRED.
Blinding is structural. JudgeRequest fields cannot carry generator identity, tier, or case metadata.
Order-sensitive judges are excluded. A judge who prefers A in forward and B in reverse is excluded from the voting majority but logged in excluded_judge_ids.
Evaluation is additive and tenant-owned. Cases and judgements persist in ai_research_evaluation_cases / ai_research_evaluation_judgements (migration 073), scoped per owner_user_id, with append-only judgements and immutable terminal verdicts.

Exit codes

| Code | Meaning | |---|---| | 0 | PASS — report meets tier thresholds | | 1 | FAIL — report below total minimum, below critical-dimension minimum, or regression too large | | 2 | REVIEW_REQUIRED — insufficient judges, critical spread too wide, or no majority | | 3 | Input/config error — bad paths, unparseable JSON, adaptation failure |

Cost transparency

Each judgement persists tokens, cost_usd, latency_ms, and prompt_hash alongside the full scorecard JSON, so every comparison's budget is auditable. Costs are never rolled into the evaluated report — they stay in the evaluation-only tables.

Running a comparison

python -m arb_bot.portfolio_app.ai_research.v3.evaluation.cli \
  --left tests/portfolio_app/ai_research/v3/evaluation/fixtures/v2_strong.json \
  --right tests/portfolio_app/ai_research/v3/evaluation/fixtures/v2_strong.json \
  --tier STANDARD --seed 42 --output result.json

Reading order

In practice an operator should read a dossier in this order:

Dashboard — what is the rating, what action is implied, how confident is the model.
Active baseline banner — am I looking at the active thesis or a superseded one.
Segment breakdown + peer table + valuation basis chip — does the thesis hold up across the disclosed segments and peers, and what lens am I being asked to apply.
Bull / bear / catalysts / risks / invalidations — the full case.
Claims + sources — provenance for every material claim.
Stages — what ran and where the budget went.
Thesis baselines — the full pin history, not just the active one.
Outcomes — 5/20/60-day realised returns once the schedule has evaluated them.

V3 evidence acquisition, authority, and attestation

The V3 pipeline (Phase 6A–6D) turns raw evidence into ResearchFactV3 entries in a FactLedger, then grades the report by the authority of the facts behind each required section. The whole design rests on two principles: authority comes from who published the evidence, never from how we fetched it, and the institutional bar for coverage is fixed in code — there is no flag that lowers it.

Authority tiers

| Tier | Meaning | |---|---| | FILING | Exchange-filed document fetched from BSE/NSE (bse_filing / nse_filing) or a per-issuer filed annual report | | REGULATOR | Corporate actions and listing metadata from BSE/NSE feeds | | COMPANY | Issuer-authored documents fetched from the company's own IR domain | | OFFICIAL | Exchange/broker market-data feeds (Dhan) | | AGGREGATOR | Redistributed issuer data (Yahoo Finance) | | DISCOVERED | Web/codex discovery — never sufficient for coverage |

A cached copy inherits its publisher's authority; caching never raises it. Authority is persisted as the enum name, never the integer.

Evidence-family policy matrix

Coverage requires each cited fact to meet its family's minimum:

| Evidence family | Minimum gating authority | |---|---| | Financial statements | COMPANY (FILING preferred) | | Exchange prices / technicals | OFFICIAL | | Corporate actions | REGULATOR | | Company guidance | COMPANY | | Derived facts | Inherit the weakest input authority | | Discovered / web evidence | Never sufficient |

The matrix is a module constant. An AGGREGATOR revenue fact is admitted to the ledger but is_validated=False, so a section grounded only on it emits INSUFFICIENT_AUTHORITY_FOR_COVERAGE and the run terminates REVIEW_REQUIRED — never COMPLETED.

What `REVIEW_REQUIRED` means

REVIEW_REQUIRED is a terminal, persisted, downloadable report that is complete and internally sound but not approved for publication: its required sections rest on aggregator-grade evidence. A COMPLETED report means every required section met its family's authority minimum on filing/company/official evidence. No configuration converts an aggregator run into COMPLETED; only better evidence does.

Every run must still produce a readable report

A run that cannot be rated is not a run that produces nothing. Four independent defects combined to make that untrue for every report ever published, and each is now pinned by tests/…/v3/test_report_renders_without_a_rating.py and tests/…/ai_research/test_exchange_resolution.py:

Acquisition was disabled by a blank registry field. providers._india_only_unavailable gated price history and the other India-only providers on exchange in {NSE, BSE, NS}, and portfolio.instruments.exchange is blank on 97 of 98 production rows. Nearly every symbol got UNAVAILABLE: unsupported_exchange and acquired almost nothing — TCS 19 facts from 4 sources, TATASTEEL 17 from 10, MINDSPACE 10 from 8, against BEL's 414 from 7435. BEL alone worked, because it happens to have a second registry row carrying exchange='NSE'. An unknown exchange is no longer treated as a non-Indian one: the exchange is recovered from the ticker suffix or the ISIN country prefix, a row with none of those is probed against yfinance (.NS, then .BO, then bare) so US listings and ETFs in the same table keep their own tickers, and the fetch — which already degrades to no packets — decides. Failing closed on missing metadata silently destroyed the report; failing closed on missing data is the fetch's job.
The evidence view starved the decision-maker. It capped at 50 facts and filled them first-obligation-first, so the investment committee — which has no capability_ids and therefore views the whole ledger — received revenue, assets, equity and market cap out of 414 facts and never reached price, valuation ratios, leverage, cash flow or the technicals. Selection is now breadth-first (every obligation's latest period, then second, …) and the cap is 160, leaving max_tokens as the real ceiling it was always meant to be.
A declining committee was blanked. When rating is None the orchestrator replaced the whole output with an empty CommitteeDecisionOutput, discarding the committee_disagreement / coverage_assessment / why_not_* / thesis_requirements paragraphs that explain what evidence would have to exist to rate the name. Only the rating is overridden now. When there is no rating, the reasoning is the deliverable.
PDF preflight disagreed with QA about "required". It demanded Catalysts, Risk and Committee — three sections quality.py had already dropped from its own required list as structurally unsatisfiable (their metric families have no MetricDefs, so no obligation is ever compiled). Preflight now hard-fails on exactly the four _REQUIRED_SECTIONS, a parity test pins the two lists together, and a failed render records its reason on run.error instead of only a log line — a missing PDF forces partial via classify_review_required's first condition and leaves nothing to download, which is far too quiet to hide in a log.warning.

Resilience: a run must terminate with an artifact

Four further failure modes each produced a run with nothing to download, none of them visible from the run row before the diagnostics below existed:

One flaky specialist call cost the whole report. The graph hard-blocks on missing dependencies, so a single node failing skipped seven downstream nodes (blocked by missing deps), emptied the remaining required sections and failed preflight. SpecialistGraphEngine now retries an invocation once with a short backoff. RepairEngine only ever covered validation failures.
The failure was undiagnosable. codex exec exited 1: with nothing after the colon — codex writes diagnostics to its --json event stream on STDOUT and leaves stderr empty. CodexRunner now falls back to the stdout tail, which immediately revealed the real cause: Selected model is at capacity. Please try a different model.
A model at capacity is not a reason to lose a report. The upstream error says what to do, so CodexSpecialistInvoker takes fallback_models and the orchestrator wires the configured quick model behind the deep one. Only capacity signals fall through; a schema or auth failure would fail identically on any model and is surfaced instead of burning the fallbacks.
A QA-FAIL verdict suppressed the document entirely unless the lifecycle was exactly REVIEW_REQUIRED, so every legitimately partial run produced no artifact — including runs that completed all 12 specialist nodes and filled every required section. Suppressing the PDF never fixed the defect, it only hid it. Structural gates still apply; beyond them the renderer marks an unapproved report unmistakably (DRAFT — NOT APPROVED FOR PUBLICATION — COVERAGE n% on every non-COMPLETED page, plus a "QA Issues" list of every blocking defect), matching the documented V2 contract.

Orphaned runs. The worker only ever claimed rows in status queued, so a worker that died mid-run left its row running forever — never completing, never failing. Any container restart does this, including a routine deploy. The worker now also reclaims a running run whose updated_at is older than _STALE_RUN_SEC (one hour) and logs the reclamation loudly. Queued work is still claimed first, so one repeatedly-failing run cannot starve the queue. The threshold is deliberately generous: _set_stage mutates the ORM object without committing per stage, so updated_at is effectively the claim time and cannot support a tight lease.

Broad except blocks hide the bug you are looking for. The peer-universe query used Session/get_engine without importing them (this module keeps DB imports method-local), raising NameError on every call — which its own except Exception reported as peer universe unavailable; curated peers only, a message indistinguishable from a real data condition. The universal peer fallback never ran in production, and the tests missed it because they exercised peer_set.resolve directly and only inspected the orchestrator method's source text. Failure handlers here log exc_info, and the tests assert on the failure log rather than the return value — an empty list looks exactly like an empty database.

Peer groups

peer_set.lookup(symbol) is load-bearing, not decoration: the blueprint compiles peer- and industry-scope obligations for every comparison metric, and AcquisitionService can only satisfy those from peer packets — never from the subject's own numbers. No peers means roughly 35 of a ~91-obligation catalogue are uncovered before the run starts.

It was a flat four-entry map (RELIANCE, HDFCBANK, INFY, BEL) read only in the forward direction, so a TCS run resolved zero peers despite TCS being listed inside INFY's own comparable set, and every other registry symbol resolved zero as well. Two changes, no new data source and no network call:

Membership is a group, not a one-way pointer — a symbol's peers are its group minus itself, so listing INFY's comparables also answers TCS, WIPRO, HCLTECH and TECHM. A symbol may appear in exactly one group; a second membership raises at import, because it would make resolution asymmetric (the symbol resolves group A while every member of group B still names it).
PEER_GROUPS covers the sectors actually present in the instrument registry — banks, NBFCs, IT, metals, power, EPC, consumer, autos, REITs/InvITs, market infrastructure — rather than four hand-picked names. Every group has at least four members so _derive_industry clears its _MIN_PEERS_FOR_INDUSTRY floor of 3.

PORTFOLIO_AI_RESEARCH_V3_PEER_COUNT now bounds the returned list. It was parsed in PortfolioRuntimeConfig and read by nothing; every peer costs provider calls in collect_peer_packets, so the budget flag has to be real.

Universal fallback. Curation only answers for symbols someone has hand-listed, and for the long tail of the registry that meant [] — every peer- and industry-scope obligation uncovered regardless of how complete the rest of the acquisition was. peer_set.resolve() screens the candidate universe for names sharing the subject's industry when curation misses, widening to its sector only far enough to clear the three-contributor floor below which a "sector median" is not a median. Fewer than three comparables returns [] rather than a short group, so the obligation stays honestly uncovered.

The candidate universe is the registry filtered to asset_class in {Indian Equity, US Equity} — a mutual fund must never become the peer of an operating company. (sector, industry) comes from the registry first and the yfinance profile second (the same source blueprint/identity.py uses), and is memoised per orchestrator instance because screening asks about every candidate; unmemoised it would be one profile fetch per registry row per run.

A curated group still wins wherever one exists — "same GICS industry" routinely pairs businesses no analyst would compare.

Fact ids, periods, and hashing

Fact ids are {metric_id}.{scope_slug}.{period_slug} (e.g. revenue_ttm.company.fy2025, revenue_ttm.seg-consumer.fy2025).
Periods are canonical PeriodKeys (Indian FY, quarterly, latest, TTM); they sort chronologically, never lexically.
Evidence hashes are full sha256: over the normalized payload (including normalized_facts), recomputed at adaptation. A packet whose recomputed hash fails is rejected outright — zero facts, counted hash_mismatch. Acquisition does not trust the legacy V2 content_hash (a truncated 32-hex digest over reference|excerpt only): a packet carrying that old format — or none — is verified by the recomputed canonical hash over its own content, so a real V2 provider packet is admitted and stamped with the canonical hash.
Collected packets are deduplicated by source_id before admission (first occurrence wins), so a provider emitting the same source twice — e.g. Codex web discovery returning two items with the same URL — cannot collide on the (run_id, source_id) / (run_id, fact_id) unique constraints and fail the whole acquisition with IntegrityError.
yfinance statement rows are normalized onto the V3 extractor keys before the packet is built (revenue → revenue_ttm, net_income → net_income_ttm, equity → total_equity, capital_expenditure → capex), so a collected financial-statements packet actually yields facts. The financial-statements branch also falls back to "now" when no cutoff is supplied (the orchestrator passes cutoff=None), instead of silently dropping the packet. The acquisition request leaves provider_symbol empty so each provider applies its own configured symbol (yfinance's "BEL.NS" suffix) instead of being clobbered by the plain canonical symbol — without this yfinance statements resolve to "BEL" (0 rows) and the packet is never emitted.
Every codex --output-schema (blueprint proposer + specialist invokers) is sanitized to strip regex lookaround assertions and force every property into required before the model call. codex-cli ≥ 0.146 rejects schemas with lookaround (invalid_json_schema: regex lookaround is not supported) and schemas whose required omits a defaulted property (Missing 'severity'); pydantic's Decimal schema emits the first and defaulted fields produce the second — without the sanitizer every model call fails with codex exec exited 1 and the run degrades to an empty report.
Acquisition extracts flat-dict fundamentals as well as statement rows: a packet whose normalized_facts is a flat dict without annual/quarterly keys (exactly what yfinance:fundamentals emits — {roe, ep_yield, ...}) is passed straight to a fundamental extractor instead of being split into an annual list (which yielded zero facts). yfinance fundamental keys are mapped onto obligation metrics (op_margin → operating_margin, eps_ttm → reported_eps, fcf → free_cash_flow, and inverse yields ep_yield → pe_ttm, bp_ratio → pb_ratio).
Source-family rules now map the collected sources onto the obligation families: persisted:quote/persisted:valuation → market_data, persisted:sector → exchange_listing, price:/yfinance:price: → market_data+exchange_prices, technical: → exchange_technicals, and yfinance:fundamentals → fundamentals+market_data+annual_report +sector_benchmark+consensus_estimates. Previously persisted:* and price:* mapped to empty families and market_data obligations (market_cap, pe_ttm, dividend_yield, roe, ...) were never satisfied, so the specialist graph's fundamental inputs were empty.
The company_industry_analysis node now also carries the financial_history capability, so its evidence view includes revenue / net-income / asset / equity facts alongside the identity facts — enough for a real business_model_assessment instead of an empty UNAVAILABLE. The worker's sector_lookup deliberately does not fall back to asset_class: "Indian Equity" is a meaningless sector placeholder that would win the acquisition dedupe (OFFICIAL authority) over a real sector from yfinance:profile (AGGREGATOR), starving the identity node of real sector evidence.
Catalyst facts come from BSE corporate-action events: the catalyst_analysis capability requires a catalyst metric, and a bse:* packet (family corporate_actions, REGULATOR authority) produces catalyst.company.latest so the catalysts_event_timeline section can cite a family-matching, authority-sufficient fact. Codex discovery stays DISCOVERED and never satisfies the section on its own (spec §5.3.4).
Blueprint company identity (v3/blueprint/identity.py) resolves in a fixed order: instrument registry → yfinance profile longName → the ticker. cover.company feeds the UNRESOLVED_COMPANY_IDENTITY hard gate, and it used to come from instruments.display_name alone — with 95 of 98 production rows holding that column null, empty, or equal to canonical_symbol, the gate failed for nearly every symbol. A display_name that merely restates its own ticker counts as unset, so the profile lookup gets its turn. Because portfolio.instruments is not unique on canonical_symbol and a run can be bound by instrument_id to the bare half of a duplicate pair, identity fields are coalesced across sibling rows rather than read from one row. Identity is never resolved by asking a model for the company name — a hard gate must not depend on a field a model can invent. When nothing resolves, the name stays the ticker and the gate correctly fires.
Coverage counts derived facts. _derive must run last (sector_median needs the peer facts), so coverage rows measured during extraction predate every formula-produced fact. Acquisition therefore re-measures the uncovered rows against the final fact set before returning. Covered rows are untouched, and a row with genuinely no fact keeps its original reason — no_source_family / no_peer_data / no_segment_document / segment_attestation_failed are the operator's diagnosis of why and are never flattened to a generic "uncovered".
Three derivation shapes, dispatched per obligation in acquisition/service.py:
- single — one fact per named input, per period and per entity (net_margin, roa, …). Grouping is keyed on (scope_detail, period): at peer scope there are N companies in the fact list, and keying on period alone would let each peer's inputs overwrite the last, producing one ratio built from whichever peers came last and mislabelled as the whole set.
- series — an ordered multi-period series of ONE metric, keyed y0..yN (CAGRs: revenue_growth_3y, eps_growth_3y). revenue_growth_3y was registered, wired and reachable since it was written, and returned None on every call because the single-arity path hands each formula one period at a time. It needs the whole series at once. Fewer observations than the horizon derives nothing — a "3-year" growth rate computed off two years is a fabrication, and neither is a CAGR whose endpoint went negative. Grouped per entity, for the same reason single is: a flat sort by period at peer scope puts the four most recent observations of four different peers into one window and computes a "3-year CAGR" across unrelated issuers.
- industry — sector_median across the peer-scope facts of the same metric. sector_median was registered from the start with no caller anywhere in the codebase. Industry obligations run last, over a working set that already includes the peer ratios derived in the same pass; contributions are one-per-peer (a repeated scope_detail would let one company vote twice), with a floor of 3 peers. The median is an addition — an industry-scope number the section can cite. It does not change coverage semantics: an industry obligation has always been covered by the peer-scoped facts themselves.
Peer ratios get their inputs without extra fetches. The blueprint declares peer obligations for the comparison metrics (roa-peer, …) but never for their statement-line inputs, so total_assets / capex / operating_cash_flow were never extracted at peer scope and every peer ratio reported no_peer_data — even though the peer's statements packet had already been fetched. _extract_peer_formula_inputs re-reads the packets the peer loop already holds: no extra provider call, no budget impact.
Company formula inputs that are not obligations. The company extraction loop runs EXTRACTORS[obligation.metric_id], so a formula input only reaches the ledger when it happens to be a declared obligation itself. Every formula written before eps_growth_3y was satisfied by that accident (roa needs net_income_ttm and total_assets, both obligations). The annual diluted_eps series is nobody's obligation, so _derive saw an empty series. _extract_company_formula_inputs is the company-scope twin of the peer version — same packets, no extra fetch. Its facts are admitted to the ledger but kept out of the obligation's coverage row: attributing four fiscal years of diluted_eps to the eps_growth_3y obligation would report it covered at FY2022–FY2025 while 3Y, the only period it requires, stayed missing. Because two obligations can now legally produce the same fact_id, acquire() tracks emitted_fact_ids so facts stays one row per id (the snapshot is unique on (run_id, fact_id)).
roic and debt_to_ebitda gate the terminal status. Both back historical_financial_performance, a REQUIRED section (roic literally, debt_to_ebitda via the debt_ prefix in _SECTION_METRIC_FAMILIES), so an uncovered row for either keeps _authority_shortfall_only false and terminates the run partial no matter what else was acquired. roic had a V3 extractor reading a key yfinance_provider._derive_rows never emitted; debt_to_ebitda had neither extractor nor producer. Both are now derived point-in-time from figures the provider already downloaded — roic as NOPAT ÷ (debt + equity − cash), with the effective tax rate clamped to [0, 1] so a loss-making period's tax credit cannot invert NOPAT's sign and report a profitable ROIC for a company that lost money.
beta / max_drawdown_1y / var_95 have a producer. All three had MetricDefs and extractors from the start and reported "no data" on every run ever published: the worker's technicals_lookup — the only producer of the technical:snapshot packet they read — emitted sma/rsi/volume_ratio/week52 and nothing else, and the three MetricDefs additionally declared source_families=("market_data",) while technical: maps to exchange_technicals, so the per-obligation family intersection could never match. Both halves are fixed; beta is measured against ^NSEI and fails closed to no fact when the benchmark history is unavailable.
A registered formula must be reachable. tests/…/test_formula_liveness.py walks the registry itself and fails if any registered formula has no dispatch path (_FORMULA_SPECS or _AGGREGATE_FORMULAS), if a spec names an unregistered formula, or if any spec cannot produce a fact from inputs built from its own declaration. A formula that is present and dead looks identical to a working one from every vantage point except the fact table, so do not register a formula whose inputs are not yet extracted. eps_growth_3y was held back under this rule until the annual diluted_eps series existed (reported_eps is TTM-only); it is registered now that _extract_company_formula_inputs supplies that series.
industry_coverage_pct is computed, not defaulted. ObligationCoverage carries the obligation's scope, so AcquisitionResult derives industry_coverage_pct and industry_obligation_count from the industry-scope rows and the orchestrator passes the percentage into the publication. The count distinguishes "0% of 8" (a real gap) from "0% of 0" (not applicable). Checkpoints written before scope existed rehydrate with scope="" and are excluded rather than miscounted.
Identity-resolution evidence comes from the instrument registry, not fetched sources: the worker's identity_lookup surfaces the registry's exchange/isin/sector/industry/display_name as a persisted:identity packet (family exchange_listing+market_data, authority OFFICIAL). Identity extractors (exchange, isin, sector, industry) emit presence-token facts whose string value rides in the fact's unit field (the ledger's value is a Decimal); the section grader reads fact_id + authority, not the token. market_cap is now computed point-in-time in yfinance_provider._derive_rows as close × shares_outstanding and persisted in research_fundamentals (db/090), so the market_data obligations that company_industry_analysis needs are satisfied from the same provider that supplies roe/pe_ttm. When the instrument registry lacks a sector, a yfinance:profile packet (sector/industry/longName from yfinance info, family exchange_listing+sector_benchmark, AGGREGATOR authority) covers the sector obligation instead — so company_industry_analysis never runs on an empty identity handoff. exchange/isin stay OFFICIAL from the registry when present.
Acquisition dedupes facts by fact_id across packets before admission (best-authority wins), so when two packets cover the same obligation metric — e.g. persisted:identity (OFFICIAL) and yfinance:profile (AGGREGATOR) both emit sector.company.latest — exactly one fact is admitted and the (run_id, fact_id) unique constraint never collides.
The UNRESOLVED_MATERIAL_CONFLICT QA gate only blocks a conflict that cites two facts (fact_a_id + fact_b_id) — a genuine numerical contradiction from acquisition detect_conflicts. The fact_reconciliation specialist's free-text notes ("no direct conflict, but the caveat limits interpretation") carry no fact ids and are surfaced but never block. The orchestrator folds both sources: acquisition conflicts (fact ids → blocking) and reconciliation notes (empty ids → non-blocking).

Atomic snapshot & resume

Acquisition persists in one transaction: sources, facts, conflicts, calculations, and a mutable checkpoint row commit together or not at all. Resume reads the checkpoint: a COMPLETED snapshot with a matching input_hash is rehydrated in full; a changed input_hash returns NEW_RUN_REQUIRED and mutates nothing. Artifact rows are immutable.

Section relevance

A required section is satisfied only by a cited fact whose metric belongs to that section's allowed family — a revenue_ttm fact cannot satisfy the risk section. Per-section grades compose to the overall evidence_grade (FILING_GRADE / INSTITUTIONAL_GRADE / AGGREGATOR_GRADE / INSUFFICIENT_EVIDENCE).

The required section set is aligned to what the acquisition pipeline can actually evidence: business_model_position, historical_financial_performance, cash_flow_balance_sheet, and valuation_peers_sensitivities. catalysts_event_timeline, risks_invalidation_monitoring, and committee_disagreement_decision are not required: their metric families (catalyst, risk_flag/risk_factor/invalidation_trigger, committee/disagreement/vote) have no catalogue MetricDefs, so no obligation is ever compiled, no extractor ever produces a fact, and a fact coverage gate on them could never be satisfied by any provider set — the catalyst family additionally depends on REGULATOR BSE corporate actions, which can be unavailable when the BSE announcements feed is unreachable. They remain QA'd (cited source/fact-id validation) and rendered, but they no longer gate the terminal status. The §6.5 required-period coverage condition is likewise scoped to the obligations the required sections actually cite (company scope only), so an uncovered peer/industry/segment obligation cannot force a partial run.

Report source map, units, and metric formatting

The publication source map contains only sources that back a cited fact (the union of each cited fact's source_ids), not the full acquisition source list. The price-history provider emits one source per daily observation (hundreds of price:YYYY-MM-DD rows per run); those raw observations are market-data inputs, not citable report sources, and are dropped from the PDF's Source Map. The map stays at the handful of real evidence documents (yfinance:financial-statements, persisted:identity, ...).
Statement packets carry packet-level currency/scale (yfinance emits currency="INR" beside the annual list). These are merged into each row before extraction so the resulting facts carry the currency — otherwise the specialist model hallucinates a currency (writing $ for INR figures). The specialist prompt serializes each fact with its currency and scale so the model prices the narrative in the right unit.
PDF metric tables render scaled, currency-prefixed values (₹60.62bn, ₹9.85bn) instead of raw 60620000000.0 absolute.
technical_condition_market is grounded by the monitoring_framework capability's technical metrics (rsi_14, sma_50, sma_200, week52_high, week52_low, volume_ratio), extracted from the technical:snapshot source (AGGREGATOR authority — computed from the yfinance price history, never DISCOVERED). Peer/industry coverage for valuation and industry sections comes from the curated tier map in arb_bot/portfolio_app/ai_research/peer_set.py.
Identity facts (isin, exchange, sector, industry) store their string value in the fact's unit (the fact model has no string-value column, so the identity extractor uses a placeholder numeric value of 1). The specialist prompt serializes these metrics with the string as the value (isin: INE263A01024), so the model anchors on the right company — without it a BEL run once produced a Siemens report because the model could not see the ISIN.
Peer symbols absent from the instrument registry are suffixed for yfinance (HAL → HAL.NS) so peer statements/fundamentals are fetched in INR instead of defaulting to USD.
management_governance_forensic falls back to catalyst_risk_governance.governance_notes when the deep-tier forensic_review node did not run (standard runs never execute it).
industry_structure_kpis runs for every company via the optional generic_industry_analysis capability, so the industry section is populated even when no industry-specific business-model overlay (banking/saas/...) applies. segment_geographic_economics falls back to the business-model assessment when no explicit segment breakdown exists.

Report quality remediation (Phases 0–5)

The pipeline's report-quality controls are designed so an honest artifact is the only artifact:

No fabricated comparables. The derive stage (_derive) only satisfies an obligation with facts of the SAME scope, so a peer/industry net-margin obligation is never met by the company's own revenue/income. Fundamentals are forward-filled at most 2 quarters per column and the provider's operating margin comes from TTM EBIT, not net income.
A failing report cannot score 100. quality_score clamps to 0 on any FAIL and is scaled by evidence coverage on a PASS; INSUFFICIENT_AUTHORITY_FOR_COVERAGE maps into the evidence-citation dimension.
available means available. Empty or placeholder-only sections render unavailable and land in data_gaps; build-status text ("not yet produced by this pipeline stage") is never rendered.
A directional rating must be anchored. BUY/OVERWEIGHT/UNDERWEIGHT/SELL require a target price, a stated valuation method, a horizon and ≥1 cited fact (MISSING_TARGET_PRICE). A committee that declines to rate emits rating=None + portfolio_action=INSUFFICIENT_EVIDENCE and renders a no-view note — never a fabricated blank HOLD.
No prompt leaks. Section text that opens with a prompt-vocabulary imperative (Challenge, Assess, Evaluate, ...) is rejected (PROMPT_INSTRUCTION_LEAK); the bull/base/bear section carries per-scenario target prices and driver assumptions.
Numbers are formatted, dates are populated. Ratios render one decimal
- x, percentages two decimals + %, LC resolves to the real currency, identity facts use a text_value (never the unit field), and source-map entries carry observed_at/effective_at.
Costs are real. Per-node model-call counts are persisted and summed on the rehydrate path, so a resumed run reports its actual total_calls / total_cost_usd.
Presentation. Financial summary (with CAGR), peer comparison, valuation-sensitivity tables and a close+SMA50/SMA200 price chart are built from the ledger; every non-COMPLETED PDF carries a DRAFT — NOT APPROVED FOR PUBLICATION — COVERAGE {n}% watermark. REVIEW_REQUIRED still renders a reviewable PDF — it is a designed state, now externally distinguishable from a publishable note.

Attested filings (Phase 6C)

Filed documents (annual reports, segment notes) are fetched under a strict security contract: HTTPS-only, host allowlist, at most 3 redirects with per-hop re-validation, resolve once / vet every A and AAAA answer / connect to a pinned vetted address with TLS hostname verification intact, 25 MB and 400-page caps, and unparseable_document for malformed, encrypted, or image-only PDFs. Claims are verified deterministically — the quote must appear at the stated page/offset, the number re-parsed from the quote must match after normalization, and the document hash must match — before a fact is admitted with verification="VERIFIED_FETCHED" and the document's authority. Provenance (quote, page, char_offset, document_hash) is stored in dedicated fact columns, never in scope_detail. A failing check yields zero facts and a counted rejection.

Discovered-source enrichment (Phase 6D)

Behind AI_RESEARCH_V3_DISCOVERY_FACTS_ENABLED, codex:* packets may produce facts only for non-statement metrics and always at is_validated=False. Discovered facts never satisfy a required section.

Feature flags

| Flag | Default | Gates | |---|---|---| | AI_RESEARCH_V3_ENABLED | 0 | The whole V3 pipeline (worker startup) | | PORTFOLIO_AI_RESEARCH_V3_BLUEPRINT_ENABLED | 0 | Blueprint compilation | | PORTFOLIO_AI_RESEARCH_V3_PEER_COUNT | 6 | Peer-set size | | PORTFOLIO_AI_RESEARCH_V3_FILINGS_ENABLED | 0 | Attested extraction + segment scope | | PORTFOLIO_AI_RESEARCH_V3_FILINGS_SYMBOLS | (empty) | Per-symbol filings allowlist (comma-separated) | | PORTFOLIO_AI_RESEARCH_V3_DISCOVERY_FACTS_ENABLED | 0 | Discovered-source facts |

Only AI_RESEARCH_V3_ENABLED is unprefixed — it is read by the dashboard's worker-startup hook, not by PortfolioRuntimeConfig.

Filings are doubly gated. PORTFOLIO_AI_RESEARCH_V3_FILINGS_ENABLED=1 alone is not enough: PORTFOLIO_AI_RESEARCH_V3_FILINGS_SYMBOLS is a per-symbol allowlist, and a symbol is only eligible when it is both listed there and the global flag is on. An empty allowlist (the default) keeps every symbol off. Filing discovery (acquisition/discovery.py) queries the BSE/NSE announcement APIs — both hosts already on the fetch allowlist — for (url, doc_type, period, published_at) candidates and returns them under the same §7.1 security contract: disallowed hosts are dropped up front and the fetcher re-vets every hop. Discovery is wired into the orchestrator (_filing_document_for), which fetches the first annual-report candidate as the segment_document for the attested segment path when the gate is open.

Wiring status

AcquisitionService.acquire() is the production entry point and reads the subject company's own provider packets, resolving company-scope obligations. The orchestrator now resolves the curated peer set (peer_set.lookup) and passes it to acquire(), which calls acquire_peer_facts() to cover peer + industry obligations from peer packets when peers exist (unknown symbols stay uncovered with no_peer_data). Segment obligations route through acquire_segment_facts(): with PORTFOLIO_AI_RESEARCH_V3_FILINGS_ENABLED off (or no document/claims supplied) they are retained as explicitly uncovered (no_segment_document), never satisfied from the subject's own numbers. Segment attestation (acquire_attested_facts + the filings provider and claims extractor) still needs to be wired into the orchestrator before _FILINGS_ENABLED produces real segment coverage; _PEER_COUNT drives the peer set size when the universe-based resolver is used in place of the curated peer_set map.

A partial run's detail response surfaces the top 20 uncovered obligations (id, reason, missing periods) from the persisted checkpoint. A read-only smoke script (scripts/v3_acquisition_smoke.py) runs blueprint → acquisition for one symbol and prints the coverage table, evidence_grade, and fact count without enqueuing anything or calling a model.

Provider chain. The V3 pipeline (specialist graph invokers, blueprint proposer, and the evaluation runner) and the V2 web-discovery stage now honor the per-user / default provider chain. The V3 worker resolves the run owner's provider via resolve_user_provider and falls back to resolve_default_provider(feature="v3_pipeline") on error; each model call site builds a CodexRunnerShim when a provider is present instead of shelling out to codex exec directly. V2 web-discovery resolves through resolve_default_provider(feature="ai_research_web_discovery"), which stays codex (premium tier) because web search requires the Codex CLI.

Deleting a run

Both the Dossier and V3 report views expose a permanent-delete action with a type-to-confirm dialog (type the symbol to confirm).

DELETE /ai-research/runs/{run_id} and DELETE /ai-research/v3/runs/{run_id} both delegate to the same repository cascade.
The cascade removes the run and every row referencing it in one transaction: all ai_research_v3_* child tables (publication QA, publications, specialist outputs, committee decisions, facts, conflicts, calculations, sources, acquisition runs, obligations, blueprints, the V3 queue row) and all V2 child tables (thesis comparisons/baselines, claims (+ claim sources via cascade), sources, stages, reports, outcomes, run links), then the ai_research_runs row itself.
Self-references (replaced_by_run_id, retry_of_run_id) are NULLed first so the delete cannot FK-fail. A V3 delete removes the shared parent row too, so the run disappears from both the V3 list and the Dossier.
Deletes are tenant-scoped: an operator can only delete their own runs (404 otherwise).

V3 canonical research state

Every downstream stage reads the same reconciled facts. Before this existed, each specialist rebuilt its own partial view by sampling the ledger, and the sampler decided what a decision-maker saw.

The defect it replaces

EvidenceViewBuilder caps an evidence view at 50 facts, and the control nodes (fact_reconciliation, investment_committee, publication_review) declare capability_ids=(), so build_handoff hands them every obligation and lets the cap choose. The obligations arrive in the blueprint's capability order, and the round-robin spends the budget on the first ~50 of them.

Replaying the persisted ledger of production run 30c18fa3bb964463a3eca415397f33d1 (BEL, 414 facts, 91 obligations) through that path:

| Capability | Obligations | Running total | |---|---:|---:| | identity_resolution | 5 | 5 | | financial_history | 7 | 12 | | growth_analysis | 8 | 20 | | margin_analysis | 13 | 33 | | earnings_quality | 5 | 38 | | cash_generation | 4 | 42 | | leverage_liquidity | 9 | 51 — budget gone | | valuation_framework | 12 | never reached | | risk_assessment | 3 | never reached | | monitoring_framework | 7 | never reached |

The committee received 50 facts covering 30 metrics — revenue, margins, returns, leverage, cash flow — and not one of last_price, pe_ttm, pb_ratio, ev_to_ebitda, beta, max_drawdown_1y, var_95, sma_50, sma_200 or rsi_14. Every one was in the ledger, and every one was printed elsewhere in the same PDF. Its verdict, verbatim: "the evidence lacks current price, valuation multiples, forward estimates and catalyst support needed for an investable committee rating." That was an accurate report of what it had been handed.

Raising the cap is not the fix. The evidence view is bounded because the model invocation is bounded: a 160-fact view killed cash_flow_balance_sheet with codex exec exited 1, which hard-blocked seven dependent nodes, emptied both remaining required sections and produced a run with no PDF.

How it works

arb_bot/portfolio_app/ai_research/v3/canonical.py reduces the whole ledger to a fixed ~50-metric snapshot (CANONICAL_METRICS) plus per-peer values and computed peer statistics. HandoffBuilder attaches it to every handoff as TypedHandoff.canonical, unsampled, in addition to whatever the evidence view selected. It is rebuilt per handoff so a node that runs late never sees a staler state than one that ran early.

Availability is machine-determined: capability flags are computed from the presence of facts, never inferred by a model from prose. Availability is also not binary — current, relative, historical, forward, intrinsic and sensitivity valuation each get their own flag, because on BEL the first, second and last were available the whole time while only the forward and intrinsic legs were missing.

| Flag family | Examples | |---|---| | Presence | has_current_price, has_pe, has_peer_valuation, has_risk_metrics, has_governance_data | | Derived capability | can_do_current_valuation, can_do_relative_valuation, can_do_forward_valuation, can_do_dcf, can_do_multiple_sensitivity, can_issue_rating |

The specialist prompt carries canonical, capabilities, peers and peer_stats, and instructs the model never to describe a metric present in canonical as unavailable — the sampled facts list is explicitly a sample.

Contradiction gate

publication/contradictions.py re-reads the published narrative against the same canonical state. A sentence that asserts a family is unavailable while the state holds it produces a REPORT_CONTRADICTION blocking defect, so the run cannot reach review_required or completed.

Not firing is the harder half. "Forward valuation is unavailable" was true on BEL, and a checker that flags it would block every honest report. Absence claims qualified by something the evidence genuinely lacks (forward, consensus, projected, DCF, historical valuation, target price, governance, catalysts, segments) are exempt, and a claim about peer valuation is judged against the peer set rather than the company's own multiples.

Post-QA prose repair

A prose gate that fires is not the end of the run — it is a sentence to fix. publication/repair.py maps a blocking model-error defect (REPORT_CONTRADICTION, COMPARISON_DIRECTION_WRONG, CAPABILITY_CONTRADICTION, PERIOD_LABEL_MISMATCH, REJECTED_FACT_USED_IN_PUBLICATION) back to the specialist node whose output wrote the offending section, and the orchestrator re-invokes that node with the defect text before rebuilding and re-assessing. The pass is bounded (_PROSE_REPAIR_MAX_ATTEMPTS), so a stubborn error leaves the run partial rather than looping; structural defects (EMPTY_REQUIRED_SECTION, MISSING_RATING, INSUFFICIENT_AUTHORITY_FOR_COVERAGE, …) are never repaired because re-invoking a specialist cannot fix them.

Evidence availability matrix

publication/evidence_matrix.py computes one availability answer per evidence family, rendered as a table immediately after the executive recommendation. The weakest source in a family grades it — one FILING-grade number does not upgrade three aggregator ones — and a family with no facts is stated as unavailable rather than omitted.

Reproducible confidence

v3/confidence.py computes confidence from evidence coverage under CONFIDENCE_WEIGHTS (summing to 100), minus penalties for unresolved conflicts (8 each) and published contradictions (15 each). The committee's own number is discarded.

Confidence means how much of the evidence a rating needs this run obtained — not a probability the call is right; nothing here is calibrated against realised outcomes. So a band (HIGH ≥ 70, MEDIUM ≥ 40, LOW) leads on the cover and the percentage is the audit trail. confidence_basis carries the full decomposition and the formula version into the PDF.

Forward evidence is weighted 10 and no metric in the registry supplies it today, so 90 is the ceiling a real run can reach. That gap is deliberately visible rather than normalised away.

One decision object

Rating and confidence are computed once, by v3/decision.py, and every surface renders that. Before this, BEL_V3_final.pdf said three things about one decision:

cover:      HOLD · HIGH CONFIDENCE (85%)
executive:  HOLD · Confidence HIGH (85%)
committee:  Rating: HOLD (confidence: 72)

85 is the deterministic coverage score. 72 is what the model wrote: publication/builder.py formatted committee.confidence into the committee section's findings at build time, and _attach_evidence_layer later overwrote only the executive recommendation. Nothing compared them.

build_decision_state owns rating, confidence, band, model version, components and the rating drivers; apply_decision writes them to the executive recommendation and rewrites the committee section's rating findings. No module reads committee.confidence — a test asserts that structurally, because a new call site would only surface in a rendered PDF. DECISION_CONSISTENCY blocks publication if any section states a rating or a confidence that differs from the report's own.

Four quantities, kept apart

A reader conflates these unless the report separates them, so the cover and the executive block state each on its own line:

| Quantity | Means | Source | |---|---|---| | Decision confidence | share of the evidence a rating needs that this run obtained | v3/confidence.py | | Evidence coverage | share of declared obligations satisfied at all | evidence_obligation_coverage_pct | | Evidence authority | who published the evidence | evidence_grade | | Publication status | whether it may ship | lifecycle_status |

Decision confidence is explicitly not a probability that the call is right — nothing here is calibrated against realised returns. A fully-covered report on aggregator data is an ordinary state: coverage 88%, authority AGGREGATOR_GRADE, status REVIEW_REQUIRED.

Rating drivers

_decision_factors derives positive / negative / uncertainty factors from the canonical state — peer-relative returns and margins, valuation premium, leverage, cash conversion, and what was never acquired (governance, forward estimates, risk metrics). The same set reaches the committee prompt and the executive block, so the two can word a driver differently but cannot give different factual reasons for the same rating.

Reconciliation and plausibility

v3/reconciliation.py recomputes supplied growth from the underlying series (RECONCILIATION_CONFLICT beyond 1.5 percentage points) and range-checks returns, multiples and margins. The rule is flag, never discard: a −177% peer ROIC is the visible end of a real formula defect (almost certainly a negative invested-capital denominator), and dropping it hides the bug. Both values are recorded with a probable cause; neither is silently preferred. An exact 0.0x leverage ratio is flagged as a likely missing value rendered as zero.

Extreme readings are interpreted, not just flagged. Interest coverage of 1,238× on an issuer with debt/equity of 0.01 is arithmetic, not bad data — the denominator approaches zero. That case is classified EXTREME_BUT_EXPLAINABLE with the leverage evidence named. The warning is never suppressed; only its interpretation changes, and a levered issuer showing the same ratio still gets a plain NEEDS_VALIDATION.

The executive page carries one line ("N metric reconciliation issues require review"); the full technical detail renders as Metric Reconciliation Notes at the back, including the RAW reading and its unit so the normalisation itself can be checked rather than trusted.

Findings are weighted by severity, not counted

A count is not a measurement. One report deducted 24 confidence points for three findings, one of which read "the values agree once normalised; this is a unit label defect, not a disagreement about the number" — nothing about the company was in doubt — and it cost exactly what an irreconcilable metric costs. Two of the three were free-text notes carrying no fact ids, which _extract_unresolved_conflicts itself documents as "not a material contradiction".

| Severity | Applies to | Penalty | |---|---|---| | RESOLVED_NORMALIZATION | UNIT_CONFLICT, EXTREME_BUT_EXPLAINABLE | 0 | | VALID_DIFFERENT_PERIOD | PERIOD_CONFLICT | 0 | | OPEN_NON_MATERIAL | METRIC_DEFINITION_CONFLICT, NEEDS_VALIDATION, notes citing no facts | 2 | | OPEN_MATERIAL | SOURCE_VALUE_CONFLICT, RECONCILIATION_CONFLICT, conflicts citing two facts | 8 | | DECISION_BLOCKING | any metric marked usable_for_decision = false | 15 |

Confidence is therefore rating-evidence coverage less unresolved-conflict penalties, and the report says so in those words — describing the number as pure coverage while deducting 24 points from it was a smaller version of the self-contradiction the decision layer exists to remove.

Conflict taxonomy

A metric is not identified by its name. roe alone cannot distinguish an FY2026 figure computed from the statements from a TTM figure a vendor defined its own way, and treating them as one number reported a period difference as a unit error. classify_metric_conflicts normalises first, then names what actually differs:

| Status | Meaning | Usable for a decision | |---|---|---| | UNIT_CONFLICT | same value, same period, two unit labels | yes — a producer labelling defect | | PERIOD_CONFLICT | different windows; both may be correct | yes, with the window stated | | METRIC_DEFINITION_CONFLICT | same period, one computed one provider-defined | yes, with the basis stated | | SOURCE_VALUE_CONFLICT | same period, same basis, different value | no — nothing explains the gap |

Metric identity is carried on dimensions the ledger already records: metric, entity, period, period type, unit, currency, source authority, and calculation basis (a fact carrying a formula_version was computed here; anything else came ready-made). Statement basis (standalone vs consolidated) is not recorded by any producer today and is reported as unknown rather than guessed.

Each conflict selects a canonical value with a machine-readable reason — most current period type wins, then a statement-computed reading over a provider-defined one — except SOURCE_VALUE_CONFLICT, which yields no canonical value and marks the metric usable_for_decision = false.

Unit normalisation

Percentage metrics are normalised to percentage points before anything is compared (normalize_to_percent). Run 30c18fa3bb964463a3eca415397f33d1 persists roe as both percentage (25.27, FY2026) and percentage_ratio (0.28769, TTM) because two producers disagree: the yfinance extractors label their fractions percentage_ratio, while the derived-formula path emits percentage after multiplying by 100.

Raw, that pair looks 100× apart and was reported as a unit conflict — telling a reader to distrust an extractor that was working. Normalised it is 25.27% and 28.77%: two different, defensible readings over two different windows, which is a PERIOD_CONFLICT.

Normalisation is driven by the metric, never by magnitude: PERCENT_METRICS lists the metrics whose natural unit is a percentage, so a P/E of 51.7 ratio is never turned into 5,170%. Anything outside that set with two unit labels stays incomparable and is excluded rather than guessed at.

One producer bug was fixed rather than worked around: revenue_growth_3y and eps_growth_3y were labelled percentage_ratio while _cagr_3y returns percentage points, so a 32.16% CAGR read as 3,216% to anything that normalised it.

Downstream consequences:

Peer statistics normalise both sides, so a peer whose ROE arrives as 0.24 now joins a median with a company at 24.0 instead of being dropped. A blank unit counts as unknown, not a mismatch.
Plausibility ranges remain unit-aware: roce at percentage_ratio 0.364 is judged against 0–1, not 0–100, or the check passes everything.
Company and peer values are both selected by latest period.

Committee-safe fact projection

Conflicted metrics stay in the canonical payload and additionally appear under excluded_from_decision with the reason. Dropping them silently made the committee reason as though the data had never been acquired — a different, and wrong, conclusion from "we hold two readings and cannot yet say which".

The BEL report used a provider 1-year revenue growth of 1.99% as evidence of slowing growth while printing FY2025 and FY2026 revenues implying 15.9%. Both may be right; until that is established neither may be quoted as the growth rate, so the metric is marked usable_for_decision = false and the 3-year CAGR — which reconciles — remains usable.

The publication builder routes every fact into its section's metric list, and the rejected set used to ride along — a non-reconciling revenue_growth_1y fell into the fallback bucket absorbed by historical_financial_performance, so REJECTED_FACT_USED_IN_PUBLICATION fired on every run regardless of what the model wrote. The builder now receives the usable_for_decision = false fact ids from the canonical state and skips them when routing, so the rejection stays visible in the reconciliation flags without becoming a section metric.

Valuation output types

"Valuation" was one word doing three jobs. The report published a sensitivity grid (40× → ₹318.80 … 60× → ₹478.20) and then said no ledger-supported per-share valuation output existed. Both halves were defensible; together they read as a contradiction. The capabilities separate them:

can_do_multiple_sensitivity — implied price at an assumed multiple. Needs EPS only. build_sensitivity_grid(eps, multiples) multiplies an EPS by assumed multiples; a current price appears nowhere in that arithmetic. The flag used to require one, so a run that lost its quote printed the grid and, two pages later, wrote that the evidence "does not support multiple-sensitivity analysis" — the prose was faithfully reporting a flag that was wrong.
can_do_price_scenarios — upside/downside against the traded price. This is the one that needs the price.
can_publish_fair_value / can_publish_target_price — needs a justified target multiple or forward EPS. False on every run today, because nothing in the metric registry supplies a forward estimate.

The correct sentence is "a TTM multiple sensitivity can be calculated, but the evidence does not support choosing a target multiple or forward EPS" — never "no per-share valuation output exists".

CAPABILITY_CONTRADICTION blocks publication when a section denies an analysis the canonical state grants. It is matched per sentence, and a sentence that grants availability elsewhere ("a TTM multiple sensitivity is available, but no price-relative downside case can be published") is not a denial.

Peer statistics and the peer table

peer_stats computes median, mean, min, max, count, the company's percentile and its premium/discount to the median, per metric, in code. The BEL narrative reached "premium to peers" by naming Mazagon Dock alone while five peers sat in the ledger; arithmetic done in prose came out differently in different sections of the same document.

The rendered table reads from the canonical state, not raw facts, so it cannot print a peer's FY2023 margin beside the company's TTM one. Cells are formatted, not raw: ROE 24.0%, never ROE 0.24. Every column names its window — Revenue CAGR (3Y), P/E (TTM) — because the published table had a column headed Growth whose underlying fact was a 1-year revenue figure the same report flagged as unreconciled. A column no peer carries is dropped rather than rendered as em-dashes, and a company metric marked usable_for_decision = false is withheld from the subject row.

Peer evidence is a selected set, and the language is bounded to match: a claim of "sector-leading" raises UNBOUNDED_PEER_CLAIM (non-blocking — the comparison is real, only its generalisation overreaches). The supportable form is "highest in the selected peer set".

Calculation appendix

Every rendered row states what it computed and what it produced. The published appendix ran five pages of ID | Version with Description and Output blank on every row: nothing ever set CalculationLineageV3.formula_description, and the appendix never looked up the output fact.

FORMULA_DESCRIPTIONS in v3/acquisition/formulas.py carries the definition in words for every registered formula (a test fails if one is missing), and the builder resolves the output value and unit from the ledger. A row that can state neither is not rendered, and CALCULATION_APPENDIX_INCOMPLETE blocks publication if one slips through.

Every input fact id is rendered, prefixed with the count (n=5: …). Truncating the list printed a five-peer median beside four peer fact ids — an audit trail that cannot reproduce its own number — and n is the first thing anyone checking a median asks.

FCF conversion is free cash flow ÷ net income × 100. The metric registry names the metric "FCF Conversion" and the formula computed operating cash flow ÷ net income, which is CFO-to-PAT — a different quantity, roughly 2.7x larger for a capex-heavy issuer. The report then said "FY2026 free cash flow was only 25.4% of net income" while printing FCF of INR 5.56bn against PAT of INR 60.62bn (9.2%), and its own reconciliation warned the figure "should not be characterized as 25.4% of the listed FY2026 net income". The registry name is authoritative because it is what a reader acts on.

free_cash_flow is itself derived (OCF − capex), so _formula_rank orders the derivation pass from the specs — a formula built on another orders itself rather than resolving on no run because the blueprint happened to emit its obligation first.

Pagination

paged_table splits a long table into blocks that reportlab places whole, repeating the header, and merges a tail shorter than three rows back into the block before it. The published Source Map spilled a single row onto page 14 and left the rest blank — nothing was wrong with the table; a LongTable splits wherever the page boundary falls. Section headings are kept with their first block so a heading never strands at the foot of a page.

Scenarios

The section is titled "Bull, Base & Bear" and shipped two of the three, with a bull case that opened by arguing against itself: "The current premium valuation leaves limited room for execution shortfalls."

That was not a weak model. The graph node is bull_challenge and its label is the task statement the model receives — it was told to challenge the bull thesis and did, and the builder rendered the result under the heading "Bull Case". The labels now state which side to argue ("steel-man the upside: what must go right, which metrics improve, and why the valuation would hold").

The base case is composed deterministically from the committee's own conclusion — strategic view, base valuation, and the conditions the thesis holds under. No third model call, and the three scenarios cannot contradict the rating printed above them. Without forward EPS they stay qualitative; nothing invents a price target.

Non-blocking defects are still reported

Severity decides whether a defect stops publication, never whether it is visible. Non-blocking hard-gate findings used to be computed, counted against the dimension scores, and then dropped on the floor — no operator ever saw one. They now reach non_blocking_defects.

Rating gate

can_issue_rating is has_current_price and has_financials and can_do_current_valuation — all three, no partial credit:

| Input | Why it gates the rating | |---|---| | Current price | A rating is a statement about the price a reader can trade at. Multiples alone rank a company; they do not say buy or sell at today's quote. | | Financials (revenue_ttm / net_income_ttm) | Without them the multiples have no earnings base to be a multiple of. | | A current-valuation leg (P/E, P/B or EV/EBITDA) | Something has to be the valuation method the rating cites. |

A run holding P/E, P/B, EV/EBITDA and a full peer set but no quote therefore publishes NO RATING ISSUED, not a relative-value rating — deliberately.

A committee that issues a rating without them has it withdrawn to INSUFFICIENT_EVIDENCE, with RATING_GATE_FAILED recorded and its reasoning retained. The gate only ever removes a rating — it can never manufacture one — and NO_RATING remains a valid successful outcome. decision.py zeroes confidence alongside it, so no surface shows a number that outlived its rating.

A number keeps the period its fact was measured over

Page 3 of one report read "FY2026 ROCE of 36.4% and ROE of 28.8%". The ledger measures both over TTM — and that report's own methodology section listed the FY2026 characterisation under Rejected. Rejecting a claim in one section and repeating it in another catches nothing.

PERIOD_LABEL_MISMATCH blocks publication. It anchors on the value, not on metric names: a number matching a ledger fact, in a sentence naming exactly one period, must be a period that fact was measured over. A sentence naming two periods is a comparison ("from 15.5% in FY2023 to 18.2% in FY2026") and is left alone.

A direction needs a difference the reader can see

"Debt-to-equity is 0.003x, below the selected peer median of 0.003x" is true at full precision (0.0027 against 0.0030) and false at the precision it is printed in. ROUNDED_COMPARISON_MISLEADING blocks it. Either show the digits that separate the two values, or say they are approximately in line.

Aggregators may index evidence, never supply it

Every V3 report carries the same standing blocker:

INSUFFICIENT_AUTHORITY_FOR_COVERAGE: cash_flow_balance_sheet is grounded on
AGGREGATOR-grade evidence for financial_statements, which requires COMPANY

AGGREGATOR is 10 and COMPANY is 30, so adding a second aggregator changes nothing — a Screener or Tickertape number lands at exactly the level yfinance already occupies, and buys only a fresh source of SOURCE_VALUE_CONFLICT when two aggregators round differently.

What an aggregator can do is say which primary document to fetch. One Screener company page links 54 documents on bseindia.com — annual-report PDFs and corporate-filing attachments — and that host is already on documents.DEFAULT_ALLOWLIST. ScreenerDocumentDiscovery returns those URLs and nothing else; DocumentFetcher fetches them under the §7.1 contract, and authority_for resolves the result from its document_origin (bse_filing → FILING, annual_report → COMPANY) before it ever looks at the source id.

The provenance stays honest in both directions:

| | | |---|---| | source_id | screener:<n> — absent from _SOURCE_AUTHORITY_TABLE, so anything reading it alone resolves to DISCOVERED. Fail-closed. | | document_origin | Where the document actually lives. This is what grants FILING/COMPANY. |

Discovery drops anything not HTTPS-and-allowlisted before returning it, and the fetcher re-vets independently — a link to screener.in itself, or to an issuer's own IR domain, never becomes evidence. Page anchors (#page=30) are stripped: the same 13 MB filing is linked at several anchors, and without stripping, dedupe fails and it is fetched once per anchor.

The provider is inert until a page fetcher is injected, and still sits behind the per-symbol filings gate — PORTFOLIO_AI_RESEARCH_V3_FILINGS_ENABLED plus an explicit symbol allowlist, both off by default.

Governance needs a primary source too

The evidence-family authority matrix had no entry for shareholding_pattern or annual_report, and an absent family is treated as no authority requirement — so an aggregator's promoter-holding or pledged-share figure would have been admitted on the strength of the table not naming it.

Both families have primary sources that exist and are now fetchable: a shareholding pattern is filed with the exchanges every quarter, and board composition is in the issuer's own annual report.

| family | minimum | why | |---|---|---| | shareholding_pattern | REGULATOR | filed quarterly with BSE/NSE, published by the exchange | | annual_report | COMPANY | the issuer's own document |

This only tightens. No fact in the ledger carried either family when the entry was added, so nothing that previously passed now fails — it closes the hole before governance evidence starts arriving, not after.

One metric, one evidence family

Six metrics can be minted two ways. The aggregator's flat fundamentals snapshot carries a ready-made TTM value, and the formula pass derives the same metric from statement lines at fiscal-year periods. The extractor stamped fundamentals, the formula stamped financial_statements — so one ledger held roe.company.ttm in one family and roe.company.fy2025 in another. Same quantity, two gradings, decided by nothing more than which packet answered.

evidence_family answers what the fact is; who published it is source_authority, recorded separately. All six are statement quantities however they arrive, so EVIDENCE_FAMILY_BY_METRIC (acquisition/extractors.py) now owns the label for both paths:

| metric | family | |---|---| | free_cash_flow, net_margin, roe, roa | financial_statements | | revenue_growth_3y, eps_growth_3y | financial_statements |

An aggregator-supplied copy then faces the financial_statements COMPANY bar it should always have faced, instead of sheltering under a family the authority matrix does not name. Measured against three persisted ledgers before shipping: the same eight sections fail authority before and after, so this corrects the label without moving any published outcome.

A gap names the source that would close it

Every unavailable row of the Evidence Availability table read no fact acquired — the same three words whether the family needs a source nobody has wired, needs one that does not exist for this market at all, or simply came back empty this run. Three different facts about the product, rendered identically, and a reader cannot act on any of them.

Measured across three persisted ledgers (BEL, PFC and one full-schema run), the reports carry 5–8 reader-visible gaps each, and all of them trace to exactly two of the thirteen evidence families. The families are missing for different reasons, and the row now says which:

| family | detail | kind of gap | |---|---|---| | Governance | needs the exchange-filed quarterly shareholding pattern; no provider here reads one | unwired — the primary source exists | | Forward estimates | needs analyst consensus estimates; no provider here supplies them | structural — no such source in this pipeline |

The distinction is what makes the row actionable. Governance is pinned at REGULATOR by the authority matrix above, so the natural repair — pointing an aggregator such as Screener or Tickertape at it — mints a fact that the authority gate then rejects, leaving the row blank and a fetch spent learning that. Forward estimates have no source at all, so a reader who knows that stops waiting for a target price that is never coming.

The measurement still leads and the reason follows it (no fact acquired — …); naming the reason adds to the row rather than replacing what it already said correctly. The strings describe this pipeline's inputs, not a roadmap: they say what a family needs and that nothing here supplies it, never when that might change.

Reaching BSE at all

The document fetcher identified itself honestly and was refused. Measured against bseindia.com with the fetcher's own header set:

| User-Agent | Result | |---|---| | PortfolioPlannerResearch/3.0 (+…; contact: …) | 403 at the edge | | Mozilla/5.0 … Chrome/126.0.0.0 Safari/537.36 | 404 — reached the origin |

The 404 is the informative half: with a browser UA the request arrives and is answered on its merits, so the User-Agent is the whole of the gate. Every filings fetch had been failing closed on that 403, which is why the evidence families that depend on filed documents could never be covered.

corporate_actions/bse_source.py reached the same conclusion for the same host in July 2026 and already shipped a browser UA. This aligns the document fetcher with that decision rather than leaving one of the two BSE paths silently dark.

Contactability was not dropped, it moved: the From header (RFC 9110's header for a human contact behind an automated agent) carries the address the UA used to, and no CDN gates on it. Both remain overridable via PORTFOLIO_AI_RESEARCH_V3_FETCH_USER_AGENT.

NSE is still closed. It does not 403 — it completes TLS, accepts the HTTP/2 request and then stalls until timeout, and a browser UA does not change that (its bot management fingerprints cookies, header order and TLS, not just the UA). Nothing in this pipeline reaches nseindia.com.

Forward estimates, and whose target price it is

Forward estimates were the one evidence family with no source at all. The report could not read the growth and margin assumptions embedded in the current price, and can_publish_target_price — which gates on valuation.forward_pe / valuation.consensus_target — was never true, so valuation, committee and scenarios each said a target price was not publishable.

Sell-side consensus closes it, from a fetch the pipeline was already making: get_profile reads yfinance .info for sector and industry, and forwardPE / targetMeanPrice / numberOfAnalystOpinions are keys on that same object. No new host, no allowlist entry, no fundamentals-store column.

A forecast is not a fact about the company, so the wiring is deliberately narrow:

| guard | why | |---|---| | yfinance:consensus maps to consensus_estimates only | a forecast must never satisfy an obligation asking for reported history, and cannot stand in for a quote | | consensus_estimates pinned at AGGREGATOR | an unnamed family means no authority requirement — the hole the shareholding_pattern entry closed. Not a relaxation: nobody files a mean of other people's forecasts, so a higher bar would be unreachable by construction | | analyst_count is a required metric | a mean of 2 estimates and a mean of 31 are different claims | | company scope only | a peer's consensus target is not evidence about this issuer |

The target price is attributed rather than merely printed. _consensus_target_basis writes, deterministically from the ledger and never by the model:

Target price basis: sell-side consensus mean of 31 analyst estimates (aggregator-grade evidence). This is the market's published forecast, not a valuation produced by this report.

It returns "" — and the report says nothing — unless the ledger holds both a company-scope consensus target and the analyst count behind it. "Consensus" with no number attached sounds like sourcing while supplying none.

forward_expectations is a universal capability, so a company nobody forecasts now carries an uncovered obligation and a lower coverage percentage. That is the intended reading: a stock thirty analysts follow and one nobody covers are different propositions, and the evidence matrix now says no sell-side analyst coverage found for this symbol rather than blaming a missing provider.

The aggregator attestation prints once

grade_evidence writes the aggregator-source attestation into the methodology section. It wrote it into both summary and narrative, and the renderer prints both — so the paragraph shipped twice, verbatim, back to back. The builder's own summary/narrative dedupe runs earlier and never saw it. The attestation is now the summary; the narrative keeps whatever the reconciliation specialist wrote, unless that is the same sentence again.

Canonical industry vs aggregator profile

Three different questions used to share one cover label:

| Line | Question | Source | |---|---|---| | Industry: | What industry is this issuer in? | An industry fact in the ledger | | Aggregator profile: | What did the profile provider call it? | blueprint.company.industry — printed only when the ledger has nothing, under Canonical industry: unverified | | Research overlay: | Which industry-specific KPI set applies? | The blueprint overlay; General means no specialised set exists |

General was never a claim about an industry, and the aggregator profile is never promoted into the evidence line.

Numeric fields the model fills

Specialist contracts are strict=True, so Pydantic will not coerce. JSON has no decimal type, so a model answering a Decimal field can only send a float — and strict mode rejects it. Production run 9c1ab7752c924f8886c92c6ac741fc90 died exactly there:

fair_value_range_low
  Input should be an instance of Decimal [input_value=203.91, input_type=float]

Because the graph hard-blocks on missing dependencies, that single failure skipped bull_challenge, bear_challenge, fact_reconciliation, investment_committee and publication_review. The one output the valuation specialist exists to produce was guaranteed to kill the report, and CommitteeDecisionOutput.target_price had the same shape — so a committee stating a target price died the same way.

Neither existing safety net covered it: the retry is for transient failures and this is deterministic; RepairEngine only runs after a successful invoke whose output fails validation, and here validation raises inside invoke().

Model-supplied numeric fields therefore use LenientDecimal, a BeforeValidator that converts via str (so 203.91 stays 203.91, not 203.909999999999996589…). Booleans are still rejected — bool is an int subclass and a boolean in a price field is a defect — non-numeric strings still fail, and strictness is untouched everywhere else.

Citations must match the quantity

Every canonical and peers entry carries its own fact_id, and the prompt requires citing them verbatim. Supplying peer values without citable ids made a specialist construct one (pb_ratio.peer-mazdock.ttm for a fact that is .mrq), which QA correctly blocked as UNKNOWN_FACT_ID.

A cited fact must also be the same quantity as the value it supports. A fair-value price in INR may not cite a P/B ratio fact — QA catches that as FACT_VALUE_MISMATCH (value 269 != fact value 8.540; unit 'INR' != 'ratio'). A derived number with no matching ledger fact belongs in the narrative, not in a metric: the multiple × EPS grid already ships as a Valuation Sensitivity table, and a sensitivity is not a fair value.

Markup in the PDF

safe_markup escapes everything, which is correct — narrative text comes from a model and must never inject markup into the document. But callers wanting a bold lead-in wrote the tags into the string:

paragraph(f"<b>Strategic View:</b> {narrative}", style)

and the tags were escaped along with the narrative, so every V3 report rendered a literal <b>Strategic View:</b> to the reader. Use rich_paragraph(style, label=…, text=…, detail=…) instead: the component supplies the label markup, model text stays escaped, and detail renders the small print (a confidence decomposition, a formula) underneath.