Lanes, verdicts and outcomes

Frazer Reviewer is one product built from many lanes. A lane is a typed review program: what it reads, when it applies, a static stage that runs first, a model stage that reads only what the static stage passed, the findings it may raise, its cost class and its owner. Frazer ships lanes in nine groups, from security to “making sure it’s not a mistake”, organizations write their own as files, and lane packs install others’. A change is split into items, and squads of reviewers read each item in parallel, mostly without context, so each gives an honest opinion. Every review is remembered with what it did, what it cost and what it found; when something breaks later, the incident links back to the review of the change that caused it, and what was missed becomes a better lane.

The short version

Lanes are data: versioned, backtested, in shadow until they prove themselves. Static checks run first and are free; models read small items at the lowest context rung that answers. The verdict auto-merge reads needs evidence the author did not produce, reviewers of another model family, and every mandatory lane run. An incident is classified told (a finding was raised and not fixed), missed, partial or outside, and every miss gets an analysis whose remedy a person seals.

What a lane is

Part	What it holds
Name, group, kind	security/authz; one of the nine groups (a view of the catalog); one kind, which is its price line: code review, security review or cold review
Targets and items	What it reviews (layers, stacks, plans and records, repository snapshots for scans) and which item kinds it reads
Trigger	A CEL expression over Fir’s facts (paths and their tags, change classes, languages, symbols touched, risk class, plan type), never a model’s guess
Static stage	Deterministic checks first: CEL rules, structural code patterns, analyzers, fitness functions, Frazer CI tasks. Their results are evidence and findings of their own; a failure holds the model stage and says what it saved
Model stage	A short checklist with the evidence each line must read, what not to flag, the default context rung, and the roles (reviewer, verifier of another family)
Findings	The classes it may raise
Cost class	static, light, standard or heavy (execution on CI or a preview), with the estimate before a run
Owner and harness	An OWNERS group answers for it; it runs on a harness profile (frazer:review-lane by default, or any harness you assign)
Version and state	draft, shadow, active, retiring; its precision per class with its interval, and its escapes

pellwyn/.frazer/review/lanes.star

load("@frazer//review:v1", "review_lane", "cel", "pattern", "checklist")

review_lane(
    name = "pellwyn/tenant-scope", kind = "security", group = "security",
    items = ["symbol"],
    applies = cel("self.repository.tags.exists(t, t == 'multi-tenant') && " +
                  "self.items.exists(i, i.kind == 'symbol' && i.touches.exists(s, s.startsWith('db.')))"),
    static = [pattern("rules/query-without-tenant.yml")],   # a query on a tenant table without tenant_id
    model = checklist(file = "lanes/tenant-scope.md", rung = 1,
                      roles = {"reviewer": "standard", "verifier": "other_family"}),
    raises = ["security.authz"], cost = "standard",
    owner = "@pellwyn/security", state = "shadow",
)

The same lane can be written as YAML, which compiles to the same thing. Lanes never receive a person’s principles: a lane reads the change and its evidence.

The catalog

Frazer ships built-in lanes for code and security in the nine groups, plus the same groups for plans in cold review. Mandatory lanes run on every layer they apply to. You can turn any lane off, but a layer whose mandatory lane did not run gets no pass, so it never auto-merges.

Group	Lanes at launch (examples)
Security	security/authz (input to sink, tenancy, roles), security/injection (taint rules), security/secrets (static only, every layer), security/supply-chain (lockfile diff, release age, provenance, install scripts, and that every new package exists and is not a fresh look-alike), security/agent-config (instructions aimed at agents), security/privacy (personal-data fields in logs and flows)
Business	business/requirements (every acceptance criterion has a test that targets it; does the item meet its criteria and the decisions in force), business/money (minor units, currencies, rounding, limits, idempotent charges)
Design	design/ui (design tokens, a visual diff in the preview), design/accessibility (an accessibility checker in the preview, keyboard paths), design/copy (strings and wording against the glossary)
System design	architecture/boundaries (import rules, layering, cycles, ownership; the decisions and records in force), api/design (naming, pagination, errors, compatibility)
Code	code/correctness, code/concurrency, code/tests (no assertion weakened; a change that weakens CI is a blocker), code/maintainability (a summary only, never inline), docs/accuracy (comments the code contradicts)
Benchmarks	performance/benchmarks (affected micro-benchmarks, base and head interleaved on one runner; macro suites after each test batch lands, with change points and a bisect), performance/queries (N+1 and query plans on a database branch)
Integration	integration/contracts (breaking-change checks on protobuf, OpenAPI and schemas, with the consumers Fir knows), integration/e2e (the preview’s flows for what the change affects)
Production safety	production/readiness (configuration, flags on risky paths, limits and probes, rollout, rollback, alerts, runbooks), data/migrations (locks, rewrites, reversibility, a rehearsal on a database branch), reliability/idempotency, reliability/timeouts, observability, production/expected-effects (what should move and what must not, checked after deploy), production/risk (a risk score learned from your own incidents, an input and never a verdict alone)
Intent (“is it a mistake?”)	intent/conformance (the diff against the plan item), intent/reconstruction (a model of another family restates what the diff does from the diff alone, and Fir compares it with the criteria), intent/mistakes (debug code, stray files, generated churn, unexpected version bumps, merge leftovers)

Lanes as files. Built-in frazer: lanes ship with the engine. Organizations, projects and repositories write lanes in .frazer/review/lanes.star (or the YAML form), layered organization → project → repository → path, the stricter winning, with locked floors. Lane packs from the Marketplace are signed, pinned by digest, show the reads they need and their price before install, and never get a write tool. A want becomes a lane: “check that every payout path writes an audit record” compiles to the most deterministic lane that can decide it (a CEL rule, an analyzer, a CI task, and only then a model checklist), shown with its cost and a backtest.

Lanes you can carry. A lane follows a published contract with a JSON Schema. frazer reviewer lanes validate checks a file, lanes export writes a lane as lane JSON, an Agent Skill or SARIF rules, and lanes import --dry-run turns a REVIEW.md, an AGENTS.md or a skill into lane drafts; frazer reviewer lanes list --repo acme/storefront --path services/payments/refund.go says which lanes apply to a file and why. A built-in security/ci-workflows lane, static and mandatory on workflow and pipeline files, refuses a workflow that checks out untrusted code with write access, interpolates untrusted text into a script step, uses an unpinned third-party action, prints a secret, grants write-all permissions or runs a public repository on a self-hosted runner; no model’s all clear overrules it.

Coverage, apart from the result. Each run records what it covered (frazer reviewer coverage get <rev>): the items and bundles read, the lanes run and skipped with why. A bundle is files that must be read together, such as a handler and its route registration, one message key in every locale, or a migration and the model it changes.

From shadow to blocking. A new lane or a new version runs in shadow (it records and posts nothing) until its backtest on the last 50 merged pull requests in its scope passes and its frozen seeded set of known defects and known-good code shows recall of 95 % or more. It posts once active, and blocks only when its classes’ precision lower bound reaches 0.8 on at least 50 labels. A revision of a lane (its checklist, trigger, rung, static rules, or the model its profile names) is admitted only by probes on fresh, held-out merged work, with the safety suite, and a person seals it.

Built-in lanes follow Frazer’s lane updates. When Frazer improves a built-in lane (a new checklist or model, sealed by Frazer’s lane owners), your organization’s lanes follow it once it replays clean in shadow on your organization’s own last 90 days. To stay on the current lane, pin it (builtin_lane_updates: pin); a pinned lane changes only when one of your people accepts the update.

Squads that review each item

Items. Fir’s structured diff splits each version into items: a symbol (a changed function, method or type, 20 to 40 changed lines by default), a file, a test, a contract, migration, configuration, dependency, screen or benchmark, a concern (every authorization check, every SQL query, every outbound call or every write with a retry in the change), the plan item with its criteria, and the whole change (only its interfaces). Each item carries its contract: the plan item’s criteria and the intent lines that cover it. Review quality falls fast with diff size, which is why items are small.

The lane plan and dispatch. Before any model runs, a deterministic lane plan lists the mandatory lanes for the risk class, then every lane whose trigger holds, each paired with the items it reads, and each skip with its reason, with the estimate first. The review then passes the same admission service as all model work, so a campaign never starves its own second opinions. Screens run in call mode (one request, schema output, no tools); deep reads that use tools run as sessions; one review run is one session of kind review, with tool-using item reviews as its child sessions. Static stages run first and in parallel, then screens, deep reads, verification, the pass over the whole change and aggregation; no step waits on another item’s model call.

Blind by default, and rungs. On a first pass a reviewer never sees the description and discussion, the author’s transcript or reasoning, other reviewers’ findings, prior scores or who the author is; a blind manifest (the digests of everything a reviewer was given) is stored per item review, so blindness can be audited. Code context comes by rungs:

Rung	A reviewer may read	Default for
L0	The item and its contract	Intent reconstruction, copy, observability
L1	Plus enclosing symbols, direct callers and callees	Most lanes
L2	Plus read-only repository tools: search, files, the history of the touched lines	Deep reads that need them
L3	Plus other repositories (consumers), the decision history, runtime signals	Contracts, architecture
L4	Plus execution: Frazer CI tasks, a preview	Benchmarks, UI, end-to-end tests, reproductions

A reviewer climbs one rung by a tool call inside its budget; every climb is on the receipt and in the lane’s calibration. At deep, a lane also runs one blind L0 read beside its default rung, because the two together find more than either.

Screen, deep read, verify, reproduce. A fast screen per item clears it or flags it, erring toward flagging (always deep for security lanes at R2 and R3). A deep read runs at the standard tier, or the max tier with two families at R3. A verifier of another family checks medium-or-higher candidates against code evidence; the tester reproduces high ones as CI tasks; security findings in loops need an exploit or a failing test, never a reading alone. At R2 and R3, when every lane calls an item clean, a consensus challenger of another family looks for evidence against that shared claim; when findings conflict, a disagreement resolver settles them against the environment (a test, a query, a run), never by vote. A lane that looks for something missing (a test for a criterion, a metric for a new endpoint, a rollback, an alert) lists what must be there and checks each line; no lane asks a model “is anything missing?”. Agreement is shown with its effective number of independent opinions, never the member count.

Aggregation and levels. Findings merge by fingerprint into one ledger; a pattern finding is one finding with many locations; a scan looks for text aimed at the author. The whole-change pass reads only the interfaces. A stack and a plan level above stacks get the rollup of their parts’ verdicts plus one pass over the interfaces between them, never one review of the whole group, because one review over many changes catches far less than reviews of each.

Campaign shards: the pattern reviewed once. A campaign applies one change across many repositories. The Reviewer reviews the pattern once, at the campaign’s approval; each shard then gets its static lanes, Fir’s check that the shard’s diff is the pattern applied (byte-identical for a patch, syntax-tree equal for a transformation), and screens only on the hunks that deviate. A shard whose deviation is material leaves the pattern and takes a full review.

The verdict auto-merge reads

One verdict per pull request version is what auto-merge reads:

Field	What it says
result, reason	pass, fail or no_verdict with its reason
version	The digest of the exact version reviewed; a new version voids it
risk_class	R0 to R3, with its reasons
p_clean	Calibrated for the class and repository, with its 95 % interval and the count behind it
blockers[]	Each high finding, failing proof or scope breach that stops a pass
lanes_run[], lanes_skipped[]	Every lane in the plan, and why each skipped one did not run
independent_evidence[]	Each source the author did not produce: its kind, its reference, whether it was executed
independence	The author’s model family and harness; each reviewer’s family and harness; the effective number of independent opinions; the blind manifest’s digest
evidence, calibration, cost, time, signature	Hashes of the inputs, recordings and CI results; the calibrator’s version; what the run cost and took; the signature

A pass needs all of: no blocker; no open finding at medium or above; p_clean at or above the class’s threshold; every mandatory lane ran; for R1 and above at least one independent evidence source, and for R2 and R3 an executed one; and every reviewer of another model family than the author. Another harness than the author’s is preferred and attested on every verdict, and required at R3 or by your policy. Every claim the verdict accepts has its evidence event in the author’s session; a claim without one (“added tests for the retry path”) is unverified and never counts.

Independent evidence	How the Reviewer gets it	Executed
Acceptance tests from the criteria	The tester writes them from the plan item’s criteria, never shows them to the author, and runs them in CI	Yes
A reproduction	A test that fails at an earlier version and passes now, for a finding raised in the loop	Yes
Intent reconstruction	intent/reconstruction, compared field by field with the criteria	No
A preview run	A test run on the affected flows, graded on checked backend state; a run that could not happen is a no_verdict, never a pass	Yes
A benchmark run	performance/benchmarks on the affected benchmarks	Yes

Class	Mandatory lanes (where they apply)	Depth	Independent evidence
R0	intent/conformance, intent/mistakes, business/requirements, code/correctness, security/secrets	light	None required; the verdict says “shared evidence only”
R1	R0’s, plus intent/reconstruction	standard	One, of any kind
R2	R1’s, plus security/authz, security/injection, security/privacy, production/readiness, data/migrations, integration/contracts	standard, with a verifier of a third family	One executed
R3	Every lane that applies	deep: two families per deep read, the max tier for the verifier	One executed, and a reproduction of every high finding

No verdict. no_funds (“Not run: no funds. Static lanes ran (14 of 16 green). Balance $0.00; top up or raise the review budget.”), provider_down (a family the lane plan needs is unavailable; never a substitute family), held (a static stage failed, so the model stages wait, and it says what that saved), too_large (split suggested) and lane_error. A no_verdict never merges and is never replaced by a cheaper run; the static lanes still run.

frazer review verdict pellwyn/payouts#15012 --json
fir pr second-opinion 15012 --wait

Every review remembered

Every review run (rev-…) is an immutable record, written as it happens:

Part	Recorded
What was reviewed	The target and its version’s digest; the trigger (policy, a request, a loop’s turn, a lookback); every item’s digest
What ran	The lane plan: every lane and its version, the items it read, why each skipped lane did not run; for each item review the rung reached and every climb, the harness profile, the model and its family, tokens, cost, time and result
What it found	Every finding, its evidence and grade, and its fate (posted, kept silent and why, refuted)
What it decided	The verdict
What it cost and who paid	The total and each stage’s cost, the budget it drew on, and “Not run: no funds” when it did not start
Proof	Hashes of the inputs, recordings and CI results; the blind manifests; the signature; a timestamp per stage

A finding’s headline (its title, class, severity, anchor, fingerprint, state, and who resolved it and how) and each run’s summary are kept for as long as the repository exists, so a hotfix years later can still find the warning; the finding’s full text and evidence keep the usual retention, extended while the finding stays open or dismissed on code that is still on main unchanged. Organizations with zero retention keep hashes, classes and states, never text. Across runs, versions and lookbacks a finding is in one of six states: new, persisting, reopened, not observed, resolved or unknown. It is resolved only by evidence (a fix whose re-check no longer raises its fingerprint, or a person’s close with a reason); not observed, when a later run covered the lines and did not raise it, is never resolved, and a High finding that disappears without a fix raises an alert. Headlines search and export as JSON Lines or SARIF (frazer reviewer headlines search, frazer reviewer headlines export). Separately, a finding ends one of five ways: fix (the anchored code changed), ask (a question or decision for a person), accept risk (Won’t fix, with a reason, a scope and an expiry), out of scope, or dismiss as wrong (which labels the lane’s precision); each end records who chose it, their words and the version. The Reviews subsection and frazer reviewer runs list read the ledger in the query language of pull requests: lane:security/authz verdict:fail risk:R2.. cost:>1 since:7d.

Outcomes linked both ways

Incidents and error-budget burns (through the issue’s caused_by), reverts, rollbacks and failed canaries, hotfixes (a pull request with fixes:), escapes found later by a scan or a lookback, failed fixes, performance regressions and test gaps all find the review of the change behind them. Fir classifies each link from the causal lines and the review runs that covered them; a model of another family summarizes, never classifies:

Class	The rule	Shown as
told	A finding at medium or above on the causal lines was raised and not fixed: dismissed, Won’t fix, ignored or overridden	“Flagged on Nov 9 by reliability/idempotency · Won’t fix (‘internal endpoint, no retries’)”
missed	The runs that covered the causal lines found nothing there	“Missed by the review of Oct 12 · miss analysis ma-0417”
partial	A finding existed but did not reach anyone, or not at its weight: kept silent under the threshold, the wrong class or severity, refuted by the verifier, or held	“Raised but kept silent under quiet”
outside	The cause is outside any reviewed diff: configuration applied outside Fir, data, a third party, infrastructure	“Outside the review: a provider change”

“We told you” shows on the incident, the hotfix, the issue and the original pull request, to everyone who can see them, as facts with dates and the dismissal’s own words, and never as a ranking of people. The person who dismissed or overrode the finding and the plan’s accountable person are told in their digest, blamelessly; a lane’s owners are told about missed and partial links. A dismissal later linked to an outcome recalibrates how much dismissals in that class and repository are trusted, and two told links in 90 days on one cause draft a learned check. The reverse is counted too: findings fixed before merge in classes that cause incidents.

Demo (Pellwyn, product UI, demo data): on Mon Nov 16 at 11:42 incident INC-0231 releases a payout batch twice after a new caller retried. The hotfix pellwyn/payouts#15040 changes the release’s key; the issue’s caused_by is pellwyn/payouts#15001 v2, merged Nov 9, found by an automatic bisect. Its review had raised a medium reliability.idempotency finding (“batch release has no idempotency key; a caller’s retry releases again”), marked Won’t fix with “internal endpoint, no retries”. Outcome link ol-0031 is told: the incident and the hotfix show the finding with its date and the dismissal’s words, and the remedy proposed is a lane revision that makes “no retries” an answer that needs a citation of the callers Fir lists.

Misses become better lanes

For every missed and partial link, an agent of another family runs a miss analysis (ma-…) within 24 hours: it replays the lanes on the causal change at its reviewed version, with the fix now known, at each rung, and records which lane at which rung would have caught it and why the review did not:

Cause	Remedy proposed
The lane’s trigger did not hold	A trigger change
The defect spanned two items no concern grouped	A split or concern rule
The rung was too low	A higher default rung for that item kind
The model missed it at the right rung	A checklist line, a seeded case, a deeper depth for the class
Below the threshold, or refuted by the verifier	A calibration change for the class and repository
Dismissed by a person	A label, and a rule that makes that dismissal need a citation
Held, no funds or a provider outage	Nothing about the lane; the report says the review did not run

A static check is always tried first: a rule that reproduces the defect on history at a precision of 0.9 or more becomes a draft static stage. Remedies go through the improvement step as lane revisions, probed on fresh merged work (never the miss’s own case alone), with the safety suite, then shadow, and a person seals each one (the lane’s owners). The miss’s case joins the lane’s frozen seeded set either way.

Demo (product UI, demo data): on Fri Oct 16 a scheduled scan of pellwyn/admin finds a report endpoint that returns another tenant’s rows; blame sends it to pellwyn/admin#2291, merged Oct 12, whose review passed: missed. Miss analysis ma-0417: the split put the handler and its route registration in two items, and security/authz read the handler at L0, which never showed that the route bypassed the tenant middleware; replayed at L1 the lane finds it in 3 of 3 draws. Remedies: a static check that every handler under admin/reports/** calls the tenant check, security/authz revision 8 reading handlers at L1 by default, and a seeded case. The revision waits for its owners’ seal.

Lookbacks. When a lane revision finds more, it re-runs in shadow over the last 90 days of merged code in its trigger, within a budget shown before it starts. Its findings never land as a flood: they become one ranked backlog (by severity, the code’s traffic and its owners’ load), worked by agents as a campaign, with people seeing only what only a person can settle. Lookbacks are on by default within their budget.

Learning number per lane	Target
Precision: upheld findings over judged findings, with its 95 % lower bound	A lower bound of 0.8 or more on at least 50 labels before the lane blocks
Escape rate: missed or partial links per 100 layers reviewed, over 90 days	0.5 or less at maturity
Told, not fixed: told links over findings dismissed in the lane	Reported; its fall after a revision measures dismissal quality
Seeded recall	95 % or more
Link to analysis; analysis to a revision in shadow	24 hours; 7 days

Loops by default

Every layer an agent writes gets a review loop by default, from the first day: author, critic and tester until a stop rule fires, then a held-out final pass (Frazer Reviewer). A benchmark tunes each loop’s depth, turns, lanes and stop rules for accuracy first, with cost breaking ties; budgets are guards, set so that nearly every loop ends on a quality rule. When funds run out, a loop stops between turns as “Paused: no funds”, keeps its best version and says so; the layer’s verdict is no_verdict: no_funds and it does not merge. A person’s own pull request gets one review with fixes offered, never pushed: the person takes one with one key, or “take the safe ones”.

In the console

Frazer Reviewer has nine subsections: Inbox, Reviews, Lanes (the catalog and your lanes, with their states and numbers), Security, Loops, Outcomes (outcome links, miss analysis reports and lookbacks), Autonomy and audit, Quality (per lane) and Settings (lanes per repository and risk class).

Reference

CLI

Command	What it does
frazer review verdict	The second opinion auto-merge reads for a pull request version; exits 0 pass, 11 fail, 13 when a person must unblock it, 7 when it has no verdict yet.
frazer reviewer lanes list	The lanes in scope by group and kind, with their state, owner, harness profile and quality numbers.
frazer reviewer lanes create	Create a lane from a file; it starts in shadow until its backtest and seeded set pass.
frazer reviewer lanes test	Backtest a lane on recent merged pull requests in its scope and on its frozen seeded set.
frazer reviewer lanes install	Install a lane pack from the Marketplace; its lanes start in shadow.
frazer reviewer lanes revise	Draft a revision of a lane from a miss analysis or a want; a person seals it.
frazer reviewer lanes lookback	Re-run a lane revision in shadow over recent merged code, with its estimate first, filing a ranked backlog.
frazer reviewer lanes validate	Check a lane file against the published lane contract.
frazer reviewer lanes export	Write a lane as lane JSON, an Agent Skill or SARIF rules.
frazer reviewer lanes import	Turn a REVIEW.md, an AGENTS.md or a skill into lane drafts; --dry-run returns the drafts.
frazer reviewer coverage get	What one review run covered, the items, bundles and lanes, and why each skipped lane did not run.
frazer reviewer headlines search	Search finding headlines across the repository's life, by state, repository and time.
frazer reviewer quality get	The Reviewer's measured quality, per lane and period.
frazer reviewer lane-revisions seal	Seal a lane revision, for people only; --dry-run shows the plan.
frazer reviewer runs list	Review runs in the query language of pull requests, by lane, verdict, risk, cost or outcome.
frazer reviewer items list	The items of a review run, each with the lanes that read it, the rung reached and the result.
frazer reviewer outcomes list	Outcome links (told, missed, partial, outside) between incidents, reverts or hotfixes and the reviews of their changes.
frazer reviewer misses get	A miss analysis: which lane at which rung would have caught a defect, why it did not, and the remedies.

All of them: CLI reference.

MCP tools

Tool	What it does
reviews_verdict	The verdict for a pull request version, with its lanes, independent evidence and independence.
lanes_list	List the lanes in scope.
lanes_test	Backtest a lane.
reviewer_runs_get	A review run as recorded.
items_list	The items of a review run and their results.
reviewer_outcomes_list	Outcome links for a change, an incident or a lane.
misses_get	A miss analysis with its remedies.
lanes_validate	Check a lane file against the lane contract.
lanes_export	Export a lane as lane JSON, a skill or SARIF rules.
lanes_import	Turn a REVIEW.md, an AGENTS.md or a skill into lane drafts (a dry run returns them).
reviewer_coverage_get	What one review run covered.
headlines_search	Search finding headlines across the repository's life.
reviewer_quality_get	The Reviewer's measured quality, per lane and period.

All of them: MCP tools reference.

Events

Event type	When it is sent
evt.reviewer.verdict.issued	A verdict was issued for a pull request version.
evt.reviewer.outcome.linked	An incident, revert or hotfix was linked to the review of the change behind it.
evt.reviewer.miss.analyzed	A miss analysis finished, with its remedies.
evt.reviewer.lane.revised	A lane revision entered shadow or was sealed.

All of them: Events reference.

The API is frazer.reviewer.v1: lanes and their revisions, lane plans, item reviews, verdicts, outcome links, miss analysis reports and lookbacks. Lane revisions are sealed only by people: the MCP tool lane_revisions_seal is hidden from agents, the same verb as frazer agents revisions seal.

For agents

Read the verdict (reviews_verdict, or fir pr second-opinion --wait) and act on its blockers; a no_verdict is not a pass. You may propose a lane or a lane revision, never seal one, and never dismiss a blocking finding. A claim in your pull request counts only when your session shows the event behind it.

Related

• Frazer Reviewer: kinds, settings, findings, loops and prices
• Auto-merge and the merge queue: what the verdict decides
• Frazer QA: independent evidence from test runs
• Harnesses: call mode and the harness each lane runs on