Pular para conteúdo

AUR-448 — no discard is silent

Documento histórico do board. Pode descrever componentes removidos; para uso atual, consulte o guia do produto.

Card AUR-448 closes a confidently-wrong-answer defect in aurumcode review: a user who followed the no-provider-configured message literally could get No issues found. with exit 0 for a fixture that actually planted a finding, because the message's example shape omitted rule_id, which the AUR-434 rule gate (internal/review.enforceRuleCitations) requires. Two fixes, both required:

  1. The no-provider message now shows the complete fixture shape the engine accepts, rule_id included, with a real id from the embedded catalog as the example.
  2. When the rule gate discards one or more findings, the command names how many and why on stderr. It never happens silently again.

Command

aurumcode review --base <ref>

Unchanged from AUR-430/AUR-434 (see docs/specs/AUR-430.md and docs/specs/AUR-434.md): diffs <ref> against HEAD, sends the diff to the configured LLM provider, prints the findings. No new flags.

1. The no-provider message shows the complete fixture shape

Before this card (selectProvider, cmd/aurumcode/main.go):

no LLM provider configured: set AURUMCODE_LLM_FIXTURE=<path> to a JSON file
shaped like {"issues":[{"file":"<path>","line":<n>,"severity":"error|warning|info","message":"<text>"}]}
for offline use (see tests/fixtures/review/known-problem-response.json for
a worked example), or set LLM_API_KEY and LLM_BASE_URL for a live provider

A user who builds a fixture from exactly that shape gets a finding with no rule_id. The rule gate discards it, and — before this card's second fix — aurumcode review printed No issues found., exit 0: the code looked clean when it was not. The referenced example file also does not exist outside this repository's own checkout, so a user running the released binary elsewhere could not open it either.

After this card:

no LLM provider configured: set AURUMCODE_LLM_FIXTURE=<path> to a JSON file
shaped like {"issues":[{"file":"<path>","line":<n>,"severity":"error|warning|info",
"rule_id":"<id from the embedded rule catalog, e.g. security/hardcoded-secret>",
"message":"<text>"}]} for offline use -- a finding whose rule_id is missing
or unknown is discarded, never shown, so rule_id is not optional -- if you
have the AurumCode source checked out, tests/fixtures/review/known-problem-
response.json is a worked example -- or set LLM_API_KEY and LLM_BASE_URL
for a live provider

(wrapped here for readability; the command prints it as one line).

  • rule_id is shown with a real id from the embedded catalog (security/hardcoded-secret, internal/review/rules/security.yml), not a placeholder, so copying it verbatim into a fixture works.
  • The message explains, inline, why rule_id matters: a finding without one is discarded, never shown. A reader does not need to already know about the rule gate to avoid the trap.
  • The example-file pointer is explicitly conditional ("if you have the AurumCode source checked out"): the message is self-sufficient without it, because the file it names is not guaranteed to exist in the environment where the released binary runs. This wording keeps the pointer (still useful to a contributor working in this checkout) without making it load-bearing for anyone else — see AUR-443's tests/acceptance/AUR-443.sh (no-provider-missing-fixture-pointer), which still asserts the literal path substring is present in this message and continues to pass unmodified.

reportModelUnavailable (the --modelo <name> unavailable message, cmd/aurumcode/main.go) is a distinct message, reachable only when --modelo is given. It already points at the same fixture file and was not part of this card's measured defect (the achado's repro is aurumcode review --base HEAD~1, no --modelo); this card's Non-goals excludes behavior the declared command cannot observe, so it is left unchanged. Recorded here, not silently skipped.

2. A discard is never silent

internal/review.enforceRuleCitations (AUR-434) already discarded a finding whose rule_id was missing or unknown before this card; that gate is unchanged. What is new is discardSummary and formatDiscardWarning (internal/review/reviewer.go): GenerateReview now tallies why each discard happened (a missing rule_id, an unknown one, or both, naming the distinct unknown ids) and stores the rendered one-line explanation in result.Metadata["discard_warning"], "" when nothing was discarded.

cmd/aurumcode's runReview prints that string to stderr, verbatim, prefixed like every other canonical stderr line here ("aurumcode review: %s\n"), immediately after the review runs and before any finding prints:

aurumcode review: 2 finding(s) discarded: 1 with no rule_id, 1 citing an unknown rule_id (security/definitely-not-a-rule)

Singular case:

aurumcode review: 1 finding(s) discarded: 1 with no rule_id

stdout is byte-identical to the published AUR-434/AUR-426 contract in every case. The discard already hid the finding from stdout before this card (that gate is unchanged); this card only adds the stderr explanation. When every reported finding is discarded, stdout is still exactly No issues found. — now with the reason on stderr instead of nothing.

When rejected == 0 (every finding the model reported cited a resolvable rule, or the model reported none), discard_warning is "" and the if warning != "" guard in runReview never executes: stderr receives zero additional bytes. This is the byte-identical happy path tests/acceptance/AUR-448.sh's nominal_case measures directly against the unmodified known-problem-response.json fixture (one grounded finding, zero discards) before asserting anything about the discard case.

Exit codes

Unchanged from AUR-430/AUR-434:

Exit Meaning
0 Review completed (with or without surviving findings, with or without a discard).
1 Runtime failure: not a git repository, diff failure, no provider configured, unparseable model response, broken embedded rules catalog.
2 Usage error (unknown command, missing --base).

Discarding a finding, with or without the new stderr warning, never changes the exit code by itself — exactly like AUR-434's own gate. (The independent --fail-on gate, AUR-431, still evaluates the surviving findings; a discarded finding never participates in it, unchanged.)

Offline, secret-free example

cd tests/fixtures/repos/git-demo/repo.git
cat > /tmp/mixed.json <<'EOF'
{
  "issues": [
    {"file": "config/demo-tokens.txt", "line": 3, "severity": "error",
     "rule_id": "security/hardcoded-secret", "message": "grounded finding"},
    {"file": "config/demo-tokens.txt", "line": 4, "severity": "error",
     "message": "no rule_id at all"},
    {"file": "config/demo-tokens.txt", "line": 5, "severity": "warning",
     "rule_id": "security/definitely-not-a-rule", "message": "unknown rule_id"}
  ],
  "summary": "example for docs/specs/AUR-448.md"
}
EOF
AURUMCODE_LLM_FIXTURE=/tmp/mixed.json aurumcode review --base HEAD~1

stdout:

config/demo-tokens.txt:3: [error] grounded finding (rule security/hardcoded-secret: Hardcoded Secrets)

stderr:

aurumcode review: 2 finding(s) discarded: 1 with no rule_id, 1 citing an unknown rule_id (security/definitely-not-a-rule)

Diagnosing a failure

Symptom Likely cause
No issues found. with a stderr line starting aurumcode review: N finding(s) discarded: ... The model reported N finding(s) this run, and every one of them failed the rule gate (missing or unknown rule_id). This is the exact case that used to be indistinguishable from "the code is clean" — read the stderr line, then either fix the fixture/model output's rule_id or, for a real provider, check that it is following the review prompt's schema.
No issues found. with no stderr discard line The model genuinely reported zero findings this run. Nothing was discarded.
A finding you expected does not print, but stderr names it discarded Same cause as AUR-434's own table: the rule_id it cited is missing or not in the embedded catalog (internal/review/rules/*.yml).
no LLM provider configured: ... No AURUMCODE_LLM_FIXTURE, LLM_API_KEY+LLM_BASE_URL configured. The message shows the complete fixture shape, rule_id included, with a real catalog id as the example.

For everything else (repository backends, provider selection, parser fallback, diff limits, --modelo, --limite, --seguranca, --pr, --check, the review cache) see docs/specs/AUR-430.md and the other cards' specs it and this file cross-reference.

Security

  • result.Metadata["discard_warning"] is built entirely from data already redacted at the model boundary: enforceRuleCitations runs strictly after redactReviewResult in GenerateReview (see reviewer.go's own ordering note), so every RuleID the discard message can quote was already passed through the single AUR-009 redaction filter before this card's code ever reads it. The discard warning line also passes through cmd/aurumcode's errW (the AUR-432 redaction writer wrapping stderr) like every other canonical stderr line.
  • The AURUM_SECRET_CANARY value never appears in the discard warning: it is built only from counts and already-redacted rule ids, never from a finding's Message, File, or Suggestion.

A note on this card's own TDD-proof identifiers

The card's TDD proof section names the unit/integration/e2e selectors TestAUR435 / IntegrationAUR435 / E2EAUR435. tests/unit/AUR-435.go and tests/integration/AUR-435.go already declare functions with exactly those names, in the same unit/integration packages every card's bridge test shares — reusing them here would be a duplicate-symbol compile error, not a naming preference. This is a numeral copy artifact in the card text, not an instruction this card can satisfy literally without breaking the build. tests/unit/AUR-448.go, tests/integration/AUR-448.go, and tests/e2e/AUR-448.sh instead declare TestAUR448 / IntegrationAUR448 / E2EAUR448, following the same numeral-matches-file convention every sibling card in this office already uses (TestAUR434, TestAUR443, ...). The acceptance program (tests/acceptance/AUR-448.sh) still answers to the selectors AC-001, TestAUR448, IntegrationAUR448, E2EAUR448, and AC-001-MUT-001, and still proves exactly what the TDD proof section describes in prose (Unit/Integration/E2E layers plus the mutation).

Validation

ulimit -v 8388608
GOMEMLIMIT=2GiB GOFLAGS='-mod=mod -p=1' bash tests/acceptance/AUR-448.sh

./.board/bin/oci-run --profile bootstrap-readonly-v1 --card AUR-448 runs the same program inside the sealed, read-only, network-denied profile.