Pular para conteúdo

AUR-016: Definir uso evidencial de ISO 25010

Documento histórico do board. Pode descrever componentes removidos; para uso atual, consulte o guia do produto.

This is a research/vocabulary card. It does not ship an executable feature or a scoring engine; it fixes a vocabulary and an abstention rule that later quality-evidence cards implement against. The observable behavior is the acceptance lint itself: tests/acceptance/AUR-016.sh deterministically resolves the artifacts below and either passes or names exactly which one is incomplete.

Behavior

tests/acceptance/AUR-016.sh (selector AC-001) reads four artifacts this card owns and checks that they resolve to one consistent, evidence-honest vocabulary, without making any network request:

  1. .board/research/quality-standards.md — literally contains the string ISO/IEC 25010:2023, contains, case-insensitively, the phrase does not claim conformity, and carries a ## Sources examined table whose rows each cite a [label](url) source with an allowlisted host, a version token, and an ISO-8601 date.
  2. standards/quality/README.md — states the abstention rule in fixed terms: the literal status name not_assessed, and a sentence forbidding a score without a wired metric.
  3. standards/quality/characteristics.yaml — one entry for each of the nine ISO/IEC 25010:2023 characteristics, each carrying name, measurable, status, score, metric, and metric_source.
  4. tests/specs/AUR-016/characteristics.txt and tests/specs/AUR-016/allowed-source-domains.txt — the fixed nine characteristic names and the source-host allowlist the lint resolves against.

The lint parses characteristics.yaml structurally (no YAML library — the acceptance program runs alone inside a minimal, network-denied container) and enforces the abstention rule as data, not prose: every entry with measurable: false must have status: not_assessed and score: null; every entry with status: measured must have measurable: true, a non-null metric, a non-null metric_source, and a numeric score.

Contract

  • tests/specs/AUR-016/characteristics.txt and tests/specs/AUR-016/allowed-source-domains.txt are the single source of truth for, respectively, the exact nine characteristic names the standards table must resolve, and the hostnames a research-document Source link may cite. The shell acceptance test reads these two files instead of duplicating the lists.
  • A ## Sources examined row's Version cell must match Edition N, vN(.N)(.N), revYYYY, or post YYYY-MM-DD; its Date cell must be an ISO-8601 calendar date.
  • A characteristic entry may carry a numeric score only if it is measurable: true and cites both a metric and a metric_source. Any entry with measurable: false (equivalently status: not_assessed) that also carries a non-null score fails with the typed code AUR-016/AC-001/score-without-signal.
  • Naming a metric without a metric_source (or the reverse) fails with AUR-016/AC-001/malformed-entry.
  • A characteristic name outside the fixed nine, a duplicate, or a missing required characteristic all fail with a typed code (unknown-characteristic, duplicate-characteristic, or missing-characteristic) naming the offending entry.
  • This card produces no quality score for the AurumCode repository and no approval threshold. Every characteristic in the shipped characteristics.yaml is not_assessed today; wiring a real, sourced, computed metric for any one characteristic is left to a dependent card that changes only that characteristic's row.

Limits

There is no numeric ceiling in this card — its only "limit" is the structural invariant standards/quality/README.md states in prose and characteristics.yaml encodes in data: a characteristic without an observable, currently wired metric MUST report not_assessed, and a numeric score MUST NOT appear without one.

Example (no secret)

Running the acceptance test directly, with no network access and no secret material of any kind:

$ ./.board/bin/oci-run --profile bootstrap-readonly-v1 --card AUR-016
{"card":"AUR-016","scenario":"AC-001","selector":"AC-001","characteristics":9,"measured":0,"not_assessed":9,"result":"pass"}

A failing example — a characteristic entry assigned a numeric score with no wired metric behind it (MUT-001):

$ tests/acceptance/AUR-016.sh
AUR-016/AC-001/score-without-signal: Functional suitability: measurable is false but score is not null: 87

A failing example — a researched source replaced by a non-allowlisted host (MUT-002):

$ tests/acceptance/AUR-016.sh
AUR-016/AC-001/unversioned-source: source host is not allowlisted: evil.example.com

Diagnosis

  • AUR-016/AC-001/behavior-missing: required artifact absent: <path> — one of the research document, standards/quality/README.md, standards/quality/characteristics.yaml, or a tests/specs/AUR-016/* fixture is missing or empty. Restore it from the card's owned paths.
  • AUR-016/AC-001/behavior-missing: ... missing the literal version string ISO/IEC 25010:2023 or ... missing the case-insensitive phrase 'does not claim conformity' — the research document lost one of the two literal markers its own Given clause requires. Restore the exact string/phrase.
  • AUR-016/AC-001/unversioned-source: ... — a ## Sources examined row is missing a [label](url) link, cites a non-allowlisted host, has no parseable version token, or has a non-ISO-8601 date. Fix the offending cell.
  • AUR-016/AC-001/unknown-characteristic / duplicate-characteristic / missing-characteristic — the characteristics.yaml entry set no longer resolves 1:1 against tests/specs/AUR-016/characteristics.txt. Fix the entry name or the fixture.
  • AUR-016/AC-001/malformed-entry — measurable or status is not one of its two literal values, or metric/metric_source are inconsistently null. Fix the offending field.
  • AUR-016/AC-001/score-without-signal — a measurable: false (or status: not_assessed) entry carries a non-null score, or a status: measured entry is missing measurable: true, metric, metric_source, or a numeric score. This is the abstention-rule violation the card exists to prevent: remove the unsupported score, or wire a real metric and source before claiming measured.
  • AUR-016/AC-001/infrastructure: ... — a required host tool (awk/grep/sed/sort/wc) is missing or the repository root could not be resolved; this is never a behavioral result and never counts as RED evidence for this card.