Continuous Verification Engine, Matrix Scorecard & Evidence Classification Subsystem Operational Architecture Guide
1. Executive Architectural Summary
The Continuous Verification Engine, Matrix Scorecard & Evidence Classification Subsystem (pkg/verification and pkg/verification/matrix) provides the machine-verifiable, objective validation foundation for the ZQK Knowledge Kernel.
In modern software delivery and autonomous multi-agent environments, quality claims frequently suffer from subjective developer optimism, transient test flakiness, and narrative inflation. Under core governance policy POL-DEFAULT-142277d3:
"Done means re-runnable predicate evidence; narrative is never a gate."
The continuous verification subsystem operationalizes this principle by providing an objective, blind validation harness that evaluates software artifacts against strict mechanical proofs rather than conversational assurances. The subsystem unifies four universal verification strategies alongside a content-addressed verification matrix that tracks file classifications, literal duplication inventories, and multi-dimensional diamond quality scorecards.
Core Architectural Principles
- Blind Verification Execution: Verification strategies evaluate code artifacts independently of the agent or developer claiming completion. Strategies evaluate commands, AST structures, database graph queries, and system state negations without contextual bias.
- Universal Verification DSL: Verification rules declare declarative execution strategies:
-
command_exit_code: Executes external shell commands and linters, verifying zero-exit status and capturing diagnostic output. -ast_semantic_match: Walks Go syntax trees and filesystem hierarchies to detect forbidden patterns, banned imports, or missing AST annotations. -query_metric: Executes Cypher-like pattern queries across the Knowledge Kernel graph, evaluating count and metric thresholds against relational boundaries. -state_negation: Verifies that invalid states (such as orphaned tasks or failed prerequisites) do not exist within persistent storage. - Content-Addressed Verification Matrix: Tracks evaluation records across file content SHA-256 hashes. If file contents remain unmodified, verification results are retrieved in \(\mathcal{O}(1)\) time without redundant test or linter execution.
- Global Literal Inventory & Deduplication: Extracts string literals repository-wide, enforcing deduplication policies (\(\le 1\) unique occurrence allowed globally for magic constants) to eliminate hardcoded logic.
- Multi-Dimensional Diamond Scoring: Evaluates findings across seven standard quality dimensions, mapping critical errors, warnings, and violations to an objective 1-to-5 diamond rating scale.
- Elevated Test Coverage Invariant: The subsystem enforces a strict statement test coverage invariant (\(\ge 85.0\%\), verified at \(93.9\%\) in
pkg/verificationand \(92.3\%\) inpkg/verification/matrix).
2. Component Topology & Subsystem Architecture
The subsystem consists of two primary packages working in tight coordination: pkg/verification (the core execution engine and universal DSL strategies) and pkg/verification/matrix (the content-addressed matrix, polyglot classifier, and scorecard evaluator):
flowchart TD
subgraph ExecutionCore["Continuous Verification Core (pkg/verification)"]
RunVerify["RunVerification()\nStrategy Dispatcher"]
CmdStrat["CommandExitCodeStrategy\n(CLI & Subprocess Runner)"]
ASTStrat["ASTSemanticMatchStrategy\n(AST & Token Matcher)"]
QueryStrat["QueryMetricStrategy\n(Graph Query Evaluator)"]
StateStrat["StateNegationStrategy\n(Negative Boundary Proof)"]
end
subgraph MatrixCore["Verification Matrix Subsystem (pkg/verification/matrix)"]
MatrixEngine["Matrix Engine\n(EvaluateFile / EvaluateAll / StampFileCheck)"]
Classifier["Evidence Classifier\n(Polyglot Profile & Glob Matcher)"]
CommandRunner["CommandCheckRunner\n(Tool Substitution & Timeout Guard)"]
Inventory["Literal Inventory\n(Deduplication & Location Tracker)"]
ScorecardMgr["Scorecard Evaluator\n(Threshold Evaluation & Diamond Scoring)"]
LedgerStore["Matrix Ledger\n(Content-Addressed SHA-256 Cache)"]
end
subgraph StorageLayer["Storage & Artifacts (.matrix/)"]
LedgerFile["verification_matrix.json"]
InventoryFile["literal_inventory.json"]
ScorecardsDir["scorecards/*.json"]
end
RunVerify --> CmdStrat
RunVerify --> ASTStrat
RunVerify --> QueryStrat
RunVerify --> StateStrat
MatrixEngine --> Classifier
MatrixEngine --> CommandRunner
MatrixEngine --> Inventory
MatrixEngine --> ScorecardMgr
MatrixEngine --> LedgerStore
LedgerStore --> LedgerFile
Inventory --> InventoryFile
ScorecardMgr --> ScorecardsDir
Subsystem Component Breakdown
| Component | Package | Primary Responsibility |
|---|---|---|
RunVerification |
pkg/verification |
Blind router dispatching verification steps to registered strategies. |
CommandExitCodeStrategy |
pkg/verification |
Executes subprocess commands with repository root resolution and environment propagation. |
ASTSemanticMatchStrategy |
pkg/verification |
Walks filesystem trees and evaluates Go source code against forbidden token patterns. |
QueryMetricStrategy |
pkg/verification |
Queries object storage via Cypher expressions, asserting metric count thresholds. |
StateNegationStrategy |
pkg/verification |
Queries storage to verify that zero objects exist in an invalid or failing state. |
Engine |
pkg/verification/matrix |
Coordinates file evaluation, check stamping, ledger persistence, and cache invalidation. |
Classifier |
pkg/verification/matrix |
Classifies source files into language and structural classes (go_prod, go_test, script, config, docs, other). |
CommandCheckRunner |
pkg/verification/matrix |
Executes external linters or analysis tools with file path substitutions and bounded timeouts. |
LiteralInventory |
pkg/verification/matrix |
Tracks string literals across the codebase, flagging duplication violations. |
FileScorecard |
pkg/verification/matrix |
Records findings, calculates diamond scores, formats markdown reports, and coordinates auto-minted remediations. |
3. Verification Execution Lifecycle
The verification execution lifecycle proceeds through six deterministic phases, ensuring complete isolation and reproducible evaluation:
sequenceDiagram
autonumber
participant Caller as Orchestrator / CI / Agent
participant Engine as Verification Engine
participant Classifier as Evidence Classifier
participant Runner as CommandCheckRunner
participant Inventory as Literal Inventory
participant Scorecard as Scorecard Evaluator
participant Ledger as Matrix Ledger
Caller->>Engine: EvaluateFile(ctx, relPath, force)
Engine->>Ledger: isCacheSatisfied(relPath, currentSHA256)
alt Cache Hit (Hash Unmodified & Clean)
Ledger-->>Engine: Cached FileEntry
Engine-->>Caller: Stored Results (O(1))
else Cache Miss / Invalidation
Engine->>Classifier: Classify(relPath)
Classifier-->>Engine: FileClass (e.g. go_prod)
Engine->>Runner: executeRequiredChecks(ctx, entry)
Runner-->>Engine: CheckResults & Findings
opt Hardcoded Logic Scan
Engine->>Inventory: Record(literal, path, line)
Inventory-->>Engine: count, isViolation
end
Engine->>Scorecard: EvaluateScorecard(ctx, scorecard, autoMint)
Scorecard-->>Engine: DiamondScore & PolicyEvaluation
Engine->>Ledger: UpdateEntry & SaveLedger()
Engine-->>Caller: Evaluated FileEntry & Scorecard
end
Lifecycle Stage Details
- Target Resolution & SHA-256 Hashing: The engine verifies that the requested target exists on disk and is a regular file. It computes the cryptographic SHA-256 content hash of the raw bytes.
- Ledger Cache Lookup (\(\mathcal{O}(1)\)): The engine checks the persistent
MatrixLedger. If the file hash matches the recorded hash and all required check evaluations remain valid, evaluation short-circuits with cached results. - Polyglot Classification: If execution is forced or content has changed, the
Classifierinspects the path against glob patterns and file extensions to categorize the file into its designatedFileClass. - Required Check Execution: The registered evaluators and command runners matching the file's class are executed concurrently with token budgets and execution deadlines.
- Literal Extraction & Deduplication: Discovered string literals are submitted to the
LiteralInventory. Any string literal occurring \(\ge 2\) times across the repository triggers a deduplication policy failure. - Scorecard Compilation & Remediation: Findings are evaluated against dimension thresholds. If violations occur and auto-remediation is active, a
RemediationBlockis minted and persisted alongside the scorecard.
4. Evidence Classification & Scorecard Taxonomy
File Class Hierarchy
The evidence classifier organizes repository paths into distinct structural categories to ensure appropriate validation suites are applied:
| File Class | Default Extensions | Default Match Patterns | Target Verification Dimensions |
|---|---|---|---|
ClassGoProd |
.go |
(non-test Go files) | HCODE, ERRHYG, EFFPERF, CONCURR, SECOBS |
ClassGoTest |
.go |
*_test.go, tests/**/*.go |
HCODE, ERRHYG, CONCURR |
ClassScript |
.sh, .bash, .zsh |
scripts/**, tools/** |
LINT, SECOBS |
ClassConfigFile |
.yaml, .yml, .json, .toml |
config/**, *.config.* |
LINT, SECOBS |
ClassDocs |
.md, .markdown, .rst |
docs/**, *.md |
DOCSIG |
ClassOther |
(all others) | (fallback) | LINT |
Evaluation Dimensions
The scorecard measures code quality across seven distinct engineering dimensions:
| Dimension Code | Name | Focus Area | Mandatory Policy Invariant |
|---|---|---|---|
HCODE |
Hardcoded Logic | Eliminates magic strings, embedded credentials, unmapped endpoints, and duplicate literals. | Max 1 occurrence per string literal globally. |
EFFPERF |
Efficiency & Performance | Identifies algorithmic anti-patterns, memory allocations in tight loops, and unbuffered streams. | Zero unbounded hot-path allocations. |
ERRHYG |
Error Hygiene | Prohibits swallowed errors (_ = ...), naked error returns, and missing context wraps. |
Zero unhandled or discarded error returns. |
CONCURR |
Concurrency Safety | Enforces named goroutines (goroutinelabels.NewGoroutine), context propagation, and race-free channels. |
Prohibits naked go func() invocations. |
SECOBS |
Security & Observability | Verifies structured logging, metric instrumentation, path traversal sentinels, and secret masking. | Zero plaintext secrets or unsanitized paths. |
DOCSIG |
Documentation Signature | Verifies public API documentation, architecture guide references, and payload purity. | Zero forbidden internal IDs in public docs. |
LINT |
Standard Linter & Hygiene | Language-specific syntax rules, static analysis, and code style compliance. | Clean static linter exit codes. |
Diamond Quality Scoring Scale
Findings are compiled into an objective 1-to-5 diamond quality rating:
◆◆◆◆◆ (5/5 Flawless) Zero findings across all dimensions.
◆◆◆◆◇ (4/5 Minor Polish) <= 2 warnings/medium findings; zero errors or criticals.
◆◆◆◇◇ (3/5 Remediation Needed) > 2 warnings; zero errors or criticals.
◆◆◇◇◇ (2/5 Failing) >= 1 error or high-severity finding.
◆◇◇◇◇ (1/5 Critical Failure) >= 1 critical-severity finding.
◇◇◇◇◇ (Unrated) File has not undergone evaluation.
5. Concurrency, Thread Safety & Subsystem Invariants
1. Named Goroutine Concurrency (goroutinelabels)
In accordance with system policy POL-CONCURRENCY, naked go func() statements are strictly prohibited across both production code and test suites. Concurrent verification tasks and test workers must use named goroutine wrappers:
goroutinelabels.NewGoroutine(
"verification_worker",
fmt.Sprintf("worker-%d", workerID),
).StartSimple(func() {
defer wg.Done()
res, err := RunVerification(ctx, secCtx, sp, step)
// process result...
})
2. Thread-Safe Subsystem Mutexes
Engine: Usessync.RWMutexto guard the in-memory ledger, active classifier, and registered remediation hooks during concurrent evaluations and stamping.LiteralInventory: Employssync.RWMutexaround literal maps and occurrence counts, ensuring race-free deduplication checks across parallel test execution.- Atomic File Writing: Persists ledger, inventory, and scorecard files using
fileutil.WriteSecureFileto prevent partial or corrupted writes.
3. Hermetic Directory & Cleanup Invariant
All test fixtures and matrix repositories must execute within hermetic directories allocated via t.TempDir(). Background workers and timers must be joined and halted prior to test completion to prevent directory cleanup races. Direct os.* filesystem calls in test code are prohibited; tests must use fileutil.*.
4. Statement Test Coverage Invariant
Under quality governance policy POL-DEFAULT-f746cccc03ce5694, the continuous verification subsystem maintains a static coverage floor strictly \(\ge 85.0\%\):
pkg/verification: 93.9% statement coverage.pkg/verification/matrix: 92.3% statement coverage.
6. Programmatic API & Integration Recipes
Initializing the Verification Matrix Engine
package main
import (
"context"
"fmt"
"log"
"github.com/zqk-os/zqk/pkg/verification/matrix"
)
func main() {
repoRoot := "."
storageDir := ".matrix"
registry := matrix.NewCheckRegistry()
engine, err := matrix.NewEngine(repoRoot, storageDir, registry)
if err != nil {
log.Fatalf("failed to initialize verification engine: %v", err)
}
// Evaluate all repository files
ctx := context.Background()
summary, err := engine.EvaluateAll(ctx, false)
if err != nil {
log.Fatalf("verification evaluation failed: %v", err)
}
fmt.Printf("Evaluation Complete: %d files evaluated (%d clean, %d failed, %d pending)\n",
summary.TotalFiles, summary.CleanFiles, summary.FailedFiles, summary.PendingFiles)
}
Executing a Universal Verification DSL Step
step := map[string]any{
"verification_strategy": "command_exit_code",
"command": "go test -v -timeout 30s ./pkg/verification/...",
"task_id": "TASK-VERIFY-001",
}
result, err := verification.RunVerification(ctx, secCtx, storageProvider, step)
if err != nil {
log.Fatalf("verification execution error: %v", err)
}
if !result.Passed {
log.Printf("Verification Failed: %s\n", result.Feedback)
} else {
log.Printf("Verification Passed: %s\n", result.Feedback)
}
Stamping an External Verification Result
req := matrix.StampRequest{
Path: "pkg/verification/engine.go",
Dimension: matrix.DimensionERRHYG,
ContentHash: "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
Evaluator: "agent-hygiene-auditor",
DiamondScore: matrix.ScoreFlawless,
Status: matrix.CheckStatusPassed,
Feedback: "Zero unhandled errors; all return values verified.",
}
result, err := engine.StampFileCheck(ctx, req)
if err != nil {
log.Fatalf("failed to stamp verification: %v", err)
}
7. Policy Compliance & Quality Assurance
| Governance Invariant | Implementation Mechanism | Verification Test Reference |
|---|---|---|
| POL-DEFAULT-142277d3 (Objective Evidence Done-Gate) | Four universal verification strategies evaluate artifacts blindly without chat narrative. | pkg/verification/verification_coverage_elevation_cycle237_test.go |
| POL-CONCURRENCY (Named Goroutine Concurrency) | All async tasks use goroutinelabels.NewGoroutine. Naked go statements prohibited. |
TestCycle237_Verification_ConcurrentExecution, TestCycle237_Matrix_ConcurrentStamping |
| POL-AST-HYGIENE (Zero Swallowed Errors) | All errors checked and asserted. Direct os.* prohibited in tests; fileutil.* enforced. |
Static lint and pre-commit check valves. |
| POL-DOCS-PUBLIC-PURITY (Zero Literal Nanosecond IDs) | Documentation uses symbolic identifiers. Checked via check-public-release-payload.sh. |
sh scripts/open-core/check-public-release-payload.sh . |
| POL-COVERAGE-FLOOR (\(\ge 85.0\%\) Statement Coverage) | Elevated unit and boundary test suites covering all edge cases, errors, and fallbacks. | ZQK_ALLOW_FOREGROUND_GO_TEST=1 go test -cover ./pkg/verification ./pkg/verification/matrix |