Process Supervision, Engine Registry & Metrics Extractor Subsystem Architectural & Operational Guide
1. Executive Summary
The Process Supervision, Engine Registry & Metrics Extractor Subsystem (pkg/supervision) provides structured telemetry extraction, contrastive retry prompt formatting, negative hypothesis boundary synthesis, and canonical exemplar catalog resolution across the ZQK Knowledge Kernel.
Governed by core system policy POL-DEFAULT-142277d3, autonomous agent swarms and task runners must never engage in cyclical retry thrashing or blind, unguided iterations upon task failure. The subsystem transforms raw terminal failures (stderr/stdout streams, non-zero exit codes, panic dumps, and compiler errors) into actionable, localized telemetry wrapped in a Supervision Envelope. This envelope illuminates the failure delta, invalidates failed execution hypotheses, injects idiomatic design patterns, and constrains execution scope to narrow sub-gates before re-attempting full verification suites.
The package achieves 100.0% statement test coverage, strictly exceeding the mandatory 85.0% static floor invariant, and guarantees thread-safe, concurrent execution across high-throughput agent swarms.
2. Architectural Overview & Core Principles
Autonomous AI software engineering agents operating in large codebases often fail due to cyclical retry loopsβrepeatedly applying the same failing patch, ignoring root-cause compiler warnings, or thrashing against full test suites without localized reproduction.
The pkg/supervision subsystem solves this by implementing Adaptive Task Supervision structured around four complementary pillars:
+-----------------------------------------------------------------------------------+
| Failed Task Execution Event |
| (Raw stderr, stdout, exit code, phase, duration) |
+-----------------------------------------------------------------------------------+
|
v
+-----------------------------------------------------------------------------------+
| Telemetry & Metrics Extractor (pkg/supervision/extractor.go) |
| 1. ExtractFailureAnchor: Pinpoints file:line:col or test failure symbol |
| 2. ExtractFocusedDiagnostic: Strips build noise, preserves error context |
| 3. BuildFailureDiagnostic: Constructs structured FailureDiagnostic telemetry |
+-----------------------------------------------------------------------------------+
|
v
+-----------------------------------------------------------------------------------+
| Adaptive Supervision Envelope Construction (types.go) |
+-----------------------------------------------------------------------------------+
| | |
v v v
[Pillar 1: Diagnostic Delta] [Pillar 2: Negative Hypotheses] [Pillar 3: Exemplars]
- Failing Phase & Exit Code - Invalidated Approaches - Canonical Patterns
- Pinned Failure Anchor - Forbidden Action Guardrails - Anti-Pattern Rules
- High-Signal Diagnostic - Anti-Thrashing Constraints - Idiomatic Models
| | |
+--------------------------------+-----------------------------+
|
v
[Pillar 4: Progressive Scope Narrowing]
- Localized sub-gate verification command
- Narrowed iteration target & rationale
|
v
+-----------------------------------------------------------------------------------+
| Contrastive Retry Formatter (pkg/supervision/formatter.go) |
| FormatContrastiveRetryPrompt(basePrompt, envelope) |
+-----------------------------------------------------------------------------------+
|
v
+-----------------------------------------------------------------------------------+
| Illuminated Autonomous Agent Prompt |
| (Guiding targeted fix with clear constraints and reference patterns) |
+-----------------------------------------------------------------------------------+
The Four Pillars of Task Supervision
- Focused Diagnostic Delta: Filters verbose build logs to isolate critical error markers (
FAIL,Error,panic:,undefined:,cannot use), capturing adjacent context while pruning compilation banners and noise. - Negative Hypotheses (Invalidated Paths): Derives explicit forbidden actions from prior attempts, preventing the agent from repeating the same code modifications or strategies that triggered failures.
- Canonical Exemplars: Injects curated, category-specific code references from an in-memory catalog, showing both idiomatic implementations and common anti-patterns to avoid.
- Progressive Scope Narrowing: Directs the retry toward a tight, sub-second feedback loop (e.g., a single package test or file lint) before escalating to the full integration test suite.
3. Component Topology & Subsystem Architecture
The subsystem is organized into five modular functional components engineered for high concurrency, zero swallowed errors, and language-agnostic resilience:
flowchart TD
subgraph ExecutionLayer["Execution & Verification Layer"]
Runner["Task / Test Runner"]
end
subgraph SupervisionSubsystem["Process Supervision Subsystem (pkg/supervision)"]
Runner -->|"Failure Output (stderr/stdout)"| Extractor["Telemetry Extractor (extractor.go)"]
Extractor --> ExtAnchor["ExtractFailureAnchor()"]
Extractor --> ExtSignal["ExtractFocusedDiagnostic()"]
Extractor --> BuildDiag["BuildFailureDiagnostic()"]
BuildDiag --> DiagObj["FailureDiagnostic Telemetry"]
DiagObj --> Synthesizer["Hypothesis Synthesizer"]
Synthesizer --> NegHypo["SynthesizeNegativeHypothesis()"]
CatalogReq["Category Request"] --> Registry["Exemplar Registry (registry.go)"]
Registry --> ResolveFn["Resolve(categories...)"]
DiagObj --> Envelope["SupervisionEnvelope Container (types.go)"]
NegHypo --> Envelope
ResolveFn --> Envelope
Envelope --> Formatter["Contrastive Formatter (formatter.go)"]
Formatter --> PromptOut["FormatContrastiveRetryPrompt()"]
end
PromptOut --> AgentSession["Autonomous Agent Retry Session"]
Component Manifest
| Component | Source File | Core Responsibility |
|---|---|---|
FailureDiagnostic |
types.go |
Encapsulates structured attempt telemetry, exit codes, failure phase, primary source anchor, raw streams, and filtered diagnostics. |
SupervisionPolicy |
types.go |
Configures attempt limits, hypothesis pruning toggles, exemplar injection flags, progressive scope thresholds, and watchdog timeouts. |
ExemplarRegistry |
registry.go |
Thread-safe in-memory catalog mapping categories to CanonicalExemplar records. Provides seed defaults, case-insensitive resolution, and deduplication. |
Extractor Engine |
extractor.go |
High-efficiency regex parsing engine that pinpoints source anchors and extracts high-signal diagnostics from noisy compiler outputs. |
Contrastive Formatter |
formatter.go |
Assembles formatted markdown prompts illuminating failure deltas, forbidden actions, exemplars, and sub-gate execution targets. |
4. Supervision Lifecycle & Engine Lookup Sequence
The end-to-end lifecycle from task failure detection to retry prompt formulation is detailed in the sequence below:
sequenceDiagram
autonumber
actor Agent as Autonomous Agent / Worker
participant Runner as Execution Runner
participant Extractor as Telemetry Extractor
participant Registry as Exemplar Registry
participant Formatter as Contrastive Formatter
Agent->>Runner: Execute Task Command (e.g. Test / Build)
Runner-->>Agent: Task Failure (Exit Code != 0, Stderr / Stdout)
Agent->>Extractor: ExtractFailureAnchor(stderr, stdout)
Extractor-->>Agent: Return Failure Anchor (e.g., "pkg/core/handler.go:42:15")
Agent->>Extractor: ExtractFocusedDiagnostic(stderr, stdout, exitCode, maxLines)
Extractor-->>Agent: Return Filtered Error Excerpt (Noise Removed)
Agent->>Extractor: BuildFailureDiagnostic(attempt, exitCode, phase, stderr, stdout, anomalyDesc)
Extractor-->>Agent: Return FailureDiagnostic
Agent->>Extractor: SynthesizeNegativeHypothesis(attempt, diag, triedApproach)
Extractor-->>Agent: Return NegativeHypothesis with ForbiddenActions
Agent->>Registry: Resolve("testing", "error_handling")
Registry-->>Agent: Return Deduplicated Canonical Exemplars
Agent->>Formatter: FormatContrastiveRetryPrompt(basePrompt, envelope)
Formatter-->>Agent: Return Illuminated Markdown Retry Prompt
Agent->>Runner: Execute Narrowed Sub-Gate Command (Progressive Scope)
Runner-->>Agent: Verification Succeeded
5. State Transitions & Failing Phase Taxonomy
Every execution failure is classified into a deterministic FailingPhase. This prevents downstream components from applying compile-time remedies to runtime timeouts or verification discrepancies:
stateDiagram-v2
[*] --> PhaseUnknown: Initial Attempt
state ExecutionLifecycle {
[*] --> PhaseCompile: Compiler / Build Error
[*] --> PhaseExecution: Non-Zero Exit / Crash
[*] --> PhaseTimeout: Watchdog Duration Exceeded
[*] --> PhaseVerification: Unit / Integration Test Failure
[*] --> PhaseValidation: Exit Code 0 with Semantic Invariant Failure
}
PhaseCompile --> Resolved: Syntax / Symbol Fixed
PhaseExecution --> Resolved: Runtime Exception Resolved
PhaseTimeout --> Resolved: Algorithm / Resource Optimized
PhaseVerification --> Resolved: Test Assertions Satisfied
PhaseValidation --> Resolved: Domain Policy Invariant Passed
Resolved --> [*]
Phase Definitions
PhaseCompile: The build or compilation step failed due to syntax errors, missing symbols, or mismatched types.PhaseExecution: The target process crashed, exited abnormally, or encountered an unhandled panic during active runtime.PhaseTimeout: Execution exceeded the configuredExecutionWatchdogGracewindow, requiring process group termination.PhaseVerification: Automated test runners reported test case assertion failures (--- FAIL:).PhaseValidation: The process terminated with exit code 0, but domain contracts or post-execution verification checks failed.PhaseUnknown: Unclassified failure category defaulting to generic telemetry analysis.
6. Concurrency & Thread-Safety Guarantees
The pkg/supervision subsystem is engineered to support high-throughput, multi-agent swarms operating concurrently:
- Read/Write Mutex Protection: The
ExemplarRegistrysynchronizes all catalog modifications and lookup operations using async.RWMutex. Multiple reader goroutines can resolve patterns simultaneously without contention, while registration locks the catalog exclusively. - Immutable Telemetry Instances: Once constructed by
BuildFailureDiagnostic,FailureDiagnosticandSupervisionEnvelopestructs are passed by value or treated as read-only references, eliminating race conditions across worker boundaries. - Goroutine Labeling Compliance: All background routines and concurrent test workers utilize
pkg/goroutinelabels(goroutinelabels.NewGoroutine) to preserve runtime traceability and prevent resource leaks. - Deterministic Resolution: Category queries in
Resolve()normalize inputs by trimming whitespace and folding case (strings.ToLower), ensuring consistent pattern resolution across diverse agent hosts.
7. Operational Recipes & Implementation Walkthrough
Recipe 1: Constructing a Supervision Envelope on Verification Failure
package main
import (
"context"
"fmt"
"time"
"github.com/zqk-os/zqk/pkg/supervision"
)
func SuperviseFailedTask(ctx context.Context, taskID string, attempt int, stderr, stdout string, exitCode int) string {
// 1. Build diagnostic telemetry
diag := supervision.BuildFailureDiagnostic(
attempt,
exitCode,
supervision.PhaseVerification,
stderr,
stdout,
"Test assertion failed during contract verification",
)
// 2. Synthesize anti-thrashing negative boundaries
negativeHypothesis := supervision.SynthesizeNegativeHypothesis(
attempt,
&diag,
"Inlined mock data directly in production handler",
)
// 3. Resolve relevant canonical exemplars
exemplars := supervision.DefaultExemplarRegistry().Resolve("testing", "error_handling")
// 4. Assemble the supervision envelope
envelope := supervision.SupervisionEnvelope{
TaskID: taskID,
Attempt: attempt + 1,
MaxAttempts: 3,
Diagnostic: &diag,
NegativeBoundaries: []supervision.NegativeHypothesis{negativeHypothesis},
Exemplars: exemplars,
ProgressiveScope: &supervision.ProgressiveScope{
SuggestedSubGateCommand: "go test -v -timeout 10s ./pkg/example/...",
FocusedTarget: diag.FailureAnchor,
IterationRationale: "Verify targeted package test before running whole-repo integration suite.",
},
}
// 5. Generate illuminated retry prompt
basePrompt := "# Task: Refactor Service Invariants"
return supervision.FormatContrastiveRetryPrompt(basePrompt, envelope)
}
Recipe 2: Registering Custom Exemplars
package main
import "github.com/zqk-os/zqk/pkg/supervision"
func RegisterCustomPatterns(registry *supervision.ExemplarRegistry) {
registry.Register("resilience", supervision.CanonicalExemplar{
Category: "resilience",
Title: "Exponential Backoff with Jitter",
Description: "Avoid thundering herd retry storms by applying jitter to exponential delays.",
PatternCode: "backoff := time.Duration(1<<attempt)*time.Millisecond + randJitter()",
AntiPatternCode: "time.Sleep(5 * time.Second) // Static unjittered delay",
})
}
8. Verification & Coverage Invariants
In accordance with kernel quality standards and POL-DEFAULT-142277d3, pkg/supervision adheres to the following verification invariants:
- Static Statement Coverage Floor: Minimum 85.0% statement coverage required across all routines.
- Achieved Coverage: 100.0% statement test coverage (
go test -cover ./pkg/supervision). - Zero Swallowed Errors: All error returns and diagnostic extractions are explicitly validated without blank assignments (
_ = ...). - Hermetic Unit Testing: All test suites run with strict 30-second timeouts (
-timeout 30s) and require zero external network or filesystem dependencies.
| Verification Target | Test Suite | Proof Type | Coverage |
|---|---|---|---|
| Source Anchor Extraction | supervision_coverage_elevation_cycle215_test.go |
Regex matching across Go, Rust, Python, and Windows formats | 100.0% |
| Focused Diagnostic Filtering | supervision_coverage_elevation_cycle215_test.go |
Signal marker detection, noise pruning, and tail fallback | 100.0% |
| Negative Hypothesis Synthesis | supervision_coverage_elevation_cycle215_test.go |
Approach preservation, exit code guardrails, and anchor checks | 100.0% |
| Exemplar Registry Operations | supervision_coverage_elevation_cycle215_test.go |
Case-folding, deduplication, and concurrent reader/writer safety | 100.0% |
| Contrastive Prompt Formatter | supervision_coverage_elevation_cycle215_test.go |
Complete vs minimal envelope formatting and markdown structure | 100.0% |
9. Related Documentation & Subsystem Links
- Diagnostics, Auto-Remedies & Self-Healing Guide: 4-layer health checks and automated system remediation.
- Error Formatting, Actionable Diagnostics & Poison Pill Protection Guide: Structured error formatting and poison pill defense.
- Goroutine Identity & Concurrency Labels Subsystem Guide: Profiler labeling and worker pool governance.
- Task Decision Envelope & Ingestion Subsystem Guide: Boundary contracts for autonomous agent decisions.