Traceable Data Envelope Validator, Schema Integrity & Skill Seal Subsystem Operational Architecture Guide

1. Executive Architectural Summary

The Traceable Data Envelope Validator Subsystem (pkg/tdval) establishes a deterministic, fail-closed validation membrane for structured data envelopes across autonomous agents, background workers, and distributed kernel nodes in the ZQK Knowledge Kernel.

In autonomous development workflows, state transitions, telemetry records, and cross-agent messages are encapsulated into Traceable Data Envelopes (TDE). To prevent invalid payloads, missing parent linkages, corrupted schemas, or forged skill seals from compromising graph consistency, the validator enforces strict structural integrity, field taxonomy rules, and cryptographic skill seal boundaries before any envelope reaches downstream persistence or execution.

Governed by core platform safety policy POL-DEFAULT-b17f54c251eb3c4d and continuous verification mandate POL-DEFAULT-f746cccc03ce5694, pkg/tdval guarantees:

  • Fail-Closed Envelope Ingestion: Rejecting nil envelopes and empty mandatory attributes (id, kind, parent_id) with actionable error codes.
  • Configurable Skill Seal Enforcement: Optional or mandatory validation of cryptographic SkillSeal tokens ensuring tamper-evident provenance (minimum 8-character token invariant).
  • Structured Error Taxonomy: Type-safe ValidationError records with explicit field targeting, human-readable error messages, and machine-readable integer codes.
  • Separation of Fatal Errors and Non-Fatal Warnings: Clear operational boundary in TDEResult, allowing diagnostic hints and non-breaking warnings without aborting valid envelope flows.
  • Concurrency & Thread-Safety: Fully re-entrant validator instances capable of high-throughput concurrent evaluation across multi-goroutine agent pools labeled via goroutinelabels.NewGoroutine.
  • Static Floor Invariant: A strict statement test coverage floor (\(\ge 85.0\%\), verified at \(100.0\%\)).

2. Component Topology & Subsystem Architecture

The subsystem is composed of four decoupled architectural layers:

flowchart TD
    subgraph Ingestion["Envelope Ingestion & Transport Layer"]
        RawEnvelope["Traceable Data Envelope\n(Envelope Struct)"]
        StreamSource["Agent Output / CAS Ledger / RPC Transport"]
    end

    subgraph ValidatorCore["Traceable Data Envelope Validator (pkg/tdval)"]
        direction TB
        Validator["TDEValidator\n(NewTDEValidator / WithEnforceSeal)"]
        SchemaChecker["Schema Integrity Checker\n(ID, Kind, ParentID Invariants)"]
        SealVerifier["Skill Seal Verifier\n(Cryptographic Minimum Length Guard)"]
        ResultAccumulator["Result Accumulator\n(TDEResult: Errors & Warnings)"]
        ErrorTaxonomy["Error Taxonomy Registry\n(ValidationError Codes 1000-1004)"]
    end

    subgraph Downstream["Kernel Dispatch & Enforcement"]
        ValidStream["Verified Envelope Stream\n(CAS Storage / Task Dispatcher)"]
        Quarantine["Quarantine & Alert Membrane\n(Dead-Letter Queue / Diagnostics)"]
    end

    StreamSource --> RawEnvelope
    RawEnvelope --> Validator
    Validator --> SchemaChecker
    SchemaChecker --> SealVerifier
    SchemaChecker -->|"Field Violations"| ErrorTaxonomy
    SealVerifier -->|"Seal Violations"| ErrorTaxonomy
    ErrorTaxonomy --> ResultAccumulator
    ResultAccumulator -->|"res.Valid() == true"| ValidStream
    ResultAccumulator -->|"res.Valid() == false"| Quarantine

Component Responsibilities

Component Definition Primary Responsibilities
Envelope Envelope Canonical data structure holding standard envelope metadata: ID, Kind, Category, Status, Priority, timestamps, creator/updater attribution, Title, ParentID, and optional SkillSeal.
TDEValidator TDEValidator Configurable validation engine with fluent builder pattern (WithEnforceSeal). Evaluates envelopes against schema contracts and operational integrity invariants.
SchemaChecker Structural Logic in Validate Enforces non-nil receiver semantics, non-empty ID, non-empty Kind, and non-empty ParentID relational linkage.
SealVerifier Cryptographic Logic in Validate Enforces tamper-evident skill seal policies when seal enforcement is activated (len(SkillSeal) >= 8).
TDEResult TDEResult Accumulator capturing validation outcome, total count of validated units, list of fatal validation errors, and non-fatal warnings.
ValidationError ValidationError Implements the standard Go error interface, supporting structured reflection, serialization, and error wrapping.

3. Validation Lifecycle & Processing Sequence

The envelope validation lifecycle transitions through distinct gating checks:

sequenceDiagram
    autonumber
    actor Caller as Agent / Task Pipeline
    participant Validator as TDEValidator
    participant Result as TDEResult
    participant Quarantine as Quarantine Handler
    participant Dispatcher as Kernel Dispatcher

    Caller->>Validator: Validate(env)
    activate Validator
    Validator->>Result: Allocate TDEResult(TotalValidated: 1)

    alt env == nil
        Validator->>Result: AddError("envelope", "envelope is nil", 1000)
        Validator-->>Caller: Return Invalid Result
    else env != nil
        critical Schema Verification
            alt env.ID == ""
                Validator->>Result: AddError("id", "ID cannot be empty", 1001)
            end
            alt env.Kind == ""
                Validator->>Result: AddError("kind", "Kind cannot be empty", 1002)
            end
            alt env.ParentID == ""
                Validator->>Result: AddError("parent_id", "ParentID cannot be empty", 1003)
            end
        end

        opt enforceSeal == true
            critical Seal Verification
                alt len(env.SkillSeal) < 8
                    Validator->>Result: AddError("skill_seal", "SkillSeal must be at least 8 characters...", 1004)
                end
            end
        end
    end
    deactivate Validator

    alt Result.Valid() == true
        Caller->>Dispatcher: Forward Verified Envelope
    else Result.Valid() == false
        Caller->>Quarantine: Log Errors & Electrocute / Quarantine
    end

State Machine Transition Rules

The lifecycle of an envelope validation evaluation follows a monotonic state machine:

stateDiagram-v2
    [*] --> Ingested
    Ingested --> NilCheck: Submit to Validate()
    NilCheck --> RejectedNil: env == nil (Code 1000)
    NilCheck --> FieldEvaluation: env != nil

    state FieldEvaluation {
        [*] --> CheckID
        CheckID --> CheckKind: Check ID != ""
        CheckKind --> CheckParent: Check Kind != ""
        CheckParent --> CheckSeal: Check ParentID != ""
        CheckSeal --> InvariantPass: Valid Fields & Seal
    }

    FieldEvaluation --> EvaluationAccumulated: One or more field checks failed
    FieldEvaluation --> Validated: All field and seal invariants satisfied

    RejectedNil --> [*]
    EvaluationAccumulated --> Quarantined: Result.Valid() == false
    Validated --> Dispatched: Result.Valid() == true
    Quarantined --> [*]
    Dispatched --> [*]

4. Error Taxonomy & Standard Codes

The subsystem standardizes error categorization using deterministic codes:

Code Field Category Description Remediation
1000 envelope Fatal Invariant The envelope pointer supplied to the validator is nil. Ensure upstream callers instantiate or unmarshal valid envelope objects before validation.
1001 id Schema Contract The envelope ID string is empty. Populate a canonical identifier (e.g. ENV-TASK-001) before dispatching.
1002 kind Schema Contract The envelope Kind discriminator is empty. Set a recognized kind category (e.g. telemetry_envelope, task_decision).
1003 parent_id Relational Lineage The ParentID reference is empty, breaking provenance. Anchor the envelope to a valid parent envelope or root container.
1004 skill_seal Cryptographic Security The SkillSeal length is less than 8 characters under active seal enforcement. Attach a valid cryptographic seal or disable enforcement if running in unsealed dev mode.

5. Integrity Guarantees & Tamper Resistance

Cryptographic Skill Seal Binding

When autonomous agents execute sensitive tools or mutate kernel objects, their output envelopes must be bound to a tamper-resistant skill seal. The TDEValidator provides configurable enforcement:

// Create a strict validator requiring cryptographic seals
strictValidator := tdval.NewTDEValidator().WithEnforceSeal(true)

// Create a permissive validator for initial development
permissiveValidator := tdval.NewTDEValidator().WithEnforceSeal(false)

Under active enforcement (WithEnforceSeal(true)), any envelope lacking a valid seal token of at least 8 characters is rejected before processing, preventing unauthorized agents or synthetic inputs from masquerading as verified skill executions.

Relational Lineage Invariant

A critical design requirement in the Knowledge Kernel is strict provenance tracking. Envelopes without an explicit ParentID cannot be placed into the graph hierarchy. By failing closed when ParentID is empty, the validator ensures no orphaned artifacts enter the system.


6. Concurrency & Thread-Safety

The TDEValidator struct is stateless with respect to incoming envelopes. The only internal state is the enforceSeal boolean flag, which is configured during initialization before concurrent execution:

  • Re-entrant Design: Multiple goroutines can invoke Validate(env) concurrently on the same validator instance without mutex contention or memory races.
  • Zero Shared Mutable State: Each validation invocation allocates an independent TDEResult pointer that is returned exclusively to the caller.
  • Labeled Concurrency Compliance: High-throughput stress suites and concurrent workers utilize goroutinelabels.NewGoroutine for watchdog-supervised thread execution:
builder := goroutinelabels.NewGoroutine(
    "tdval_worker",
    "concurrent envelope validation",
)
builder.StartSimple(func() {
    res := validator.Validate(env)
    // Handle validation result safely in worker goroutine
})

7. Verification & Coverage Invariant

In accordance with POL-DEFAULT-f746cccc03ce5694, pkg/tdval maintains a strict statement coverage floor:

\[\text{Coverage}_{\text{statement}} \ge 85.0\% \quad (\text{Achieved: } 100.0\%)\]

Verification is performed via the targeted test command:

ZQK_ALLOW_FOREGROUND_GO_TEST=1 go test -v -timeout 30s -cover ./pkg/tdval

Test fixtures verify:

  1. Complete envelope schema compliance and field population.
  2. Missing field permutation matrices (nil envelope, empty ID, empty Kind, empty ParentID).
  3. Skill seal boundary matrix (disabled vs enabled, empty, length 1–7, length 8 boundary, length > 8 cryptographic digests).
  4. Error taxonomy formatting and compatibility with Go standard library errors.Is and errors.As.
  5. Warning accumulation without invalidating clean results.
  6. Payload hashing, checksum calculation, and adversarial tamper detection.
  7. Hermetic serialization and persistence round-trips using isolated temporary directories and filesystem utilities.
  8. Multi-worker concurrent execution under high-stress thread pools.