TPM Strategic Readiness, Three-Fold Phi Metrics & Gantt Matrix View Subsystem Operational Architecture Guide (pkg/tpm)
1. Architectural Overview & Design Philosophy
The TPM Strategic Readiness, Three-Fold Phi Metrics & Gantt Matrix View Subsystem (pkg/tpm) provides the core strategic intelligence and telemetry engine governing technical program execution within the ZQK Knowledge Kernel.
Governed by core platform policies (POL-DEFAULT-142277d3 for strategic alignment invariants and POL-AGENT-TPM-TRACE-PIPELINE-001 for continuous traceability verification), the subsystem bridges high-level requirements with real-time runtime execution. In high-velocity autonomous swarm environments, engineering backlogs frequently suffer from three classic failure modes:
- Unspecified Requirements: Requirements exist without formal acceptance criteria, leading to hallucinated or divergent implementations.
- Untested Assertions: Requirements declare criteria but lack automated test cases, creating phantom progress without empirical verification.
- Unfueled Initiatives: Fully specified and tested requirements languish because they have not been attached to active execution lanes or discrete backlog items in the active priority plan.
To eliminate these vulnerabilities, pkg/tpm implements a real-time reactive graph accumulator that continuously evaluates the Three-Fold Phi Metrics (\(\Phi_{\text{req}}\), \(\Phi_{\text{trace}}\), \(\Phi_{\text{fuel}}\)), groups requirements into architectural domain clusters, computes execution runway depth, and triggers autonomous runway replenishment when active execution capacity drops below minimum thresholds.
flowchart TD
KernelStore["Knowledge Kernel Graph<br/>(Requirements, Criteria, Test Cases, Plans, BLIs)"] -->|Scan / WAL Stream| SRV["StrategicReadinessView<br/>(pkg/tpm)"]
SRV --> EvalPhi["Three-Fold Phi Metric Engine<br/>(Phi_req, Phi_trace, Phi_fuel)"]
EvalPhi --> Classify["Readiness Node Classification<br/>(Shovel-Ready vs. Fully-Aligned)"]
Classify --> Cluster["Domain Clustering & Aggregation<br/>(Storage, Scheduler, Mesh, Governance, etc.)"]
Cluster --> Runway["Runway Depth Watermark Check<br/>(Active / In-Progress Plans)"]
Runway -->|Runway Depleted: Depth <= 1| CandidateGen["Autonomous Synthesis Candidate Generator"]
Runway -->|Runway Healthy: Depth > 1| Persist["Zero-Hot-Path Lite File Materialization<br/>(strategic_readiness_lite.json)"]
CandidateGen --> Persist
Persist --> CLI["CLI & MCP Telemetry Queries<br/>(Sub-5ms SLA)"]
2. Component Topology
The subsystem is organized around a set of cooperating structures maintaining in-memory graph state, persisting serialized snapshots, and reacting to Write-Ahead Log (WAL) lifecycle transitions:
classDiagram
class StrategicReadinessView {
-mu sync.RWMutex
-projectRoot string
-requirements map~string, RequirementReadinessNode~
-critToReqs map~string, []string~
-critToTests map~string, []string~
-reqToBLIs map~string, []string~
-bliToPlan map~string, string~
-planStatuses map~string, string~
-domainClusters map~string, DomainCluster~
-synthesisCandidates []SynthesisCandidate
-recentEvents []StrategicLifecycleEventSummary
-lastUpdated time.Time
-engine Engine
+ApplyEvent(ev) bool
+BuildPayload() StrategicReadinessLitePayload
+ScanFromStorage(ctx, sp) error
+ScanFromStorageWithSecurity(ctx, sp, secCtx) error
+SaveToLiteFile() error
+LoadFromLiteFile() (bool, error)
+CheckRunwayReplenishment() (bool, []SynthesisCandidate)
+GetSnapshot() StrategicReadinessLitePayload
}
class RequirementReadinessNode {
+ID string
+Title string
+Domain string
+Status string
+CriteriaRefs []string
+HasCriteria bool
+HasAttestationSuites bool
+HasExecutionLane bool
+PriorityPlanRef string
+BacklogItemRef string
+IsShovelReady bool
+IsFullyAligned bool
+MissingElements []string
}
class DomainCluster {
+Domain string
+TotalRequirements int
+AlignedRequirements int
+UnmappedRequirements int
+Requirements []RequirementReadinessNode
}
class SynthesisCandidate {
+Domain string
+RequirementIDs []string
+Title string
+EstimatedBLIs int
+SuggestedPlanTitle string
}
class StrategicReadinessLitePayload {
+SchemaVersion string
+MaterializedAt time.Time
+RunwayDepth int
+IsRunwayDepleted bool
+TotalActiveRequirements int
+AlignedRequirements int
+GapRequirementsCount int
+DomainClusters map~string, DomainCluster~
+SynthesisCandidates []SynthesisCandidate
+Requirements map~string, RequirementReadinessNode~
+RecentEvents []StrategicLifecycleEventSummary
+PlanStatuses map~string, string~
}
StrategicReadinessView *-- RequirementReadinessNode
StrategicReadinessView *-- DomainCluster
StrategicReadinessView *-- SynthesisCandidate
StrategicReadinessView --> StrategicReadinessLitePayload
Key Topology Elements:
StrategicReadinessView: Central coordinator that satisfies the genericaccumulator.Accumulator[*StrategicReadinessLitePayload]interface. It encapsulates an adjacency index between requirements, criteria, test cases, backlog items, and priority plans.RequirementReadinessNode: Normalized graph node tracking the static completeness, traceability state, execution fuel status, and missing element diagnostics for an individual requirement.DomainCluster: Subsystem partition aggregating requirements by domain (e.g.,storage,scheduler,mesh_agent,tpm_orchestration,governance,performance,qa_traceability, and fallbackcore_kernel).SynthesisCandidate: High-value autonomous planning bundle derived when execution runway is depleted, clustering unmapped requirements in a domain to seed the next priority plan cycle.StrategicReadinessLitePayload: The zero-cost materialized state persisted to disk at.zqk/state/strategic_readiness_lite.json.
3. Strategic Readiness & Materialization Lifecycle
The subsystem operates under the Zero Hot-Path Scan Invariant: read queries must resolve in sub-5ms from disk without querying the storage layer. Background synchronization occurs incrementally through Write-Ahead Log (WAL) events and periodic reconciliation.
sequenceDiagram
autonumber
actor Operator as CLI / Agent / MCP
participant View as StrategicReadinessView
participant Engine as Accumulator Engine
participant Disk as State File (.zqk/state/strategic_readiness_lite.json)
participant Storage as Kernel Object Storage (CAS)
participant WAL as Lifecycle WAL (lifecycle_events.wal)
Note over Operator, Disk: Fast-Path Read Flow (Hot Path)
Operator->>View: LoadFromLiteFile()
View->>Engine: LoadFromLiteFile()
Engine->>Disk: Read atomic JSON payload
Disk-->>Engine: Payload Envelope (fresh)
Engine-->>View: Hydrate in-memory graph
View-->>Operator: Return true (Ready in <5ms)
Note over Operator, Storage: Cold Boot or Stale Cache Flow
Operator->>View: EvaluateAndReplenishRunway(ctx, sp, root)
View->>Disk: Check freshness (mtime > 5min or empty)
Disk-->>View: Cache Miss / Stale
View->>Storage: ScanFromStorageWithSecurity(ctx, sp, secCtx)
Storage-->>View: QueryResult (Plans, BLIs, TCs, Reqs)
View->>View: evaluateAllPredicatesLocked()
View->>View: clusterAndSynthesizeLocked()
View->>Engine: SaveToLiteFile()
Engine->>Disk: Write tmp file & atomic swap
View-->>Operator: Return Snapshot + Synthesis Candidates
Note over WAL, View: Real-Time Incremental Event Flow
WAL->>View: ApplyLifecycleEvent(ev)
View->>View: Update affected nodes & appendRecentEventLocked()
View->>View: clusterAndSynthesizeLocked()
View->>Engine: SaveToLiteFile()
Engine->>Disk: Atomic write to disk
4. Three-Fold Phi Metrics Formulation
The mathematical and operational foundation of the subsystem rests on the Three-Fold Phi Metrics, evaluating requirement readiness across static, dynamic, and execution planes:
graph TD
subgraph "Three-Fold Phi Proof Plane"
PhiReq["Phi_req (Specification Completeness)<br/>len(CriteriaRefs) > 0 AND status == active"]
PhiTrace["Phi_trace (Traceability Proof)<br/>FORALL c in CriteriaRefs: len(critToTests[c]) > 0"]
PhiFuel["Phi_fuel (Execution Fuel)<br/>EXISTS bli -> plan: plan.status in {active, in_progress, shovel_ready, grooming}"]
end
PhiReq --> ShovelReady{"Shovel-Ready?<br/>Phi_req AND Phi_trace"}
PhiTrace --> ShovelReady
ShovelReady -->|No| BacklogGrooming["Backlog Grooming Required<br/>(missing_criteria / missing_test_attestation)"]
ShovelReady -->|Yes| FullyAligned{"Fully Aligned?<br/>ShovelReady AND Phi_fuel"}
PhiFuel --> FullyAligned
FullyAligned -->|No| ReadyForStaging["Ready for Swarm Staging<br/>(unmapped_execution_lane)"]
FullyAligned -->|Yes| ActiveInFlight["Active In-Flight Execution<br/>(Fully Fueled & Verified)"]
Formal Metric Definitions:
-
Specification Completeness (\(\Phi_{\text{req}}\)):
\[\Phi_{\text{req}}(r) = (\text{status}(r) = \text{active}) \land (|C_r| > 0)\]where \(C_r\) is the set of criterion references attached to requirement $r$. An active requirement without defined criteria fails \(\Phi_{\text{req}}\) and is tagged withmissing_criteria. -
Traceability Proof (\(\Phi_{\text{trace}}\)):
\[\Phi_{\text{trace}}(r) = (|C_r| > 0) \land \left(\forall c \in C_r, \; |T_c| > 0\right)\]where \(T_c\) is the set of test cases verifying criterion $c$. If even a single criterion lacks automated verification, the requirement fails \(\Phi_{\text{trace}}\) and is tagged withmissing_test_attestation. -
Execution Fuel (\(\Phi_{\text{fuel}}\)):
\[\Phi_{\text{fuel}}(r) = \exists b \in B_r, \; \exists p = \text{plan}(b) \;\text{s.t.}\; \text{status}(p) \in \{\text{active}, \text{in\_progress}, \text{shovel\_ready}, \text{grooming}\}\]where \(B_r\) are backlog items referencing requirement $r$. Requirements with complete specs and tests but without an active execution container fail \(\Phi_{\text{fuel}}\) and are tagged withunmapped_execution_lane.
Compound States:
- Shovel-Ready: Requirement satisfies both \(\Phi_{\text{req}}\) and \(\Phi_{\text{trace}}\). It is fully specified and attested, ready to be scheduled.
- Fully-Aligned: Requirement satisfies all three: \(\Phi_{\text{req}} \land \Phi_{\text{trace}} \land \Phi_{\text{fuel}}\). It is actively moving forward in an execution cycle.
5. Runway Depth & Autonomous Replenishment
Execution capacity is quantified by Runway Depth (\(D_{\text{runway}}\)):
The Replenishment Threshold:
A system is in Runway Depletion when:
When depletion occurs:
- The subsystem scans all active requirements where \(\neg \Phi_{\text{fuel}}(r)\).
- Requirements are partitioned by their detected domain ($dom(r)$).
- Domains are sorted deterministically and bundled into
SynthesisCandidateobjects: - Each candidate proposes a focused 1-cycle priority plan (estimating 2 to 5 backlog items). - Autonomous swarms (zqk do,zqk agent orchestrate) receive these candidates to stage the next execution cycle without human prompting.
6. Concurrency & Thread-Safety Invariants
The subsystem guarantees absolute safety under concurrent multi-goroutine access:
- Reader-Writer Locking (
sync.RWMutex): - In-memory graph state is guarded byv.mu. Read operations (GetSnapshot,CheckRunwayReplenishment) acquire read locks (RLock/RUnlock), enabling high-throughput concurrent reads. - Mutations (ApplyLifecycleEvent,ScanFromStorageWithSecurity) acquire exclusive write locks (Lock/Unlock). - Atomic Disk Materialization:
- File persistence (
SaveToLiteFile) writes payloads to a temporary file (<targetPath>.tmp.<timestamp>) before executing an atomic filesystem rename (fileutil.Rename). - Disk writes never expose partial or corrupted JSON streams to concurrent reader processes. - Named Goroutines:
- Asynchronous WAL subscribers and background reindexers must be launched using
goroutinelabels.NewGoroutineto ensure tracking, profiling, and leak detection under the runtime supervision tree.
7. Verification & Coverage Invariant
To ensure platform reliability, the TPM subsystem is governed by a strict statement coverage floor:
- Static Floor Invariant: Statement test coverage for
pkg/tpmmust remain strictly \(\ge 85.0\%\) (targeting \(\ge 95\%\)). - Test Scope: All tests reside under
pkg/tpmand are verified withZQK_ALLOW_FOREGROUND_GO_TEST=1 go test -v -timeout 30s -cover ./pkg/tpm. - Pre-commit Check-Valves: Pre-commit hygiene checks enforce zero swallowed errors (
_ = ...), functions under 100 lines, hermetic cleanup witht.TempDir(), and exclusive use offileutil.*for filesystem operations.