Kernel Subsystem Operational & Architectural Guide
Overview
The ZQK Kernel Subsystem (pkg/kernel) implements the foundational cellular microkernel architecture, relational graph membrane, transactional plane isolation, and extensible runtime host for the Zen Quantum Knowledge Kernel.
Unlike conventional flat-file task systems or transient chat harness contexts where goals and execution state evaporate, ZQK anchors all system entitiesβfrom timeless strategic goals down to atomic backlog items and cryptographic verification proofsβinto an immutable, relational graph. The kernel subsystem governs object containment, edge validation across transactional isolation planes, cyclic graph analysis, Model Context Protocol (MCP) tool execution, and pluggable runtime extension lifecycles.
Architectural Taxonomy & Component Layout
pkg/kernel/
βββ kernel.go # Concrete Kernel runtime, MCP mediation, extension registry, and shutdown
βββ knowledge_kernel.go # Core KnowledgeKernel interface contract
βββ adjacency.go # AdjacencyEngine, Edge relations, PlaneBoundaryEnforcer, and cycle detection
βββ extension.go # Pluggable Extension interface contract
βββ intake/ # Intake membrane, clustering, and candidate validation
βββ mutation/ # Mutation gates, shockwave enforcers, and CAS boundary transitions
βββ steward/ # Runway steward daemons and lifecycle supervisors
βββ verification/ # Verification engine, execution organizers, and criteria attestation
Subsystem Architecture & Interaction Flow
flowchart TD
Client["Client / Autonomous Swarm Agent / CLI"]
subgraph KernelRuntime["Knowledge Kernel Runtime (pkg/kernel)"]
KK["KnowledgeKernel Engine"]
ExtReg["Extension Registry (Pluggable Extensions)"]
MCPServer["MCP Mediation Bridge (CallTool / ListTools)"]
subgraph GraphMembrane["Relational Graph Membrane"]
PBE["PlaneBoundaryEnforcer"]
PR["PlaneResolver (Draft / Staged / Promoted / Apoptotic)"]
Adj["AdjacencyEngine (MemoryAdjacencyEngine)"]
Cycle["Cycle Detection & Graph DFS Engine"]
end
subgraph StoragePlane["Storage Plane"]
CAS[("Content Addressable Storage (CAS)")]
WAL["Write-Ahead Event Log"]
end
end
Client -->|ExecuteMCPTool / ListMCPTools| MCPServer
Client -->|RegisterExtension / Shutdown| KK
KK --> ExtReg
KK --> MCPServer
KK --> CAS
Client -->|AddEdge / RemoveEdge| Adj
Adj -->|ValidateEdge| PBE
PBE -->|ResolvePlane| PR
Adj --> Cycle
1. Knowledge Kernel Core Principles
The kernel subsystem adheres to four foundational architecture invariants:
- Immutable CAS Persistence: Every domain entity exists as an immutable, content-addressed object in
.zqk/process/. Mutations produce discrete cryptographic revisions rather than in-place overwrites. - Relational Graph Membrane: Relationships between entities (causal dependencies, semantic references, implementations) are first-class directed edges governed by deterministic validation invariants.
- Decoupled Cellular Architecture: The kernel acts as a minimal microkernel, delegating domain-specific processing to modular subsystems (
intake,mutation,steward,verification) and pluggable extensions. - Transactional Plane Isolation: Objects operate across discrete lifecycle planes with strict boundary validation preventing unpromoted contamination of authoritative state.
2. The 5-Layer Relational Cascade
The Knowledge Kernel orchestrates work across a formal five-layer ontological hierarchy:
flowchart TD
VM["1. Vision & Mission<br/>(Timeless Intent Compass)"] --> GOAL["2. Strategic Goals<br/>(Timeless Strategic Contracts)"]
GOAL -->|Decomposes into 3-5| REQ["4a. Requirements<br/>(Whole Feature Contracts)"]
MILE["3a. Milestones<br/>(Chronological Boundary Anchors)"] --> PRI["3b. Priority Plans<br/>(1-Cycle Execution Batches)"]
PRI -->|Groups 2-5| BLI["5. Backlog Items<br/>(Atomic Units of Effort)"]
REQ -->|Satisfies 1-3 Criteria| CRIT["4b. Acceptance Criteria<br/>(Three-Fold Proof: Static/Dynamic/Adversarial)"]
BLI -.->|Satisfies| CRIT
TST["Test Cases<br/>(Unified Verification Groups)"] -->|Verifies 1:1| REQ
Layer Roles and Relational Rules
- Vision & Mission (Level 1): Defines the foundational purpose, values, and operating principles of the organization or project.
- Strategic Goals (Level 2): Timeless strategic contracts anchoring high-level intent. Goals decompose into 3β5 child
requirementcontracts (requirement.goal_refs). Goals do not declare sprint constraints or priority plan links. - Chronological Anchors & Execution Cycles (Level 3):
- Milestones: Time-bounded anchors representing release events or roadmap targets.
- Priority Plans: Time-bounded execution batches (sprints or kanban iterations) grouping 2β5 atomic backlog items toward a milestone. Priority plans lock scope upon entering
in_progress. - Epics: Thematic containers providing human-legible semantic structure across multi-plan initiatives. - Functional Contracts & Three-Fold Proof (Level 4):
- Requirements: Whole feature contracts owning verifiable
criteria_refs. Requirements exist outside calendar time and decompose into criteria. - Criteria: The Three-Fold Proof model requiring a Static Floor Invariant, an Operational Dynamic Proof, and a Negative Adversarial Boundary. - Test Cases: Linked 1:1 to requirements, wrapping and verifying all criteria in a single execution. - Backlog Items (Level 5): Atomic, token-budgeted units of effort assigned to engineering agents. Each backlog item directly satisfies 1 to 3 specific criteria.
3. Transactional Isolation Planes & Boundary Enforcer
Entities in the Knowledge Kernel belong to one of four transactional planes:
| Plane | Constant | Description | Permitted Inbound Causal Source |
|---|---|---|---|
| Draft | PlaneDraft (0) |
Gestation phase; speculative, unverified, or in-flight objects. | Draft |
| Staged | PlaneStaged (1) |
Metabolism & Attestation; active execution under verification gates. | Draft, Staged |
| Promoted | PlanePromoted (2) |
Equilibrium; complete, verified, authoritative baseline state. | Staged, Promoted |
| Apoptotic | PlaneApoptotic (3) |
Apoptosis; archived, quarantined, or tombstoned objects. | None (Strictly Isolated) |
Boundary Enforcement Rules (PlaneBoundaryEnforcer)
The PlaneBoundaryEnforcer intercepts all edge creation requests before mutations are committed to the graph:
flowchart TD
EdgeReq["AddEdge Request (Source -> Target, Kind)"] --> ChkResolver{"Resolver Available?"}
ChkResolver -->|No| Allow["Allow Edge (Unchecked)"]
ChkResolver -->|Yes| ResolvePlanes["Resolve Source & Target Planes"]
ResolvePlanes --> ChkApoptosis{"Either Node in PlaneApoptotic?"}
ChkApoptosis -->|Yes| RejectApop["Reject: Apoptotic objects cannot participate in active edges"]
ChkApoptosis -->|No| ChkCausal{"Kind is causal or depends_on?"}
ChkCausal -->|"No (Informational)"| Permit["Permit Edge"]
ChkCausal -->|Yes| ChkDraftToPromoted{"Source = Draft AND Target = Promoted?"}
ChkDraftToPromoted -->|Yes| RejectDraft["Reject: Draft cannot link causal to Promoted without staging"]
ChkDraftToPromoted -->|No| ChkPromotedToUnpromoted{"Source = Promoted AND Target = Draft or Staged?"}
ChkPromotedToUnpromoted -->|Yes| RejectContam["Reject: Promoted cannot depend on unpromoted object"]
ChkPromotedToUnpromoted -->|No| Permit
- Apoptotic Isolation: Neither the source nor target entity may reside in
PlaneApoptotic. Quarantined or deleted entities cannot participate in active edges. - Draft Causal Isolation: Entities in
PlaneDraftcannot formcausalordepends_ondependencies toPlanePromotedentities without first advancing throughPlaneStaged. Informational references (e.g.references,relates_to) are permitted. - Upstream Contamination Guard: Entities in
PlanePromotedcannot declare causal dependencies on unpromoted entities (PlaneDraftorPlaneStaged), guaranteeing that the authoritative baseline remains closed and self-consistent.
4. Graph Mechanics & Memory Adjacency Engine
The AdjacencyEngine interface governs directed relations across the system:
type AdjacencyEngine interface {
AddEdge(ctx context.Context, edge Edge) error
RemoveEdge(ctx context.Context, source, target dna.URN, kind string) error
GetOutbound(ctx context.Context, source dna.URN) ([]Edge, error)
GetInbound(ctx context.Context, target dna.URN) ([]Edge, error)
DetectCycles(ctx context.Context) ([][]dna.URN, error)
HasCycle(ctx context.Context) (bool, error)
}
Implementation Details (MemoryAdjacencyEngine)
- Dual-Map Indexing: Outbound edges (
map[string][]Edge) and inbound edges (map[string][]Edge) are indexed simultaneously under a reader-writer mutex (sync.RWMutex), enabling \(O(1)\) lookup for both upstream and downstream neighbors. - Defensive Slice Cloning: Inbound and outbound queries return decoupled slices via
cloneEdges(). Callers cannot mutate internal kernel adjacency collections. - Idempotent Addition: Attempting to add an edge identical in
Source,Target, andKindsucceeds idempotently without duplicating edge slices. - Kind-Specific & Wildcard Pruning:
RemoveEdgesupports targeting a specificKindor providing an empty string""to remove all relations between source and target nodes.
Cycle Detection Algorithm
The DetectCycles algorithm executes depth-first traversal with path tracking:
- Iterates all nodes in deterministic lexicographical order.
- Maintains a
visitedset and an activerecStack(recursion stack). - If an edge leads to an unvisited node, DFS recurses.
- If an edge leads to a node currently in
recStack, a directed cycle is identified and sliced fromcurrentPath. - Returns all elementary cycles as slices of
dna.URN.
5. Pluggable Extension Registry & Lifecycle
The Knowledge Kernel provides an extension membrane allowing custom subsystems, analyzers, or daemon workers to register cleanly:
type Extension interface {
Name() string
Init(ctx context.Context, k KnowledgeKernel) error
Shutdown(ctx context.Context) error
}
Extension Lifecycle Sequence
sequenceDiagram
participant Host as Kernel Bootstrap / Host
participant K as Kernel Runtime
participant Ext as Extension
participant Storage as Storage Provider
Host->>K: RegisterExtension(ext)
K->>K: Validate non-nil, non-empty name, uniqueness
K->>Ext: Init(ctx, k)
Ext-->>K: Init Result (success / error)
K-->>Host: Registration Confirmed
Note over K,Ext: Normal Operation (GetExtension, ListExtensions)
Host->>K: Shutdown(ctx)
loop Every Registered Extension
K->>Ext: Shutdown(ctx)
Ext-->>K: Shutdown Result
end
opt Storage Closer Present
K->>Storage: Shutdown(ctx)
Storage-->>K: Storage Shutdown Result
end
K-->>Host: Joined Errors via errors.Join
- Thread-Safe Registry: Protected by
sync.RWMutex. - Lexicographical Enumeration:
ListExtensions()returns sorted names for deterministic inspection and CLI formatting. - Fault-Tolerant Shutdown: Kernel shutdown invokes
Shutdown(ctx)on all registered extensions and the underlyingObjectStorageProvider. Any errors encountered are collected and joined viaerrors.Join, ensuring that failure in one extension does not abort teardown of remaining extensions or storage resources.
6. Model Context Protocol (MCP) Integration
The Kernel struct embeds native support for the Model Context Protocol:
ExecuteMCPTool(ctx, secCtx, toolName, params): Routes tool execution directly to the boundmcp.MCPServer, packaging parameters into standardToolCallParams.ListMCPTools(ctx, secCtx): Queries the tool registry, returning available tools and parameter schemas for LLM agent discovery.
7. Verification & Statement Coverage Invariant
To ensure reliability across concurrent agent swarms and mission-critical execution loops, the Kernel Subsystem enforces a strict verification standard:
- Statement Coverage Floor:
pkg/kernelstatement coverage must be maintained at >= 85.0% at all times. - AST Quality Standards:
- Zero swallowed errors: All error return values must be explicitly checked and asserted (
require.NoError,assert.Error). - Modular function sizing: Test and production functions must not exceed 100 lines.
- Managed concurrency: All background goroutines in tests and production must use
goroutinelabels.NewGoroutine(...)orgoroutinelabels.Pool. - Hermetic testing: Tests must allocate isolated temporary environments via
testkit.PrepareIsolatedTempProjectort.TempDir(). - Bounded test execution: Test suites must complete reliably within
-timeout 30s.
Running Kernel Verification Commands
# Execute focused Cycle test suite
ZQK_ALLOW_FOREGROUND_GO_TEST=1 go test -v -timeout 30s ./pkg/kernel -run 'TestCycle'
# Run statement coverage report
ZQK_ALLOW_FOREGROUND_GO_TEST=1 go test -short -cover ./pkg/kernel
# Run codebase hygiene and tree validation
./bin/zqk-vet -suite hygiene,tree
# Run public release payload verification
sh scripts/open-core/check-public-release-payload.sh .