CLI Mesh Subsystem Architectural Guide
Overview
The ZQK CLI Mesh Subsystem (cmd/zqk/mesh) provides the decentralized federation, capability exchange, and cross-kernel resource leasing surface within the Zen Quantum Kernel (ZQK).
Unlike centralized cloud brokers or transient prompt-harness frameworks that rely on opaque cloud routers, ephemeral API sessions, and unrecorded peer interactions, ZQK roots all mesh interactions directly in the persistent Knowledge Kernel. Every advertised capability, peer identity, and active lease is materialized as a versioned, verifiable graph object in Content Addressable Storage (CAS) under .zqk/process/.
The Mesh Subsystem enables sovereign autonomous kernels to:
- Discover & Inspect Peer Capabilities: Query the sovereign marketplace via
zqk mesh marketwith optional capability filtering. - Broadcast Local Capacity: Announce available compute, agent seats, storage, or specialized agent skills via
zqk mesh advertise. - Establish Cryptographically Authorized Leases: Secure time-bounded leases and generate authorization tokens via
zqk mesh lease. - Coordinate Across Transports: Seamlessly bridge peer kernels using local CLI execution harnesses and Model Context Protocol (MCP) transports.
Subsystem Architecture & File Layout
The mesh subsystem spans the CLI presentation layer (cmd/zqk/mesh/), the core federation engine (pkg/federation/), and the distributed mesh broker (pkg/federation/meshbroker/):
cmd/zqk/mesh/
├── mesh.go # Root 'zqk mesh' command builder & kernel ID resolution
├── market.go # 'zqk mesh market': marketplace querying & formatted display
├── advertise.go # 'zqk mesh advertise': capability announcement & CAS sync commit
├── lease.go # 'zqk mesh lease': authorization token generation & lease creation
├── mesh_test.go # Smoke and subcommand topology tests
├── mesh_extended_test.go # In-process project fixture & CLI execution tests
└── mesh_coverage_elevation_cycle95_test.go # Statement coverage elevation & error boundary test suite
pkg/federation/
├── identity.go # Kernel ID derivation, public keys & IdentityManager
├── handshake.go # Handshake negotiation structures & capability exchange
├── handshaker_mcp.go # MCP-based handshake protocol implementation
├── transport.go # Transport abstraction & LocalCLITransport
├── query_engine.go # Cross-kernel federated query resolution
└── meshbroker/
├── broker.go # MeshBroker lifecycle & provider registration
├── compute_broker.go # Compute resource reservation & capacity accounting
├── lease_token.go # Cryptographic lease token validation
└── transport_mcp_client.go # Remote MCP client transport & tool execution
Sovereign Mesh Interaction Topology
The following diagram illustrates how the CLI Mesh commands interface with local kernel storage, identity resolution, the mesh broker, and remote peer kernels:
flowchart TD
subgraph CLI ["CLI Command Surface (cmd/zqk/mesh)"]
CMD_MKT["zqk mesh market [--type]"]
CMD_ADV["zqk mesh advertise <resource_type>"]
CMD_LEASE["zqk mesh lease <resource_id> --from --duration"]
end
subgraph IDENTITY ["Federation Identity (pkg/federation)"]
ID_MGR["IdentityManager"]
KER_ID["Local Kernel ID (KER-<brand>-<hash>)"]
end
subgraph STORAGE ["Kernel CAS Storage (.zqk/process/)"]
ADV_STORE["capacity_advertisement Objects"]
LEASE_STORE["skill_lease Objects"]
SYNC_FLAG["storage.WithSyncCreateForKind"]
end
subgraph BROKER ["Mesh Broker & Transport (pkg/federation/meshbroker)"]
BROKER_ENGINE["MeshBroker Engine"]
TOKEN_GEN["Cryptographic Token Generator (TOK-*)"]
MCP_TRANSPORT["TransportMCPClient (JSON-RPC)"]
CLI_TRANSPORT["LocalCLITransport (In-Process / CLI)"]
end
subgraph REMOTE ["Remote Peer Kernel"]
REMOTE_KERNEL["Remote Kernel (e.g. KER-COMMUNITY-002)"]
REMOTE_MCP["Remote MCP Daemon (:8765)"]
end
CMD_MKT -->|Reads with ListFilter| ADV_STORE
CMD_ADV -->|Resolves Local Identity| ID_MGR
ID_MGR -->|Derives| KER_ID
CMD_ADV -->|Synchronous Commit| SYNC_FLAG
SYNC_FLAG -->|Persists| ADV_STORE
CMD_LEASE -->|Resolves Local Identity| KER_ID
CMD_LEASE -->|Generates Authorization| TOKEN_GEN
TOKEN_GEN -->|Creates Object| LEASE_STORE
CMD_LEASE -->|Dispatches RPC| BROKER_ENGINE
BROKER_ENGINE -->|Executes via| MCP_TRANSPORT
BROKER_ENGINE -->|Executes via| CLI_TRANSPORT
MCP_TRANSPORT -->|Federated Tools| REMOTE_MCP
REMOTE_MCP -->|Serves Resources| REMOTE_KERNEL
1. Command Taxonomy (zqk mesh)
The zqk mesh command group provides the operator entry points for participating in the sovereign mesh network.
Root Command: zqk mesh
Constructed by NewMeshCmd() in cmd/zqk/mesh/mesh.go, this parent command aggregates all mesh subcommands:
zqk mesh --help
Subcommands registered:
market: Browse available capacity advertisements across the network.advertise: Broadcast local capabilities and capacity offers to the mesh.lease: Secure a time-bounded lease for a specific remote resource.
Marketplace Query: zqk mesh market
The market command scans Content Addressable Storage for active capacity_advertisement objects, formatting them into an ANSI-highlighted console dashboard.
Command Syntax
zqk mesh market [--type <resource_type>]
Supported Flags
--type(string): Filters displayed advertisements by resource kind (e.g.,compute,agent_seat,gpu,storage,agent_skill).
Execution Flow
- Filter Assembly: Configures a
storage.ListFilterwithKind: objects.KindCapacityAdvertisement. If--typeis specified, adds an exact-match filter onobjects.FieldKeyResourceType. - Storage Query: Invocates
proc.Storage().List()to retrieve all matching advertisements. - Robust Value Normalization: Advertised quantities can arrive as
float64,int, orint64depending on client serialization. The display loop safely converts these types to uniform float representations. - Visual Presentation: Renders headers, provider references, units, and resource identifiers using terminal color palettes.
Example Output
SOVEREIGN MESH MARKETPLACE
============================================================
compute [ADV-DEMO-001]
Provider: KER-COMMUNITY-001
Capacity: 16.0 cores
Resource: comp-cluster-01
agent_seat [ADV-DEMO-002]
Provider: KER-COMMUNITY-002
Capacity: 3.0 instances
Resource: seat-craftsman-senior
Capability Advertising: zqk mesh advertise
The advertise command allows a kernel operator or background daemon to offer local capacity to other peer kernels in the mesh.
Command Syntax
zqk mesh advertise <resource_type> [flags]
Supported Flags
--amount(string): Numerical capacity available (parsed as 64-bit float; defaults to0.0).--units(string): Capacity units (e.g.,cores,instances,GB,hours; defaults toinstances).--resource-id(string): Optional identifier for the underlying resource or cluster instance.
Execution Flow
- Identity Resolution: Obtains the deterministic local kernel identifier via
resolveLocalKernelID(proc.ProjectRoot()). - Object Formulation: Constructs a
capacity_advertisementmap: -kind:capacity_advertisement-title:Offer: <type> (<amount> <units>)-provider_kernel_ref: Local Kernel ID -resource_type:<resource_type>-quantity:<amount>-units:<units>-resource_id:<resource-id>(if provided) -status:implemented - Synchronous CAS Commit: Uses
storage.WithSyncCreateForKindto bypass background write queues, ensuring immediate index visibility to subsequent marketplace queries. - Operator Feedback: Writes confirmation containing broadcasted metrics.
Example Invocation
zqk mesh advertise compute --amount 32.0 --units cores --resource-id comp-node-alpha
# Output:
# ✅ Broadcasting compute capacity: 32.0 cores
Resource Leasing: zqk mesh lease
The lease command establishes a formal mutual agreement between the local consumer kernel and a remote provider kernel.
Command Syntax
zqk mesh lease <resource_id> --from <provider_kernel_ref> [flags]
Supported Flags
--from(string, required): The remote provider's kernel identifier (KER-...).--duration(duration): Lease validity duration (e.g.,30m,2h,24h; defaults to0sfor indefinite/session-bounded).
Execution Flow
- Local Identity Derivation: Queries
IdentityManagerto establish the consumer's provenance. - Token Generation: Generates a cryptographic authorization token using
crypto/randformatted asTOK-<HEX16>(e.g.,TOK-7F3A2B1C9E4D...). - Object Formulation: Constructs a
skill_leasekernel object: -kind:skill_lease-title:Lease: <resource_id> from <provider_kernel_ref>-provider_kernel_ref: Target provider -consumer_kernel_ref: Local kernel ID -resource_ref: Leased resource ID -token_id: Generated authorization token -status:implemented-expires_at: RFC3339 timestamp (if--duration> 0) - CAS Persistence: Commits the lease synchronously into
.zqk/process/skill_lease/with immediate index freshness. - Token Delivery: Prints the authorization token for inclusion in downstream MCP tool invocations.
Example Invocation
zqk mesh lease comp-node-alpha --from KER-COMMUNITY-001 --duration 4h
# Output:
# ✅ Lease established for comp-node-alpha.
# Authorization Token: TOK-8C49E1F033BA21D9
2. Mesh Discovery & Identity Management
Every participating node in the sovereign mesh requires a stable, tamper-evident cryptographic identity. ZQK decouples identity from network addresses or DNS records, using deterministic root hashing.
Identity Derivation
The IdentityManager (pkg/federation/identity.go) computes the canonical kernel identifier:
absRoot, err := filepath.Abs(m.projectRoot)
hash := sha256.Sum256([]byte(absRoot))
shortHash := fmt.Sprintf("%x", hash)[:12]
kernelID := fmt.Sprintf("%s-%s-%s", KernelIDPrefix, brand.Prefix(), shortHash)
- Prefix:
KER(canonical kernel prefix). - Brand: System brand prefix (e.g.
ZQK). - Short Hash: Truncated 12-character SHA-256 digest of the absolute project root.
This guarantees:
- Project Isolation: Separate project directories on the same host maintain isolated identities and authorization boundaries.
- Portability: Re-opening the same project repository yields the identical Kernel ID, ensuring durable lease ownership and continuity.
Public Key Management
Each kernel maintains a sovereign keypair in .zqk/identity/. The public key (PUB-...) is shared during federation handshakes, allowing peer kernels to verify signed messages, leases, and state attestations without central certificate authorities.
3. Heartbeat & Lease Supervision
Lease agreements in a distributed mesh must handle node disconnections, process failures, and graceful reclamation.
sequenceDiagram
autonumber
participant Consumer as Consumer Kernel (KER-COMMUNITY-002)
participant Storage as Kernel Storage (skill_lease)
participant Scheduler as Scheduler Lease Supervisor
participant Provider as Provider Kernel (KER-COMMUNITY-001)
Consumer->>Storage: zqk mesh lease comp-01 --from KER-COMMUNITY-001 --duration 2h
Storage-->>Consumer: Confirmed (TOK-9A8B7C6D)
loop Every Lease Heartbeat Interval (e.g. 30s)
Scheduler->>Provider: SendHeartbeat(ctx, endpoint, consumerKernelID)
Provider-->>Scheduler: Heartbeat ACK / Refresh
end
alt Lease Expiration or Heartbeat Timeout
Scheduler->>Storage: Update Lease Status (expired / revoked)
Provider->>Provider: Reclaim Resource Allocation
end
Key Supervision Guarantees
- Deterministic Expiration: If
expires_atpasses, the lease supervisor marks the lease asexpired, revoking access to the resource. - Heartbeat Protocol:
Transport.SendHeartbeat(ctx, endpoint, kernelID)maintains active session state between kernels. - Idempotent Reclamation: Leased resources are automatically returned to the provider's unreserved pool upon timeout or revocation.
4. Transport Protocols & Execution Abstraction
The mesh subsystem abstracts communication channels behind the federation.Transport interface:
type Transport interface {
SendHandshake(ctx context.Context, endpoint string, req HandshakeRequest) (*HandshakeResponse, error)
SendHeartbeat(ctx context.Context, endpoint string, kernelID string) error
ExecuteTool(ctx context.Context, endpoint string, toolName string, arguments map[string]any) (json.RawMessage, error)
}
Transport Implementations
| Transport | Implementation File | Usage Context | Characteristics |
|---|---|---|---|
LocalCLITransport |
pkg/federation/transport.go |
Local testing, CI simulations, single-host pipelines | Invokes local zqk binary via subprocess; zero network dependencies |
TransportMCPClient |
pkg/federation/meshbroker/transport_mcp_client.go |
Distributed nodes, multi-host swarms, remote agent seats | JSON-RPC 2.0 over standard I/O or TCP SSE stream; Model Context Protocol compatible |
Handshake Protocol
When two kernels federate, they exchange a structured HandshakeRequest:
{
"protocol_version": "1.0",
"kernel_id": "KER-COMMUNITY-002",
"namespace": "default",
"public_key": "PUB-92A8E1BC...",
"capabilities": [
{
"kind": "agent_skill",
"id": "skill-qa-verifier",
"name": "QA Verification Specialist"
}
],
"timestamp": "2026-10-20T03:00:00Z"
}
Upon verification, the receiving kernel responds with an accepted HandshakeResponse detailing mutual capabilities and shared namespaces.
5. Security & Isolation Boundaries
- Content-Addressable Provenance: Every advertisement and lease object in CAS contains cryptographic hashes and author signatures, preventing retroactive modification.
- Fail-Closed Permissions: The storage layer enforces read and write permissions per object kind. Unauthenticated callers default to a guest context, blocking unauthorized lease creations.
- Capability Scoping: Leases are strictly bounded by resource ID and time. A token issued for
comp-node-alphacannot authenticate operations againstcomp-node-beta. - Zero Ambient Authority: Remote tool execution over
TransportMCPClientrequires an active, unexpired lease token matching the provider's local lease index.
Summary of Verification & Standards
- Command Syntax: Strictly adhere to
zqk mesh [market|advertise|lease]. - Statement Test Coverage: Maintained at >= 98.0% across
cmd/zqk/mesh/viamesh_test.go,mesh_extended_test.go, andmesh_coverage_elevation_cycle95_test.go. - Identifier Standards: Prohibits ephemeral or raw studio nanos-hex values; employs canonical prefixes (
KER-,TOK-,ADV-,MESH-DEMO-001).