CLI New Subsystem Architecture & Operational Guide
Executive Overview
The CLI New Subsystem (zqk new) serves as the canonical shift-left entry point for intent intake, object drafting, and CLI DNA authoring across the ZQK Knowledge Kernel. Unlike ephemeral, prompt-promiscuous chat harnesses where requirements, constraints, and architecture drift in transient LLM memory buffers, ZQK objectifies all engineering intent into persistent, verifiable graph entities from moment zero.
The zqk new subsystem bridges human and agentic intent with the Knowledge Kernel's Content-Addressable Storage (CAS) membrane through:
- First-Class Ontological Command Taxonomy: Ergonomic, shorthand CLI verbs (
zqk new goal,zqk new req,zqk new bli,zqk new plan) enforcing the 5-layer cascade. - Draft Plane Architecture: Safe isolation of in-progress drafts in
.zqk/drafts/and transient draft-plane states before immutable CAS sealing. - Just-In-Time (JIT) Ontological Guidance: Built-in AST and schema invariants rendered directly in interactive terminal sessions to prevent scope-stacking and architectural anti-patterns.
- CLI DNA Scaffolding (
command-spec): Automated introspection and generation of declarative command specifications conforming to JSON schemas. - Background Promotion Pipeline: One-time scheduler lifecycle integration (
enqueueMintPromoteJob) enabling seamless transition from draft to canonical CAS.
flowchart TD
subgraph Intent ["Human & Agent Intent Intake"]
CLI["zqk new <kind>"]
CSPEC["zqk new command-spec"]
BNDL["zqk new bundle / swarm"]
end
subgraph DraftingPlane ["Draft Plane & Template System"]
DP[".zqk/drafts/ & last-draft.yaml"]
TMPL["Schema Default Injection"]
JIT["JIT Shift-Left Guidance"]
end
subgraph CASMembrane ["CAS Crossing Membrane"]
VAL["Schema & Field Validation"]
SCHED["Scheduler Job: SCH-mint-promote-*"]
PROM["zqk object promote"]
end
subgraph KernelGraph ["Knowledge Kernel (CAS Engine)"]
GRAPH[("Immutable Knowledge Graph\nGoals / Reqs / Plans / BLIs")]
end
CLI --> JIT
CLI --> TMPL
TMPL --> DP
DP --> VAL
CSPEC --> VAL
BNDL --> DP
VAL -->|--promote opt-in| SCHED
SCHED --> PROM
PROM --> GRAPH
VAL -->|manual promote| GRAPH
1. Command Taxonomy & Shorthand Conventions
The zqk new subsystem implements a hierarchical command taxonomy designed for maximum discovery, ergonomic terminal interaction, and multi-agent automation.
1.1 Root Command & Discovery
Executing zqk new --help displays the full taxonomy of draftable entities and command generators:
zqk new [command]
The root command acts as a gateway to:
- Generic Object Drafting:
zqk new object <kind> - Convenience Kind Verbs:
zqk new goal,zqk new req,zqk new bli,zqk new plan, etc. - DNA Scaffolding:
zqk new command-spec <command-path> - Scenario Bundles:
zqk new bundle - Swarm Packages:
zqk new swarm - Ontology Extension:
zqk new kind-spec <ontology>
1.2 First-Class Kind Convenience Commands
To eliminate friction during high-velocity development, zqk new registers top-level convenience verbs for all core ontological entities:
| Command | Aliases | Ontological Purpose | Example Invocation |
|---|---|---|---|
zqk new goal |
— | Timeless strategic compass theme | zqk new goal --title "Enterprise Observability" |
zqk new vision |
vis |
High-level foundational vision | zqk new vision --title "Zero-Downtime Migration" |
zqk new mission |
mis |
Core organizational mission | zqk new mission --title "Sub-15ms Index Latency" |
zqk new roadmap |
— | Multi-horizon strategic roadmap | zqk new roadmap --title "H2 Performance Runway" |
zqk new workstream |
ws |
Dedicated domain track | zqk new workstream --title "Storage Optimization" |
zqk new epic |
— | Thematic multi-plan container | zqk new epic --title "Storage Engine Refactor" |
zqk new plan |
pri, priority_plan |
1-cycle time-bounded execution batch | zqk new plan --title "Storage Sharding Batch" |
zqk new req |
requirement |
Timeless functional whole-feature contract | zqk new req --title "Append-Only WAL Sharding" |
zqk new crit |
criteria |
Three-fold verification contract | zqk new crit --title "Static Floor: Sync Latency < 2ms" |
zqk new tc |
test_case |
1:1 Unified test suite wrapper | zqk new tc --title "Verify WAL Sharding Contracts" |
zqk new bli |
backlog_item |
Atomic unit of effort (1–3 criteria) | zqk new bli --title "Implement Shard Router Hash" |
zqk new mil |
milestone |
Chronological boundary anchor | zqk new mil --title "v2.4 Production Candidate" |
zqk new techdebt |
td, technical_debt |
Non-functional behavior plane entry point | zqk new techdebt --title "Eliminate Memory Leak in WAL" |
zqk new policy |
— | Verification gate and architectural rule | zqk new policy --title "POL-STORAGE-SHARD-001" |
zqk new que |
question |
Architectural inquiry probe | zqk new que --title "Consensus Protocol Selection" |
zqk new dec |
decision |
Architectural Decision Record (ADR) | zqk new dec --title "Adopt Raft-based Lease State" |
2. Draft Plane Architecture & Storage Invariants
The ZQK Knowledge Kernel enforces a strict boundary between mutable draft templates and immutable CAS records.
2.1 The Two Draft Planes
The system maintains two distinct drafting mechanisms:
-
File-Based Template Drafting (
.zqk/drafts/): - Generated when usingzqk new bundle,zqk new swarm,zqk new kind-spec, orwriteDraft. - Files follow the deterministic naming format:.zqk/drafts/<sanitized-name>-<timestamp>.yaml. - The file.zqk/drafts/last-draft.yamlrecords the pointer to the most recently generated template, including scope, kind, and absolute filesystem path. - Allows human developers or autonomous agents to inspect, fill out complex YAML properties, and review prior to committing. -
Kernel Draft-Plane State (
objects.FieldKeyStatus = "draft"): - Generated when invokingzqk new object <kind>or convenience verbs (zqk new bli). - The entity is assigned a canonical identifier (e.g.BLI-DEMO-001), initialized with standard metadata, and stored with draft status. - Isolated from release reports and milestone completion metrics until verified and promoted.
2.2 Template Materialization via writeDraft
The internal engine function writeDraft provides resilient destination routing:
- Default Pathing: Writes into
.zqk/drafts/, sanitizing base names (replacing/,\,:with-) and appending UTC log-rotate timestamps. - Stdout Streaming: Specifying
--output -directly writes the generated YAML payload to standard output, enabling Unix pipe integration (e.g.zqk new bundle -o - | yq .). - Targeted Output: Passing
--output path/to/file.yamlensures secure file materialization at arbitrary filesystem locations.
2.3 Auto-Trace Pipeline Generation
When minting timeless requirement feature contracts (zqk new req), the subsystem supports optional auto-trace pipeline generation:
- Traceability Linkage: Automatically binds the newly minted requirement to downstream criteria and test cases via
workflow.ApplyGeneratedTracePipelineFromCmd(cmd, id). - Fail-Closed Verification: If trace pipeline generation fails, the command fails closed with an error (
minted <id> but trace pipeline failed; promote will fail closed without traceability links). - Opt-In Discipline: Auto-trace pipeline is disabled by default to prevent ghost criteria clutter and graph pollution. It is explicitly opted into via
--trace-pipelineor subsequent workflow execution (zqk workflow gen-trace-pipeline <REQ-id>), and can be explicitly bypassed with--skip-trace-pipeline. - Test Isolation Protection: Under test execution environments, auto-trace pipeline is automatically bypassed to keep unit test suites fast, isolated, and deterministic.
3. Just-In-Time (JIT) Ontological Guidance
To prevent architectural entropy, zqk new inspects invocation context and automatically outputs cyan-highlighted guidance tips when invoked in interactive human/text console modes:
JIT guidance is automatically silenced when structured output formats are requested (--format json, --format yaml, --format jsonrpc, --format stream) to prevent stdout/stderr contamination for downstream tools.
Guidance Rules by Entity Kind
-
Goals (
KindGoal):💡 Shift-Left Tip: Goals represent timeless strategic compass themes (outside of sprint/time boundaries) and decompose into 3–5 requirements (child requirement.goal_refs). Goals do NOT declare criteria_refs directly.
-
Requirements (
KindRequirement):💡 Shift-Left Tip: Requirements are timeless functional contracts (never bound to time or priority plans). They own criteria_refs (\(\ge 3\) satisfying three-fold proof: static invariant, operational proof, negative boundary) verified 1:1 by a test_case.
-
Milestones (
KindMilestone):💡 Shift-Left Tip: Milestones introduce chronological and time constraints — acting as time-bounded subgoals across delivered reality that priority plans aim toward.
-
Priority Plans (
KindPriorityPlan):💡 Shift-Left Tip: Priority plans bound exactly 1 cycle of work (sprint/kanban batch, 2–5 BLIs aiming toward a milestone). Plans scope-lock upon entering in_progress to prevent drift.
-
Backlog Items (
KindBacklogItem):💡 Shift-Left Tip: Backlog items are atomic units of effort satisfying 1–3 criteria. Decompose work across 2–5 BLIs per plan to avoid scope-stacking.
-
Epics (
KindEpic):💡 Shift-Left Tip: Epics give human-legible semantic shape to the overall initiative, grouping 2–5+ priority plans under a recognizable umbrella. Permissive by default (plans can be added in progress unless execution_locked=true).
-
Technical Debt (
KindTechnicalDebt):💡 Shift-Left Tip: Technical debt provides a tactical entry point into the non-functional behavior plane (code smells, test flakes, performance hotpaths, bug rollups). Rollups of tech debt items tell the anti-pattern story to institute lasting architectural and coding best practices.
4. CAS Membrane Crossing Rules & Promote Invariants
Before any entity can cross from the draft plane into permanent, immutable CAS storage, the CAS membrane evaluates strict validation rules:
4.1 Membrane Validation Gates
- Title and Body Requirement:
- Every object requires a non-empty
title. - Atomic effort units (backlog_item,technical_debt) require non-empty descriptions before crossing into CAS storage. - Schema Invariant Checking:
- Entities must conform to JSON schemas registered under
.zqk/schemas/. - Prohibited metadata mutations (such as manual alteration of system-computed provenance fieldsupdated_atorupdated_by) are blocked. - Traceability Proofs: - Requirements must link to parent goals and declare verification criteria. - Backlog items must map to active Priority Plans and satisfy explicit criteria references.
4.2 Background Promotion via Scheduler (--promote)
Passing the --promote flag triggers an asynchronous, fail-closed promote job enqueued directly with the ZQK Scheduler:
zqk new bli --title "Implement Router Hash" --description "Consistent hash ring for shard router" --promote
Under the hood:
zqk newcreates the draft object in storage.enqueueMintPromoteJob(projectRoot, objectID)builds a scheduler job entity (SCH-mint-promote-<id>).- Job configuration:
- Job Type:
run_wrapper- Command:zqk object promote <id>- Execution Mode:one_time- Category:lifecycle- Max Runtime: 600 seconds - Retry Count: 1 (15-second delay) - If an existing promote job is found for the same object,
enqueueMintPromoteJobperforms an idempotent update of execution parameters without mutating restricted system provenance fields. - A trigger request is dispatched to the background scheduler daemon queue.
5. CLI DNA Command Scaffolding (zqk new command-spec)
The zqk new command-spec tool provides declarative, spec-driven authoring of new CLI commands. Rather than writing repetitive Cobra boilerplate by hand, developers define high-level YAML specifications that code generators translate into type-safe Go AST command builders.
5.1 Command Line DNA Architecture
flowchart LR
CS["zqk new command-spec <path>"] --> SPEC[".zqk/cli/specs/*_command.yaml"]
SPEC --> VAL["Schema Validation\n(command_spec.schema.json)"]
VAL --> GEN["zqk system generate-command-builders"]
GEN --> GO["cmd/zqk/bldr_cli_cmd_v1/<name>.go"]
GO --> BIN["Native Binary Build (./bin/zqk)"]
5.2 Scaffolding Flags & Options
zqk new command-spec <command-path> [flags]
--short <text>: Short one-line summary (required unless--from-cmdis used).--description <text>: Detailed command description and long help.--from-cmd: Introspect an existing registered Cobra command in the binary, harvesting flags, argument specifications, aliases, and examples.--aliases <list>: Comma-separated list of short command aliases (e.g.er,el-run).--args-type <type>: Argument validation rules (no_args,exact,minimum,maximum,range).--args-count <n>,--args-min <n>,--args-max <n>: Numeric limits for argument validation.--flag <def>: Custom flag definition inname:type:default:description[:shorthand]format.--example <def>: Usage examples incomment:commandformat.--run-e <func>: Name of the Go execution function (defaults torunPascalCaseName).--async: Wrap execution function withcli.BindAsyncProgressfor background progress bars.--force: Overwrite existing spec file on disk.
5.3 Introspection from Registered Tree (--from-cmd)
When migrating existing handcrafted commands to declarative CLI DNA specifications, --from-cmd automatically extracts metadata:
- Preserves command usage patterns (
<target>or[items...]). - Filters out standard common flags (
--format,--output,--verbose,--help). - Translates Cobra flag types (
bool,int,stringSlice,duration) into canonical spec types. - Parses comments and invocations from existing
Examplestrings.
6. Output Formats & Composability
The zqk new command suite conforms to the Universal ZQK CLI Ergonomics contract, supporting multiple standard presentation formats:
| Format Flag | Output Characteristics | Typical Use Case |
|---|---|---|
| (default human) | Formatted key-value pairs with colorized JIT tips and status alerts | Interactive developer terminal |
--format json |
Structured JSON object receipt with machine-readable metadata | Automated script ingestion, CI/CD |
--format yaml |
Clean YAML payload | GitOps pipelines, configuration files |
--format stream |
Line-delimited event stream | Long-running orchestrations and daemons |
Example JSON Receipt (zqk new bli --format json)
{
"id": "BLI-DEMO-001",
"kind": "backlog_item",
"status": "draft",
"title": "Implement Consistent Hash Ring",
"description": "Consistent hash ring for shard router",
"created_at": "2026-10-20T19:45:00Z",
"created_by": "USR-DEV-001",
"promote_enqueued": true,
"job_id": "SCH-mint-promote-BLI-DEMO-001"
}
7. Verification & Operational Reference
Quick Reference Workflows
# 1. Mint a strategic goal outside of time constraints
zqk new goal --title "Enterprise Storage Resilience"
# 2. Mint a whole-feature functional contract with opt-in trace pipeline
zqk new req --title "Multi-Region Read Replicas" --trace-pipeline
# 3. Mint a 1-cycle priority plan aiming toward a release milestone
zqk new plan --title "Read Replica Sharding Batch"
# 4. Mint an atomic unit of effort with immediate promote enqueuing
zqk new bli --title "Add Shard Consensus Ping" \
--description "Periodic health ping for active consensus shards" \
--promote
# 5. Scaffold declarative CLI command specification from existing command
zqk new command-spec storage/sync-status --from-cmd
# 6. Draft a benchmark scenario bundle directly to stdout
zqk new bundle --name storage-perf --output - | yq .
Verification & Statement Coverage Invariant
All components of the CLI New subsystem are verified against strict continuous quality invariants:
- Statement Test Coverage Floor (\(\ge 85.0\%\)): The package
cmd/zqk/newenforces a strict statement test coverage floor of \(\ge 85.0\%\) across all command builders (NewCmd,NewObjectCmd,NewCommandSpecCmd,NewBundleCmd,NewSwarmCmd), flag processors, output writers, and promotion enqueuers. Coverage is continuously measured and verified viago test -v -timeout 30s -cover ./cmd/zqk/new. - AST Quality Invariant — Zero Swallowed Errors: Error returns from all system and standard library invocations must be checked fail-closed (
if err != nilort.Fatalf). Discarding error values using the blank identifier (_ = ...) is prohibited by static analysis check-valves (zqk-vet). - AST Function Complexity Bounds: All production and unit test functions adhere to strict line count boundaries (under 100 lines per function, targeting under 80 lines) to maintain modularity, readability, and clean test separation.
- CAS Non-Contamination Invariant: Zero manual mutation of restricted system provenance fields (
created_at,created_by,updated_at,updated_by). Provenance fields are system-computed; attempts to manually supply or overwrite them during object updates fail closed. - Project Root Test Isolation: Test suites isolate their execution environments using temporary project roots bootstrapped via test environment helpers, guaranteeing hermetic verification with zero leakage into the host workspace.