CLI Command Taxonomy, Operations & Extensible Architecture Guide
Comprehensive operational and architectural guide for CLI command composition, internal object processing, atomic bulk operations, lifecycle hooks, and the extensible CLI generation engine in the ZQK Knowledge Kernel.
1. Executive Architectural Summary & Command DNA
The pkg/zqkcli package implements the operational foundation for CLI object processing and internal subsystem commands within the ZQK ecosystem. It bridges human operator interactions (via interactive CLI and TUI) and autonomous AI agent operations (via headless JSON/YAML streams and Model Context Protocol tools) with Content-Addressed Storage (CAS) and the Knowledge Kernel graph membrane.
Architectural Invariants & Guarantees
- Strict Statement Test Coverage (\(\ge 85.0\%\) Invariant): All CLI command handlers, bulk mutation pathways, grouping engines, and formatting transformers enforce rigorous unit and integration test coverage.
- Atomic Bulk Operations: Multi-object mutations (
bulk create,bulk update,bulk delete) operate under transactional boundaries to prevent partial corruption and ensure deterministic rollback on failure. - Fail-Closed Scheduler Guards: High-risk write operations verify daemon and scheduler readiness unless explicitly overridden by authorized maintenance actors via
--ignore-scheduler-down. - Unified Command DNA: All commands inherit standardized flag parsing, context propagation, structured error wrapping (
pkg/errfmt), and multi-format output streaming (table, JSON, JSONL, YAML, CSV, Markdown, HTML, and JSON-RPC).
2. Command Architecture & Subsystem Taxonomy
The CLI command hierarchy is organized into cohesive operational categories:
zqk internal
├── list # List internal and built-in objects with filters, grouping, and pagination
├── get # Retrieve single or multiple objects by ID or spec definition
├── create # Create internal objects from file or inline YAML/field inputs
├── update # Mutate fields or transition status on existing objects
├── delete # Purge objects with cascade or GhostRef unlink guards
├── count # Fast aggregate counting with parallel workers and group-by
├── fields # Inspect schema field specs, types, and descriptions
├── process # Execute declarative multi-operation reference files
└── bulk # High-throughput batch operations
├── create # Batch ingestion from YAML object arrays
├── update # Batch field mutation with transactional commit
├── get # Multi-ID batch retrieval
└── delete # High-throughput multi-object erasure with cascade
Command Roles & Responsibilities
| Command | Primary Use Case | Output Formats | Concurrency / Safety Guards |
|---|---|---|---|
list |
Inspecting cataloged internal objects across storage and memory | Table, JSON, YAML, JSONL, CSV, Markdown | Pagination limits, memory cap, source type filtering |
get |
Fetching authoritative state of internal objects, specs, and lifecycles | Table, JSON, YAML, Raw | Fallback to disk specs, schema validation |
create |
Originating single internal objects | Table, JSON, YAML | ID validation, scheduler guard, schema compliance |
update |
Applying field-level patches or lifecycle status changes | Table, JSON, YAML | Dry-run preview, write-behind CAS flush |
delete |
Removing internal entities from storage | Table, JSON, YAML | GhostRef refusal, built-in object warnings |
count |
Rapid quantitative aggregation across object kinds | Table, JSON, YAML | Bounded worker pools, parallel kind execution |
bulk |
High-throughput batch mutations | Table, JSON, YAML | Transactional chunking (500 items/chunk), rollback safety |
process |
Replaying complex multi-step recipe files | Table, JSON, YAML | Per-operation error handling, continue-on-error |
fields |
Discovering schema fields, descriptions, and rules | Table, JSON, YAML | Field registry caching, dynamic completion |
3. High-Throughput Bulk Operations & Transactional Semantics
The bulk command family provides high-performance data processing pipelines designed for batch migration, graph bootstrapping, and scheduled retention sweeps.
Batch Chunking & Memory Ceilings
To adhere to memory hygiene policies, bulk mutations chunk operations into discrete atomic transactions:
- Chunk Size Limit: Operations are processed in batches of 500 items.
- Rollback Isolation: If any operation within an atomic chunk fails, the entire chunk rolls back via
ObjectTransaction.Rollback(). - Memory Footprint Optimization: Results accumulate lightweight identifiers (
map[string]any{"id": ID}) rather than full object states during bulk passes, eliminating heap spikes.
Incoming Update Items [1..N]
│
▼
┌─────────────────────────────────┐
│ Chunk Slicer (Max 500 items) │
└─────────────────────────────────┘
│
▼
┌─────────────────────────────────┐
│ BeginTransaction(ctx) │
├─────────────────────────────────┤
│ • Apply op[0] │
│ • Apply op[1] │
│ • ... │
│ • Apply op[499] │
└─────────────────────────────────┘
│ │
(Success) (Failure)
│ │
▼ ▼
┌─────────┐ ┌──────────┐
│ Commit │ │ Rollback │
└─────────┘ └──────────┘
Dry-Run Verification
All mutating commands (bulk create, bulk update, create, update, delete) support the --dry-run flag. In dry-run mode:
- Schema rules and ID formats are strictly evaluated.
- Pre-mutation state is compared against requested mutations to render a unified diff.
- Storage writes, WAL appends, and cache invalidations are completely bypassed.
4. Lifecycle Hooks & Scheduler Guard Enforcement
Scheduler Guard Architecture
The scheduler guard (pkg/zqkcli/scheduler_guard.go) acts as a check-valve to prevent desynchronization between persistent storage and background scheduler worker loops:
- Pre-Execution Check: Mutating commands check if the daemon scheduler is operational.
- Lockdown Refusal: If the scheduler is in a degraded or locked-down state, mutating commands fail closed to protect consistency.
- Emergency Override: Operators performing offline maintenance can pass
--ignore-scheduler-downto explicitly bypass the guard.
Fail-Closed Delete & GhostRef Sentinel
Deleting objects in a relational graph carries the danger of dangling references (GhostRefs). The delete command enforces fail-closed safety:
- Refusal Without Flags: Running
zqk internal delete <id>without flags is immediately refused. - Explicit Invariant Resolution: Operators must specify either
--cascade(recursively deleting dependents) or--unlink-references(cleaning incoming edge references before erasure). Both flags cannot be combined. - Built-in Object Warning: Deleting entities with
built_in: truemetadata emits an explicit operator warning prior to execution.
5. Extensible CLI Example Generation Engine
The CLIExampleGenerator (pkg/zqkcli/cli_example_generator.go) dynamically compiles verifiable CLI recipes from live schema definitions and object specifications.
Capabilities
- Command Syntax Synthesis: Automatically generates executable bash/zsh command examples for all standard ontological kinds.
- Synthetic Fixture Ingestion: Employs field generators to synthesize valid, schema-compliant sample data for tutorials and developer documentation.
- Bootstrapping Scripts: Assembles multi-object graph creation scripts ensuring parent-child dependencies (e.g., mission -> goal -> requirement) are sequenced in valid topological order.
6. Coverage & Quality Assurance Invariants
In accordance with kernel verification policies, pkg/zqkcli maintains continuous automated verification:
| Metric / Dimension | Requirement | Verification Strategy |
|---|---|---|
| Statement Coverage | \(\ge 85.0\%\) Floor | In-process integration suites exercising all commands, error paths, and formatters |
| AST Concurrency | Named goroutines | Strict enforcement of goroutinelabels across all parallel workers |
| Resource Hygiene | Zero descriptor leaks | Strict cleanup of all test temp directories and file handles |
| Payload Integrity | Zero leaked IDs | Verified against scripts/open-core/check-public-release-payload.sh |
7. Operational Recipes & Command Examples
1. Batch Creating Internal Objects
# Ingest multiple kind synonyms from a YAML array
zqk internal bulk create kind_synonym --file /tmp/synonyms.yaml
# Perform dry-run preview before creating
zqk internal bulk create kind_synonym --file /tmp/synonyms.yaml --dry-run
2. Updating Objects with Dynamic Fields
# Update title and status via inline flags
zqk internal update OBJ-001 --field title="Updated Title" --field status=active
# Update using a structured YAML delta
zqk internal update OBJ-001 --data "title: New Title\npriority: high\n"
3. High-Throughput Aggregations
# Count objects grouped by status
zqk internal count audit_event --group-by status
# Count multiple kinds in parallel with JSON output
zqk internal count audit_event,change_journal_entry --format json
4. Executing Multi-Step Reference Operations
# Process a declarative operations manifest
zqk internal process --file /path/to/manifest.yaml
# Preview operations execution in dry-run mode
zqk internal process --file /path/to/manifest.yaml --dry-run