CLI Documentation Management Subsystem Operational & Architectural Guide
Overview
The ZQK Documentation Management Subsystem (cmd/zqk/docman, backed by pkg/docman) provides a content-addressed, cryptographically verified documentation registration and integrity engine engineered for human pair programmers and autonomous AI agent swarms operating within the Zen Quantum Kernel.
In conventional software projects and AI agent harnesses, documentation quickly decays into an unindexed, drifted graveyard of stale Markdown files. AI agents lack reliable ways to discover relevant operational guides, while continuous autonomous refactoring silently breaks file references, contracts, and architecture runbooks without triggering build errors.
The ZQK Documentation Management Subsystem solves this through four core capabilities:
- Automated Subtree Discovery: Fast recursive scanning of markdown repositories with configurable subtrees and intelligent exclude filters.
- Metadata & Frontmatter Extraction: Automatic extraction of document titles, summaries, categories, and ontological graph links from YAML frontmatter and Markdown headings.
- Content Addressable Storage (CAS) Registration: Registering markdown files as persistent, versioned
doc_entrygraph objects within the Knowledge Kernel. - Cryptographic Drift Verification & Auto-Sealing: Continuous verification of SHA-256 hashes and byte lengths against on-disk files, detecting drift and auto-healing document signatures via fail-closed check-valves.
Subsystem Command Taxonomy
The zqk docman command group provides an integrated suite of documentation operations:
| Command | Category | Purpose | Primary Flags & Options | Example |
|---|---|---|---|---|
zqk docman register |
Registration Pipeline | Discovers markdown documents and registers them as doc_entry graph objects |
--subtrees, --shipped-only, --dry-run, --update-existing, --format, --context |
zqk docman register --shipped-only |
zqk docman verify |
Integrity & Drift | Cryptographically verifies doc_entry objects against disk and detects drift |
--ids, --subtrees, --shipped-only, --auto-seal, --strict, --format, --context |
zqk docman verify --strict |
1. zqk docman register Flags & Options
| Flag | Short | Type | Default | Description |
|---|---|---|---|---|
--dry-run |
- | bool |
false |
Inspect candidate files and show what would be registered without creating graph objects |
--shipped-only |
- | bool |
false |
Only scan and register core shipped documentation subtrees (docs/architecture, docs/best-practices, docs/onboarding) |
--subtrees |
- | []string |
docs/ |
Specific relative subtree directories to scan (comma-separated or repeated) |
--update-existing |
- | bool |
false |
Backfill missing reference fields on existing doc_entry objects to comply with schema contracts |
--format |
-f |
string |
table |
Output format: table, json, yaml, jsonl, raw, md |
--output |
-o |
string |
stdout | Write output to target file path |
--context |
- | string |
human |
Execution context profile: human, ai-agent, debug |
--quiet |
-q |
bool |
false |
Suppress non-essential informational messages |
--verbose |
-v |
bool |
false |
Enable verbose debug logging during discovery and parsing |
2. zqk docman verify Flags & Options
| Flag | Short | Type | Default | Description |
|---|---|---|---|---|
--ids |
- | []string |
all | Specific doc_entry IDs to verify (e.g. DOC-001,DOC-002) |
--subtrees |
- | []string |
all | Restrict verification to documents under specified relative subtrees |
--shipped-only |
- | bool |
false |
Only verify shipped documentation subtrees |
--auto-seal |
- | bool |
false |
Compute current SHA-256 hash and byte length on disk and update stored doc_entry hashes (heals drift) |
--strict |
- | bool |
true |
Fail closed with a non-zero exit code if any drift, unsealed, or missing documents are detected |
--format |
-f |
string |
table |
Output format: table, json, yaml, raw |
--output |
-o |
string |
stdout | Write verification report to target file path |
Core Architecture
flowchart TD
subgraph ClientPlane["Execution Plane (Human Developer & Autonomous AI Agents)"]
CLI["CLI Commands<br/>(zqk docman register / verify)"]
end
subgraph DocmanSubsystem["Docman Command Subsystem (cmd/zqk/docman)"]
EnvInit["initDocmanEnv<br/>Context, flags & project root resolution"]
RunReg["runRegister<br/>Registration orchestrator"]
RunVer["runVerify<br/>Verification & drift orchestrator"]
end
subgraph DocmanEngine["Core Documentation Engine (pkg/docman)"]
Discoverer["Discoverer<br/>Subtree traversal & exclude filtering"]
Parser["Parser<br/>YAML frontmatter & header extraction"]
Registry["Registry<br/>ID assignment & object synthesis"]
Verifier["Verifier<br/>SHA-256 & byte size validation"]
Updater["Updater<br/>Reference schema normalization"]
end
subgraph KnowledgeKernel["Knowledge Kernel Storage Layer"]
StorageProvider["ObjectStorageProvider<br/>(File CAS / Object Store)"]
DocEntries["doc_entry Graph Objects<br/>(DOC-001, DOC-002, ...)"]
end
subgraph Filesystem["Local Repository Workspace"]
MarkdownFiles["Markdown Files<br/>(docs/architecture, docs/guides, ...)"]
end
CLI --> EnvInit
EnvInit --> RunReg
EnvInit --> RunVer
RunReg --> Discoverer
RunReg --> Parser
RunReg --> Registry
RunReg --> Updater
RunVer --> Verifier
Discoverer -->|Scans| MarkdownFiles
Parser -->|Reads & Parses| MarkdownFiles
Verifier -->|Inspects & Hashes| MarkdownFiles
Registry -->|Persists| StorageProvider
Updater -->|Mutates| StorageProvider
Verifier -->|Reads & Updates| StorageProvider
StorageProvider --> DocEntries
1. Subtree Discovery & Directory Pruning
The discovery engine (docman.Discoverer) scans the repository starting from the project root:
- Default Scope: Scans all Markdown files under
docs/when no specific subtrees are configured. - Targeted Subtrees: Restricts scanning to one or more directory paths (e.g.
docs/architecture,docs/guides). - Shipped Documentation Baseline: The package defines canonical shipped documentation subtrees (
docman.ShippedInitDocSubtrees): docs/architecturedocs/best-practicesdocs/onboarding- Exclude Directories: Automatically prunes non-documentation and cache directories, including
.git,node_modules,__pycache__,.zqk, andarchive.
2. Frontmatter Parsing & Metadata Extraction
The parser (docman.Parser) extracts structured metadata from discovered files:
- YAML Frontmatter: Reads fenced YAML blocks (
---) at the top of markdown files. Extracted fields includetitle,summary,group,category, and ontological reference links. - Header Fallback: If YAML frontmatter is omitted, the parser derives the document title from the first
# Titlemarkdown heading and infers default categories based on subtree location. - Normalization: Paths are normalized to repository-relative Unix slashes (
paths.NormalizeDocEntryPathForKey) ensuring cross-platform stability across Windows, Linux, and macOS.
3. Registration Pipeline & Graph Object Synthesis
The registry (docman.Registry) transforms discovered files into versioned kernel objects:
- Deterministic ID Generation: Assigns monotonically increasing identifier keys formatted as
DOC-###(e.g.DOC-001). - Duplicate Prevention: Queries existing
doc_entryobjects in storage before creating new ones; files already registered are automatically skipped. - Initial Cryptographic Sealing: Computes the initial SHA-256 hash and byte size of the document, sealing them onto the
doc_entrypayload upon registration. - Schema Backfilling (
--update-existing): Synchronizes existingdoc_entryrecords with required reference schema arrays (goal_refs,workstream_refs,milestone_refs,requirement_refs) without modifying document content.
4. Cryptographic Drift Verification & Self-Healing
The verifier (docman.Verifier) guarantees documentation integrity:
- Double Cryptographic Anchor: Validates both the SHA-256 content hash and the document size in bytes.
- Drift Classification:
Passed: On-disk file exactly matches the stored cryptographic signature.Drifted: File has been altered on disk; hash or size does not match stored values.Unsealed:doc_entryexists in storage but lacks content hash or size metadata.Missing:doc_entryreferences a target path that does not exist on disk.- Auto-Sealing (
--auto-seal): When files are intentionally updated by developers or agents, running verify with--auto-sealre-computes hashes and sizes, updating the CAS storage and restoring pristine status. - Fail-Closed Gate (
--strict): In CI/CD pipelines and pre-commit checks, strict mode exits with an error if any drift or unsealed documents are present, preventing unverified changes from reaching production.
Operational Workflows & Recipes
Recipe 1: Initial Discovery & Shipped Documentation Seeding
Seed the Knowledge Kernel with core architectural and onboarding guides:
# Register shipped documentation entries into the kernel
zqk docman register --shipped-only
# Verify registration output
zqk object list --kind doc_entry
Recipe 2: Preflight Dry-Run Auditing
Inspect what files will be registered before committing changes to storage:
# Preview documentation registration across all subtrees
zqk docman register --dry-run
# Output formatted as JSON for automated agent ingestion
zqk docman register --dry-run -f json
Recipe 3: Targeted Subtree Scanning
Register documentation files within specific domain subtrees:
# Register architecture and operational guides only
zqk docman register --subtrees docs/architecture,docs/guides
Recipe 4: Cryptographic Integrity Verification & CI Gate
Enforce that no documentation has drifted or been deleted:
# Verify shipped documentation integrity with strict fail-closed enforcement
zqk docman verify --shipped-only --strict
# Verify specific document IDs
zqk docman verify --ids DOC-001,DOC-002
Recipe 5: Automated Drift Remediation & Sealing
Heal drifted documents after intentional content updates:
# Recompute SHA-256 signatures and seal all updated documents in storage
zqk docman verify --shipped-only --auto-seal
Verification & Statement Coverage Invariant
To ensure reliability across continuous multi-agent swarm operations, the Documentation Management subsystem is bound by strict automated testing invariants:
- >=85.0% Statement Test Coverage Floor: Every package routine (
resolveDocmanContext,resolveSubtreesAndShipped,initDocmanEnv,withDocmanEnv,NewDocmanCmd,NewRegisterCmd,NewVerifyCmd,runRegister,runVerify) is thoroughly exercised across both standard execution paths and adversarial edge conditions. - Hermetic Test Isolation: Tests execute within isolated project workspaces generated via
testkit.PrepareIsolatedTempProject, ensuring clean working directories and zero state leakage across test suites. - Storage Provider Cleanup: Every test fixture registers a cleanup routine invoking
cli.UnregisterStorageForProjectRootto prevent cached storage handles from interfering with subsequent runs. - Zero Swallowed Errors: All test cases rigorously assert error returns (
require.NoError,require.Error,assert.Contains) without discarding errors via blank identifiers. - Modular Function Boundaries: All unit test helper routines and test functions remain concise (<100 lines), maintaining readability and maintainability.