Predicate DSL & AST Evaluation Engine Subsystem Architectural & Operational Guide
Overview
The Predicate DSL & AST Evaluation Engine Subsystem (pkg/predicate) provides the declarative expression grammar, abstract syntax tree (AST) validation pipeline, legacy prose compilation membrane, and lexical salience analysis engine powering the ZQK Knowledge Kernel.
In the ZQK architecture, state transitions, lifecycle stage advancements, and verification-driven specification (VDS) gates are governed by verifiable invariants rather than transient prompt assumptions. The Predicate DSL provides a deterministic, machine-checkable language for expressing structural constraints, entity relationships, metrics, filesystem invariants, and lexical cohesion across kernel objects.
Subsystem Architecture & Component Topology
The subsystem operates as a layered verification pipeline:
flowchart TD
InputExpr["Raw Expression (DSL or Legacy Prose)"] --> PreCompiler["Legacy Prose Overlay Compiler (CompilePrecondition)"]
PreCompiler -->|Canonical DSL String| Splitter["Compound Splitter (SplitPredicates)"]
InputExpr -->|Direct DSL Expression| Splitter
subgraph PredicateEngine["Predicate DSL Subsystem (pkg/predicate)"]
Splitter -->|Individual Clauses| Validator["Syntax Validator (ValidatePredicateSyntax)"]
Validator --> RegistryCheck{"Predicate Category"}
RegistryCheck -->|Zero-Arg Gates| GateRegistry["Standalone Gate Registry (StandaloneGatePredicates)"]
RegistryCheck -->|Field & Entity| FieldValidator["Field Invariant Validator (validateFieldPredicate)"]
RegistryCheck -->|Links & Patterns| MatchValidator["Link & Pattern Validator (validateLinkAndMatchPredicate)"]
RegistryCheck -->|AST & Metrics| MetricValidator["AST & Metric Validator (validateAstAndNumericPredicate)"]
MetricValidator --> MetricParser["Metric Grammar Parser (ParseMetricPredicate)"]
end
subgraph SalienceEngine["Lexical Salience Engine (salience.go)"]
RawText["Text Content (Title / Body)"] --> Tokenizer["Unicode Tokenizer (Tokenize)"]
Tokenizer --> Filter["Stop-Word Elimination"]
Filter --> Stemmer["Morphological Stemmer (Stem)"]
Stemmer --> TFIDF["Sublinear TF Scorer (ExtractSalientStems)"]
TFIDF --> Cohesion["Cohesion Prover (VerifyTitleBodyCohesion)"]
end
PredicateEngine --> DownstreamEval["Evaluation Runtime (pkg/vds & pkg/validation)"]
Component Breakdown
| Component | Source File | Core Responsibility |
|---|---|---|
| DSL Syntax Validator | dsl.go |
Validates clause structure, identifiers, field paths, regexes, and operators. |
| Compound Splitter | dsl.go |
Splits compound expressions by ; or , while preserving quotes and nested lists. |
| Precondition Compiler | dsl.go |
Deterministically maps natural language preconditions into canonical DSL clauses. |
| Metric Parser | dsl.go |
Parses numerical metric assertions (name, op, threshold). |
| Lexical Tokenizer & Stemmer | salience.go |
Extracts words, eliminates stop-words, and applies Porter-derived stemming. |
| Salience & Cohesion Prover | salience.go |
Computes Szymkiewicz-Simpson overlap and verifies semantic cohesion. |
DSL Grammar Specification
The Predicate DSL conforms to a formal grammar designed for safe, deterministic AST evaluation.
Formal EBNF Grammar
Expression ::= CompoundPredicate ;
CompoundPredicate ::= SinglePredicate ( ( ";" | "," ) SinglePredicate )* ;
SinglePredicate ::= StandaloneGate
| ParameterizedPredicate ;
StandaloneGate ::= Identifier ;
ParameterizedPredicate ::= FieldPredicate
| LinkPredicate
| MatchPredicate
| MetricPredicate
| AstPredicate
| CountPredicate ;
FieldPredicate ::= "object_exists" ":" Identifier
| "field_nonempty" ":" [ Identifier ":" ] Identifier
| "field_cleared" ":" [ Identifier ":" ] Identifier
| "any_nonempty" ":" IdentifierList
| "role_is" ":" Identifier
| "active_ref" ":" Identifier ;
LinkPredicate ::= "link_back" ":" Identifier ":" Identifier ;
MatchPredicate ::= "field_matches" ":" Identifier ":" RegexPattern
| "path_exists" ":" FilePath
| "content_hash_matches" ":" FilePath
| "content_size_positive" ":" FilePath ;
MetricPredicate ::= "query_metric" ":" Identifier Operator Number ;
AstPredicate ::= "ast_semantic_match" ":" FilePath ":" Constraint [ ":" Identifier ] ;
CountPredicate ::= "at_least" ":" NonNegativeInteger ":" Identifier
| "title_body_cohesion" ":" NonNegativeInteger ;
Operator ::= "==" | "!=" | "<=" | "<" | ">=" | ">" ;
IdentifierList ::= Identifier ( ( "," | ":" ) Identifier )* ;
Identifier ::= [a-zA-Z0-9_-]+ ;
FilePath ::= [a-zA-Z0-9_\-\/\.]+ ;
Operator Precedence Rules
- Parentheses & Enclosures: Quoted strings (
'...',"...",`...`) escape delimiters (semicolons and commas). - Compound Conjunction: Delimiters (
;and,) bind clauses sequentially with logicalANDsemantics. - Disjunction in Lists:
any_nonemptyaccepts comma- or colon-separated fields with logicalORsemantics. - Relational Operators: In
query_metric, relational operators (==,!=,<=,<,>=,>) bind metric names to float64 constants.
Supported Predicate Clauses
1. Canonical Zero-Argument Standalone Gates
The subsystem registers 31 canonical gates evaluated without arguments:
| Gate Identifier | Description |
|---|---|
standard_checks_pass |
Standard baseline hygiene checks succeed. |
tests_ok_per_customization |
Custom test command suite passes. |
lint_ok_per_customization |
Custom linter passes with zero violations. |
criteria_linked_or_acceptance_present |
Entity links to valid criteria or defines acceptance considerations. |
git_diff_nonempty_or_waiver |
Working tree exhibits git modifications or explicit waiver. |
smoke_or_integration_evidence_present |
Automated test or integration execution evidence recorded. |
ci_required_checks_green_or_na |
CI check suites report green status. |
security_gate_ok_or_na |
Security scanner passes or marked not applicable. |
performance_gate_ok_or_na |
Performance benchmark passes or marked not applicable. |
publish_ack_present_if_public |
Operator acknowledgment recorded for public release. |
path_exists |
Target document exists on filesystem. |
content_hash_matches |
Cryptographic SHA-256 matches sealed document. |
content_size_positive |
Target file length is greater than zero bytes. |
shovel_ready |
Priority plan has met all planning preconditions. |
tdd_test_red_phase |
Linked criteria bound to active test case references. |
criteria_active_test_case |
Criteria verified by at least one active test case. |
ready_backlog_references_plan |
At least one ready backlog item references active plan. |
linked_backlog_ready_or_later |
All linked backlog items are in ready or later status. |
linked_backlog_all_terminal |
All linked backlog items have reached terminal state. |
no_linked_backlog_in_progress_or_complete |
Zero linked backlog items are active or complete. |
workflow_constraints_if_set |
Declared workflow state constraints satisfied. |
priority_plan_validated |
Plan has been validated against active governance goals. |
team_or_persona_dispatch_refs |
Work item assigned to designated persona or team. |
git_mutation_evidence_present |
Git commit hash recorded in closure evidence. |
branch_is_ancestor_of_trunk |
Integration branch descends from origin trunk. |
machine_checkable_closure_evidence |
Machine-verifiable scheduler fingerprint recorded. |
linked_criteria_validated_or_complete |
All linked criteria items are verified or complete. |
priority_plan_archived_when_set |
Referenced plan archived when linked. |
priority_plan_execution_facing |
Plan is currently active or in-progress. |
work_done |
Work execution completed. |
title_body_cohesion |
Title and body share salient semantic keywords. |
2. Parameterized Predicates
object_exists:<id>: Proves target object ID exists within kernel CAS.field_nonempty:[<id>:]<field>: Proves field is present, non-nil, and non-empty.field_cleared:[<id>:]<field>: Proves field is absent, nil, or empty string.any_nonempty:<field1>,<field2>: Proves at least one field from list is non-empty.role_is:<role>: Proves entity role matches specified identifier.active_ref:<ref_field>: Proves referenced relationship targets active entity.link_back:<subject_field>:<target_field>: Proves bidirectional graph edge continuity.field_matches:<field>:<regex>: Proves field string matches regular expression.query_metric:<name><op><value>: Evaluates numeric metrics (coverage>=85).ast_semantic_match:<path>:<constraint>[:<symbol>]: Inspects Go AST for symbol presence/absence or panic freedom.at_least:<count>:<field>: Asserts collection field contains at leastcountelements.
Evaluation Pipeline & Truth Tables
sequenceDiagram
participant Caller as Runtime Caller
participant Splitter as SplitPredicates
participant Validator as ValidatePredicateSyntax
participant Eval as Evaluator Engine
Caller->>Splitter: Split compound expression
Splitter-->>Caller: []string predicates
loop For each predicate
Caller->>Validator: Validate single syntax
Validator-->>Caller: nil (Valid) / error
Caller->>Eval: Evaluate against Object / Context
alt Clause Evaluation Fails
Eval-->>Caller: Result{OK: false, Detail: "..."}
Note over Caller: Short-circuit termination
else Clause Evaluation Passes
Eval-->>Caller: Result{OK: true}
end
end
Caller-->>Caller: All predicates satisfied (OK: true)
Truth Table Evaluation Matrix
| Clause Type | Expression | Object State | Result |
|---|---|---|---|
field_nonempty |
field_nonempty:title |
{"title": "Valid Title"} |
true |
field_nonempty |
field_nonempty:title |
{"title": " "} |
false |
field_nonempty |
field_nonempty:title |
{} (key absent) |
false |
field_cleared |
field_cleared:plan_ref |
{} (key absent) |
true |
field_cleared |
field_cleared:plan_ref |
{"plan_ref": ""} |
true |
field_cleared |
field_cleared:plan_ref |
{"plan_ref": "REF-1"} |
false |
query_metric |
query_metric:coverage>=85 |
{"coverage": 92.5} |
true |
query_metric |
query_metric:coverage>=85 |
{"coverage": 84.9} |
false |
any_nonempty |
any_nonempty:a,b |
{"a": "val", "b": ""} |
true |
any_nonempty |
any_nonempty:a,b |
{"a": "", "b": ""} |
false |
Lexical Salience & Cohesion Engine
The salience engine (salience.go) prevents superficial title-body drift in kernel objects.
Algorithmic Pipeline
- Tokenization: Extracts alphanumeric tokens and preserves underscores.
- Stop-Word Removal: Strips English grammatical terms and generic process words (
ensure,must,verify,check). - Morphological Stemming: Normalizes word forms using a 3-step Porter-style algorithm:
- Suffix removal for plurals and past tenses (
sses,ies,eed,ing). - Derivational suffix normalization (ational,tional,ation,tion,izer,alism,ality,ment,ness,able,ible). - Trailing vowel cleanup. -
Sublinear Term Weighting: Computes stem salience via:
\[\text{Score}(t) = 1 + \ln(\text{count}(t))\] -
SzymkiewiczβSimpson Overlap: Measures overlap coefficient between title stems and salient body stems:
\[\text{Overlap} = \frac{|\text{Stems}_{\text{title}} \cap \text{Stems}_{\text{body}}|}{\min(|\text{Stems}_{\text{title}}|, 1)}\]
Concurrency & Thread-Safety Model
The predicate subsystem is engineered for high-concurrency evaluation across swarm sessions:
- Stateless Validation:
ValidatePredicateSyntax,SplitPredicates,CompilePrecondition, andParseMetricPredicateare purely functional with zero shared mutable state. - Immutable Registries: Standalone gate maps and alias lookup tables are populated during package
init()and remain read-only. - Managed Concurrency: In multi-threaded validation loops, concurrent workers are launched using
goroutinelabels.NewGoroutineto maintain label identity and budget tracking. - Race Freedom: Verified under the Go race detector (
-race) with parallel goroutines performing simultaneous compile, validate, and salience operations.
Verification & Coverage Invariant
Statement test coverage for pkg/predicate is governed by a strict static floor invariant:
Current coverage achieved across the package: 96.3% of statements.
Verification Command
To execute the test suite and verify statement coverage:
ZQK_ALLOW_FOREGROUND_GO_TEST=1 go test -v -timeout 30s -cover ./pkg/predicate