Ambient Compute, EventMesh & Anticipatory Intent Subsystem (pkg/ambience)
1. Executive Summary & Architectural Overview
The Ambient Compute, EventMesh & Anticipatory Intent Subsystem (pkg/ambience) provides low-latency, event-driven perception and proactive intent inference across the ZQK Knowledge Kernel runtime. By capturing ambient development signalsβsuch as filesystem mutations, editor focus shifts, and local test execution failuresβthe subsystem enables autonomous AI agents and kernel daemons to anticipate engineering intent, pre-warm caches, and invoke predictive verification workflows before explicit human command submission.
Rather than polling storage layers or executing expensive full-repository scans, pkg/ambience implements a reactive three-tier pipeline:
- In-Memory EventMesh Fabric (
EventMesh): High-throughput, thread-safe publish-subscribe backbone routing discrete ambient signals with zero external message broker dependencies. - Signal Aggregator (
SignalAggregator): Time-windowed and count-bounded aggregation engine smoothing transient event bursts (such as multi-file editor saves or build artifact churn) into high-confidence signal batches. - Anticipatory Heuristic Engine (
FSEventsEngine/AnticipatoryEngine): Lightweight, deterministic intent prediction assigning actionable operations (run_tests,build,mod_tidy,edit) and normalized confidence scores to ambient events.
flowchart TD
subgraph SignalSources["Ambient Signal Sources"]
FS["File System Watcher\n(fsevents / inotify)"]
UI["Editor / IDE Focus\n(focus_changed)"]
Test["Test Runner / Sentinel\n(test_failed)"]
end
subgraph MeshFabric["pkg/ambience EventMesh"]
Mesh["InMemoryEventMesh\n(sync.RWMutex Router)"]
Audit["Audit Event Sourcing\n(ambient_audit goroutine)"]
end
subgraph ConsumerEngines["Anticipatory Consumer Engines"]
Agg["SignalAggregator\n(Sliding Window & Flush Count)"]
FSEng["FSEventsEngine\n(PredictIntent Heuristics)"]
end
subgraph Outcomes["Predictive Actions & Downstream Swarms"]
TestPred["Action: run_tests\n(Confidence: 0.8)"]
BuildPred["Action: build\n(Confidence: 0.7)"]
ModPred["Action: mod_tidy\n(Confidence: 0.9)"]
BatchFeed["Aggregated Signal Feed\n(Aggregated Channel)"]
end
FS -->|EventFileModified| Mesh
UI -->|EventFocusChanged| Mesh
Test -->|EventTestFailed| Mesh
Mesh -->|Audit Enabled| Audit
Mesh -->|Subscribe| Agg
Mesh -->|Subscribe| FSEng
Agg -->|Window/Count Flush| BatchFeed
FSEng -->|Predict: _test.go| TestPred
FSEng -->|Predict: .go| BuildPred
FSEng -->|Predict: go.mod/sum| ModPred
2. Architectural Components & Topology
2.1 The Event Model (AmbientEvent & EventType)
Every ambient signal captured in the runtime is normalized into a strongly-typed AmbientEvent envelope:
| Field | Type | Description |
|---|---|---|
ID |
string |
Unique identifier generated for the discrete ambient signal. |
Type |
EventType |
Canonical event type (file_modified, focus_changed, test_failed). |
URI |
string |
Target resource path or URI associated with the signal (e.g., pkg/kernel/kernel.go). |
Timestamp |
int64 |
Unix epoch timestamp in seconds recording signal origination. |
Payload |
[]byte |
Optional structured JSON or raw byte telemetry payload describing mutation context. |
Canonical event types are strongly typed string constants:
type EventType string
const (
EventFileModified EventType = "file_modified"
EventFocusChanged EventType = "focus_changed"
EventTestFailed EventType = "test_failed"
)
2.2 In-Memory EventMesh Fabric (InMemoryEventMesh)
InMemoryEventMesh implements the EventMesh contract:
type EventMesh interface {
Publish(ctx context.Context, event AmbientEvent) error
Subscribe(ctx context.Context, types []EventType) (<-chan AmbientEvent, error)
}
Key architectural mechanics:
- Reader-Writer Lock Protection: Internal subscriber topologies (
map[EventType][]chan AmbientEvent) are guarded bysync.RWMutex. - Fan-Out Delivery:
Publishcopies the subscriber channel slice under read-lock, then multiplexes events to all registered channels using context-guarded select operations. - Managed Subscription Teardown: Upon cancellation of the subscriber context, an asynchronous goroutine safely removes the subscription channel from all registered type slices, preventing stale subscriber leaks.
- Audit Event Sourcing: When event sourcing is enabled via
EnableEventSourcing(projectRoot, secCtx), ambient events trigger an asynchronous, namedambient_auditgoroutine that classifies and records persistent audit records in storage without blocking delivery pipelines. - Deterministic Teardown (
WaitForAudits): Exposes an explicit wait-group barrier allowing callers and test harnesses to drain all in-flight audit emissions prior to workspace cleanup.
2.3 Signal Aggregator (SignalAggregator)
High-frequency signals (e.g., typing bursts, rapid test re-runs) frequently generate noise. The SignalAggregator buffers events across a configurable sliding duration (window) and batch threshold (flushCount):
type SignalAggregator struct {
mesh EventMesh
mu sync.Mutex
events []AmbientEvent
window time.Duration
flushCount int
cancel context.CancelFunc
running bool
flushCh chan []AmbientEvent
}
- Dual-Trigger Flush:
- Count-Triggered: When accumulated events reach
flushCount, the buffer flushes immediately toflushCh. - Window-Triggered: If
windowelapses with buffered events, the timer loop flushes accumulated signals. - Non-Blocking Channel Ingress: Flushes to
flushCh(buffer size 10) use non-blockingselectwithdefault:drops to prevent stalling the aggregator loop if downstream consumers lag.
2.4 Anticipatory Intent Engine (FSEventsEngine)
FSEventsEngine implements the AnticipatoryEngine contract:
type Intent struct {
Action string
Target string
Confidence float64
}
type AnticipatoryEngine interface {
Start(ctx context.Context) error
Stop() error
PredictIntent(event AmbientEvent) (Intent, error)
}
- Atomic Lifecycle: Uses
sync/atomic.Boolto guarantee atomic, idempotent transitions acrossStartandStop. - Heuristic Pattern Inference:
*_test.go→ Action:run_tests(Confidence: 0.8)*.go→ Action:build(Confidence: 0.7)go.mod,go.sum→ Action:mod_tidy(Confidence: 0.9)- Generic files → Action:
edit(Confidence: 0.5) - Non-file events → Action:
""(Confidence: 0.0)
3. Core Invariants & Boundaries
3.1 Concurrency & Named Goroutines Invariant
All background processing must use explicitly named goroutines via goroutinelabels.NewGoroutine:
ambience.aggregator_loop: Goroutine executing the time/count aggregation ticker.ambience.fsevents_loop: Goroutine consuming file events and triggering intent prediction.ambient_audit: Goroutine dispatching audit entries to persistent storage.
Unlabeled go func() invocations are strictly prohibited to maintain trace hygiene, leak profiling, and supervision compliance.
3.2 Context Cancellation & Bounded Blocking Invariant
Every channel send operation during Publish or loop execution is bounded by context cancellation:
select {
case <-ctx.Done():
return ctx.Err()
case ch <- event:
}
If a subscriber channel buffer saturates and the context expires, the mesh fails cleanly without indefinite deadlocks.
3.3 Security Context & Audit Decoupling Invariant
Event sourcing requires a non-nil SecurityContext and non-empty projectRoot. Audit events are created using storage.CreateAuditEventWithBuilder in a background worker, ensuring storage I/O latency never blocks in-memory event dispatch.
3.4 Idempotent Lifecycle Invariant
Both SignalAggregator and FSEventsEngine guarantee safe, idempotent execution of Start and Stop. Redundant Start() calls return nil without launching duplicate consumer goroutines; redundant Stop() calls return nil without panic or null dereference.
4. Operational Runbook & Verification
4.1 Verification via The Three-Fold Proof
Subsystem integrity is continuously validated through the canonical Three-Fold Proof:
- Static Floor Invariant: The package statement test coverage must achieve ≥ 85.0% across all files (
aggregator.go,eventmesh.go,fsevents_engine.go,anticipatory.go), currently reaching 100.0%. - Operational Dynamic Proof: Validated by integration tests exercising multi-subscriber fan-out, sliding window flushes, count threshold bursts, and event sourcing classification.
- Negative Adversarial Boundary: Confirmed by error injection testing subscriber registration failures, full channel buffer saturation timeouts, empty URI payloads, and non-JSON payloads.
4.2 Verifying Subsystem Coverage
Execute the test suite with statement coverage profiling:
go test -coverprofile=coverage.out -v ./pkg/ambience/...
go tool cover -func=coverage.out
Expected output:
github.com/zqk-os/zqk/pkg/ambience/aggregator.go:24: NewSignalAggregator 100.0%
github.com/zqk-os/zqk/pkg/ambience/aggregator.go:33: Start 100.0%
github.com/zqk-os/zqk/pkg/ambience/aggregator.go:58: loop 100.0%
github.com/zqk-os/zqk/pkg/ambience/aggregator.go:91: flush 100.0%
github.com/zqk-os/zqk/pkg/ambience/aggregator.go:98: Stop 100.0%
github.com/zqk-os/zqk/pkg/ambience/aggregator.go:108: PredictIntent 100.0%
github.com/zqk-os/zqk/pkg/ambience/aggregator.go:114: Aggregated 100.0%
github.com/zqk-os/zqk/pkg/ambience/eventmesh.go:49: NewInMemoryEventMesh 100.0%
github.com/zqk-os/zqk/pkg/ambience/eventmesh.go:56: EnableEventSourcing 100.0%
github.com/zqk-os/zqk/pkg/ambience/eventmesh.go:64: Publish 100.0%
github.com/zqk-os/zqk/pkg/ambience/eventmesh.go:149: Subscribe 100.0%
github.com/zqk-os/zqk/pkg/ambience/eventmesh.go:180: WaitForAudits 100.0%
github.com/zqk-os/zqk/pkg/ambience/fsevents_engine.go:19: NewFSEventsEngine 100.0%
github.com/zqk-os/zqk/pkg/ambience/fsevents_engine.go:26: Start 100.0%
github.com/zqk-os/zqk/pkg/ambience/fsevents_engine.go:46: loop 100.0%
github.com/zqk-os/zqk/pkg/ambience/fsevents_engine.go:60: Stop 100.0%
github.com/zqk-os/zqk/pkg/ambience/fsevents_engine.go:68: PredictIntent 100.0%
total: (statements) 100.0%
4.3 Race Detector Gate
Verify thread safety across concurrent publishers and subscribers:
go test -race -v ./pkg/ambience/...
5. Configuration & Environment
| Parameter | Type | Default | Description |
|---|---|---|---|
window |
time.Duration |
Dynamic (e.g. 50ms) |
Sliding duration window for SignalAggregator ticker. |
flushCount |
int |
Dynamic (e.g. 10) |
Maximum events accumulated before immediate aggregation flush. |
channelBufferSize |
int |
100 |
In-memory subscriber channel buffer size in InMemoryEventMesh. |
flushChBufferSize |
int |
10 |
Aggregated event slice channel buffer size in SignalAggregator. |
6. Anti-Patterns & Operational Hazards
6.1 Blocking Subscribers on Publish
- Anti-Pattern: Using unbuffered subscriber channels or performing slow synchronous operations directly inside an event subscriber callback.
- Hazard: Blocks
EventMesh.Publishloop, backing up other consumers and triggering context deadline timeouts. - Remedy: All mesh subscriptions use buffered channels (
make(chan AmbientEvent, 100)). Heavy processing must be dispatched to isolated background worker pools.
6.2 Arbitrary Sleep Teardown in Tests
- Anti-Pattern: Using
time.Sleep(...)to wait for asynchronous audit event storage emissions before cleaning up temporary directories. - Hazard: Causes flaky test failures in fast or heavily loaded CI environments due to file write races against directory deletion.
- Remedy: Always invoke
mesh.WaitForAudits()followed bystorage.ScrubProjectRootForTempCleanup(...)before exiting test scopes.
6.3 Unbounded Intent Speculation
- Anti-Pattern: Triggering full build compilation or broad test execution suites synchronously on every single raw keystroke.
- Hazard: CPU saturation, compiler lock contention, and thermal throttling.
- Remedy: Always route raw signals through
SignalAggregatorprior to triggering heavier anticipatory pipelines.