Architecture Specification: Shockwave Subscriber and Invalidation Bus Event Cascading
1. Executive Summary & Problem Statement
In distributed multi-agent systems and asynchronous execution kernels, state mutations (e.g. task completions, requirement transitions, criteria satisfactions, and scheduler job outcomes) occur continuously. Historically, dependent agents and query subsystems relied on polling loops or timed refresh intervals to detect state changes. Polling introduces two critical failure modes:
- Agent Stalls & Resource Depletion: Frequent disk and memory scans consume excessive CPU cycles and I/O bandwidth, causing latency spikes.
- Context Latency & Clock Drift: Changes are only recognized after interval expiration, stalling high-velocity autonomous agent loops.
The Knowledge Kernel addresses this by introducing the ShockwaveSubscriber, a reactive callback component that bridges asynchronous scheduler callbacks directly to the kernel's InvalidationShockwaveBus. State changes immediately trigger reactive mutation invalidations, evicting stale caches and waking up downstream listeners across the kernel graph without polling overhead.
2. Component Architecture & Event Cascading Topology
+-----------------------------------------+
| Asynchronous Callback Producer |
| (Scheduler, Subagent, Worker, Command) |
+-----------------------------------------+
|
v
+-----------------------------------------+
| MultiSubscriberDispatcher |
+-----------------------------------------+
| |
+----------------+ +----------------+
| |
v v
+-----------------------+ +-----------------------+
| FileLogSubscriber | | KernelWALSubscriber |
| (Audit Trails) | | (Lifecycle WAL) |
+-----------------------+ +-----------------------+
|
v
+-----------------------------------------+
| ShockwaveSubscriber |
| - Interface: CallbackSubscriber |
| - Nil & Boundary Event Validation |
| - Semantic Kind & ID Resolution |
| - Non-Blocking Async / Sync Dispatch |
+-----------------------------------------+
|
v
+-----------------------------------------+
| InvalidationShockwaveBus |
| (storage.GetGlobalInvalidationBus()) |
+-----------------------------------------+
| | |
v v v
+--------------+ +--------------+ +--------------+
| ParseCache | | ListCache | | ObjectIDCache|
+--------------+ +--------------+ +--------------+
| | |
+-----------------+-----------------+
|
v
[Reactive Cache Eviction & Listener Wakeup]
3. Core Contracts & Structural Implementation
3.1 Interface Compliance
ShockwaveSubscriber implements the canonical CallbackSubscriber (aliased to Subscriber) contract:
type Subscriber interface {
Name() string
Notify(ctx context.Context, entry *CallbackEntry) error
}
type CallbackSubscriber = Subscriber
3.2 Constructor & Functional Options
The subscriber can operate in synchronous mode (default) or asynchronous buffered mode:
// Synchronous initialization linking to the global or custom invalidation bus
sub := callback.NewShockwaveSubscriber(bus)
// Non-blocking asynchronous initialization with bounded ring-buffer capacity
asyncSub := callback.NewAsyncShockwaveSubscriber(bus, 256,
callback.WithLogger(logger),
callback.WithName("shockwave"),
)
defer asyncSub.Close()
4. Boundary Condition Handling & Resilience
4.1 Nil Event Rejection
To prevent undefined behavior, Notify rejects uninitialized entries and payloads early:
entry == nil-> ReturnsErrNilCallbackEntryentry.Payload == nil-> ReturnsErrNilPayload- Calling on an uninitialized receiver or bus -> Returns
errfmt.Errorf/ErrNilBus - Calling on a terminated subscriber -> Returns
ErrSubscriberClosed
4.2 Semantic Kind and ID Inference
Payloads from disparate subsystems often omit explicit object kinds. ShockwaveSubscriber resolves them deterministically:
- Identifier Resolution: Inspects
object_id, thenid, thenjob_id. - Kind Inference:
- Inspects
objects.FieldKeyKind. - If missing, callsstorage.InferKindFromID(objID)matching canonical prefixes (BLI-->backlog_item,REQ-->requirement,CRIT-->criteria,GOAL-->goal,PRI-->priority_plan,MIL-->milestone). - If ID represents a scheduler job, falls back toscheduler_job. - Otherwise defaults tounknown. - Operation Resolution: Maps
"op"toMutationOpPutorMutationOpDelete.
5. Concurrency Lifecycle & Zero-Leak Guarantees
5.1 Named Goroutine Supervision
In asynchronous mode, worker goroutines are registered via goroutinelabels.NewGoroutine:
goroutinelabels.NewGoroutine("shockwave_subscriber_worker", "async invalidation shockwave worker").StartSimple(sub.runWorker)
5.2 Graceful Termination and Drain
When Close() is invoked:
- Atomically sets the closed flag (
s.closed.CompareAndSwap(false, true)). - Cancels context signaling the worker goroutine.
- Drains remaining events in
eventChto ensure zero message loss. - Waits for
s.workerWg.Wait()to complete, guaranteeing zero channel or goroutine leaks.
6. Verification Matrix
The implementation is verified by automated test suites in cmd/zqk/callback/shockwave_subscriber_test.go:
TestShockwaveSubscriber_InterfaceComplianceAndDefaults: Asserts interface and naming invariants.TestShockwaveSubscriber_NilEventAndPayloadRejection: Verifies fail-closed rejection of nil inputs.TestShockwaveSubscriber_KindInferenceResilience: Verifies prefix-based kind deduction across ontology kinds.TestShockwaveSubscriber_MissingObjectIDFallback: Verifies fallback tojob_idandunknown.TestShockwaveSubscriber_AsyncNonBlockingDispatch: Confirms non-blocking queueing, graceful drain, and closed rejection.TestShockwaveSubscriber_ConcurrentLoadAndZeroLeaks: Stresses the subscriber under 50 concurrent goroutines with-raceenabled.TestShockwaveSubscriber_DispatcherIntegration: Proves seamless fanout when orchestrated byMultiSubscriberDispatcher.