Architecture Specification: Reactive WAL Lifecycle Callback Listener and Zero-Idle Waker
1. Overview and Problem Statement
In multi-agent and autonomous distributed operations, background processes (such as scheduled tasks, long-running test suites, compilation steps, or subagent tasks) execute out-of-band. Historically, callers and agent orchestrators resorted to polling loops (time.Sleep or periodic tickers), which either wasted CPU cycles or caused agents to yield control and enter idle wait states (waiting_for_input).
To guarantee continuous autonomous execution and zero idleness, the Knowledge Kernel implements the Reactive WAL Lifecycle Callback Listener and Zero-Idle Waker. When background scheduler jobs complete, their callback execution synthesizes an EventTypeSchedulerCallback event into the write-ahead log (lifecycle_events.wal). The Lifecycle Listener immediately processes this record and dispatches it to registered JobWakerRegistry listeners, unblocking waiting tasks reactively with sub-millisecond latency.
2. System Architecture
┌───────────────────────┐
│ Background Scheduler │
│ Execution │
└──────────┬────────────┘
│ --callback-completion "zqk callback notify"
▼
┌───────────────────────┐
│ Callback Processor │
└──────────┬────────────┘
│ MultiSubscriberDispatcher
▼
┌───────────────────────┐
│ KernelWALSubscriber │
└──────────┬────────────┘
│ Append(EventTypeSchedulerCallback)
▼
┌───────────────────────┐
│ Lifecycle Event WAL │ (.zqk/wal/lifecycle_events.wal)
└──────────┬────────────┘
│ ReplayFromCursor / Poller
▼
┌───────────────────────┐
│ Lifecycle Listener │
└──────────┬────────────┘
│ Wake(ev)
▼
┌───────────────────────┐
│ JobWakerRegistry │
└──────────┬────────────┘
│ ch <- ev (non-blocking)
▼
┌───────────────────────┐
│ Reactive Agent Task │ (Zero-Idle AwaitJob)
└───────────────────────┘
3. Core Components
3.1 EventTypeSchedulerCallback
A first-class lifecycle event type in pkg/lifecycle:
const (
EventTypeSchedulerCallback EventType = "scheduler_callback"
)
Carries the job_id in ev.ID, target status (completed or failed) in ev.ToStatus, and execution metadata in ev.Scope.
3.2 JobWakerRegistry
Thread-safe waker coordinator in pkg/lifecycle/waker.go:
Register(jobID string) (<-chan *LifecycleEvent, func()): Registers a channel for a specific job ID.RegisterWildcard() (<-chan *LifecycleEvent, func()): Registers for all scheduler callback events.Dispatch(ev *LifecycleEvent) int: Non-blocking broadcast to all matching registered wakers.AwaitJob(ctx context.Context, jobID string) (*LifecycleEvent, error): Blocks reactively until the specific job completes or the context expires, cleaning up channel registration on return.AwaitJobWithWAL(ctx, projectRoot, jobID, pollInterval): Integrates continuous WAL polling with reactive waker dispatch.
3.3 Lifecycle Listener Integration
Listener implements CallbackWaker registration:
- In
NewListener, automatically registersGetGlobalJobWakerRegistry(). - In
processEvent(), matchesEventTypeSchedulerCallbackand dispatches directly to all registeredCallbackWakerinstances.
4. Concurrency, Reliability, and Error Boundaries
- Non-Blocking Delivery: Dispatches to buffered channels using
select { case ch <- ev: default: }, preventing slow or hanging consumers from blocking the WAL replay thread. - Context and Timeout Safety:
AwaitJobstrictly respects context deadlines (context.WithTimeout), returningcontext.DeadlineExceededwithout leaking goroutines or channels. - Idempotent Cleanup: Waker unregistration functions are safe to invoke multiple times and immediately remove dead channels from the active registry.
- Race-Free Concurrency: Verified under the Go race detector (
go test -race ./pkg/lifecycle/...) across high-concurrency worker fan-out.