Zero-Copy Shared-Memory IPC and Vectorized Batch Callback Delivery
Technical Specification:
TSP-SHM-IPC-VECTORIZED-001
Governing Policy:POL-CONCURRENCY-SHM-001
Status: Implemented & Verified
Target Audience: Core Systems Craftsmen, Event Bus Architects, IPC & Daemon Engineers
1. Executive Summary & Core Mandate
In high-throughput agent swarms and continuous autonomous loops, event dispatch systems face critical performance bottlenecks:
- Serialization Overhead & Heap Pressure: Continually allocating and marshaling individual callback payloads across process or goroutine boundaries induces substantial garbage collector (GC) pauses and memory churn.
- Lock Contention Under Heavy Concurrency: Mutex-synchronized queue dispatchers suffer from lock convoying when multiple producer and consumer threads access the event stream simultaneously.
- False Sharing Across Cache Lines: Multiple CPU cores updating adjacent sequence counters on the same hardware cache line trigger cache line bouncing and degrade bus throughput.
To eliminate these bottlenecks, ZQK deploys a Zero-Copy Shared-Memory IPC Ring Buffer and a Vectorized Batch Callback Subscriber. This architecture provides:
- Lock-Free MPMC Operation: Bounded circular ring buffer with Dmitry Vyukov turn-based atomic sequence coordination.
- Hardware Cache Line Isolation: 64-byte padded head and tail sequence indices (
atomic.Uint64with[8]uint64padding) eliminating false sharing. - True Zero-Copy Slot Reservation: Direct memory-mapped slice reservation (
Reserve) with steady-state 0.00 heap allocations per operation. - Adaptive Vectorized Batching: Dual-triggered batch dispatching (batch capacity threshold or sub-millisecond ticker flush) delivering amortized, zero-allocation callback streaming.
- Fail-Closed Segment Integrity: Cryptographic IEEE CRC32 checksum verification, magic byte validation, and strict file truncation defense.
2. Memory-Mapped Ring Buffer Layout & Binary Topology
The shared-memory ring buffer segment file uses a deterministic, cache-aligned binary layout supporting both cross-process POSIX memory mapping (mmap) and portable in-memory byte buffers:
flowchart TD
subgraph SegFile["Shared Memory Segment Binary Layout"]
H["Segment Header (Offset 0..63, 64 Bytes)<br/>Magic ('ZQKSHM01') | Version (uint32) | HeaderSize (64) | Capacity | SlotPayloadSize | SlotSize | CRC32 Checksum"]
HeadArea["Head Sequence Area (Offset 64..127, 64 Bytes)<br/>Consumer Sequence (uint64) + 56-byte Cache Line Padding"]
TailArea["Tail Sequence Area (Offset 128..191, 64 Bytes)<br/>Producer Sequence (uint64) + 56-byte Cache Line Padding"]
subgraph SlotsArea["Contiguous Slots Area (Offset 192..End)"]
S0["Slot 0 (64-byte aligned)<br/>Turn (uint64) | Len (uint32) | Res (uint32) | Payload Bytes"]
S1["Slot 1 (64-byte aligned)<br/>Turn (uint64) | Len (uint32) | Res (uint32) | Payload Bytes"]
SN["Slot N-1 (64-byte aligned)<br/>Turn (uint64) | Len (uint32) | Res (uint32) | Payload Bytes"]
end
end
H --> HeadArea --> TailArea --> SlotsArea
2.1 Segment Header Specification (64 Bytes)
| Offset Range | Field Name | Data Type | Description |
|---|---|---|---|
0x00 - 0x07 |
Magic |
[8]byte |
Fixed segment magic bytes (ZQKSHM01) |
0x08 - 0x0B |
Version |
uint32 |
Binary layout version (1) |
0x0C - 0x0F |
HeaderSize |
uint32 |
Header byte length (64) |
0x10 - 0x13 |
SlotCapacity |
uint32 |
Power-of-two slot count (e.g. 1024) |
0x14 - 0x17 |
SlotPayloadSize |
uint32 |
Maximum payload bytes per slot (e.g. 4096) |
0x18 - 0x1B |
SlotSize |
uint32 |
Total 64-byte aligned slot size (SlotHeaderSize + SlotPayloadSize) |
0x1C - 0x1F |
Flags |
uint32 |
Reserved segment flags (0x00000000) |
0x20 - 0x23 |
Checksum |
uint32 |
IEEE CRC32 checksum computed over bytes 0x00..0x1F |
0x24 - 0x3F |
Reserved |
[28]byte |
Zero padding ensuring 64-byte alignment |
2.2 Cache Line Isolation Invariant
Modern x86_64 and ARM64 CPUs utilize 64-byte cache lines. When two CPU cores concurrently mutate distinct atomic variables residing within the same 64-byte line, cache line invalidation cycles (false sharing) degrade memory bus bandwidth by up to 10x.
To enforce hardware isolation:
Head Sequenceresides at offset64and is padded with56bytes to offset128.Tail Sequenceresides at offset128and is padded with56bytes to offset192.- Each discrete slot is padded to a multiple of 64 bytes (
(16 + slotPayloadSize + 63) &^ 63), ensuring slot turn counters never share a cache line with adjacent slots.
3. Lock-Free Multi-Producer Multi-Consumer (MPMC) Protocol
The ring buffer coordinates concurrent producers and consumers without mutex locks using Dmitry Vyukov's turn-based sequencing algorithm:
sequenceDiagram
autonumber
actor Producer as Producer Thread
participant Ring as SharedMemoryRingBuffer
participant Slot as Slot [tail & mask]
actor Consumer as Consumer Thread
Note over Producer,Slot: Slot Reservation Phase (Write)
Producer->>Ring: Reserve(payloadSize)
Ring->>Slot: Inspect turn == tail
alt Slot is Ready (turn == tail)
Ring->>Ring: CAS tail -> tail + 1
Ring-->>Producer: Return direct slot slice & preallocated commitFn
Producer->>Slot: Write payload directly into memory
Producer->>Producer: commitFn()
Producer->>Slot: Store turn = tail + 1 (Release Barrier)
else Slot Saturated (turn < tail)
Ring-->>Producer: Return ErrRingBufferFull (Fail-Closed)
end
Note over Consumer,Slot: Slot Consumption Phase (Read)
Consumer->>Ring: ReserveRead()
Ring->>Slot: Inspect turn == head + 1
alt Entry is Committed (turn == head + 1)
Ring->>Ring: CAS head -> head + 1
Ring-->>Consumer: Return direct payload slice & preallocated releaseFn
Consumer->>Consumer: Process payload zero-copy
Consumer->>Producer: releaseFn()
Consumer->>Slot: Store turn = head + capacity (Release Slot for Producers)
else Buffer Empty (turn < head + 1)
Ring-->>Consumer: Return ErrRingBufferEmpty
end
3.1 Mathematical Turn Invariants
For a ring buffer of capacity \(C\) and slot index \(i = s \pmod C\):
- Initial State: \(\text{turn}_i = i\).
- First Write (\(s = i\)): Producer checks \(\text{turn}_i == s\). Upon commit, updates \(\text{turn}_i \leftarrow s + 1\).
- First Read (\(s = i\)): Consumer checks \(\text{turn}_i == s + 1\). Upon release, updates \(\text{turn}_i \leftarrow s + C\).
- Second Write (\(s = i + C\)): Next cycle producer checks \(\text{turn}_i == s\). Upon commit, updates \(\text{turn}_i \leftarrow s + 1\).
- Progress Invariant: \(s_{\text{tail}} - s_{\text{head}} \le C\) strictly bounded at all times.
4. Zero-Copy Slot Reservation Architecture
Traditional IPC mechanisms copy data twice: from caller buffer into IPC buffer, and from IPC buffer to consumer. SharedMemoryRingBuffer eliminates both copies through reservation:
// Producer zero-copy write:
slot, commit, err := ring.Reserve(payloadSize)
if err != nil {
return err // ErrRingBufferFull or ErrPayloadTooLarge
}
// Caller writes directly into backing memory (zero allocation, zero copy)
copy(slot, encodedPayload)
commit() // Sets slot turn and advances commit sequence atomically
// Consumer zero-copy read:
slot, release, err := ring.ReserveRead()
if err != nil {
return err // ErrRingBufferEmpty
}
processPayloadDirectly(slot)
release() // Returns slot to producer pool with zero allocation
Preallocated Closures for Zero-Allocation Execution
In standard Go, returning a closure func() dynamically allocates heap memory if free variables escape. To guarantee testing.AllocsPerRun == 0.00, each Slot is initialized with pre-bound, immutable commitFn and releaseFn function pointers allocated once during buffer creation:
slotPtr.commitFn = func() {
seq := slotPtr.curSeq.Load()
size := slotPtr.curSize.Load()
slotPtr.len.Store(size)
slotPtr.turn.Store(seq + 1)
}
Returning slotPtr.commitFn merely copies a 16-byte interface value onto the caller stack, achieving absolute zero heap allocation.
5. Vectorized Batch Callback Delivery Protocol
The VectorizedBatchSubscriber coordinates bulk event delivery from the single-event callback stream into batched vectors:
flowchart LR
E1["CallbackEntry 1"] --> Q["Buffered Queue<br/>(Capacity: 1024)"]
E2["CallbackEntry 2"] --> Q
EN["CallbackEntry N"] --> Q
subgraph WorkerLoop["Vectorized Worker (goroutinelabels)"]
W1["Worker Loop"]
T["Ticker Flush<br/>(5ms default)"]
CapCheck{"Batch Count >= 64?"}
end
Q --> W1
W1 --> CapCheck
CapCheck -->|Yes| Dispatch["Vectorized Dispatch<br/>(WriteBatch / Handler)"]
T -->|Timer Fire| Dispatch
Dispatch --> Ring["SharedMemoryRingBuffer"]
5.1 Dual-Trigger Flush Mechanics
- Fullness Trigger (Throughput Mode): When incoming callbacks reach
BatchCapacity(default 64 entries), the worker flushes the batch immediately without waiting for the timer. - Ticker Trigger (Low-Latency Mode): When callback arrival is sparse, an adaptive interval timer (
FlushInterval, default 5ms) flushes partial batches, guaranteeing an upper bound on delivery latency. - Clean Teardown: During shutdown,
Close()stops accepting notifications, drains the internal channel, delivers any lingering partial batch, and terminates the background goroutine cleanly (zero goroutine leaks).
6. Fail-Closed File Integrity & Memory Protection
When backed by persistent filesystem segments (.seg), the subsystem enforces rigorous fail-closed validation:
- Truncated Header Rejection: Files smaller than
SHMHeaderSize(64 bytes) are rejected withErrTruncatedFile. - Magic Byte Verification: File headers lacking the
ZQKSHM01signature fail withErrInvalidMagic. - Checksum Verification: The first 32 bytes are verified against
header.Checksumusing IEEE CRC32. Any flipped bit immediately yieldsErrCorruptedHeaderbefore parsing or mapping segment bodies. - Capacity Alignment Check: Slot capacities that are non-powers-of-two fail with
ErrInvalidCapacity. - Total Boundary Rejection: If file size is smaller than
192 + SlotCapacity * SlotSize, initialization aborts fail-closed.
7. Requirement Traceability Matrix
| Symbolic ID | Category | Description | Verification Target |
|---|---|---|---|
REQ-SHM-IPC-DELIVERY |
Whole Feature | Zero-Copy Shared-Memory IPC and Vectorized Batch Delivery | cmd/zqk/callback/shm_ipc.go |
CRIT-SHM-RING-LAYOUT |
Static Invariant | Memory-mapped ring buffer layout, 64-byte cache-line isolation, and header validation | TestSHM_SingleProducerSingleConsumer_FIFOAndZeroCopy, TestSHM_ErrorBoundaries_HeaderAndCorruption |
CRIT-VECTORIZED-BATCH-DELIVERY |
Operational Proof | Vectorized batch delivery dual-triggers, concurrent lock-free MPMC ring, and 0.00 alloc hot path | TestSHM_VectorizedBatchDelivery_Triggers, TestSHM_ConcurrentMultiProducerMultiConsumer_RaceDetector, TestSHM_ZeroAllocationHotPath |
CRIT-SHM-CORRUPTION-RESILIENCE |
Negative Boundary | Defensive validation against corrupted checksums, truncated files, invalid magic, and clean shutdown | TestSHM_ErrorBoundaries_HeaderAndCorruption, TestSHM_CleanShutdownAndZeroLeaks |
TST-SHM-IPC-VERIFICATION |
Test Suite | Comprehensive unit, concurrent race-detector, and micro-allocation test suite | cmd/zqk/callback/shm_ipc_test.go |