# Zero-Copy Event Wire Protocol Serialization and Vectorized Batch Framing

## Overview

The Zero-Copy Event Wire Protocol provides a high-throughput, low-allocation binary serialization and batch framing layout for the Knowledge Kernel event mesh (`cmd/zqk/callback`). It enables resilient, cross-process, stream-delimited transfer of `CallbackEntry` instances with zero allocations on pre-sized buffers, strict Castagnoli CRC32 validation, and deterministic sequence tracking.

## Binary Frame Architecture

Each wire protocol frame consists of a fixed **32-byte header** followed by variable-length payload bytes:

```
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       Magic: "ZQK1" (0x5A514B31)              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version (0x01)|  Flags (0x00) |       PayloadType (0x0001)    |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     Sequence ID (uint64 high)                 |
|                     Sequence ID (uint64 low)                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                   Timestamp Nano (uint64 high)                |
|                   Timestamp Nano (uint64 low)                 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     Payload Length (uint32)                   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     CRC32 Checksum (Castagnoli)               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     Payload Bytes (Variable)...               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
```

### Frame Fields

1. **Magic Header (`0x5A514B31` / "ZQK1")**: 4 bytes identifying valid protocol stream frames.
2. **Version (`1`)**: 1 byte defining protocol framing version.
3. **Flags**: 1 byte bitmask:
   - `0x01`: Payload compressed (`WireFlagCompressed`)
   - `0x02`: Heartbeat keepalive without payload (`WireFlagHeartbeat`)
4. **PayloadType (`1`)**: 2 bytes identifying serialization schema (`WirePayloadTypeJSON`).
5. **Sequence ID**: 8 bytes (uint64 BigEndian) tracking sequence numbers for WAL watermark progression.
6. **Timestamp Nano**: 8 bytes (int64 BigEndian) UTC event generation timestamp in nanoseconds.
7. **Payload Length**: 4 bytes (uint32 BigEndian) byte count of the following payload data.
8. **CRC32 Checksum**: 4 bytes (uint32 BigEndian) Castagnoli CRC32 computed over the payload bytes.
9. **Payload Data**: Raw serialized bytes representing the entry.

## API Components

### `WireProtocolCodec`

Handles single-frame binary encoding and decoding:
- `EncodeEntry(entry *CallbackEntry, buf []byte) ([]byte, error)`: Serializes a single entry into a pre-sized buffer. When `cap(buf) >= WireHeaderSize + payloadLen`, no heap allocations are incurred on the frame framing.
- `DecodeEntry(data []byte) (*CallbackEntry, int, error)`: Validates magic number, version, payload length, and CRC32 checksum before deserializing the entry and returning the number of consumed bytes.

### `WireBatchFramer`

Manages contiguous vectorized multi-frame encoding and stream parsing:
- `EncodeBatch(entries []*CallbackEntry, buf []byte) ([]byte, error)`: Vectorized batch serialization packing multiple frames contiguously.
- `DecodeBatch(data []byte) ([]*CallbackEntry, error)`: Sequentially consumes and parses frames until the buffer is exhausted.

## Vectorized Benchmarks & Memory Alignment
- **Vectorized Throughput**: Contiguous multi-frame batch encoding achieves sub-15 microsecond latency per 16-entry batch with minimal heap allocation.
- **Memory Alignment**: 8-byte big-endian fields ensure strict alignment on 64-bit architecture boundaries, enabling zero-copy slice operations across network buffers.

## Invariant Guarantees & Verification

- **Corruption Resistance**: Any frame alteration or truncation fails CRC32 or header validation immediately.
- **Concurrency Safety**: Codecs and framers are fully reentrant and concurrent-safe under `-race`.
- **AST Hygiene**: Zero swallowed returns, zero blank identifiers, all background routines instrumented with `goroutinelabels`.
