AppleDouble Resource-Fork Sidecar & Filesystem Hygiene Subsystem Architectural & Operational Guide
1. Executive Architectural Summary
The AppleDouble Resource-Fork Sidecar & Filesystem Hygiene Subsystem (pkg/appledouble) provides high-performance, deterministic detection and filtering primitives for macOS AppleDouble resource-fork sidecar files across the ZQK Knowledge Kernel.
When engineers and autonomous agents interact with git workspaces, shared network volumes, or external archives, macOS file operations frequently synthesize auxiliary sidecars prefixed with ._. These sidecars are binary AppleDouble wrappers containing legacy resource forks or extended attributes. If ingested by the Knowledge Kernel's automated scanners, these non-text sidecars contaminate YAML deserializers, trigger parser panics, distort file metrics, and pollute version-controlled trees.
The pkg/appledouble package provides:
- Zero-Allocation Filename Detection: High-speed classification of file basenames via
IsSidecarFileNameand its semantic aliasSkipNameInReadDir. - Segment-Aware Path Traversal: Deep segment inspection via
PathHasSidecarSegmentandSkipPathInTreeWalkthat detects sidecars nested anywhere within directory hierarchies. - Cross-Platform Path Normalization: Seamless handling of Unix slashes and Windows backslashes ensuring identical behavior across Darwin, Linux, and Windows runtimes.
- Composite Tree-Walk Evaluator: Unified pruning logic via
SkipDirOrSidecarfor concise, readable filesystem traversal routines. - Strict 100% Statement Coverage Invariant: Exhaustive boundary, integration, and concurrency verification guarantees absolute reliability on hot scanning paths.
2. The macOS Finder / SMB Resource Fork Artifact Anti-Pattern
2.1 The AppleDouble Dual-Fork Architecture
The AppleDouble format (RFC 1740) was introduced to preserve Macintosh dual-fork files (data fork and resource fork) on filesystems that natively support only single-fork, unstructured byte streams (such as FAT32, ext4, SMB, or tar archives).
On modern macOS systems (APFS/HFS+), file metadata and extended attributes (such as com.apple.quarantine, ACLs, and FinderInfo tags) are stored directly within filesystem inode structures. However, when files are copied or synced across network mounts, FAT32/exFAT storage media, or unflattened tar archives:
flowchart TD
subgraph APFS ["APFS / HFS+ Source Volume"]
Original["spec.yaml\n[Data Fork + Inode Extended Attributes]"]
end
subgraph Transfer ["Copy / Sync / Tar Operation"]
Bridge["Finder / SMB Client / BSD tar"]
end
subgraph NonMac ["Non-APFS Target Volume / Archive"]
DataFork["spec.yaml\n(Pure UTF-8 YAML Content)"]
ResourceFork["._spec.yaml\n(AppleDouble Binary Header + Attributes)"]
end
Original --> Bridge
Bridge --> DataFork
Bridge --> ResourceFork
When macOS performs this split:
- Data Fork (
spec.yaml): Contains the actual user payload (e.g., UTF-8 YAML, Go source code, or JSON specs). - Resource Fork Sidecar (
._spec.yaml): A synthesized file beginning with the 4-byte AppleDouble magic number0x00051607, followed by binary entry descriptors and attribute payloads.
2.2 AppleDouble Sidecar Directory Artifacts
In addition to individual file sidecars, macOS Finder frequently creates sidecar directory shadows when moving directories over network filesystems. For instance, copying a directory system/ may create ._system/ containing folder attributes. Any files nested within ._system/ are themselves tainted by the sidecar parent segment and must be disqualified from kernel discovery.
3. Impact on Knowledge Kernel
The ZQK Knowledge Kernel continuously discovers, validates, and parses structured objects (goals, requirements, specifications, and execution policies) from disk. Unfiltered AppleDouble sidecars introduce severe operational failures:
3.1 YAML Parser Contamination
YAML parsers expect valid UTF-8 streams. When a parser attempts to decode ._command.yaml, it encounters binary header bytes (0x00051607), failing immediately with control character or invalid UTF-8 errors:
yaml: control characters are not allowed at line 1, column 1
3.2 Spec Discovery Duplication & Ghost Objects
Spec builder scanners identifying files matching *.yaml will register both command.yaml and ._command.yaml. If error recovery ignores the binary parse failure, the spec registry may report missing attributes or register corrupted phantom entries.
3.3 Git Workspace Poisoning
If an engineer on macOS edits a project stored on an external or network drive, Finder may drop ._* sidecars into the workspace. If committed, these pollute git repositories, break CI pipelines on Linux runners, and trigger tree police check-valve failures.
3.4 Legitimate Dotfiles vs. AppleDouble Sidecars
A naive filter that skips all files starting with a dot (.) is overly destructive because the Knowledge Kernel relies on essential dotfiles and dot-directories:
.zqk/(runtime process directory, CAS storage, and WAL logs).gitignore(git tracking policies).git/(repository metadata)
pkg/appledouble enforces surgical discrimination: it targets only entries prefixed with SidecarPrefix (._), preserving legitimate dotfiles while eradicating AppleDouble noise.
| Path Entry | Classification | Subsystem Action |
|---|---|---|
._repair_command.yaml |
AppleDouble Sidecar | Filtered / Skipped |
._system/config.yaml |
Nested in Sidecar Directory | Filtered / Skipped |
.zqk/process.yaml |
Legitimate Dot-Directory | Accepted & Visited |
.gitignore |
Legitimate Dotfile | Accepted & Visited |
foo._bar.yaml |
Internal Dot-Underscore File | Accepted & Visited |
script._ |
Trailing Dot-Underscore File | Accepted & Visited |
4. Component Topology & Subsystem Architecture
The pkg/appledouble package is structured into primitive detectors, traversal analyzers, and semantic policy wrappers:
flowchart TD
subgraph Consumers ["Kernel Subsystems & Callers"]
SpecDiscovery["Spec Builder & Loader\n(pkg/specbuilder)"]
TreeWalker["Workspace Tree Scanner\n(filepath.WalkDir)"]
DirReader["Flat Directory Scanners\n(os.ReadDir)"]
PoliceEngine["Tree Police & Vet Check-Valves\n(pkg/validation)"]
end
subgraph Primitives ["Primitive Analyzers (pkg/appledouble)"]
IsSidecar["IsSidecarFileName(name)\nPrefix Matcher on Base"]
PathHasSidecar["PathHasSidecarSegment(path)\nSegment Normalizer & Splitter"]
end
subgraph Policies ["Semantic Policy Facades"]
SkipName["SkipNameInReadDir(name)"]
SkipPath["SkipPathInTreeWalk(path)"]
SkipDirSidecar["SkipDirOrSidecar(isDir, path)"]
end
DirReader --> SkipName
TreeWalker --> SkipPath
SpecDiscovery --> SkipDirSidecar
PoliceEngine --> SkipPath
SkipName --> IsSidecar
SkipPath --> PathHasSidecar
SkipDirSidecar --> SkipPath
Component Responsibilities
| Primitive / Facade | Signature | Core Responsibility |
|---|---|---|
SidecarPrefix |
const string = "._" |
Canonical prefix constant defining AppleDouble sidecar identity. |
IsSidecarFileName |
func(name string) bool |
Extracts filepath.Base(name) and evaluates strings.HasPrefix(base, "._"). |
SkipNameInReadDir |
func(name string) bool |
Semantic facade for flat directory readers; clarifies intent at call sites. |
PathHasSidecarSegment |
func(path string) bool |
Normalizes path separators (converting Windows \ to /), splits into segments, and checks each non-empty segment for SidecarPrefix. |
SkipPathInTreeWalk |
func(path string) bool |
Semantic facade for recursive tree traversals; drops any file or directory containing a sidecar segment. |
SkipDirOrSidecar |
func(isDir bool, path string) bool |
Composite helper evaluating isDir || SkipPathInTreeWalk(path), standardizing file-only visitor logic. |
5. Filesystem Walk Integration Patterns
5.1 Idiomatic Recursive Tree Traversal (filepath.WalkDir)
When walking directory trees to find configuration specs, YAML definitions, or source files, callers should prune entire subtrees when encountering sidecar directories, and ignore individual sidecar files:
package specdiscovery
import (
"io/fs"
"path/filepath"
"github.com/zqk-os/zqk/pkg/appledouble"
)
// DiscoverSpecs recursively finds valid YAML specs while skipping AppleDouble sidecars.
func DiscoverSpecs(rootDir string) ([]string, error) {
var specs []string
err := filepath.WalkDir(rootDir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
// Prune sidecar directories immediately from traversal.
if d.IsDir() {
if appledouble.SkipPathInTreeWalk(path) {
return filepath.SkipDir
}
return nil
}
// Skip individual sidecar files.
if appledouble.SkipPathInTreeWalk(path) {
return nil
}
// Collect legitimate YAML specs.
if filepath.Ext(path) == ".yaml" || filepath.Ext(path) == ".yml" {
specs = append(specs, path)
}
return nil
})
return specs, err
}
sequenceDiagram
autonumber
participant W as filepath.WalkDir
participant Filter as appledouble.SkipPathInTreeWalk
participant Consumer as Spec Collector
W->>Filter: Evaluate "system/._dir" (isDir=true)
Filter-->>W: true (Contains "._" segment)
Note over W: Prune directory via filepath.SkipDir
W->>Filter: Evaluate "system/._config.yaml" (isDir=false)
Filter-->>W: true (Basename matches "._*")
Note over W: Ignore entry and continue
W->>Filter: Evaluate "system/config.yaml" (isDir=false)
Filter-->>W: false (Clean path)
W->>Consumer: Deliver clean spec to collector
5.2 Idiomatic Flat Directory Scanning (os.ReadDir)
When inspecting single directory contents (e.g., reading manifest collections or draft bundles), use SkipNameInReadDir:
package manifestloader
import (
"os"
"path/filepath"
"github.com/zqk-os/zqk/pkg/appledouble"
)
// ReadManifests reads all non-sidecar files from a directory.
func ReadManifests(dir string) ([]string, error) {
entries, err := os.ReadDir(dir)
if err != nil {
return nil, err
}
var validFiles []string
for _, entry := range entries {
if entry.IsDir() || appledouble.SkipNameInReadDir(entry.Name()) {
continue
}
validFiles = append(validFiles, filepath.Join(dir, entry.Name()))
}
return validFiles, nil
}
6. Edge Cases & Boundary Handling
The subsystem handles a wide spectrum of boundary inputs without panicking or allocating unnecessary heap memory:
flowchart TD
In(["Input Path String"]) --> Norm["filepath.ToSlash & Replace '\\' with '/'"]
Norm --> Split["Split on '/' Delimiter"]
Split --> Loop{"For each segment"}
Loop -- "seg == ''" --> SkipEmpty["Continue to next segment"]
Loop -- "HasPrefix(seg, '._')" --> ReturnTrue(["Return true (Sidecar Detected)"])
Loop -- "No more segments" --> ReturnFalse(["Return false (Clean Path)"])
SkipEmpty --> Loop
| Input Scenario | Input Value | Expected Result | Technical Rationale |
|---|---|---|---|
| Empty Input | "" |
false |
An empty path contains no segments matching ._. |
| Root Slashes | /, /// |
false |
Redundant slashes produce empty segments which are skipped. |
| Relative Dot Markers | ., .. |
false |
Directory markers do not match the ._ prefix. |
| Exact Prefix Only | ._ |
true |
Matches SidecarPrefix exactly. |
| Standard Sidecar | ._repair.yaml |
true |
Matches SidecarPrefix on basename. |
| Hidden Dotfiles | .git, .zqk, .DS_Store |
false |
Starting with . but not ._ preserves essential system dotfiles. |
| Internal Dot-Underscore | foo._bar.yaml |
false |
Dot-underscore inside the filename is valid and not a sidecar. |
| Trailing Dot-Underscore | file._ |
false |
Does not start with ._. |
| Windows Backslash Path | foo\._bar\baz.yaml |
true |
Backslashes are normalized to forward slashes before segment evaluation. |
| Deeply Nested Sidecar | a/b/c/._d/e/f.json |
true |
Detects sidecar directories anywhere in the path ancestor chain. |
| Consecutive Slashes | a//._b///c |
true |
Empty segments created by duplicate slashes are bypassed safely. |
7. Verification & Statement Coverage Invariant
The pkg/appledouble package maintains a strict 100.0% statement test coverage invariant.
7.1 Continuous Verification Command
To verify statement coverage in test and CI environments, run:
ZQK_ALLOW_FOREGROUND_GO_TEST=1 go test -v -timeout 30s -cover ./pkg/appledouble
7.2 Coverage Proof Matrix
| Test Suite Function | Target Functionality Verified |
|---|---|
TestIsSidecarFileName_Comprehensive |
Unit test matrix for IsSidecarFileName and SkipNameInReadDir across 18 edge cases. |
TestPathHasSidecarSegment_Comprehensive |
Unit test matrix for PathHasSidecarSegment and SkipPathInTreeWalk covering root paths, deep nesting, Windows backslashes, and consecutive slashes. |
TestSkipDirOrSidecar_Matrix |
Truth-table coverage for composite evaluation across directory and file permutations. |
TestReadDirFiltering_Hermetic |
Hermetic filesystem test simulating os.ReadDir on temporary directory trees. |
TestTreeWalkFiltering_Hermetic |
Hermetic filesystem test simulating filepath.WalkDir with sidecar file and directory pruning. |
TestConcurrencySafety |
Concurrent stress validation across 16 goroutines managed via pkg/goroutinelabels. |