Understanding walkerOS Flow

Overview

walkerOS follows a Source → Collector → Destination(s) architecture for composable, modular event processing.

Core principle: Separation of concerns. Each component has one job. Components are composable and replaceable.

The Flow Pattern

Sources → [Pre-Transformers] → Collector → [Post-Transformers] → Destinations
(Capture)  (source.next)    (Processing) (dest.before)       (Delivery)

- Browser DOM              - Validation   - Validation        - Google Analytics
- DataLayer                - Enrichment   - Enrichment        - Meta Pixel
- Server HTTP              - Redaction    - Consent check     - Custom API
- Cloud Functions                         - Routing           - Data Warehouse

Key Concepts

Composability

A Flow combines components. You can:

• Use multiple sources feeding one collector
• Route events to multiple destinations
• Swap components without changing others

The Flow Type

See packages/core/src/types/flow.ts for the canonical interface.

// Conceptual structure (see source for full type)
interface Flow {
  sources?: Record<string, Source>;
  collector: Collector;
  destinations?: Record<string, Destination>;
}

Universal Push Interface

All components communicate via push functions:

The elb() function is an alias for collector.push - used for component wiring.

startFlow Helper

See packages/collector/src/flow.ts for the startFlow function.

import { startFlow } from '@walkeros/collector';

const { collector, elb } = await startFlow({
  sources: {
    /* ... */
  },
  destinations: {
    /* ... */
  },
});

Separation of Concerns

Transformer Chains

Transformers run at two points in the pipeline, configured via next and before:

Pre-Collector Chain

Runs after source captures event, before collector processing:

sources: {
  browser: {
    code: sourceBrowser,
    next: 'validate'  // First transformer in pre-chain
  }
},
transformers: {
  validate: {
    code: transformerValidator,
    config: { next: 'enrich' }  // Chain continues
  },
  enrich: {
    code: transformerEnrich
    // No next = chain ends, event goes to collector
  }
}

Post-Collector Chain

Runs after collector enrichment, before destination receives event:

destinations: {
  gtag: {
    code: destinationGtag,
    before: 'redact'  // First transformer in post-chain
  }
},
transformers: {
  redact: {
    code: transformerRedact
    // Event then goes to destination
  }
}

Chain Resolution

• source.next → starts pre-collector chain
• transformer.config.next → links transformers together
• destination.before → starts post-collector chain per destination

Related

Skills:

• understanding-events skill - Event model
• understanding-transformers skill - Transformer interface and chaining
• understanding-destinations skill - Destination interface
• understanding-sources skill - Source interface

Package READMEs:

• packages/collector/README.md - Collector details

Source Files:

• packages/collector/src/flow.ts - startFlow implementation

Documentation:

• Website: Flow - Flow concept
• Website: Collector - Collector docs