Streaming Architecture

Kind: style

Layer: Execution Core

Record: architecture:streaming-architecture

Severity: contextual

Scope: data processing, integration

Canonical: Ontology

A convention of processing data record by record as it arrives, with backpressure, instead of collecting it first.

Listed in Architecture principles, after Queuing Theory and before Single-Pass Processing.

Requires

• Event Stream
• Backpressure

Reinforces

• Single-Pass Processing

Enables

• Continuous Processing

In tension with

• Ordering/State
• Batch-Only Processing

Tensions

• Streaming Architecture / Ordering/State
• Streaming Architecture / Batch-Only Processing

Violated by

• Full Materialization

Refactored by

• Backpressure

Severity

• contextual

Category

• Streaming / Pipeline / Dataflow Processing

Reinforced by

• Windowing

Linked from

• Event / Messaging / Asynchronous Architecture
• Error Handling / Resilience
• Streaming / Pipeline / Dataflow Processing
• Core Vocabulary
• Scalability / Performance / Optimization
• Streaming / Pipeline / Dataflow Processing
• Severity levels
• The resolutions