# The lex records whose category is streaming-pipeline-dataflow-processing

This index as JSON: https://banes-lab.com/json/api/facets/lex/category/streaming-pipeline-dataflow-processing

## Entries

- [Batch-Only Processing](https://banes-lab.com/records/lex/batch-only-processing.md): Processing data in scheduled batches rather than as a continuous low-latency stream.
- [Ordering/State](https://banes-lab.com/records/lex/ordering-state.md): The degree to which processing an unbounded stream complicates preserving event order and bounded state.
- [Forward-Only State Model](https://banes-lab.com/records/lex/forward-only-state-model.md): The requirement that processing keep only forward-moving state, never needing to revisit earlier input.
- [Global Optimization](https://banes-lab.com/records/lex/global-optimization.md): The degree to which processing data in a single pass forgoes optimizations that need a full view of the data.
- [Multi-Pass Full Materialization](https://banes-lab.com/records/lex/multi-pass-full-materialization.md): Loading a full dataset into memory and traversing it in multiple passes, rather than in a single streaming pass.
- [Error Propagation/Debugging](https://banes-lab.com/records/lex/error-propagation-debugging.md): The degree to which splitting work into pipeline stages makes an error harder to trace back to its origin.
- [Monolithic Processing Function](https://banes-lab.com/records/lex/monolithic-processing-function.md): One large function that performs every processing step at once, so stages cannot be tested or reused independently.
- [Stage Contracts](https://banes-lab.com/records/lex/stage-contracts.md): The requirement that each pipeline stage declare a typed contract for what it consumes and produces.
- [Stepwise Transformation](https://banes-lab.com/records/lex/stepwise-transformation.md): The ability to transform data through a sequence of small, composable stages.
- [Streaming](https://banes-lab.com/records/lex/streaming.md): The ability to process data continuously as it arrives rather than in complete batches.
- [Avoiding Unneeded Work](https://banes-lab.com/records/lex/avoiding-unneeded-work.md): The ability to skip computing results that are never used.
- [Debuggability/Resource Lifetime](https://banes-lab.com/records/lex/debuggability-resource-lifetime.md): The degree to which deferring computation makes execution order harder to debug and resource lifetimes harder to reason about.
- [Deferred Execution Semantics](https://banes-lab.com/records/lex/deferred-execution-semantics.md): The requirement that a computation's semantics defer its work until the result is demanded.
- [Eager Full Materialization](https://banes-lab.com/records/lex/eager-full-materialization.md): Computing and materializing a complete result up front, rather than deferring computation until parts are needed.
- [Large Data Processing](https://banes-lab.com/records/lex/large-data-processing.md): The ability to process datasets larger than memory by reading them in order, a piece at a time.
- [Lookup Performance](https://banes-lab.com/records/lex/lookup-performance.md): The degree to which reading strictly in sequence makes locating a specific item by key slow.
- [Ordered Read Model](https://banes-lab.com/records/lex/ordered-read-model.md): The requirement that data be read in a fixed forward order rather than by arbitrary index.
- [Random Access Requirement](https://banes-lab.com/records/lex/random-access-requirement.md): A need to read arbitrary items by position or key on demand rather than strictly in sequence.
- [Backtracking Algorithm](https://banes-lab.com/records/lex/backtracking-algorithm.md): A method that explores options and reverts to an earlier point when one fails, requiring the ability to look back.
- [Complex Grammar/Global State](https://banes-lab.com/records/lex/complex-grammar-global-state.md): The degree to which forbidding backtracking makes complex grammars or global-state logic hard to express.
- [No Backtracking Requirement](https://banes-lab.com/records/lex/no-backtracking-requirement.md): The requirement that processing never need to revisit earlier input to make a decision.
- [Streaming Parsers](https://banes-lab.com/records/lex/streaming-parsers.md): The ability to parse input incrementally as it streams in, without buffering the whole document.
- [Control-Flow-Centric Monolith](https://banes-lab.com/records/lex/control-flow-centric-monolith.md): A monolith driven by imperative control flow rather than data dependencies, so stages cannot run or scale independently.
- [Data Dependencies](https://banes-lab.com/records/lex/data-dependencies.md): The requirement that the data each stage needs from others be declared as explicit dependencies.
- [Parallel/Stream Processing](https://banes-lab.com/records/lex/parallel-stream-processing.md): The ability to run independent stages in parallel or stream data between them as it is produced.
- [Stages](https://banes-lab.com/records/lex/stages.md): The requirement that processing be decomposed into discrete stages connected by data flow.
- [State Coordination](https://banes-lab.com/records/lex/state-coordination.md): The degree to which a data-driven design must still coordinate shared state across concurrent stages.
- [No Hidden State](https://banes-lab.com/records/lex/no-hidden-state.md): The requirement that a processor keep no state hidden between invocations, taking all inputs explicitly.
- [Parallel Processing](https://banes-lab.com/records/lex/parallel-processing.md): The ability to process many records at once because each is handled independently of the others.
- [Stateful Business Rules](https://banes-lab.com/records/lex/stateful-business-rules.md): The degree to which rules that inherently depend on accumulated state resist a purely stateless design.
- [Stateful Hidden Accumulation](https://banes-lab.com/records/lex/stateful-hidden-accumulation.md): Accumulating state inside a processor across records, so results depend on invisible history.
- [Bounded Aggregation over Unbounded Streams](https://banes-lab.com/records/lex/bounded-aggregation-over-unbounded-streams.md): The ability to aggregate an endless stream by grouping its events into bounded windows.
- [Bounded State](https://banes-lab.com/records/lex/bounded-state.md): The degree to which processing keeps its working state within a fixed bound regardless of input size.
- [Event Time](https://banes-lab.com/records/lex/event-time.md): The time at which an event occurred, carried on the event and used to assign it to a window.
- [Late-Data Handling](https://banes-lab.com/records/lex/late-data-handling.md): The degree to which windowing by event time must reckon with events that arrive after their window has closed.
- [Unbounded Accumulation](https://banes-lab.com/records/lex/unbounded-accumulation.md): Aggregating an endless stream into ever-growing state that eventually exhausts memory.
- [Parallel Branch Processing](https://banes-lab.com/records/lex/parallel-branch-processing.md): The ability to process independent branches of work simultaneously across workers.
- [Result Aggregation](https://banes-lab.com/records/lex/result-aggregation.md): The ability to combine the outputs of parallel branches back into a single result.
- [Serial Item Processing](https://banes-lab.com/records/lex/serial-item-processing.md): Processing independent items one at a time in sequence, rather than in parallel.
- [Fitness for Purpose](https://banes-lab.com/records/lex/fitness-for-purpose.md): The degree to which the chosen processing model matches the latency and volume the problem needs.
- [Latency Requirement Clarity](https://banes-lab.com/records/lex/latency-requirement-clarity.md): The requirement that a workload's latency and freshness needs be made explicit before a processing model is chosen.
- [Latency-Appropriate Processing Model](https://banes-lab.com/records/lex/latency-appropriate-processing-model.md): The ability to choose batch or stream processing to match a workload's latency needs.
- [One-Size-Fits-All Processing](https://banes-lab.com/records/lex/one-size-fits-all-processing.md): Forcing every workload through a single processing model regardless of its latency or volume needs.
- [Operational Duplication](https://banes-lab.com/records/lex/operational-duplication.md): The degree to which supporting both batch and streaming paths duplicates operational effort and code.