Pipelinify

Learn data engineering by watching it work. Each game turns one core idea into puzzles. You build or fix part of a pipeline, run it, and watch what happens to the data.

★★★ = solved and hit the level's target

Keys in every game: R run · N next level · T try again

  1. Refinery

    Transformations

    Chain filter, cast, dedup, join and aggregate steps to turn messy input into a clean target table.

  2. DAG Doctor

    Orchestration

    Find and fix broken workflows: missing dependencies, cycles, retries that are not idempotent, and backfills.

  3. Quality Detective

    Data quality

    Place quality checks on a line of incoming records, and stop bad data before it reaches the warehouse.

  4. Stream Lab

    Streaming

    Count a live stream of events in time windows, and decide how long to wait for late data.

  5. Schema Architect

    Modelling

    Split a messy wide table into facts and dimensions, pick the grain, and keep history when things change.

  6. Partition Planner

    Storage & performance

    Choose partitions and file formats so queries skip the data they don't need and scan fewer bytes.

  7. More games and concepts on the way