Hunch
Open the board

Analytics data pipeline

hunchboard

From app events to dashboards: streaming in, warehouse, dbt models and BI.

Comments

    What is on this board

    Parts

    • Product Web app
    • Production DB Relational DB · PostgreSQL
    • Copy changes Change data capture · Debezium
    • Event collector Service
    • Events Event stream · Kafka
    • Raw Data lake · S3
    • Warehouse Data warehouse · Snowflake
    • Models ETL / data pipeline · dbt
    • Orchestration Workflow engine · Airflow
    • Dashboards Dashboard · Metabase
    • Features Feature store
    • Data tests Guardrails / evals

    How they connect

    • Product → Event collector: events
    • Event collector → Events: publish
    • Production DB → Copy changes: changes
    • Copy changes → Events: publish
    • Events → Raw: archive
    • Raw → Warehouse: load
    • Orchestration → Models: run hourly
    • Models → Warehouse: transform
    • Models → Data tests: tests
    • Warehouse → Dashboards: query
    • Warehouse → Features: features

    More like this