Visual ETL with DuckDB: Building Local-First Pipelines in Revoluzen Flow

DuckDB changed what local analytics can feel like. You can scan CSV and Parquet files quickly, join them with database extracts, and produce useful analytical tables without standing up a warehouse for every experiment.

Revoluzen Flow pairs well with that model. It gives DuckDB a visual pipeline surface: sources, transforms, previews, checks, destinations, schedules, and monitoring.

Why DuckDB belongs in visual ETL

DuckDB is excellent when:

• Data fits on one machine or one strong VM
• Files live in CSV, JSON, or Parquet
• Analysts need fast iteration
• The team wants SQL without warehouse overhead
• A pipeline should be portable across laptops, CI, and small servers

Revoluzen Flow adds the operational layer around that work. Instead of keeping a folder of scripts and ad hoc SQL files, you build the flow on a canvas and inspect each node.

Example pipeline

Build a local revenue mart from three inputs:

• orders.parquet
• customers.csv
• A small Postgres table called refunds

The target is a DuckDB table called:

mart_daily_revenue

Step 1: Install Revoluzen Flow

Revoluzen Flow is Apache 2.0 and launched as v1.0.0 on July 20, 2026.

pip install revoluzen-flow
revoluzen open

Open:

http://localhost:8001

Step 2: Add file sources

On the canvas, add:

• A Parquet Source node for orders.parquet
• A CSV Source node for customers.csv

Preview both nodes. Confirm the inferred types before writing any transform logic.

For orders.parquet, expect:

order_id, customer_id, order_ts, region, status, gross_amount

For customers.csv, expect:

customer_id, segment, lifecycle_stage

Step 3: Add the Postgres source

Add a PostgreSQL Source node for refunds:

SELECT
  order_id,
  refunded_amount,
  refunded_at
FROM refunds
WHERE refunded_at >= CURRENT_DATE - INTERVAL '30 days'

Revoluzen Flow includes database, file, storage, SaaS, operations, and vector connectors. DuckDB can sit in the middle as the execution engine for local transformation work.

Step 4: Transform with SQL

Add a SQL Transform node and join the three inputs:

WITH paid_orders AS (
  SELECT
    order_id,
    customer_id,
    DATE_TRUNC('day', order_ts) AS report_date,
    region,
    gross_amount
  FROM orders
  WHERE status = 'paid'
),
refunds_by_order AS (
  SELECT
    order_id,
    SUM(refunded_amount) AS refunded_amount
  FROM refunds
  GROUP BY 1
)
SELECT
  p.report_date,
  p.region,
  COALESCE(c.segment, 'unknown') AS segment,
  COUNT(*) AS order_count,
  SUM(p.gross_amount) AS gross_revenue,
  SUM(COALESCE(r.refunded_amount, 0)) AS refunded_revenue,
  SUM(p.gross_amount - COALESCE(r.refunded_amount, 0)) AS net_revenue
FROM paid_orders p
LEFT JOIN customers c
  ON p.customer_id = c.customer_id
LEFT JOIN refunds_by_order r
  ON p.order_id = r.order_id
GROUP BY 1, 2, 3

Preview the transform. If a column name is wrong, fix it while the graph is still local and cheap to rerun.

Step 5: Add data quality checks

Add checks that prevent bad local marts from becoming trusted outputs:

• report_date is not null
• order_count > 0
• gross_revenue >= 0
• net_revenue >= 0
• refunded_revenue <= gross_revenue

These checks are simple, but they catch the common problems: wrong joins, duplicate rows, type coercion mistakes, and refund over-counting.

Step 6: Write to DuckDB

Add a DuckDB Destination or managed local table sink and write to:

mart_daily_revenue

Use report_date, region, and segment as the upsert key. That lets you rerun the pipeline without duplicating records.

Step 7: Schedule the local job

For a daily refresh:

15 6 * * *

For development, run manually until the output is stable. Then turn on the schedule and add a notification node.

Where this pattern works

DuckDB plus Revoluzen Flow is a good fit for:

• Local analytics marts
• CSV and Parquet cleanup
• Prototype pipelines before warehouse deployment
• Small-team reporting
• Data quality checks before publishing files
• Offline or single-tenant environments

It is less ideal when:

• Data volume requires a distributed warehouse
• The pipeline depends on custom Python libraries
• You need continuous streaming beyond Kafka
• The organization already has a mature dbt and warehouse-only workflow

Add local AI when useful

With Ollama configured, the Revoluzen Flow Copilot can help write SQL against the node schemas:

Create daily net revenue by region and customer segment.
Subtract refunds by order_id.
Keep report_date, region, segment, order_count, gross_revenue, refunded_revenue, net_revenue.

The model drafts the SQL. Revoluzen Flow previews and validates it. That division of labor is what makes local AI useful without making it risky.

Bottom line

DuckDB is fast local SQL. Revoluzen Flow gives it a visual ETL workflow: 66 connectors, live previews, scheduling, monitoring, data quality checks, and Apache 2.0 licensing.

For teams that want local-first analytics without a pile of scripts, the combination is practical immediately.

---

Revoluzen Flow is Apache 2.0 and self-hosted. Install it here or explore the connector catalog.