Data Engineer Roadmap

The data engineer roadmap that actually gets you hired.

Every roadmap PNG on the internet is a wall of 60 tool logos that paralyses you. This one is different: an ordered path of 14 stages, each one a 1–2 week DataForge course, sequenced so each step unlocks the next. Follow it end to end and you'll have shipped real pipelines on Docker, Spark, Airflow, dbt and a cloud warehouse.
Start the roadmap Browse all courses

The full roadmap — in order

Stage 1 — Foundations. What data engineering is, the modern data stack, your first pipeline.

Stage 2 — Linux & Docker. Containerise everything. The skill every junior gets wrong on day one.

Stage 3 — SQL deep dive. Joins, CTEs, window functions, query plans. Non-negotiable.

Stage 4 — Python for data. Pipelines, Pydantic, requests, pytest. (See our Python for data engineering guide.)

Stage 5 — Terraform. Infrastructure as code. Stop clicking in cloud consoles.

Stage 6 — A cloud. Pick one — AWS, GCP or Azure. Don't learn three at once.

Stage 7 — A warehouse. Snowflake or BigQuery, end to end.

Stage 8 — dbt. Modern transformations and testing.

Stage 9 — Orchestration. Airflow or Dagster — schedules, retries, SLAs.

Stage 10 — Spark. Distributed processing for real volumes.

Stage 11 — Streaming. Kafka, event-driven patterns.

Stage 12 — Lakehouse & Iceberg. The open table format eating data lakes.

Stage 13 — Data quality & governance. Tests, contracts, lineage.

Stage 14 — Architecture. Designing platforms, lambda vs kappa, cost.

How fast can I go?

Aggressive: 4 months at 45 min/day — entry-level job-ready.

Realistic: 6–9 months at 20–30 min/day — full stack with portfolio.

Comfortable: 12 months at 15 min/day — sustainable, no burnout.

The DataForge streak system is tuned for the realistic and comfortable tracks — daily 5-minute exercises that compound.

What to skip

  • Learning all three clouds. Pick one. Translate later if you change jobs.
  • Hadoop / MapReduce. Historical context only — Spark replaced it everywhere.
  • NoSQL deep dives before you've mastered SQL.
  • Scala unless a specific job requires it. PySpark covers 95% of work.

FAQ

What does a complete data engineer roadmap look like in 2026?
SQL → Python → Linux/Docker → a cloud (AWS, GCP, or Azure) → a warehouse (Snowflake/BigQuery) → orchestration (Airflow/Dagster) → transformations (dbt) → distributed processing (Spark) → streaming (Kafka) → table formats (Iceberg/Delta). DataForge is built exactly along this order.
How long does it take to follow a data engineer roadmap from zero?
With 20–30 minutes a day, expect 4–6 months to entry-level job-ready and 12–18 months to mid-level. The number that matters is daily consistency, not weekend marathons.
Should I follow a roadmap or pick what's hot?
Follow a roadmap. Hot tools change every 18 months; the underlying skills — SQL fluency, container thinking, distributed systems, idempotency — compound for a decade.
Do I need certifications to land a data engineering job?
Helpful but not required. A portfolio of 2–3 end-to-end pipelines on GitHub almost always beats a certificate. AWS Data Engineer Associate or Azure DP-203 are the most respected if you do pursue one.
Is this roadmap the same for analytics engineers?
Roughly the first half. Analytics engineers stop after dbt + warehouse; data engineers continue into Spark, Kafka, streaming and platform work.

Ready to start?

7 days free. Then less than a coffee per month.

Start the roadmap