Intermediate · after Python and SQL
Data Engineering & MLOps
Beta This course is a beta version. Lessons, exercises and translations are still being reviewed and may change. Report anything that looks wrong with the feedback button.
Data scientists ask what the data says. Data engineers make sure that it arrives, correct and on time, every day. MLOps does the same for models. You'll write small versions of what the big tools do (an orchestrator, a contract checker, a registry, a drift monitor) and test them in your browser. So when you later meet Airflow, dbt, Spark, Kafka or MLflow, you'll know what problem each solves. This course doesn't teach those tools' interfaces in depth, and its browser exercises involve small data only.
- Modules
- 8
- Lessons
- 19
- Time
- ~37 hours
- Projects
- 2
What you'll be able to do
- Model data for analytics, and write idempotent, incremental pipelines
- Enforce data contracts and report data quality
- Build DAG-based orchestration with retries, and windowed stream aggregation
- Track lineage, and handle personal data responsibly
- Version datasets and models, and make training reproducible
- Monitor drift and performance, and decide on retraining and rollback
Who it's for
Programmers who can write Python and SQL and want to build reliable pipelines; data scientists whose models need to survive after the notebook.
What you need
Python with functions, classes and errors. Basic SQL with joins (the SQL level of Data Science, or similar). The MLOps chapters expect that you've trained one model.
What you'll build
- Daily Bike Pipeline
- Data-to-Model Pipeline Capstone
Curriculum
Data EngineeringModels, pipelines, contracts, orchestration and governance. 4 modules · 10 lessons
- 1 Formats and Data ModelsColumnar storage, stars and grain.
- 2 Reliable PipelinesIdempotence, increments and data contracts.
- 3 Orchestration and StreamsDAGs, retries, windows and late events.
- 4 Lineage, Privacy and GovernanceWhere each number came from, and who may see it.
Software FoundationsTests, configuration, logging, and CI. 1 module · 1 lesson
- 5 Software FoundationsTests for data code, config, logs, and CI.
MLOpsReproducible training, registries, monitoring, retraining. 2 modules · 6 lessons
- 6 Reproducible Training and RegistriesWhich data, which code, which model.
- 7 Monitoring, Drift and RetrainingModels decay quietly.
CapstoneA reproducible data-to-model pipeline. 1 module · 2 lessons
- 8 CapstoneA reproducible data-to-model pipeline.