About: What Is This Book?
The Data Foundry is a complete, free handbook for modern data engineering. It covers the full stack, from the foundational mindset of the profession through SQL, Python, orchestration, cloud warehousing, real-time streaming, and the major cloud platforms. It is written by practitioners at Morpheralabs who build and maintain data systems professionally, and it is given away without charge because the knowledge should be accessible.
Who This Book Is For
This book is written for three kinds of people.
The first is someone starting from scratch. No database experience, no Python background, no cloud account. Part 0 and Part 1 begin at the absolute beginning and build toward production-grade skills.
The second is a developer or analyst who already writes code or queries data but has not worked at the engineering level. You may know SQL but have never designed a pipeline. You may write Python but have never orchestrated it with Airflow. This book fills those gaps systematically.
The third is a working engineer who wants a reference. The ten parts are structured so you can jump to whichever chapter covers the tool or concept you are dealing with today.
What the Book Covers
Ten parts, forty-three chapters, organized in the order they are meant to be learned.
Part 0 establishes the mindset of a data engineer before touching any tool. It asks what the job actually is, how data systems work at a systems level, and what separates code that works from a system that can be trusted.
Part 1 covers SQL, the language every data engineer writes every day. Querying, filtering, joining, aggregating, windowing, pivoting, and modifying data using the BikeStores sample database throughout.
Part 2 covers Python as the engineer's primary scripting and data manipulation language. Foundations through to Pandas, with a complete Titanic dataset case study.
Part 3 covers Apache Airflow, the orchestration platform that schedules, monitors, and coordinates pipelines.
Part 4 covers the engineering practices that make pipelines production-ready: Docker, CI/CD, and Bash.
Part 5 covers agentic data engineering, the use of AI tools and agents as active participants in the engineering workflow.
Part 6 covers cloud warehousing with Snowflake and dbt, including dimensional modeling, slowly changing dimensions, and transformation pipelines.
Part 7 covers real-time data with Apache Kafka: streaming architecture, producers, consumers, and change data capture.
Part 8 covers AWS for data engineers: the core services, warehousing on Redshift, event-driven pipelines, and infrastructure as code.
Part 9 covers Azure: the Medallion Architecture, Azure Fabric, and Synapse Analytics.
How to Use This Book
Read the parts in order if you are building from foundations. The concepts in each part assume the ones before it.
If you already work with data systems, use the sidebar as a reference map and jump to the chapters that match your current project. The SQL part stands independently. The Python part stands independently. Each cloud part stands independently.
Every chapter includes concept explanations, code examples with outputs, common mistakes, and exercises. The exercises at the end of each chapter are not optional extras. They are the part where the concept becomes a skill.
Who Wrote It
The Data Foundry is written by the team at Morpheralabs: Qasim Hassan, Ayan Hussain, Abdullah Khan, and Haniya Maqsood.
Morpheralabs is a technology consultancy based in Karachi that builds and maintains data systems, AI pipelines, cloud infrastructure, and software for businesses. The book is a record of what we know from doing the work, written down and made free.