6796398924
- Move timesfm-forecast into apps/ directory - Flatten Udacity portfolio projects from deep URL-encoded paths into data-engineering/01-XX numbered directories - Remove old My-Data-Engineering-Portifolio/ parent directory - Rewrite root README.md: professional overview with badges, project table, and repo structure diagram - Create data-engineering/README.md with per-project descriptions - Add README.md for 02-cassandra-modeling (was missing) - Add README.md for 05-airflow-pipelines (was missing) - Normalize capstone readme.md -> README.md - Update .gitignore: add *.cfg, *.env, *.zip, *.sas7bdat, Jupyter checkpoints, IDE dirs; remove uv.lock exclusion - Add dwh.cfg.example and dl.cfg.example credential templates - Untrack real credential files (dwh.cfg, dl.cfg) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Data Modeling with Apache Cassandra
Udacity Data Engineering Nanodegree — Project 2
Overview
Model and query a NoSQL database using Apache Cassandra for the fictional Sparkify music streaming service. Unlike relational databases, Cassandra requires a query-first design approach: each table is purpose-built to answer one specific analytical question.
Dataset
Raw event data is sourced from CSV log files (event_data/2018-11-*.csv) representing daily user activity. These files are pre-processed and merged into a single consolidated dataset (event_datafile_new.csv) before loading into Cassandra.
Sample fields: artist, firstName, gender, itemInSession, lastName, length, level, location, sessionId, song, userId
Schema Design
Three tables are modeled to answer three specific queries:
| Table | Partition Key | Clustering Columns | Answers |
|---|---|---|---|
session_songs |
sessionId |
itemInSession |
What song was played in a given session and item? |
user_session_songs |
userId, sessionId |
itemInSession |
What songs did a user listen to in a session? |
song_listeners |
song |
userId |
Who listened to a specific song? |
Key Concepts
- Query-first modeling — schema designed around queries, not entities
- Denormalization — data is duplicated across tables to enable fast reads
- Partition keys — determine data distribution across nodes
- Clustering columns — control sort order within a partition
cassandra-driver— Python client for Cassandra
How to Run
-
Install dependencies:
pip install cassandra-driver pandas -
Open and run the notebook:
jupyter notebook "Project_1B_ Project_Template.ipynb"
The notebook walks through preprocessing the CSV data, creating Cassandra tables, inserting records, and running validation queries.