refactor: restructure monorepo for clean portfolio layout
- Move timesfm-forecast into apps/ directory - Flatten Udacity portfolio projects from deep URL-encoded paths into data-engineering/01-XX numbered directories - Remove old My-Data-Engineering-Portifolio/ parent directory - Rewrite root README.md: professional overview with badges, project table, and repo structure diagram - Create data-engineering/README.md with per-project descriptions - Add README.md for 02-cassandra-modeling (was missing) - Add README.md for 05-airflow-pipelines (was missing) - Normalize capstone readme.md -> README.md - Update .gitignore: add *.cfg, *.env, *.zip, *.sas7bdat, Jupyter checkpoints, IDE dirs; remove uv.lock exclusion - Add dwh.cfg.example and dl.cfg.example credential templates - Untrack real credential files (dwh.cfg, dl.cfg) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
45
data-engineering/02-cassandra-modeling/README.md
Normal file
45
data-engineering/02-cassandra-modeling/README.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# Data Modeling with Apache Cassandra
|
||||
|
||||
**Udacity Data Engineering Nanodegree — Project 2**
|
||||
|
||||
## Overview
|
||||
|
||||
Model and query a NoSQL database using **Apache Cassandra** for the fictional Sparkify music streaming service. Unlike relational databases, Cassandra requires a _query-first_ design approach: each table is purpose-built to answer one specific analytical question.
|
||||
|
||||
## Dataset
|
||||
|
||||
Raw event data is sourced from CSV log files (`event_data/2018-11-*.csv`) representing daily user activity. These files are pre-processed and merged into a single consolidated dataset (`event_datafile_new.csv`) before loading into Cassandra.
|
||||
|
||||
**Sample fields:** `artist`, `firstName`, `gender`, `itemInSession`, `lastName`, `length`, `level`, `location`, `sessionId`, `song`, `userId`
|
||||
|
||||
## Schema Design
|
||||
|
||||
Three tables are modeled to answer three specific queries:
|
||||
|
||||
| Table | Partition Key | Clustering Columns | Answers |
|
||||
|---|---|---|---|
|
||||
| `session_songs` | `sessionId` | `itemInSession` | What song was played in a given session and item? |
|
||||
| `user_session_songs` | `userId`, `sessionId` | `itemInSession` | What songs did a user listen to in a session? |
|
||||
| `song_listeners` | `song` | `userId` | Who listened to a specific song? |
|
||||
|
||||
## Key Concepts
|
||||
|
||||
- **Query-first modeling** — schema designed around queries, not entities
|
||||
- **Denormalization** — data is duplicated across tables to enable fast reads
|
||||
- **Partition keys** — determine data distribution across nodes
|
||||
- **Clustering columns** — control sort order within a partition
|
||||
- **`cassandra-driver`** — Python client for Cassandra
|
||||
|
||||
## How to Run
|
||||
|
||||
1. Install dependencies:
|
||||
```bash
|
||||
pip install cassandra-driver pandas
|
||||
```
|
||||
|
||||
2. Open and run the notebook:
|
||||
```bash
|
||||
jupyter notebook "Project_1B_ Project_Template.ipynb"
|
||||
```
|
||||
|
||||
The notebook walks through preprocessing the CSV data, creating Cassandra tables, inserting records, and running validation queries.
|
||||
Reference in New Issue
Block a user