refactor: restructure monorepo for clean portfolio layout

- Move timesfm-forecast into apps/ directory
- Flatten Udacity portfolio projects from deep URL-encoded paths
  into data-engineering/01-XX numbered directories
- Remove old My-Data-Engineering-Portifolio/ parent directory
- Rewrite root README.md: professional overview with badges,
  project table, and repo structure diagram
- Create data-engineering/README.md with per-project descriptions
- Add README.md for 02-cassandra-modeling (was missing)
- Add README.md for 05-airflow-pipelines (was missing)
- Normalize capstone readme.md -> README.md
- Update .gitignore: add *.cfg, *.env, *.zip, *.sas7bdat,
  Jupyter checkpoints, IDE dirs; remove uv.lock exclusion
- Add dwh.cfg.example and dl.cfg.example credential templates
- Untrack real credential files (dwh.cfg, dl.cfg)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

.gitignore

@@ -20,13 +20,27 @@ build/
 
 # OS files
 .DS_Store
+Thumbs.db
+
+# IDE
+.vscode/
+.idea/
 
 # Streamlit
 .streamlit/
 **/.streamlit/
 
-# Lock files (optional, if not needed)
+# Jupyter
-uv.lock
+.ipynb_checkpoints/
+**/.ipynb_checkpoints/
 
-# Data files
+# Credentials & config with secrets
+*.cfg
+*.env
+.env
+
+# Large / binary data files
 *.csv
+*.zip
+*.sas7bdat
+*.parquet

My-Data-Engineering-Portifolio/Capstone+Project/dl.cfg

@@ -1,10 +0,0 @@
-[AWS]
-AWS_ACCESS_KEY_ID=AKIAYZ5XCJA24K6WMN5M
-AWS_SECRET_ACCESS_KEY=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
-
-[IAM_ROLE]
-ARN=arn:aws:iam::605437446197:role/myRedshiftRole
-
-[S3]
-SOURCE_S3_BUCKET=s3a://gfp-udacity/
-DEST_S3_BUCKET=s3a://gfp-udacity/

README.md

@@ -1,29 +1,90 @@
-### Hey, I'm Gabriel! 👋 
+### Hey, I'm Gabriel! 👋
 
-- 👀 I’m interested in Data Engineer | Data Scientist | Machine Learning
+- 👀 Interested in **Data Engineering · Data Science · ML Engineering**
-- 🌱 I’m currently learning ML System Design, AI Engineering, and building scalable data pipelines for real-world corporate environments
+- 🌱 Currently learning ML System Design, AI Engineering, and building scalable data pipelines
-- 🤝 I’m looking to collaborate on open source projects
+- 🤝 Open to collaboration on open source projects
-- 📫 How to reach me: gabriel.pereira@protonmail.com
+- 📫 Reach me at: gabriel.pereira@protonmail.com
 
 <div align="center">
 
-[![Linkedin Badge](https://img.shields.io/badge/-LinkedIn-blue?style=flat-square&logo=Linkedin&logoColor=white&link=https://www.linkedin.com/in/gabrielpereira-dev)](https://www.linkedin.com/in/gabrielpereira-dev)
+[![LinkedIn](https://img.shields.io/badge/-LinkedIn-blue?style=flat-square&logo=Linkedin&logoColor=white&link=https://www.linkedin.com/in/gabrielpereira-dev)](https://www.linkedin.com/in/gabrielpereira-dev)
 
 ---
 
-### Skills  
+### Skills
 
 <p align="center">
-  <a href="https://docs.microsoft.com/en-us/cpp/?view=msvc-170" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/profileme-dev/refs/heads/main/public/icons/skills/postgresql-colored.svg"width="36" height="36" alt="SQL" /></a>
+  <a href="https://www.postgresql.org/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/profileme-dev/refs/heads/main/public/icons/skills/postgresql-colored.svg" width="36" height="36" alt="PostgreSQL" /></a>
   <a href="https://www.python.org/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/python-colored.svg" width="36" height="36" alt="Python" /></a>
   <a href="https://docs.microsoft.com/en-us/cpp/?view=msvc-170" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/cplusplus-colored.svg" width="36" height="36" alt="C++" /></a>
-  <a href="https://aws.amazon.com" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/aws-colored.svg" width="36" height="36" alt="Amazon Web Services" /></a>
+  <a href="https://aws.amazon.com" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/aws-colored.svg" width="36" height="36" alt="AWS" /></a>
-  <a href="https://aws.amazon.com" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/docker-colored.svg" width="36" height="36" alt="Docker" /></a>
+  <a href="https://www.docker.com/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/docker-colored.svg" width="36" height="36" alt="Docker" /></a>
-  <a href="https://aws.amazon.com" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/linux.svg" width="36" height="36" alt="Linux" /></a>
+  <a href="https://www.linux.org/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/linux.svg" width="36" height="36" alt="Linux" /></a>
-<a href="https://aws.amazon.com" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/raspberrypi-colored.svg" width="36" height="36" alt="RaspiberryPi" /></a>
+  <a href="https://www.raspberrypi.org/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/raspberrypi-colored.svg" width="36" height="36" alt="Raspberry Pi" /></a>
 </p>
 
+</div>
+
 ---
 
-This is my workspace, it is a monorepo for my data engineering and data science projects.
+## 📁 Repository Structure
 
+This is my personal workspace — a monorepo for data engineering and data science projects.
+
+```
+workspace/
+├── apps/               # Production applications
+├── data-engineering/   # Udacity Data Engineering Nanodegree portfolio
+└── skills/             # dbt reference templates & best practices
+```
+
+---
+
+## 🚀 Apps
+
+### [`apps/timesfm-forecast`](./apps/timesfm-forecast)
+
+A production-grade time series forecasting application powered by **Google's TimesFM** model.
+
+| | |
+|---|---|
+| **Stack** | Python · Streamlit · PyTorch · HuggingFace Transformers |
+| **Features** | Multi-key forecasting · CSV upload/download · GPU/CPU support |
+| **Package manager** | `uv` |
+
+**Quick start:**
+```bash
+cd apps/timesfm-forecast
+uv venv && source .venv/bin/activate
+uv pip install -e .
+timesfm-app
+```
+
+---
+
+## 📚 Data Engineering Portfolio
+
+Udacity Data Engineering Nanodegree projects — a hands-on progression through core DE tools.  
+[🎓 View Credential](https://graduation.udacity.com/confirm/43TR6CQZ)
+
+| # | Project | Stack | Description |
+|---|---------|-------|-------------|
+| 1 | [Postgres Data Modeling](./data-engineering/01-postgres-modeling) | Python · PostgreSQL | Star schema design for music streaming analytics |
+| 2 | [Cassandra Data Modeling](./data-engineering/02-cassandra-modeling) | Python · Apache Cassandra | Denormalized NoSQL schema for query-first design |
+| 3 | [Cloud Data Warehouse](./data-engineering/03-cloud-data-warehouse) | Python · AWS S3 · Redshift | Cloud ETL pipeline with staging and fact/dim tables |
+| 4 | [Spark Data Lake](./data-engineering/04-spark-data-lake) | PySpark · AWS S3 | Large-scale partitioned processing with Parquet output |
+| 5 | [Airflow Pipelines](./data-engineering/05-airflow-pipelines) | Apache Airflow · Redshift | Orchestrated hourly ETL with modular custom operators |
+| 6 | [Immigration Capstone](./data-engineering/06-immigration-capstone) | PySpark · AWS S3 | Multi-source data lake integrating immigration, weather & demographics |
+
+See [`data-engineering/README.md`](./data-engineering/README.md) for full details.
+
+---
+
+## 🛠️ dbt Skills
+
+The [`skills/`](./skills) folder contains reference templates and best-practice guides for dbt projects:
+
+- Staging, Transform, and Mart model patterns
+- Incremental materialization strategies
+- SQL style guide
+- YAML schema testing templates

data-engineering/02-cassandra-modeling/README.md

@@ -0,0 +1,45 @@
+# Data Modeling with Apache Cassandra
+
+**Udacity Data Engineering Nanodegree — Project 2**
+
+## Overview
+
+Model and query a NoSQL database using **Apache Cassandra** for the fictional Sparkify music streaming service. Unlike relational databases, Cassandra requires a _query-first_ design approach: each table is purpose-built to answer one specific analytical question.
+
+## Dataset
+
+Raw event data is sourced from CSV log files (`event_data/2018-11-*.csv`) representing daily user activity. These files are pre-processed and merged into a single consolidated dataset (`event_datafile_new.csv`) before loading into Cassandra.
+
+**Sample fields:** `artist`, `firstName`, `gender`, `itemInSession`, `lastName`, `length`, `level`, `location`, `sessionId`, `song`, `userId`
+
+## Schema Design
+
+Three tables are modeled to answer three specific queries:
+
+| Table | Partition Key | Clustering Columns | Answers |
+|---|---|---|---|
+| `session_songs` | `sessionId` | `itemInSession` | What song was played in a given session and item? |
+| `user_session_songs` | `userId`, `sessionId` | `itemInSession` | What songs did a user listen to in a session? |
+| `song_listeners` | `song` | `userId` | Who listened to a specific song? |
+
+## Key Concepts
+
+- **Query-first modeling** — schema designed around queries, not entities
+- **Denormalization** — data is duplicated across tables to enable fast reads
+- **Partition keys** — determine data distribution across nodes
+- **Clustering columns** — control sort order within a partition
+- **`cassandra-driver`** — Python client for Cassandra
+
+## How to Run
+
+1. Install dependencies:
+   ```bash
+   pip install cassandra-driver pandas
+   ```
+
+2. Open and run the notebook:
+   ```bash
+   jupyter notebook "Project_1B_ Project_Template.ipynb"
+   ```
+
+The notebook walks through preprocessing the CSV data, creating Cassandra tables, inserting records, and running validation queries.