6796398924
- Move timesfm-forecast into apps/ directory - Flatten Udacity portfolio projects from deep URL-encoded paths into data-engineering/01-XX numbered directories - Remove old My-Data-Engineering-Portifolio/ parent directory - Rewrite root README.md: professional overview with badges, project table, and repo structure diagram - Create data-engineering/README.md with per-project descriptions - Add README.md for 02-cassandra-modeling (was missing) - Add README.md for 05-airflow-pipelines (was missing) - Normalize capstone readme.md -> README.md - Update .gitignore: add *.cfg, *.env, *.zip, *.sas7bdat, Jupyter checkpoints, IDE dirs; remove uv.lock exclusion - Add dwh.cfg.example and dl.cfg.example credential templates - Untrack real credential files (dwh.cfg, dl.cfg) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
91 lines
4.6 KiB
Markdown
91 lines
4.6 KiB
Markdown
### Hey, I'm Gabriel! 👋
|
|
|
|
- 👀 Interested in **Data Engineering · Data Science · ML Engineering**
|
|
- 🌱 Currently learning ML System Design, AI Engineering, and building scalable data pipelines
|
|
- 🤝 Open to collaboration on open source projects
|
|
- 📫 Reach me at: gabriel.pereira@protonmail.com
|
|
|
|
<div align="center">
|
|
|
|
[](https://www.linkedin.com/in/gabrielpereira-dev)
|
|
|
|
---
|
|
|
|
### Skills
|
|
|
|
<p align="center">
|
|
<a href="https://www.postgresql.org/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/profileme-dev/refs/heads/main/public/icons/skills/postgresql-colored.svg" width="36" height="36" alt="PostgreSQL" /></a>
|
|
<a href="https://www.python.org/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/python-colored.svg" width="36" height="36" alt="Python" /></a>
|
|
<a href="https://docs.microsoft.com/en-us/cpp/?view=msvc-170" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/cplusplus-colored.svg" width="36" height="36" alt="C++" /></a>
|
|
<a href="https://aws.amazon.com" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/aws-colored.svg" width="36" height="36" alt="AWS" /></a>
|
|
<a href="https://www.docker.com/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/docker-colored.svg" width="36" height="36" alt="Docker" /></a>
|
|
<a href="https://www.linux.org/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/linux.svg" width="36" height="36" alt="Linux" /></a>
|
|
<a href="https://www.raspberrypi.org/" target="_blank" rel="noreferrer"><img src="https://raw.githubusercontent.com/danielcranney/readme-generator/main/public/icons/skills/raspberrypi-colored.svg" width="36" height="36" alt="Raspberry Pi" /></a>
|
|
</p>
|
|
|
|
</div>
|
|
|
|
---
|
|
|
|
## 📁 Repository Structure
|
|
|
|
This is my personal workspace — a monorepo for data engineering and data science projects.
|
|
|
|
```
|
|
workspace/
|
|
├── apps/ # Production applications
|
|
├── data-engineering/ # Udacity Data Engineering Nanodegree portfolio
|
|
└── skills/ # dbt reference templates & best practices
|
|
```
|
|
|
|
---
|
|
|
|
## 🚀 Apps
|
|
|
|
### [`apps/timesfm-forecast`](./apps/timesfm-forecast)
|
|
|
|
A production-grade time series forecasting application powered by **Google's TimesFM** model.
|
|
|
|
| | |
|
|
|---|---|
|
|
| **Stack** | Python · Streamlit · PyTorch · HuggingFace Transformers |
|
|
| **Features** | Multi-key forecasting · CSV upload/download · GPU/CPU support |
|
|
| **Package manager** | `uv` |
|
|
|
|
**Quick start:**
|
|
```bash
|
|
cd apps/timesfm-forecast
|
|
uv venv && source .venv/bin/activate
|
|
uv pip install -e .
|
|
timesfm-app
|
|
```
|
|
|
|
---
|
|
|
|
## 📚 Data Engineering Portfolio
|
|
|
|
Udacity Data Engineering Nanodegree projects — a hands-on progression through core DE tools.
|
|
[🎓 View Credential](https://graduation.udacity.com/confirm/43TR6CQZ)
|
|
|
|
| # | Project | Stack | Description |
|
|
|---|---------|-------|-------------|
|
|
| 1 | [Postgres Data Modeling](./data-engineering/01-postgres-modeling) | Python · PostgreSQL | Star schema design for music streaming analytics |
|
|
| 2 | [Cassandra Data Modeling](./data-engineering/02-cassandra-modeling) | Python · Apache Cassandra | Denormalized NoSQL schema for query-first design |
|
|
| 3 | [Cloud Data Warehouse](./data-engineering/03-cloud-data-warehouse) | Python · AWS S3 · Redshift | Cloud ETL pipeline with staging and fact/dim tables |
|
|
| 4 | [Spark Data Lake](./data-engineering/04-spark-data-lake) | PySpark · AWS S3 | Large-scale partitioned processing with Parquet output |
|
|
| 5 | [Airflow Pipelines](./data-engineering/05-airflow-pipelines) | Apache Airflow · Redshift | Orchestrated hourly ETL with modular custom operators |
|
|
| 6 | [Immigration Capstone](./data-engineering/06-immigration-capstone) | PySpark · AWS S3 | Multi-source data lake integrating immigration, weather & demographics |
|
|
|
|
See [`data-engineering/README.md`](./data-engineering/README.md) for full details.
|
|
|
|
---
|
|
|
|
## 🛠️ dbt Skills
|
|
|
|
The [`skills/`](./skills) folder contains reference templates and best-practice guides for dbt projects:
|
|
|
|
- Staging, Transform, and Mart model patterns
|
|
- Incremental materialization strategies
|
|
- SQL style guide
|
|
- YAML schema testing templates
|