refactor: restructure monorepo for clean portfolio layout
- Move timesfm-forecast into apps/ directory - Flatten Udacity portfolio projects from deep URL-encoded paths into data-engineering/01-XX numbered directories - Remove old My-Data-Engineering-Portifolio/ parent directory - Rewrite root README.md: professional overview with badges, project table, and repo structure diagram - Create data-engineering/README.md with per-project descriptions - Add README.md for 02-cassandra-modeling (was missing) - Add README.md for 05-airflow-pipelines (was missing) - Normalize capstone readme.md -> README.md - Update .gitignore: add *.cfg, *.env, *.zip, *.sas7bdat, Jupyter checkpoints, IDE dirs; remove uv.lock exclusion - Add dwh.cfg.example and dl.cfg.example credential templates - Untrack real credential files (dwh.cfg, dl.cfg) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -0,0 +1,42 @@
|
||||
class SqlQueries:
|
||||
songplay_table_insert = ("""
|
||||
SELECT
|
||||
md5(events.sessionid || events.start_time) songplay_id,
|
||||
events.start_time,
|
||||
events.userid,
|
||||
events.level,
|
||||
songs.song_id,
|
||||
songs.artist_id,
|
||||
events.sessionid,
|
||||
events.location,
|
||||
events.useragent
|
||||
FROM (SELECT TIMESTAMP 'epoch' + ts/1000 * interval '1 second' AS start_time, *
|
||||
FROM staging_events
|
||||
WHERE page='NextSong') events
|
||||
LEFT JOIN staging_songs songs
|
||||
ON events.song = songs.title
|
||||
AND events.artist = songs.artist_name
|
||||
AND events.length = songs.duration
|
||||
""")
|
||||
|
||||
user_table_insert = ("""
|
||||
SELECT distinct userid, firstname, lastname, gender, level
|
||||
FROM staging_events
|
||||
WHERE page='NextSong'
|
||||
""")
|
||||
|
||||
song_table_insert = ("""
|
||||
SELECT distinct song_id, title, artist_id, year, duration
|
||||
FROM staging_songs
|
||||
""")
|
||||
|
||||
artist_table_insert = ("""
|
||||
SELECT distinct artist_id, artist_name, artist_location, artist_latitude, artist_longitude
|
||||
FROM staging_songs
|
||||
""")
|
||||
|
||||
time_table_insert = ("""
|
||||
SELECT start_time, extract(hour from start_time), extract(day from start_time), extract(week from start_time),
|
||||
extract(month from start_time), extract(year from start_time), extract(dayofweek from start_time)
|
||||
FROM songplays
|
||||
""")
|
||||
Reference in New Issue
Block a user