gabspereira/workspace
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add claude skills

Browse Source
This commit is contained in:
@gabriel.pereira
2026-03-26 16:29:53 -03:00
parent 11d3cc66d5
commit 5c4e6075e1
11 changed files with 716 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

82
skills/dbt-create-mart/SKILL.md Normal file
View File

@@ -0,0 +1,82 @@
---
name: dbt_create_mart
description: "Step-by-step instructions for creating dbt Mart models (facts and dimensions) as thin wrappers over transform models. Use for any dbt + Snowflake project."
---
# Create a dbt Mart Model (Fact or Dimension)
Mart models are **thin wrappers** over transform models. They expose clean, BI-ready tables without duplicating business logic.
---
## Fact Model Steps
**1. Name the file**
- Pattern: `fct_<domain>_<metric>.sql`
- Example: `fct_sales_orders.sql`, `fct_ar_open_items.sql`
- Lives in `models/marts/facts/`
**2. Write a thin SELECT wrapper**
Option A — expose all columns (most common):
```sql
select {{ dbt_utils.star(ref('<trf_model_name>')) }}
from {{ ref('<trf_model_name>') }}
```
Option B — add a light filter or derived column:
```sql
select
*,
<derived_column> as <alias>
from {{ ref('<trf_model_name>') }}
where <filter_condition>
```
**3. Materialization**
- Default: `view` (inherited from `dbt_project.yml`).
- Override to `table` only if the BI tool performs poorly on views.
---
## Dimension Model Steps
**1. Name the file**
- Pattern: `dim_<entity>.sql`
- Example: `dim_customers.sql`, `dim_materials.sql`
- Lives in `models/marts/dim/`
**2. Write the SELECT**
- Dimensions are usually deduped/distinct entity lists from staging or transform.
```sql
select distinct
<natural_key> as <entity>_key,
<descriptive_field_1>,
<descriptive_field_2>
from {{ ref('<stg_or_trf_model>') }}
where <natural_key> is not null
```
**3. Add schema entry**
- File: `models/marts/schema.yml`
- Every fact: test `unique` + `not_null` on the primary key.
- Every dimension: test `unique` + `not_null` on the surrogate/natural key.
- Add `relationships` tests on foreign keys in fact models.
---
## Rules
| Rule | Detail |
|------|--------|
| No business logic | Mart = wrapper only; logic belongs in transform |
| Materialization | `view` by default; `table` only if needed for BI performance |
| Column naming | `snake_case` for marts; inherit from the transform model |
| `ref()` only | Always reference the transform model with `{{ ref() }}` |
| BI-facing names | Rename columns here if the BI layer needs friendlier names |
---
## Templates
See `mart_fct_template.sql` and `mart_dim_template.sql` in this folder.

10
skills/dbt-create-mart/mart_dim_template.sql Normal file
View File

@@ -0,0 +1,10 @@
-- Dimension model: deduplicated entity list from staging or transform.
-- Provide one row per unique entity (customer, product, employee, etc.)
select distinct
<natural_key> as <entity>_key,
<descriptive_field_1> as <readable_field_1>,
<descriptive_field_2> as <readable_field_2>
from {{ ref('<stg_or_trf_model>') }}
where <natural_key> is not null

8
skills/dbt-create-mart/mart_fct_template.sql Normal file
View File

@@ -0,0 +1,8 @@
-- Fact model: thin wrapper exposing all columns from a transform model.
-- Most common pattern — use dbt_utils.star() to forward all columns.
select {{ dbt_utils.star(ref('<trf_model_name>')) }}
from {{ ref('<trf_model_name>') }}
-- Optional: add a lightweight filter if needed
-- where is_active = true

70
skills/dbt-create-staging/SKILL.md Normal file
View File

@@ -0,0 +1,70 @@
---
name: dbt_create_staging
description: "Step-by-step instructions for creating a dbt staging model that maps a raw source table to clean, renamed columns. Use for any dbt + Snowflake project."
---
# Create a dbt Staging Model
Staging models are **1:1 with a source table** — no joins. Their only job is to rename SAP/raw columns to readable names and lightly clean types.
---
## Steps
**1. Determine the source table and source name**
- Identify the `source_name` and `table_name` from `src_<system>.yml` (e.g., `e1p`, `vbak`).
- Staging models live in `models/staging/<system>/`.
**2. Name the file**
- Pattern: `stg_<system>_<table>.sql`
- Examples: `stg_e1p_vbak.sql`, `stg_gx_prm_projects.sql`
**3. Add the `source` CTE**
```sql
with source as (
select * from {{ source('<source_name>', '<table_name>') }}
),
```
**4. Add the `renamed` CTE — rename and recast columns**
- Group columns by category with separator comments: `ids`, `strings`, `numerics`, `dates`, `timestamps`, `booleans`.
- Rename all cryptic column names to readable `snake_case`.
- Cast types here — never downstream.
- Keep SAP dates (stored as `YYYYMMDD` strings) as-is unless you need a `DATE` type.
**5. Add a dev data limit (if using sie_dbt_utils)**
```sql
from source
where 1=1
{{ sie_dbt_utils.dynamic_limit() }}
```
> Omit the macro if not available; add a `-- TODO: add dev limit` comment.
**6. Close with `select * from renamed`**
```sql
select * from renamed
```
**7. Add a schema entry in the matching `schema.yml`**
- File: `models/staging/<system>/schema.yml`
- Add model name, description, and `unique` + `not_null` tests on the primary key.
---
## Rules
| Rule | Detail |
|------|--------|
| No joins | Staging = 1 source table only |
| Rename once | All downstream models use the clean names from staging |
| Materialization | Always `view` (set globally in `dbt_project.yml`) |
| `ref()` / `source()` | Never hardcode schema or table names |
| Column naming | `snake_case`; booleans → `is_*` / `has_*`; dates → `*_date`; timestamps → `*_at`; amounts → `*_<unit>` |
---
## Template
See `stg_template.sql` in this folder.

35
skills/dbt-create-staging/stg_template.sql Normal file
View File

@@ -0,0 +1,35 @@
with source as (
select * from {{ source('<source_name>', '<table_name>') }}
),
renamed as (
select
---------- ids
<id_column> as <readable_id>,
---------- strings
<string_column> as <readable_name>,
---------- numerics
<amount_column> as <readable_amount>,
---------- dates
<date_column> as <event>_date,
---------- timestamps
<timestamp_column> as <event>_at,
---------- booleans
<flag_column> as is_<condition>
from source
where 1=1
{{ sie_dbt_utils.dynamic_limit() }}
-- remove the line above if sie_dbt_utils is not available
)
select * from renamed

94
skills/dbt-create-transform/SKILL.md Normal file
View File

@@ -0,0 +1,94 @@
---
name: dbt_create_transform
description: "Step-by-step instructions for creating a dbt incremental transform model with business logic, joins, and date filters. Use for any dbt + Snowflake project."
---
# Create a dbt Transform Model
Transform models contain **business logic** — joins, aggregations, date filters, and derived columns. They are almost always **incremental** to handle large datasets efficiently.
---
## Steps
**1. Name the file**
- Pattern: `trf_<domain>_<entity>.sql`
- Examples: `trf_sales_orders.sql`, `trf_ar_open_items.sql`
- Lives in `models/transform/<domain>/`
**2. Add the config block (top of file)**
```sql
{{ config(
materialized = 'incremental',
unique_key = ['key_col_1', 'key_col_2'],
on_schema_change = 'sync_all_columns',
incremental_strategy = 'delete+insert'
) }}
```
- `unique_key`: list of columns that uniquely identify a row.
- `on_schema_change = 'sync_all_columns'`: automatically adds/removes columns on model changes.
**3. (Optional) Add full-refresh protection**
```sql
{{ sie_dbt_utils.full_refresh_protection() }}
```
> Prevents accidental full-refresh data loss in production. Add on line 1 if using sie_dbt_utils.
**4. Define CTEs — one per source model**
- Name each CTE after the staging model or business concept it represents.
- Apply tenant/client filters immediately in the CTE (e.g., `WHERE mandt IN ('022', '100')`).
```sql
with
orders as (
select * from {{ ref('stg_<system>_<table>') }}
where mandt in ('022', '100') -- replace with your tenant filter
),
customers as (
select * from {{ ref('stg_<system>_kna1') }}
),
```
**5. Write the final SELECT with joins**
- Use `UPPER_CASE` for column aliases in transform models (SAP/warehouse convention).
- Qualify all columns with CTE alias when joining.
- Use `left join`; never bare `join`.
**6. Add date range filter**
- Filter to a rolling window (e.g., last 2 fiscal years) to keep the table small.
- Siemens fiscal year = calendar year + 3 months shift:
```sql
where year(dateadd(month, 3, to_date(erdat, 'yyyymmdd')))
>= year(dateadd(year, -2, dateadd(month, 3, getdate())))
```
> Adapt the date logic to your project's fiscal/calendar year convention.
**7. Add the incremental block**
```sql
{% if is_incremental() %}
and <date_column> >= (select max(<date_column>) from {{ this }})
{% endif %}
```
> Prefer `sie_dbt_utils.incremental_filter('col')` if available — it compiles the MAX watermark at build time.
**8. Add schema entry**
- File: `models/transform/<domain>/schema.yml`
- Include model description and `unique` + `not_null` on the primary key.
---
## Rules
| Rule | Detail |
|------|--------|
| Materialization | `incremental` (set in config block) |
| Column casing | `UPPER_CASE` aliases for transform/distribute layers |
| Tenant filter | Apply `mandt`/client filter in each base CTE |
| No hardcoding | Use `{{ ref() }}` and `{{ source() }}` only |
| Incremental strategy | `delete+insert` with `unique_key` |
---
## Template
See `trf_template.sql` in this folder.

44
skills/dbt-create-transform/trf_template.sql Normal file
View File

@@ -0,0 +1,44 @@
{{ config(
materialized = 'incremental',
unique_key = ['KEY_COL_1', 'KEY_COL_2'],
on_schema_change = 'sync_all_columns',
incremental_strategy = 'delete+insert'
) }}
{{- sie_dbt_utils.full_refresh_protection() -}}
-- remove the line above if sie_dbt_utils is not available
with
base_table as (
select *
from {{ ref('stg_<system>_<table>') }}
where mandt in ('022', '100') -- replace with your tenant/client filter
),
related_table as (
select *
from {{ ref('stg_<system>_<other_table>') }}
),
final as (
select
base_table.COLUMN_1 as READABLE_COLUMN_1,
base_table.COLUMN_2 as READABLE_COLUMN_2,
related_table.COLUMN_A as READABLE_COLUMN_A
from base_table
left join related_table
on base_table.join_key = related_table.join_key
where year(dateadd(month, 3, to_date(base_table.date_col, 'yyyymmdd')))
>= year(dateadd(year, -2, dateadd(month, 3, getdate())))
-- ^ Siemens fiscal year filter (FY = calendar year + 3 months)
-- Replace with your project's date filter logic
{% if is_incremental() %}
and year(dateadd(month, 3, to_date(base_table.date_col, 'yyyymmdd')))
<= year(add_months(current_date(), 3))
{% endif %}
)
select * from final

121
skills/dbt-incremental-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,121 @@
---
name: dbt_incremental_patterns
description: "Patterns and best practices for dbt incremental models: delete+insert strategy, watermark filters, is_incremental() blocks, and full-refresh protection. Reusable across any dbt project."
---
# dbt Incremental Model Patterns
Use incremental models when source tables are too large to rebuild from scratch on every run. They process only new or changed records.
---
## Standard Config Block
```sql
{{ config(
materialized = 'incremental',
unique_key = ['key_col_1', 'key_col_2'],
on_schema_change = 'sync_all_columns',
incremental_strategy = 'delete+insert'
) }}
```
| Config | What it does |
|--------|-------------|
| `unique_key` | Columns that uniquely identify a row — used to delete matching rows before re-inserting |
| `on_schema_change = 'sync_all_columns'` | Automatically adds/drops columns when the model definition changes |
| `incremental_strategy = 'delete+insert'` | Deletes matching rows then re-inserts — safe for late-arriving data |
---
## `is_incremental()` Filter Block
Limit the data processed on incremental runs using a watermark on a date/timestamp column:
```sql
where event_date >= '2020-01-01'
{% if is_incremental() %}
and event_date >= (
select max(event_date)
from {{ this }}
)
{% endif %}
```
- The subquery `(select max(...) from {{ this }})` compiles to the current table's max value.
- The outer filter (before `{% if %}`) runs on the initial full load.
- Place the `{% if is_incremental() %}` block **inside** the `where` clause.
---
## Watermark Pattern (preferred)
For better performance, compute the watermark outside the final query:
```sql
{% set max_date_query %}
select max(event_date) from {{ this }}
{% endset %}
{% if is_incremental() %}
{% set max_date = run_query(max_date_query).columns[0].values()[0] %}
{% endif %}
-- ... main query ...
where 1=1
{% if is_incremental() %}
and event_date > '{{ max_date }}'
{% endif %}
```
> If using `sie_dbt_utils`, prefer `{{ sie_dbt_utils.incremental_filter('event_date') }}` — it compiles the watermark at build time.
---
## Full-Refresh Protection
Prevent accidental full-refresh in production by adding this at the top of the model:
```sql
{{ sie_dbt_utils.full_refresh_protection() }}
-- or, without sie_dbt_utils:
{% if flags.FULL_REFRESH and target.name == 'prod' %}
{{ exceptions.raise_compiler_error("Full refresh is disabled in production.") }}
{% endif %}
```
---
## When to Use Each Materialization
| Materialization | When to use |
|-----------------|-------------|
| `view` | Staging and mart wrappers — always fresh, low overhead |
| `incremental` | Large tables with frequent appends (>1M rows, updated daily) |
| `table` | Small reference tables, distribute layer — needs stable snapshot |
| `ephemeral` | Intermediate CTEs reused across multiple models |
---
## Incremental Run Commands
```bash
# Normal incremental run (only new records)
dbt run --select <model_name>
# Force full rebuild (use with caution in prod)
dbt run --select <model_name> --full-refresh
# Run only modified models and downstream (CI pattern)
dbt run --select state:modified+
```
---
## Common Pitfalls
- **Missing `unique_key`**: Without it, `delete+insert` can't match rows → duplicates.
- **No `{% if is_incremental() %}`**: Every run re-processes the full history → slow.
- **Wrong column in watermark**: Use a column that reflects when the record was last modified, not created.
- **`on_schema_change` not set**: Schema changes will error or silently drop columns.

124
skills/dbt-sql-style/SKILL.md Normal file
View File

@@ -0,0 +1,124 @@
---
name: dbt_sql_style
description: "General SQL style guide for writing dbt models: CTE structure, formatting conventions, keywords, joins, CASE WHEN, and aggregations. Reusable across any dbt project."
---
# SQL Style Guide for dbt Models
Apply these conventions to all SQL written in dbt models. Consistent style improves readability and code review speed.
---
## Core Rules
| Rule | Convention |
|------|-----------|
| Keywords | **lowercase** (`select`, `from`, `where`, `join`, `group by`, `order by`) |
| Indentation | **4 spaces** — no tabs |
| Commas | **Trailing** (end of line), not leading |
| One column per line | Always — no inline column lists |
| Final SELECT | `select * from <last_cte_name>` |
---
## CTE Structure
Always use CTEs instead of subqueries. Name each CTE after the source or concept it represents.
```sql
with source as (
select * from {{ source('system', 'table') }}
),
renamed as (
select
---------- ids
id_column,
---------- strings
name_column as readable_name,
---------- dates
date_column as event_date,
---------- numerics
amount_column as total_usd
from source
)
select * from renamed
```
- Blank line after the opening `(` and before the closing `)` of each CTE.
- Blank line between each CTE block.
---
## Joins
```sql
from orders
left join customers
on orders.customer_id = customers.id
left join products
on orders.product_id = products.id
```
- **Always use `left join`** unless you have a specific reason for inner/cross.
- Never use bare `join` — always be explicit.
- **Always qualify columns** with table/CTE alias when joining multiple sources.
- Place `on` conditions on the next line, indented 4 spaces.
---
## CASE WHEN
```sql
case
when status = 'cancelled' and cancelled_date is not null
then cancelled_date
when status = 'expired'
then expiration_date
else created_date
end as effective_date,
```
- One `when` condition per line.
- `then` indented 4 spaces under `when`.
- Always include `else`.
- Close with `end as <alias>`.
---
## Aggregations
```sql
select
region,
order_type,
count(*) as order_count,
sum(net_value) as total_net_value_usd
from orders
group by 1, 2
```
- Use **positional references** in `group by` (e.g., `group by 1, 2`).
- Name aggregated columns clearly: `sum(x) as total_x`.
---
## Column Naming Conventions
| Type | Pattern | Example |
|------|---------|---------|
| Boolean | `is_*` / `has_*` | `is_cancelled`, `has_attachment` |
| Date | `*_date` | `created_date`, `shipped_date` |
| Timestamp | `*_at` | `created_at`, `updated_at` |
| Numeric amount | `*_<unit>` | `total_usd`, `quantity_ea` |
| Foreign key | `<entity>_id` | `customer_id`, `order_id` |

97
skills/dbt-yaml-testing/SKILL.md Normal file
View File

@@ -0,0 +1,97 @@
---
name: dbt_yaml_testing
description: "Step-by-step instructions for writing dbt schema.yml files with model descriptions, column descriptions, and data tests. Use for any dbt project."
---
# Write dbt Schema YAML & Data Tests
Every dbt model needs a schema entry with descriptions and tests. This skill covers writing `schema.yml` files correctly.
---
## Steps
**1. Locate or create the schema file**
- Naming convention: `schema.yml` (one per folder is fine) or `<model_name>.yml`.
- Place it in the **same directory** as the model.
- Source definitions use `src_<system>.yml`.
**2. Add the model block**
```yaml
models:
- name: <model_name>
description: "One sentence: what this model represents, grain, and any key filters."
```
**3. Add column definitions**
- Every column should have a `description:`.
- Use `data_tests:` (not the deprecated `tests:` key).
```yaml
columns:
- name: <primary_key_column>
description: "Unique identifier for each row."
data_tests:
- unique
- not_null
- name: <foreign_key_column>
description: "Foreign key to <other_model>."
data_tests:
- not_null
- relationships:
to: ref('<other_model>')
field: <field_name>
- name: <status_column>
description: "Status of the record."
data_tests:
- accepted_values:
values: ['active', 'inactive', 'pending']
```
**4. Set test severity (optional override)**
- Project default is usually `warn`. Override critical tests to `error`:
```yaml
data_tests:
- unique:
severity: error
- not_null:
severity: error
```
**5. Add singular tests for complex assertions**
- Place in `tests/` directory.
- A passing test returns **zero rows** (the query selects *failing* records):
```sql
-- tests/assert_<model>_<condition>.sql
select <key_column>
from {{ ref('<model_name>') }}
where <failing_condition>
```
---
## Minimum Required Tests Per Layer
| Layer | Required Tests |
|-------|---------------|
| Staging | `unique` + `not_null` on primary key |
| Transform | `unique` + `not_null` on `unique_key` columns |
| Marts (fact) | `unique` + `not_null` on PK; `relationships` on FK |
| Marts (dim) | `unique` + `not_null` on natural/surrogate key |
---
## YAML Style Rules
- **2-space indentation** (no tabs).
- Use `data_tests:` not `tests:`.
- Every model must have a `description:`.
- Quote descriptions with `"double quotes"`.
---
## Template
See `schema_template.yml` in this folder.

31
skills/dbt-yaml-testing/schema_template.yml Normal file
View File

@@ -0,0 +1,31 @@
version: 2
models:
- name: <model_name>
description: "One sentence describing the model: grain, scope, and key filters."
columns:
- name: <primary_key_column>
description: "Unique identifier for each row."
data_tests:
- unique
- not_null
- name: <foreign_key_column>
description: "Foreign key to the <entity> dimension."
data_tests:
- not_null
- relationships:
to: ref('<dim_model_name>')
field: <field_name>
- name: <status_column>
description: "Status of the record. One of: active, inactive, pending."
data_tests:
- not_null
- accepted_values:
values: ['active', 'inactive', 'pending']
- name: <optional_column>
description: "Description of what this column represents."
# no tests required for non-critical columns