98 lines
2.5 KiB
Markdown
98 lines
2.5 KiB
Markdown
---
|
|
name: dbt_yaml_testing
|
|
description: "Step-by-step instructions for writing dbt schema.yml files with model descriptions, column descriptions, and data tests. Use for any dbt project."
|
|
---
|
|
|
|
# Write dbt Schema YAML & Data Tests
|
|
|
|
Every dbt model needs a schema entry with descriptions and tests. This skill covers writing `schema.yml` files correctly.
|
|
|
|
---
|
|
|
|
## Steps
|
|
|
|
**1. Locate or create the schema file**
|
|
- Naming convention: `schema.yml` (one per folder is fine) or `<model_name>.yml`.
|
|
- Place it in the **same directory** as the model.
|
|
- Source definitions use `src_<system>.yml`.
|
|
|
|
**2. Add the model block**
|
|
```yaml
|
|
models:
|
|
- name: <model_name>
|
|
description: "One sentence: what this model represents, grain, and any key filters."
|
|
```
|
|
|
|
**3. Add column definitions**
|
|
- Every column should have a `description:`.
|
|
- Use `data_tests:` (not the deprecated `tests:` key).
|
|
|
|
```yaml
|
|
columns:
|
|
- name: <primary_key_column>
|
|
description: "Unique identifier for each row."
|
|
data_tests:
|
|
- unique
|
|
- not_null
|
|
|
|
- name: <foreign_key_column>
|
|
description: "Foreign key to <other_model>."
|
|
data_tests:
|
|
- not_null
|
|
- relationships:
|
|
to: ref('<other_model>')
|
|
field: <field_name>
|
|
|
|
- name: <status_column>
|
|
description: "Status of the record."
|
|
data_tests:
|
|
- accepted_values:
|
|
values: ['active', 'inactive', 'pending']
|
|
```
|
|
|
|
**4. Set test severity (optional override)**
|
|
- Project default is usually `warn`. Override critical tests to `error`:
|
|
```yaml
|
|
data_tests:
|
|
- unique:
|
|
severity: error
|
|
- not_null:
|
|
severity: error
|
|
```
|
|
|
|
**5. Add singular tests for complex assertions**
|
|
- Place in `tests/` directory.
|
|
- A passing test returns **zero rows** (the query selects *failing* records):
|
|
```sql
|
|
-- tests/assert_<model>_<condition>.sql
|
|
select <key_column>
|
|
from {{ ref('<model_name>') }}
|
|
where <failing_condition>
|
|
```
|
|
|
|
---
|
|
|
|
## Minimum Required Tests Per Layer
|
|
|
|
| Layer | Required Tests |
|
|
|-------|---------------|
|
|
| Staging | `unique` + `not_null` on primary key |
|
|
| Transform | `unique` + `not_null` on `unique_key` columns |
|
|
| Marts (fact) | `unique` + `not_null` on PK; `relationships` on FK |
|
|
| Marts (dim) | `unique` + `not_null` on natural/surrogate key |
|
|
|
|
---
|
|
|
|
## YAML Style Rules
|
|
|
|
- **2-space indentation** (no tabs).
|
|
- Use `data_tests:` not `tests:`.
|
|
- Every model must have a `description:`.
|
|
- Quote descriptions with `"double quotes"`.
|
|
|
|
---
|
|
|
|
## Template
|
|
|
|
See `schema_template.yml` in this folder.
|