Skip to content

Commit

Permalink
Merge pull request dbt-labs#884 from dbt-labs/add-models-selector-doc…
Browse files Browse the repository at this point in the history 
…umentation

Add toggles to differentiate between modern and legacy node selection syntax
  • Loading branch information
@joellabes
joellabes authored Oct 22, 2021
2 parents 34e27e8 + 70c0819 commit 0093f2c
Show file tree
Hide file tree
Showing 9 changed files with 943 additions and 183 deletions.
55 changes: 41 additions & 14 deletions website/docs/reference/node-selection/defer.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,32 @@ Defer requires that a manifest from a previous dbt invocation be passed to the `

### Usage

```shell
$ dbt run --select [...] --defer --state path/to/artifacts
$ dbt test --select [...] --defer --state path/to/artifacts
```
<Tabs
defaultValue="modern"
values={[
{ label: 'v0.21.0 and later', value: 'modern', },
{ label: 'v0.20.x and earlier', value: 'legacy', }
]
}>
<TabItem value="modern">

```shell
$ dbt run --select [...] --defer --state path/to/artifacts
$ dbt test --select [...] --defer --state path/to/artifacts
```

</TabItem>
<TabItem value="legacy">

```shell
$ dbt run --models [...] --defer --state path/to/artifacts
$ dbt test --models [...] --defer --state path/to/artifacts
```

</TabItem>
</Tabs>



When the `--defer` flag is provided, dbt will resolve `ref` calls differently depending on two criteria:
1. Is the referenced node included in the model selection criteria of the current run?
Expand All @@ -43,7 +65,6 @@ In my local development environment, I create all models in my target schema, `d
I access the dbt-generated [artifacts](artifacts) (namely `manifest.json`) from a production run, and copy them into a local directory called `prod-run-artifacts`.

### run

I've been working on `model_b`:

<File name='models/model_b.sql'>
Expand All @@ -53,13 +74,17 @@ select

id,
count(*)

from {{ ref('model_a') }}
group by 1
```

I want to test my changes. Nothing exists in my development schema, `dev_alice`.

### test
:::info
Before dbt v0.21, use the `--models` flag instead of `--select`.
:::
</File>

<Tabs
Expand All @@ -80,15 +105,15 @@ $ dbt run --select model_b

```sql
create or replace view dev_me.model_b as (

select

id,
count(*)

from dev_alice.model_a
group by 1

)
```

Expand All @@ -107,15 +132,15 @@ $ dbt run --select model_b --defer --state prod-run-artifacts

```sql
create or replace view dev_me.model_b as (

select

id,
count(*)

from prod.model_a
group by 1

)
```

Expand All @@ -126,8 +151,6 @@ Because `model_a` is unselected, dbt will check to see if `dev_alice.model_a` ex
</TabItem>
</Tabs>

### test

I also have a `relationships` test that establishes referential integrity between `model_a` and `model_b`:

<File name='models/resources.yml'>
Expand All @@ -147,6 +170,10 @@ models:
(A bit silly, since all the data in `model_b` had to come from `model_a`, but suspend your disbelief.)

:::info
Before dbt v0.21, use the `--models` flag instead of `--select`.
:::

</File>

<Tabs
Expand Down
28 changes: 24 additions & 4 deletions website/docs/reference/node-selection/exclude.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,29 @@ title: "Exclude"
### Excluding models
dbt provides an `--exclude` flag with the same semantics as `--select`. Models specified with the `--exclude` flag will be removed from the set of models selected with `--select`.

```bash
$ dbt run --select my_package.*+ --exclude my_package.a_big_model+
```
<Tabs
defaultValue="modern"
values={[
{ label: 'v0.21.0 and later', value: 'modern', },
{ label: 'v0.20.x and earlier', value: 'legacy', }
]
}>
<TabItem value="modern">

```bash
$ dbt run --select my_package.*+ --exclude my_package.a_big_model+
```

</TabItem>
<TabItem value="legacy">

```bash
$ dbt run --models my_package.*+ --exclude my_package.a_big_model+
```

</TabItem>
</Tabs>


Exclude a specific resource by its name or lineage:

Expand All @@ -22,4 +42,4 @@ $ dbt seed --exclude account_parent_mappings
# snapshot
$ dbt snapshot --exclude snap_order_statuses
$ dbt test --exclude orders+
```
```
92 changes: 78 additions & 14 deletions website/docs/reference/node-selection/graph-operators.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,67 @@ title: "Graph operators"
### The "plus" operator
If placed at the front of the model selector, `+` will select all parents of the selected model. If placed at the end of the string, `+` will select all children of the selected model.

```bash
$ dbt run --select my_model+ # select my_model and all children
$ dbt run --select +my_model # select my_model and all parents
$ dbt run --select +my_model+ # select my_model, and all of its parents and children
```
<Tabs
defaultValue="modern"
values={[
{ label: 'v0.21.0 and later', value: 'modern', },
{ label: 'v0.20.x and earlier', value: 'legacy', }
]
}>
<TabItem value="modern">

```bash
$ dbt run --select my_model+ # select my_model and all children
$ dbt run --select +my_model # select my_model and all parents
$ dbt run --select +my_model+ # select my_model, and all of its parents and children
```

</TabItem>
<TabItem value="legacy">

```bash
$ dbt run --models my_model+ # select my_model and all children
$ dbt run --models +my_model # select my_model and all parents
$ dbt run --models +my_model+ # select my_model, and all of its parents and children
```

</TabItem>
</Tabs>


### The "n-plus" operator
<Changelog>New in v0.18.0</Changelog>

You can adjust the behavior of the `+` operator by quantifying the number of edges
to step through.

```bash
$ dbt run --select my_model+1 # select my_model and its first-degree children
$ dbt run --select 2+my_model # select my_model, its first-degree parents, and its second-degree parents ("grandparents")
$ dbt run --select 3+my_model+4 # select my_model, its parents up to the 3rd degree, and its children down to the 4th degree
```
<Tabs
defaultValue="modern"
values={[
{ label: 'v0.21.0 and later', value: 'modern', },
{ label: 'v0.20.x and earlier', value: 'legacy', }
]
}>
<TabItem value="modern">

```bash
$ dbt run --select my_model+1 # select my_model and its first-degree children
$ dbt run --select 2+my_model # select my_model, its first-degree parents, and its second-degree parents ("grandparents")
$ dbt run --select 3+my_model+4 # select my_model, its parents up to the 3rd degree, and its children down to the 4th degree
```

</TabItem>
<TabItem value="legacy">

```bash
$ dbt run --models my_model+1 # select my_model and its first-degree children
$ dbt run --models 2+my_model # select my_model, its first-degree parents, and its second-degree parents ("grandparents")
$ dbt run --models 3+my_model+4 # select my_model, its parents up to the 3rd degree, and its children down to the 4th degree
```

</TabItem>
</Tabs>


### The "at" operator
The `@` operator is similar to `+`, but will also include _the parents of the children of the selected model_. This is useful in continuous integration environments where you want to build a model and all of its children, but the _parents_ of those children might not exist in the database yet. The selector `@snowplow_web_page_context` will build all three models shown in the diagram below.
Expand All @@ -31,7 +75,27 @@ The `@` operator is similar to `+`, but will also include _the parents of the ch
### The "star" operator
The `*` operator matches all models within a package or directory.

```bash
$ dbt run --select snowplow.* # run all of the models in the snowplow package
$ dbt run --select finance.base.* # run all of the models in models/finance/base
```
<Tabs
defaultValue="modern"
values={[
{ label: 'v0.21.0 and later', value: 'modern', },
{ label: 'v0.20.x and earlier', value: 'legacy', }
]
}>
<TabItem value="modern">

```bash
$ dbt run --select snowplow.* # run all of the models in the snowplow package
$ dbt run --select finance.base.* # run all of the models in models/finance/base
```

</TabItem>
<TabItem value="legacy">

```bash
$ dbt run --models snowplow.* # run all of the models in the snowplow package
$ dbt run --models finance.base.* # run all of the models in models/finance/base
```

</TabItem>
</Tabs>
Loading

0 comments on commit 0093f2c

Please sign in to comment.