Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/how-tos/datacoves/how_to_service_connections.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ A Service Connection consists of the following fields:
- **Environment Variables** The legacy method Datacoves used which would inject the connection credentials as environment variables into Airflow. The name of the service connection will be used to dynamically create [environment variables](/docs/reference/airflow/environment-service-connection-vars) which we inject into Airflow.
![Delivery Mode](assets/service_connection_delivery.jpg).
- **Connection Template** The connection template to base this service connection on(i.e. the defaults)
Depending on the template selected, additional fields will be displayed with the default values entered in the connection template. These default values can be overridden by toggling the indicator next to the given value. Enter the appropriate user, schema, and password. This is commonly a service account created specifically for Airflow and may differ between the development and production environment.
Depending on the template selected, additional fields will be displayed with the default values entered in the connection template. These default values can be overridden by toggling the indicator next to the given value. Enter the appropriate user, schema, and password. This is commonly a service account created specifically for Airflow and may differ between the development and production environment. **Schema is required** if the connection will be used with [`@task.datacoves_dbt`](/docs/reference/airflow/datacoves-decorators#taskdatacoves_dbt); leaving it empty causes a dbt profile validation error at runtime.

![Service Connection Connection Details](./assets/serviceconnection_editnew_details.png)

Expand Down
27 changes: 22 additions & 5 deletions docs/reference/airflow/datacoves-decorators.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,14 +58,31 @@ Datacoves dbt decorator supports all the [Datacoves dbt Operator params](/docs/r

**dbt profile generation:**

With the `connection_id` mentioned above, we create a temporary dbt profile (it only exists at runtime inside the Airflow DAG's worker). By default, this dbt profile contains the selected Service Credential connection details.
With the `connection_id` mentioned above, we create a temporary dbt profile (it only exists at runtime inside the Airflow DAG's worker).

The dbt profile `name` is defined either in Project or Environment settings, in their `Profile name` field. This can be overwritten by passing a custom `DATACOVES__DBT_PROFILE` environment variable to the decorator.
By default:

Users can also customize this dbt profile's connection details and/or target with the following params:
- **Connection details** (user, password, warehouse, database, schema, role, etc.) come from the selected service connection (`connection_id`).
- **Profile name** comes from the **Profile name** field in Project or Environment settings.
- **Target name** is `default`.

- `overrides`: a dictionary with override parameters such as warehouse, role, database, etc.
- `target`: the target name this temporary dbt profile will receive. Defaults to `default`.
:::warning
The service connection's **Schema** field must be populated when the connection will be used with `@task.datacoves_dbt`. Leaving it empty causes an error like:

```
Credentials in profile "my_dw_profile", target "dev" invalid: None is not of type 'string'
```

The Airflow Snowflake provider docs list `schema` as optional. That applies to the Airflow connection, not to the dbt profile generated from it, which requires `schema`.
:::

These can be overridden with the following params:

| What | How |
|---|---|
| Profile name | Pass a custom `DATACOVES__DBT_PROFILE` environment variable, e.g. `env={"DATACOVES__DBT_PROFILE": "prod"}` |
| Target name | `target`: the target name this temporary dbt profile will receive |
| Connection details | `overrides`: a dictionary with override parameters such as warehouse, role, database, etc. |

Basic example:

Expand Down
Loading