DBT Transform

The transform action executes a dbt transformation using the DBT runtime. A Task is created by calling run() on the Function; task parameters are passed through that call.

Overview

The DBT runtime wraps the DBT CLI and executes SQL/dbt transformations. At a high level:

1. Downloads input dataitems and loads them into temporary tables in the configured Postgres database
2. Collects the dbt project/code and generates required artifacts (profiles.yml, dbt_project.yml, etc.)
3. Runs the dbt transformation
4. Writes the resulting table back to the database and creates a Dataitem representing the output

Quick example with bare minimum parameters

import digitalhub as dh

# Create function with dbt code
function = dh.new_function(
    name="my-function",
    kind="dbt",
    code="SELECT * FROM {{ ref('my_table_ref') }}"
)

# Execute transformation
run = function.run(
    action="transform",
    inputs={"my_table_ref": dataitem.key},
    outputs={"output_table": "mapped-name"}
)

Parameters

Function Parameters

Must be specified when creating the function.

Name	Type	Description
project	str	Project name. Required only when creating from the library; otherwise MUST NOT be set.
name	str	Name that identifies the object. Required.
kind	str	Function kind. Required. Must be dbt
uuid	str	Object ID in UUID4 format.
description	str	Description of the object.
labels	list[str]	List of labels.
embedded	bool	Whether the object should be embedded in the project.
code_src	str	URI pointing to the source code.
code	str	Source code provided as plain text.
base64	str	Source code encoded as base64.
handler	str	Function entrypoint.
lang	str	Source code language (informational).

Task Parameters

Can only be specified when calling function.run().

Name	Type	Description
action	str	Task action. Required. Must be transform
node_selector	list[dict]	Node selector.
volumes	list[dict]	List of volumes.
resources	dict	Resource limits/requests.
affinity	dict	Affinity configuration.
tolerations	list[dict]	Tolerations.
envs	list[dict]	Environment variables.
secrets	list[str]	List of secret names.
profile	str	Profile template.

Run Parameters

Can only be specified when calling function.run().

Name	Type	Description
local_execution	bool	Execute the run locally instead of remotely. (Default: False)
inputs	dict	Mapping of function argument names to entity keys.
outputs	dict	Mapping of outputs. MUST BE in the form:
parameters	dict	Extra parameters passed to the function.

Entity methods

Run methods

Once the run is created, you can access its attributes and methods through the run object.

output

Get run's output by name.

Parameters:

Name	Type	Description	Default
output_name	str	Key of the result.	required
as_key	bool	If True, return result as key.	False
as_dict	bool	If True, return result as dictionary.	False

Returns:

Type	Description
Entity | dict | str | None	Result.

outputs

Get run's outputs.

Parameters:

Name	Type	Description	Default
as_key	bool	If True, return results as keys.	False
as_dict	bool	If True, return results as dictionaries.	False

Returns:

Type	Description
dict	List of output objects.