Skip to content

fix: remove databricks and spark credentials from state on 404 - #797

Open
lbarrosby wants to merge 1 commit into
dbt-labs:mainfrom
lbarrosby:fix/databricks-spark-credential-read-404
Open

lbarrosby wants to merge 1 commit into
dbt-labs:mainfrom
lbarrosby:fix/databricks-spark-credential-read-404

Conversation

@lbarrosby

Copy link
Copy Markdown

Problem

If a dbtcloud_databricks_credential or dbtcloud_spark_credential is deleted outside of Terraform (UI, API, or with its project), every following terraform plan fails with Error reading Databricks credential ... resource-not-found (or the Apache Spark equivalent). The only way out is terraform state rm.

Cause

Both Read methods turn any error from the client into a diagnostic. The other credential resources (Athena, BigQuery, Postgres, Snowflake, the semantic layer ones, ...) call helper.HandleResourceNotFound first, so a 404 removes the resource from state and Terraform plans to create it again. These two were missed.

Fix

Call helper.HandleResourceNotFound in both Read methods before returning the error, the same as the other credential resources. Added a changie entry under Fixes.

Tests

New behavior_test.go in each package, following the httptest pattern in notification_setting/behavior_test.go: the server answers the GET with a 404 and the test checks that Read returns no error and leaves the state null.

  • Before the fix both tests fail: Read failed on a 404 instead of removing the resource from state: [{Could not read Databricks credential ID 10:20: resource-not-found: ...}]
  • After the fix both pass.

go test and go vet on both packages pass. golangci-lint reports nothing on the changed files; its existing errcheck findings in the acceptance tests are unchanged. I did not run acceptance tests.

AI disclosure: written with the help of Claude Code, reviewed and tested by me.

Read on dbtcloud_databricks_credential and dbtcloud_spark_credential
turned a 404 into an error, so once the credential was deleted outside
of Terraform every plan failed. Use helper.HandleResourceNotFound like
the other credential resources so the credential is dropped from the
state and planned for creation again.

Signed-off-by: Lucas Barros <lucasbarros.kam@gmail.com>

This branch is waiting to be deployed

1 waiting deployment
cloud-tests — 5408781b Waiting Nov 7, 2026 by lbarrosby via check-changie #1725
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant