Repository navigation
feat(dcp): grant the serving account the Gemini key secret - #60
Merged
Merged
Conversation
jm-rivera
force-pushed
the
feat/dcp-v116-prep
branch
from
October 23, 2026 14:28
170b7a3 to
9c31f45
Compare
jm-rivera
force-pushed
the
feat/dcp-v116-prep
branch
from
October 23, 2026 16:57
9c31f45 to
d556c94
Compare
DCP 1.1.6 removes the serving service account's project-wide
secretmanager.secretAccessor role. null_resource.search_api_gemini_key
attaches the ${namespace}-gemini-api-key secret to the datacommons
service as GEMINI_API_KEY, and a revision whose account cannot read the
secret fails to start, so the script's rollout and the apply would
fail. A per-secret grant on data.google_secret_manager_secret.search_api_gemini
keeps the access under both module versions, and the key script waits
for it.
The key serves the DFO prototype app's /prototype/api/similarity and
/prototype/api/skill-compare routes; /api/search goes to dfo-apis.
The comments and READMEs say so.
jm-rivera
force-pushed
the
feat/dcp-v116-prep
branch
from
October 23, 2026 18:08
d556c94 to
985a5aa
Compare
jm-rivera
marked this pull request as ready for review
October 23, 2026 18:08
Terraform Plan —
|
| Init | Validate | Plan |
|---|---|---|
| pass | pass | pass |
Show plan
ephemeral.random_password.mcp_api_token: Opening...
ephemeral.random_password.studio_api_token: Opening...
ephemeral.random_password.mcp_api_token: Opening complete after 0s
ephemeral.random_password.studio_api_token: Opening complete after 0s
data.google_secret_manager_secret.langfuse_public_key: Reading...
data.google_secret_manager_secret.gemini: Reading...
data.google_secret_manager_secret.langfuse_secret_key: Reading...
google_sql_database.ai_sessions: Refreshing state... [id=projects/dev-fin-observatory/instances/dfo-dev-platform-postgres/databases/ai-sessions]
google_project_iam_custom_role.cloudsql_login_only: Refreshing state... [id=projects/dev-fin-observatory/roles/dfoDevAiCloudsqlLoginOnly]
google_storage_bucket.materialisation_store: Refreshing state... [id=dev-fin-observatory-ai-materialisation-store]
google_secret_manager_secret.studio_api_token: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-studio-api-token]
data.google_service_account.ai_deployer: Reading...
google_secret_manager_secret.mcp_api_token: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-mcp-api-token]
google_storage_bucket.definition_store: Refreshing state... [id=dev-fin-observatory-ai-definition-store]
data.google_secret_manager_secret.langfuse_secret_key: Read complete after 0s [id=projects/dev-fin-observatory/secrets/dfo-dev-langfuse-secret-key]
google_project_service.dns: Refreshing state... [id=dev-fin-observatory/dns.googleapis.com]
data.google_secret_manager_secret.langfuse_public_key: Read complete after 0s [id=projects/dev-fin-observatory/secrets/dfo-dev-langfuse-public-key]
data.google_secret_manager_secret.gemini: Read complete after 0s [id=projects/dev-fin-observatory/secrets/dfo-dev-gemini-api-key]
module.ai_service.google_service_account.runtime: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_secret_manager_secret_iam_member.studio_api_token_readers["***"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-studio-api-token/roles/secretmanager.secretAccessor/***]
google_secret_manager_secret_version.studio_api_token: Refreshing state... [id=projects/41292283861/secrets/dfo-dev-studio-api-token/versions/1]
google_secret_manager_secret_version.mcp_api_token: Refreshing state... [id=projects/41292283861/secrets/dfo-dev-mcp-api-token/versions/1]
data.google_service_account.ai_deployer: Read complete after 0s [id=projects/dev-fin-observatory/serviceAccounts/dfo-ai-deployer@dev-fin-observatory.iam.gserviceaccount.com]
module.ai_service.google_secret_manager_secret_iam_member.runtime_secret_access["dfo-dev-datasets-first-party-principal"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-datasets-first-party-principal/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
module.ai_service.google_service_account_iam_member.deployer_act_as["serviceAccount:dfo-ai-deployer@dev-fin-observatory.iam.gserviceaccount.com"]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:dfo-ai-deployer@dev-fin-observatory.iam.gserviceaccount.com]
module.ai_service.google_secret_manager_secret_iam_member.runtime_secret_access["dfo-dev-mcp-api-token"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-mcp-api-token/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
module.ai_service.google_secret_manager_secret_iam_member.runtime_secret_access["dfo-dev-gemini-api-key"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-gemini-api-key/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
module.ai_service.google_secret_manager_secret_iam_member.runtime_secret_access["dfo-dev-studio-api-token"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-studio-api-token/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
module.ai_service.google_secret_manager_secret_iam_member.runtime_secret_access["dfo-dev-live-refresh-principal"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-live-refresh-principal/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
module.ai_service.google_secret_manager_secret_iam_member.runtime_secret_access["dfo-dev-langfuse-public-key"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-langfuse-public-key/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
module.ai_service.google_secret_manager_secret_iam_member.runtime_secret_access["dfo-dev-langfuse-secret-key"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-langfuse-secret-key/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_project_iam_member.ai_cloudsql_client: Refreshing state... [id=dev-fin-observatory/roles/cloudsql.client/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_cloud_run_v2_service_iam_member.ai_apis_invoker: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-apis/roles/run.invoker/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_sql_user.ai_sessions_runtime: Refreshing state... [id=dfo-dev-ai-run@dev-fin-observatory.iam//dfo-dev-platform-postgres]
google_project_iam_member.ai_cloudsql_login: Refreshing state... [id=dev-fin-observatory/projects/dev-fin-observatory/roles/dfoDevAiCloudsqlLoginOnly/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_storage_bucket_iam_member.materialisation_store_creator: Refreshing state... [id=b/dev-fin-observatory-ai-materialisation-store/roles/storage.objectCreator/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_storage_bucket_iam_member.materialisation_store_viewer: Refreshing state... [id=b/dev-fin-observatory-ai-materialisation-store/roles/storage.objectViewer/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_storage_bucket_iam_member.definition_store_creator: Refreshing state... [id=b/dev-fin-observatory-ai-definition-store/roles/storage.objectCreator/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_storage_bucket_iam_member.definition_store_viewer: Refreshing state... [id=b/dev-fin-observatory-ai-definition-store/roles/storage.objectViewer/serviceAccount:dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com]
google_sql_provision_script.ai_sessions_schema_grant: Refreshing state... [id=dev-fin-observatory/dfo-dev-platform-postgres]
google_sql_provision_script.ai_sessions_revoke_public_connect: Refreshing state... [id=dev-fin-observatory/dfo-dev-platform-postgres]
module.psc_network.google_compute_network.this: Refreshing state... [id=projects/dev-fin-observatory/global/networks/dfo-dev-ai-psc]
module.psc_network.google_dns_managed_zone.psc: Refreshing state... [id=projects/dev-fin-observatory/managedZones/dfo-dev-ai-psc-sql]
module.psc_network.google_compute_subnetwork.this: Refreshing state... [id=projects/dev-fin-observatory/regions/northamerica-northeast1/subnetworks/dfo-dev-ai-psc-subnet]
module.psc_network.google_compute_address.psc_endpoint: Refreshing state... [id=projects/dev-fin-observatory/regions/northamerica-northeast1/addresses/dfo-dev-ai-psc-endpoint]
module.ai_service.google_cloud_run_v2_service.this: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-ai]
module.psc_network.google_dns_record_set.psc: Refreshing state... [id=projects/dev-fin-observatory/managedZones/dfo-dev-ai-psc-sql/rrsets/bbd723248b19.3niso0mm9bket.northamerica-northeast1.sql.goog./A]
module.psc_network.google_compute_forwarding_rule.psc_endpoint: Refreshing state... [id=projects/dev-fin-observatory/regions/northamerica-northeast1/forwardingRules/dfo-dev-ai-psc-fwd]
module.ai_service.google_cloud_run_v2_service_iam_member.public[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-ai/roles/run.invoker/allUsers]
module.ai_service.google_cloud_run_v2_service_iam_member.deployer_run_developer["serviceAccount:dfo-ai-deployer@dev-fin-observatory.iam.gserviceaccount.com"]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-ai/roles/run.developer/serviceAccount:dfo-ai-deployer@dev-fin-observatory.iam.gserviceaccount.com]
ephemeral.random_password.studio_api_token: Closing...
ephemeral.random_password.mcp_api_token: Closing...
ephemeral.random_password.mcp_api_token: Closing complete after 0s
ephemeral.random_password.studio_api_token: Closing complete after 0s
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.Stack ai · 985a5aa · @jm-rivera · run
Terraform Plan —
|
| Init | Validate | Plan |
|---|---|---|
| pass | pass | pass |
Show plan
ephemeral.random_password.presentation_migrator: Opening...
ephemeral.random_password.datasets_migrator: Opening...
ephemeral.random_password.presentation_migrator: Opening complete after 0s
ephemeral.random_password.datasets_migrator: Opening complete after 0s
data.google_secret_manager_secret_version.dc_key: Reading...
google_sql_database.datasets: Refreshing state... [id=projects/dev-fin-observatory/instances/dfo-dev-platform-postgres/databases/datasets]
google_secret_manager_secret.datasets_migrator_password: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-datasets-migrator-password]
google_sql_database.presentation: Refreshing state... [id=projects/dev-fin-observatory/instances/dfo-dev-platform-postgres/databases/presentation]
google_sql_user.datasets_migrator: Refreshing state... [id=dfo-dev-datasets-migrator//dfo-dev-platform-postgres]
google_storage_bucket.datasets_pinned: Refreshing state... [id=dev-fin-observatory-datasets-pinned]
google_service_account.datasets_gc: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-datasets-gc-sa@dev-fin-observatory.iam.gserviceaccount.com]
google_secret_manager_secret.presentation_migrator_password: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-presentation-migrator-password]
google_storage_bucket.dc_artifacts: Refreshing state... [id=dfo-dev-dc-artifacts-dev-fin-observatory]
google_storage_bucket.datasets_working: Refreshing state... [id=dev-fin-observatory-datasets-working]
data.google_secret_manager_secret.search_api_gemini: Reading...
google_sql_user.presentation_migrator: Refreshing state... [id=dfo-dev-presentation-migrator//dfo-dev-platform-postgres]
data.google_secret_manager_secret.search_api_gemini: Read complete after 0s [id=projects/dev-fin-observatory/secrets/dfo-dev-gemini-api-key]
google_project_service.apis["artifactregistry.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/artifactregistry.googleapis.com]
data.google_secret_manager_secret_version.dc_key: Read complete after 0s [id=projects/41292283861/secrets/dc-api-key/versions/2]
google_project_service.apis["iam.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/iam.googleapis.com]
google_project_service.apis["run.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/run.googleapis.com]
google_project_service.apis["sqladmin.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/sqladmin.googleapis.com]
google_project_service.apis["places-backend.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/places-backend.googleapis.com]
google_project_service.apis["bigqueryconnection.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/bigqueryconnection.googleapis.com]
google_project_service.apis["dataflow.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/dataflow.googleapis.com]
google_project_service.apis["workflows.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/workflows.googleapis.com]
google_project_service.apis["bigqueryreservation.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/bigqueryreservation.googleapis.com]
google_project_service.apis["vpcaccess.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/vpcaccess.googleapis.com]
google_project_service.apis["apikeys.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/apikeys.googleapis.com]
google_project_service.apis["spanner.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/spanner.googleapis.com]
google_project_service.apis["cloudresourcemanager.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/cloudresourcemanager.googleapis.com]
google_project_service.apis["secretmanager.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/secretmanager.googleapis.com]
google_project_service.apis["aiplatform.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/aiplatform.googleapis.com]
google_project_service.apis["bigquery.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/bigquery.googleapis.com]
google_project_service.apis["redis.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/redis.googleapis.com]
google_project_service.apis["workflowexecutions.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/workflowexecutions.googleapis.com]
google_project_service.apis["compute.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/compute.googleapis.com]
google_project_service.apis["maps-backend.googleapis.com"]: Refreshing state... [id=dev-fin-observatory/maps-backend.googleapis.com]
google_secret_manager_secret_version.presentation_migrator_password: Refreshing state... [id=projects/41292283861/secrets/dfo-dev-presentation-migrator-password/versions/1]
null_resource.dc_key_guard: Refreshing state... [id=6055240469338298543]
google_storage_bucket_iam_member.datasets_pinned_gc_admin: Refreshing state... [id=b/dev-fin-observatory-datasets-pinned/roles/storage.objectAdmin/serviceAccount:dfo-dev-dc-datasets-gc-sa@dev-fin-observatory.iam.gserviceaccount.com]
google_secret_manager_secret_version.datasets_migrator_password: Refreshing state... [id=projects/41292283861/secrets/dfo-dev-datasets-migrator-password/versions/2]
google_sql_provision_script.presentation_revoke_public_connect: Refreshing state... [id=dev-fin-observatory/dfo-dev-platform-postgres]
google_sql_user.datasets_gc: Refreshing state... [id=dfo-dev-dc-datasets-gc-sa@dev-fin-observatory.iam//dfo-dev-platform-postgres]
module.stack.data.google_compute_network.default: Reading...
module.stack.module.auth.random_id.api_key_suffix: Refreshing state... [id=8cdVBA]
module.stack.module.spanner[0].data.google_bigquery_default_service_account.bq_sa: Reading...
module.stack.module.ingestion_postprocessing_job[0].google_service_account.postprocessing_sa: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-pst-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_dataflow.data.google_project.project: Reading...
module.stack.module.spanner[0].google_bigquery_reservation.default[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/reservations/default]
module.stack.module.ingestion_dataflow.google_service_account.dataflow_sa[0]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-df-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.auth.google_secret_manager_secret.dc_api_key: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-dc-api-key]
module.stack.module.ingestion_helper_service.google_service_account.helper_sa[0]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-hlp-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.spanner[0].google_spanner_instance.main[0]: Refreshing state... [id=dev-fin-observatory/dfo-dev-dc-instance]
module.stack.module.auth.google_secret_manager_secret.maps_api_key[0]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-dc-google-maps-api-key-f1c75504]
module.stack.module.auth.google_apikeys_key.maps_api_key[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/global/keys/dfo-dev-dc-google-maps-key-f1c75504]
ephemeral.random_password.presentation_migrator: Closing...
ephemeral.random_password.presentation_migrator: Closing complete after 0s
module.stack.module.spanner[0].google_bigquery_reservation_assignment.project_assignment[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/reservations/default/assignments/10971338433911481598]
module.stack.module.ingestion_helper_service.google_project_iam_member.helper_dataflow_viewer[0]: Refreshing state... [id=dev-fin-observatory/roles/dataflow.viewer/serviceAccount:dfo-dev-dc-ing-hlp-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_helper_service.google_project_iam_member.helper_spanner_user[0]: Refreshing state... [id=dev-fin-observatory/roles/spanner.databaseUser/serviceAccount:dfo-dev-dc-ing-hlp-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_postprocessing_job[0].google_project_iam_member.postprocessing_bq_job_user[0]: Refreshing state... [id=dev-fin-observatory/roles/bigquery.jobUser/serviceAccount:dfo-dev-dc-ing-pst-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.spanner[0].data.google_bigquery_default_service_account.bq_sa: Read complete after 0s [id=bq-41292283861@bigquery-encryption.iam.gserviceaccount.com]
module.stack.module.ingestion_postprocessing_job[0].google_project_iam_member.postprocessing_spanner[0]: Refreshing state... [id=dev-fin-observatory/roles/spanner.databaseUser/serviceAccount:dfo-dev-dc-ing-pst-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.data.google_compute_network.default: Read complete after 0s [id=projects/dev-fin-observatory/global/networks/default]
module.stack.module.ingestion_postprocessing_job[0].google_project_iam_member.postprocessing_bq_connection_user[0]: Refreshing state... [id=dev-fin-observatory/roles/bigquery.connectionUser/serviceAccount:dfo-dev-dc-ing-pst-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_postprocessing_job[0].google_project_iam_member.postprocessing_bq_data_editor[0]: Refreshing state... [id=dev-fin-observatory/roles/bigquery.dataEditor/serviceAccount:dfo-dev-dc-ing-pst-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_dataflow.google_project_iam_member.dataflow_worker[0]: Refreshing state... [id=dev-fin-observatory/roles/dataflow.worker/serviceAccount:dfo-dev-dc-ing-df-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_dataflow.google_project_iam_member.ingestion_spanner_user[0]: Refreshing state... [id=dev-fin-observatory/roles/spanner.databaseUser/serviceAccount:dfo-dev-dc-ing-df-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_dataflow.data.google_project.project: Read complete after 1s [id=projects/dev-fin-observatory]
module.stack.module.ingestion_helper_service.google_storage_bucket_iam_member.helper_bucket_access[0]: Refreshing state... [id=b/dfo-dev-dc-artifacts-dev-fin-observatory/roles/storage.objectAdmin/serviceAccount:dfo-dev-dc-ing-hlp-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.auth.google_secret_manager_secret_version.dc_api_key_version: Refreshing state... [id=projects/41292283861/secrets/dfo-dev-dc-api-key/versions/1]
module.stack.module.spanner[0].google_project_iam_member.bq_sa_spanner_viewer[0]: Refreshing state... [id=dev-fin-observatory/roles/spanner.viewer/serviceAccount:bq-41292283861@bigquery-encryption.iam.gserviceaccount.com]
google_sql_provision_script.datasets_revoke_public_connect: Refreshing state... [id=dev-fin-observatory/dfo-dev-platform-postgres]
module.stack.google_storage_bucket_iam_member.dataflow_bucket_access[0]: Refreshing state... [id=b/dfo-dev-dc-artifacts-dev-fin-observatory/roles/storage.objectAdmin/serviceAccount:dfo-dev-dc-ing-df-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_dataflow.google_service_account_iam_member.workflows_token_creator[0]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-df-sa@dev-fin-observatory.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-41292283861@gcp-sa-workflows.iam.gserviceaccount.com]
module.stack.module.spanner[0].google_spanner_database.database[0]: Refreshing state... [id=dfo-dev-dc-instance/dfo-dev-dc-db]
ephemeral.random_password.datasets_migrator: Closing...
ephemeral.random_password.datasets_migrator: Closing complete after 0s
module.stack.module.auth.google_secret_manager_secret_version.maps_api_key_version[0]: Refreshing state... [id=projects/41292283861/secrets/dfo-dev-dc-google-maps-api-key-f1c75504/versions/1]
module.stack.module.spanner[0].google_bigquery_connection.spanner_connection[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/connections/dfo_dev_dc_spanner_connection]
module.stack.module.spanner[0].google_spanner_database_iam_member.spanner_reader[0]: Refreshing state... [id=dev-fin-observatory/dfo-dev-dc-instance/dfo-dev-dc-db/roles/spanner.databaseUser/serviceAccount:bq-41292283861@bigquery-encryption.iam.gserviceaccount.com]
module.stack.module.ingestion_helper_service.google_cloud_run_v2_service.ingestion_helper[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-dc-ingestion-helper]
module.stack.module.ingestion_postprocessing_job[0].google_cloud_run_v2_job.dc_postprocessing_job: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/jobs/dfo-dev-dc-ingestion-postprocessing-job]
module.stack.module.ingestion_preprocessing_job[0].google_service_account.preprocessing_sa: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-pre-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_preprocessing_job[0].google_secret_manager_secret_iam_member.preprocessing_secret_accessor["MAPS_API_KEY"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-dc-google-maps-api-key-f1c75504/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-dc-ing-pre-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_preprocessing_job[0].google_secret_manager_secret_iam_member.preprocessing_secret_accessor["DC_API_KEY"]: Refreshing state... [id=projects/dev-fin-observatory/secrets/dfo-dev-dc-api-key/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-dc-ing-pre-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_storage_bucket_iam_member.preprocessing_bucket_access[0]: Refreshing state... [id=b/dfo-dev-dc-artifacts-dev-fin-observatory/roles/storage.objectAdmin/serviceAccount:dfo-dev-dc-ing-pre-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_preprocessing_job[0].google_cloud_run_v2_job.dc_data_job: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/jobs/dfo-dev-dc-ingestion-preprocessing-job]
module.stack.module.datacommons_services[0].google_service_account.serving_sa: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/aiplatform.user"]: Refreshing state... [id=dev-fin-observatory/roles/aiplatform.user/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/redis.editor"]: Refreshing state... [id=dev-fin-observatory/roles/redis.editor/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/workflows.invoker"]: Refreshing state... [id=dev-fin-observatory/roles/workflows.invoker/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/compute.networkViewer"]: Refreshing state... [id=dev-fin-observatory/roles/compute.networkViewer/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/spanner.databaseUser"]: Refreshing state... [id=dev-fin-observatory/roles/spanner.databaseUser/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/storage.objectViewer"]: Refreshing state... [id=dev-fin-observatory/roles/storage.objectViewer/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/vpcaccess.user"]: Refreshing state... [id=dev-fin-observatory/roles/vpcaccess.user/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/iam.serviceAccountUser"]: Refreshing state... [id=dev-fin-observatory/roles/iam.serviceAccountUser/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_project_iam_member.serving_sa_roles["roles/secretmanager.secretAccessor"]: Refreshing state... [id=dev-fin-observatory/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_workflow.google_service_account.workflow_sa[0]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_workflow.google_cloud_run_v2_service_iam_member.helper_invoker[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-dc-ingestion-helper/roles/run.invoker/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.ingestion_workflow.google_workflows_workflow.ingestion_orchestrator[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/workflows/dfo-dev-dc-ingestion-workflow]
module.stack.module.ingestion_workflow.google_service_account_iam_member.workflow_act_as_dataflow_sa[0]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-df-sa@dev-fin-observatory.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_project_iam_member.workflow_dataflow_developer[0]: Refreshing state... [id=dev-fin-observatory/roles/dataflow.developer/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_cloud_run_v2_job_iam_member.workflow_pre_invoker[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/jobs/dfo-dev-dc-ingestion-preprocessing-job/roles/run.invoker/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_cloud_run_v2_job_iam_member.workflow_pre_viewer[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/jobs/dfo-dev-dc-ingestion-preprocessing-job/roles/run.viewer/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_cloud_run_v2_job_iam_member.workflow_post_viewer[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/jobs/dfo-dev-dc-ingestion-postprocessing-job/roles/run.viewer/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_service_account_iam_member.workflow_pre_sa_user[0]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-pre-sa@dev-fin-observatory.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_service_account_iam_member.workflow_post_sa_user[0]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-ing-pst-sa@dev-fin-observatory.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_cloud_run_v2_job_iam_member.workflow_post_developer[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/jobs/dfo-dev-dc-ingestion-postprocessing-job/roles/run.developer/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_cloud_run_v2_job_iam_member.workflow_pre_developer[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/jobs/dfo-dev-dc-ingestion-preprocessing-job/roles/run.developer/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_storage_bucket_iam_member.workflow_bucket_access[0]: Refreshing state... [id=b/dfo-dev-dc-artifacts-dev-fin-observatory/roles/storage.objectAdmin/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_project_iam_member.workflow_invoker[0]: Refreshing state... [id=dev-fin-observatory/roles/workflows.invoker/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_cloud_run_v2_job_iam_member.workflow_post_invoker[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/jobs/dfo-dev-dc-ingestion-postprocessing-job/roles/run.invoker/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_service_account_iam_member.ingestion_workflow_act_as_serving_sa[0]: Refreshing state... [id=projects/dev-fin-observatory/serviceAccounts/dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.google_project_iam_member.workflow_run_viewer[0]: Refreshing state... [id=dev-fin-observatory/roles/run.viewer/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
module.stack.module.datacommons_services[0].google_cloud_run_v2_service.dc_web_service: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-dc-datacommons-service]
module.stack.module.datacommons_services[0].google_cloud_run_v2_service_iam_member.public_access[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-dc-datacommons-service/roles/run.invoker/allUsers]
module.stack.google_cloud_run_v2_service_iam_member.workflow_serving_developer[0]: Refreshing state... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-dc-datacommons-service/roles/run.developer/serviceAccount:dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com]
null_resource.search_api_gemini_key: Refreshing state... [id=7561527881215010758]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
~ update in-place
-/+ destroy and then create replacement
Terraform will perform the following actions:
# google_secret_manager_secret_iam_member.serving_sa_gemini_key will be created
+ resource "google_secret_manager_secret_iam_member" "serving_sa_gemini_key" {
+ etag = (known after apply)
+ id = (known after apply)
+ member = "serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com"
+ project = "dev-fin-observatory"
+ role = "roles/secretmanager.secretAccessor"
+ secret_id = "dfo-dev-gemini-api-key"
}
# null_resource.search_api_gemini_key must be replaced
-/+ resource "null_resource" "search_api_gemini_key" {
~ id = "7561527881215010758" -> (known after apply)
~ triggers = { # forces replacement
~ "always_run" = "2026-10-23T18:03:34Z" -> (known after apply)
}
}
# module.stack.module.datacommons_services[0].google_cloud_run_v2_service.dc_web_service will be updated in-place
~ resource "google_cloud_run_v2_service" "dc_web_service" {
id = "projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-dc-datacommons-service"
name = "dfo-dev-dc-datacommons-service"
# (36 unchanged attributes hidden)
~ template {
# (11 unchanged attributes hidden)
~ containers {
name = null
# (8 unchanged attributes hidden)
- env {
- name = "GEMINI_API_KEY" -> null
# (1 unchanged attribute hidden)
- value_source {
- secret_key_ref {
- secret = "projects/dev-fin-observatory/secrets/dfo-dev-gemini-api-key" -> null
- version = "latest" -> null
}
}
}
# (27 unchanged blocks hidden)
}
# (1 unchanged block hidden)
}
# (2 unchanged blocks hidden)
}
Plan: 2 to add, 1 to change, 1 to destroy.
─────────────────────────────────────────────────────────────────────────────
Saved the plan to: tfplan
To perform exactly these actions, run the following command to apply:
terraform apply "tfplan"Stack dcp · 985a5aa · @jm-rivera · run
Terraform Apply —
|
| Init | Validate | Apply |
|---|---|---|
| pass | pass | pass |
Show apply output
ephemeral.random_password.studio_api_token: Opening...
ephemeral.random_password.mcp_api_token: Opening...
ephemeral.random_password.studio_api_token: Opening complete after 0s
ephemeral.random_password.mcp_api_token: Opening complete after 0s
ephemeral.random_password.studio_api_token: Closing...
ephemeral.random_password.mcp_api_token: Closing...
ephemeral.random_password.mcp_api_token: Closing complete after 0s
ephemeral.random_password.studio_api_token: Closing complete after 0s
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
runtime_service_account_email = "dfo-dev-ai-run@dev-fin-observatory.iam.gserviceaccount.com"
service_name = "dfo-dev-ai"
service_url = "https://dfo-dev-ai-zms5lj4zhq-nn.a.run.app"
studio_api_token_secret = "dfo-dev-studio-api-token"Stack ai · 5a180ba · @jm-rivera · run
Terraform Apply —
|
| Init | Validate | Apply |
|---|---|---|
| pass | pass | pass |
Show apply output
ephemeral.random_password.presentation_migrator: Opening...
ephemeral.random_password.datasets_migrator: Opening...
null_resource.search_api_gemini_key: Destroying... [id=7561527881215010758]
null_resource.search_api_gemini_key: Destruction complete after 0s
ephemeral.random_password.presentation_migrator: Opening complete after 0s
ephemeral.random_password.datasets_migrator: Opening complete after 0s
ephemeral.random_password.datasets_migrator: Closing...
ephemeral.random_password.presentation_migrator: Closing...
ephemeral.random_password.datasets_migrator: Closing complete after 0s
ephemeral.random_password.presentation_migrator: Closing complete after 0s
module.stack.module.datacommons_services[0].google_cloud_run_v2_service.dc_web_service: Modifying... [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-dc-datacommons-service]
module.stack.module.datacommons_services[0].google_cloud_run_v2_service.dc_web_service: Still modifying... [id=projects/dev-fin-observatory/locations/...ervices/dfo-dev-dc-datacommons-service, 00m10s elapsed]
module.stack.module.datacommons_services[0].google_cloud_run_v2_service.dc_web_service: Still modifying... [id=projects/dev-fin-observatory/locations/...ervices/dfo-dev-dc-datacommons-service, 00m20s elapsed]
module.stack.module.datacommons_services[0].google_cloud_run_v2_service.dc_web_service: Still modifying... [id=projects/dev-fin-observatory/locations/...ervices/dfo-dev-dc-datacommons-service, 00m30s elapsed]
module.stack.module.datacommons_services[0].google_cloud_run_v2_service.dc_web_service: Modifications complete after 31s [id=projects/dev-fin-observatory/locations/northamerica-northeast1/services/dfo-dev-dc-datacommons-service]
google_secret_manager_secret_iam_member.serving_sa_gemini_key: Creating...
google_secret_manager_secret_iam_member.serving_sa_gemini_key: Creation complete after 4s [id=projects/dev-fin-observatory/secrets/dfo-dev-gemini-api-key/roles/secretmanager.secretAccessor/serviceAccount:dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com]
null_resource.search_api_gemini_key: Creating...
null_resource.search_api_gemini_key: Provisioning with 'local-exec'...
null_resource.search_api_gemini_key (local-exec): Executing: ["/bin/sh" "-c" "\"./../../../scripts/set-search-api-gemini-key.sh\" \\\n dev-fin-observatory \\\n northamerica-northeast1 \\\n dfo-dev-dc-datacommons-service \\\n dfo-dev-gemini-api-key\n"]
null_resource.search_api_gemini_key: Still creating... [00m10s elapsed]
null_resource.search_api_gemini_key: Still creating... [00m20s elapsed]
null_resource.search_api_gemini_key: Still creating... [00m30s elapsed]
null_resource.search_api_gemini_key (local-exec): GEMINI_API_KEY set on dfo-dev-dc-datacommons-service from projects/dev-fin-observatory/secrets/dfo-dev-gemini-api-key
null_resource.search_api_gemini_key: Creation complete after 30s [id=7603058370291887627]
Apply complete! Resources: 2 added, 1 changed, 1 destroyed.
Outputs:
datacommons_service_name = "dfo-dev-dc-datacommons-service"
datacommons_service_service_account_email = "dfo-dev-dc-srvs-sa@dev-fin-observatory.iam.gserviceaccount.com"
datacommons_service_url = "https://dfo-dev-dc-datacommons-service-zms5lj4zhq-nn.a.run.app"
datasets_database_name = "datasets"
datasets_gc_db_user = "dfo-dev-dc-datasets-gc-sa@dev-fin-observatory.iam"
datasets_gc_service_account_email = "dfo-dev-dc-datasets-gc-sa@dev-fin-observatory.iam.gserviceaccount.com"
datasets_instance_connection_name = "dev-fin-observatory:northamerica-northeast1:dfo-dev-platform-postgres"
datasets_migrator_secret_id = "projects/dev-fin-observatory/secrets/dfo-dev-datasets-migrator-password"
datasets_migrator_secret_name = "dfo-dev-datasets-migrator-password"
datasets_pinned_bucket_name = "dev-fin-observatory-datasets-pinned"
datasets_working_bucket_name = "dev-fin-observatory-datasets-working"
ingestion_bucket_url = "gs://dfo-dev-dc-artifacts-dev-fin-observatory"
ingestion_input_path = "ingestion/input"
ingestion_prep_job_name = "dfo-dev-dc-ingestion-preprocessing-job"
ingestion_service_url = "https://dfo-dev-dc-ingestion-helper-zms5lj4zhq-nn.a.run.app"
ingestion_workflow_id = "projects/dev-fin-observatory/locations/northamerica-northeast1/workflows/dfo-dev-dc-ingestion-workflow"
ingestion_workflow_name = "dfo-dev-dc-ingestion-workflow"
ingestion_workflow_service_account_email = "dfo-dev-dc-ing-wf-sa@dev-fin-observatory.iam.gserviceaccount.com"
presentation_database_name = "presentation"
presentation_migrator_secret_id = "projects/dev-fin-observatory/secrets/dfo-dev-presentation-migrator-password"
presentation_migrator_secret_name = "dfo-dev-presentation-migrator-password"
project_id = "dev-fin-observatory"
region = "northamerica-northeast1"
spanner_database_id = "dfo-dev-dc-db"
spanner_instance_id = "dfo-dev-dc-instance"
storage_artifacts_bucket_name = "dfo-dev-dc-artifacts-dev-fin-observatory"Stack dcp · 5a180ba · @jm-rivera · run
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Data Commons Platform 1.1.6 removes the serving service account's project-wide
roles/secretmanager.secretAccessor.null_resource.search_api_gemini_keyattaches the${namespace}-gemini-api-keysecret (dfo-dev-gemini-api-key,dfo-prod-gemini-api-key) to the datacommons service asGEMINI_API_KEY, and a Cloud Run revision whose account cannot read a referenced secret fails to start. Under 1.1.6 the script's rollout would fail and take the apply with it.infra/stacks/dcp/dfo.tfaddsdata.google_secret_manager_secret.search_api_geminifor that secret andgoogle_secret_manager_secret_iam_member.serving_sa_gemini_keyon it formodule.stack.datacommons_service_service_account_email, an output both module versions expose, and the key script now takes the secret id from the data source and depends on the grant. The grant is additive, so it applies on 1.1.5 ahead of the 1.1.6 port in #61. When #65 moves the dcp stack to a per-caller secret, the data source'ssecret_idchanges to its variable and the grant follows.The key serves the DFO prototype app's
/prototype/api/similarity(which falls back to pseudo-embeddings without it) and/prototype/api/skill-compare./api/searchis proxied to dfo-apis and never reads it. The comments indfo.tfandscripts/set-search-api-gemini-key.sh, the dev and proddfo.README.md, and the line ininfra/dfo-dev/ai/README.mddescribe that.Testing:
terraform fmt -check,bash -n scripts/set-search-api-gemini-key.sh, andterraform init -backend=false && terraform validateon dereferenced copies ofinfra/dfo-dev/dcpandinfra/dfo-prod/dcp, all clean. Not applied.