Repository navigation
Conversation
datacommonsorg#730 meant to raise the Partitioned DML deadline from 1h to 6h, but the override it added was dropped in review in favor of passing `timeout=` to Database.execute_partitioned_dml(), which takes no such argument (it raises TypeError). Since then --deletion_timeout has been stored but never used, and every delete stream still gets the generated client's 3600s default. google-cloud-spanner has no public setting for this (still true in the latest 3.71.0), so restore the override on this database's own ExecuteStreamingSql wrapper, and fail loudly if the library internals change instead of silently falling back to 1h. deleter_timeout_test.py runs the real execute_partitioned_dml() over faked gRPC stubs and checks the deadline that is sent.
The Python CI step only ran the simple/ and ingestion-helper tests, so aggregation-helper changes, including the deadline test above, were never tested on PRs. e2e tests stay excluded via the pyproject addopts.
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| Documentation | 6 minor |
🟢 Metrics 10 complexity
Metric Results Complexity 10
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
There was a problem hiding this comment.
Code Review
This pull request configures the Spanner ExecuteStreamingSql deadline for Partitioned DML deletes by overriding the internal _timeout attribute of the google-cloud-spanner client, and adds comprehensive unit tests along with test script updates. The review feedback suggests wrapping the internal attribute traversal in a try-except block to ensure that any unexpected internal structural changes in the library fail loudly with a descriptive RuntimeError rather than raising unhandled AttributeErrors or KeyErrors.
| transport = database.spanner_api._transport | ||
| streaming_sql = transport._wrapped_methods[transport.execute_streaming_sql] | ||
| if not hasattr(streaming_sql, "_timeout"): | ||
| raise RuntimeError( | ||
| "google-cloud-spanner internals changed: cannot set the " | ||
| "ExecuteStreamingSql deadline used by Partitioned DML deletes." | ||
| ) | ||
| streaming_sql._timeout = float(self.timeout) |
There was a problem hiding this comment.
The current implementation directly accesses internal attributes (_transport, _wrapped_methods, etc.) of the google-cloud-spanner library. If any of these intermediate attributes or keys are renamed or removed in a future library version, Python will raise an AttributeError or KeyError before reaching the hasattr check, bypassing the custom RuntimeError and producing a less descriptive error. Wrapping the entire attribute traversal in a try-except block ensures that any internal structural changes always fail loudly with the intended descriptive RuntimeError message.
| transport = database.spanner_api._transport | |
| streaming_sql = transport._wrapped_methods[transport.execute_streaming_sql] | |
| if not hasattr(streaming_sql, "_timeout"): | |
| raise RuntimeError( | |
| "google-cloud-spanner internals changed: cannot set the " | |
| "ExecuteStreamingSql deadline used by Partitioned DML deletes." | |
| ) | |
| streaming_sql._timeout = float(self.timeout) | |
| try: | |
| transport = database.spanner_api._transport | |
| streaming_sql = transport._wrapped_methods[transport.execute_streaming_sql] | |
| if not hasattr(streaming_sql, "_timeout"): | |
| raise AttributeError("Missing _timeout attribute") | |
| streaming_sql._timeout = float(self.timeout) | |
| except (AttributeError, KeyError) as e: | |
| raise RuntimeError( | |
| "google-cloud-spanner internals changed: cannot set the " | |
| "ExecuteStreamingSql deadline used by Partitioned DML deletes." | |
| ) from e |
References
- Do not catch and gracefully handle exceptions for critical operations (such as data deletion or cleanup) if the system/job is designed to fail loudly and halt execution upon failure. Raising a RuntimeError ensures the deletion operation fails loudly.
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to configure the streaming timeout for Spanner ExecuteStreamingSql RPCs in the AggregationDeleter by overriding the internal _timeout attribute of the wrapped execute_streaming_sql method. It also adds a comprehensive suite of unit tests in deleter_timeout_test.py to verify this behavior and ensure robust error handling if the underlying library internals change. Additionally, the run_test.sh script is updated to execute these new tests. There are no review comments to address, and we have no further feedback to provide.
Review feedback: an AttributeError/KeyError from an earlier step of the private traversal bypassed the descriptive RuntimeError. Wrap the whole traversal, including the write (in case _timeout becomes read-only), and chain the original error. The _timeout check is a read before the write, since assigning to a renamed attribute would silently do nothing. Use the public SpannerClient.transport instead of _transport, and name google-api-core in the error since _timeout lives in its _GapicCallable. The test now covers each internal the override relies on.
#730 set out to raise the aggregation deletes' Partitioned DML deadline from 1h to 6h after the Place import delete timed out. Its first version overrode the default deadline on the Spanner client's
ExecuteStreamingSqlwrapper. In review that was swapped for passingtimeout=toDatabase.execute_partitioned_dml(), but that method takes notimeoutargument (it raisesTypeError), and thetimeout=change was never made. Since then--deletion_timeoutis stored onAggregationDeleterand never used, so every delete stream still gets the generated client's 3600s default.Changes
deleter.py: restore the override on the database's ownExecuteStreamingSqlwrapper. google-cloud-spanner has no public setting for this deadline (checked up to the latest 3.71.0). If the library internals change, the deleter raises instead of silently going back to 1h.deleter_timeout_test.py: runs the realexecute_partitioned_dml()over faked gRPC stubs and checks the deadline sent (the configured value, 21600s by default), plus the failure path. It's a separate file so it doesn't conflict with feat(spanner): set PRIORITY_LOW on batch ingestion and aggregation deletes #785's changes todeleter_test.py.run_test.sh: run the aggregation-helper unit tests in CI; until now they never ran on PRs.Context: #785 moves these deletes to
PRIORITY_LOW, which makes them slower on a busy instance, so the 1h cap is more likely to be affected. #786 raises the Java Partitioned DML timeout to 6h; this PR is the Python counterpart.Testing:
uv run --locked pytestin aggregation-helper: 81 passed. Merges with #785 without conflicts; combined tests pass.