Skip to content

[SQL] Add short-term backup retention immutability support - #34207

Open
svmicano wants to merge 9 commits into
Azure:devfrom
svmicano:svmicano/sql-str-immutability
Open

svmicano wants to merge 9 commits into
Azure:devfrom
svmicano:svmicano/sql-str-immutability

Conversation

@svmicano

@svmicano svmicano commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

🤖 PR Validation — ⚠️ Review suggested

Breaking Changes Tests
⚠️ None ️✔️ 130/130
⚠️AzureCLI-BreakingChangeTest
⚠️sql
rule cmd_name rule_message suggest_message
⚠️ 1006 - ParaAdd sql db str-policy set cmd sql db str-policy set added parameter lock_immutability
⚠️ 1006 - ParaAdd sql db str-policy set cmd sql db str-policy set added parameter yes
⚠️ 1008 - ParaPropAdd sql db str-policy set cmd sql db str-policy set update parameter diffbackup_hours: added property type=int
⚠️ 1008 - ParaPropAdd sql db str-policy set cmd sql db str-policy set update parameter retention_days: added property type=int
⚠️ 1009 - ParaPropRemove sql db str-policy set cmd sql db str-policy set update parameter retention_days: removed property required=True
⚠️ 1006 - ParaAdd sql midb short-term-retention-policy set cmd sql midb short-term-retention-policy set added parameter lock_immutability
⚠️ 1006 - ParaAdd sql midb short-term-retention-policy set cmd sql midb short-term-retention-policy set added parameter yes
⚠️ 1008 - ParaPropAdd sql midb short-term-retention-policy set cmd sql midb short-term-retention-policy set update parameter retention_days: added property type=int

Related command

  • az sql db str-policy set, show, and wait
  • az sql midb short-term-retention-policy set and show (live and restorable dropped databases)

Description

Add short-term backup immutability support using 2026-08-01-preview, with the final async SET result aligned with Azure PowerShell PR 30215.

  • Add nullable --lock-immutability and --yes/-y to both SET commands. Only true prompts, with an irreversible-action warning. Decline/non-interactive confirmation failure sends no update and emits no result; --yes bypasses confirmation.
  • Reject --lock-immutability with --deleted-time, including explicit false, before HTTP. Never serialize that property for dropped MI databases.
  • Surface the service's immutabilityStatus on SET/show for SQL Database and live/dropped MI databases, retaining flattened CLI output and nested systemData.
  • SQL Database retention remains optional for differential-only input; omitted settings are not serialized as null. MI retention remains required, including support for zero on dropped databases. SQL DB lock-only input is serializable but is rejected by the tested service; the corrected help example supplies explicit retention.
  • Generate SDK-independent STR request/response schemas and HTTP operations with AAZ. Retain the existing custom registrations, cloud endpoints, authentication, URL escaping, --ids, wait and --no-wait, with a small compatibility adapter for omitted values and completion polling. azure-mgmt-sql==4.0.0b22 is unchanged; a whole-module migration to the hybrid models in 5.0.0b2 is out of scope.

Integer differential-backup interval serialization (2026-10-16)

Commit 03307dd24188df4fce24d1223c51430d19eb144f fixes a regression found by live SQL Database validation: AAZ's generated numeric schema serialized --diffbackup-hours 12/24 as JSON 12.0/24.0, which the service rejected with InvalidResourceIdSegment. An integer-payload REST control was accepted.

The SQL Database custom operation now normalizes only a present, integral diffBackupIntervalInHours value after generated serialization, and explicitly rejects fractional input instead of truncating it. Omitted fields stay omitted; the other properties, MI operations, response schemas, confirmation, and completion polling are unchanged. Generated Python files and the upstream AAZ models were not hand-edited or changed; this compatibility override survives regeneration.

Validation for this fix:

  • Before fix: all three new regression methods failed (17 failing subtests). They assert the exact JSON integer type rather than numeric equality.
  • After fix: 50 offline unit/safety/recorded-scenario tests passed, zero failures/errors/skips, with socket connections blocked. This includes 34 CLI/HTTP unit tests, 11 safety tests, and the five existing checked-in recordings.
  • Live: eight interval updates passed on the two disposable SQL databases created during the authorized live run, one Locked and one Enabled: 12/24 hours with normal completion and --no-wait, including explicit retention/lock preservation. A transport guard checked the actual outgoing JSON integer type and permitted writes only to these two policies.
  • The live check restored both original 24-hour intervals, retention periods, and lock states. All pre-existing MI/SQL databases, their policies, and parent configurations were verified unchanged. No new databases or locks were created for this fix.
  • Passed: scoped flake8, adapter pylint (10.00/10), compilation, and whitespace checks.

The earlier broad SQL live scenarios failed before this correction; they are not being relabeled as passes. The post-fix live validation targeted the affected field and its combinations. The MI and isolated SQL locking scenarios had already passed live and offline replay.

Exact additional validation commands (from the CLI worktree root; all exited 0):

# PASS: eight real updates; integer JSON tokens verified; policy values restored.
& 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe' -u 'E:\CLI\aaz-env\validate_str_integer_live.py'

# PASS: scoped style checks.
& 'E:\CLI\aaz-env\Scripts\python.exe' -m flake8 --append-config '.flake8' 'src\azure-cli\azure\cli\command_modules\sql\_str_policy.py' 'src\azure-cli\azure\cli\command_modules\sql\tests\latest\test_sql_str_policy.py'

# PASS: adapter pylint, 10.00/10.
$env:PYTHONPATH='E:\CLI\azure-cli-str-immutability\src\azure-cli;E:\CLI\azure-cli-str-immutability\src\azure-cli-core;E:\CLI\azure-cli-str-immutability\src\azure-cli-testsdk;E:\CLI\str-env\Lib\site-packages;C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages'
& 'E:\CLI\aaz-env\Scripts\python.exe' -m pylint --rcfile 'pylintrc' --jobs 1 'src\azure-cli\azure\cli\command_modules\sql\_str_policy.py'

# PASS: compilation.
& 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe' -m compileall -q 'src\azure-cli\azure\cli\command_modules\sql\_str_policy.py' 'src\azure-cli\azure\cli\command_modules\sql\tests\latest\test_sql_str_policy.py'

The full offline command below was rerun unchanged and now reports 50 passing tests. New hosted CI results must be checked for this commit before merging.

AAZ generation (2026-10-16)

Commit 9451dbe62e486904492d957059b857d154b702a0 replaces the handwritten STR request/response schemas with six command bases generated by aaz-dev-tools 4.7.1:

  • SQL Database policy GET/PUT: sql/aaz/latest/sql/db/str_policy.
  • Live managed database policy GET/PUT: sql/aaz/latest/sql/midb/short_term_retention_policy.
  • Restorable dropped managed database policy GET/PUT: the restorable_dropped child package.

The bases remain unregistered: the existing custom handlers preserve public command names, aliases, required MI retention, database-ID handling, conditional irreversible-action confirmation, and rejection of either lock flag value for dropped databases. No new public command group is introduced. The dropped-policy model hides the shared-schema lock argument.

_str_policy.py contains compatibility adapters, including the integer wire-type correction above, not manually maintained API schemas. Its shared PUT operation override intentionally retains the final policy GET described below: the current REST specifications declare final-state-via: location, which must not undo the already-tested PowerShell alignment when regenerating.

Generation inputs: Azure/azure-rest-api-specs commit 2e2b290fd446b03bbddcdc31098ff97e7218f0cf, SQL 2026-08-01-preview; AAZ base commit f4676b10d373ec9f5c0b63b986fe2a1832682814. Generated files were not hand-edited. An unrelated generator change to the existing MI start-stop-schedule registration was excluded.

Upstream models: Azure/aaz#1130.

Final policy GET alignment (2026-10-13)

Commit 93e16b8f2253fbdccb1a8337cc2dca763d6cae83 removes the explicit final-state-via: location override from the shared STR PUT operation. For the Azure-AsyncOperation flow recorded for SQL Database and MI:

  1. PUT the policy.
  2. Poll async status until Succeeded.
  3. GET the original policy resource URL and return that policy.

This matches the PowerShell recordings, whose final policy GET belongs to the SET operation. It replaces the previous OperationResults GET; it does not add a second final read, synthesize Locked, retry until a desired immutability status, add fixed delays, or fill omitted request properties.

  • --no-wait still submits once, without polling or a final GET, and emits no result.
  • Failed/canceled operations propagate their error before any final GET. Errors from the final policy GET are also propagated.
  • Synchronous 200 responses without polling headers and Location-only responses retain standard ARM behavior: return their response without an additional policy GET. Unit tests cover these distinct paths; no unconditional readback is claimed.
  • SQL DB and MI locking scenarios now assert Locked on SET itself, as well as through independent policy GETs. All five affected scenarios were genuinely re-recorded live on 2026-10-13; response statuses and URLs were not hand-edited to pass playback.
  • Fixed the recording sanitizer to preserve non-JSON/non-object gateway responses instead of masking HTTP retry/error handling with a JSON parsing exception.
  • Isolated process-wide CLI session caches in unit-test setup/cleanup so temporary configuration directories do not contaminate later scenario tests. Help regression coverage tolerates normal line wrapping.

Wait-command compatibility

The earlier fix in 3333619a720b9deb77c784134f1e276500dda0c4 remains intact: az sql db str-policy wait --policy-name default is accepted and --policy-name remains required, matching the exact PR base cc010d2104b3ab0089c33fb7eac8816c260a11ac. A dedicated getter forwards and URL-escapes the supplied name; show/set and MI commands still use the default policy without a new public option. Regression coverage includes --ids, encoded names and missing-option rejection without HTTP. No compatibility-rule suppression or exception is requested.

Remaining service limitations and release checks

  • OperationResults can still report Disabled while the actual policy GET reports Locked. Earlier raw SQL DB recordings and 24 paired MI diagnostic reads over approximately 144 seconds demonstrated this backend inconsistency. The CLI now uses the standard final policy GET for the recorded async flow; this does not claim the backend endpoint is fixed.
  • New SQL DB policies may initially report Disabled before becoming Enabled. The new-DB locking test retains its 900-second readiness timeout and refuses to lock before Enabled; independent post-lock checks retain their 300-second timeout. A previous Oct13 initialization failure is not counted as passing. The refreshed new-DB locking recording passed.
  • The tested SQL DB backend rejects lock-only PUTs with InvalidParameterValue; including current retention succeeds. On a locked policy, retention updates with the lock flag omitted can be interpreted as unlocking. Help now documents explicit retention when locking and preserving --lock-immutability true when increasing retention. No automatic payload completion was introduced.
  • Confirm preview API rollout for intended regions/clouds and obtain maintainer approval before release. This PR is open for review; maintainer and rollout approval remain required.

Testing Guide

Latest local result (2026-10-16): 50 tests passed, zero failures/errors/skips (34 CLI/HTTP unit tests + 11 scenario safety tests + 5 recorded scenarios). Offline execution used this checkout's CLI/core/test SDK source with the installed CLI libraries on Python 3.14.6, an empty Azure configuration, live-target flags removed, socket connections blocked, and only the unrelated release-update connectivity probe disabled.

The new original-resource final-GET regressions failed against the previous Location override and passed after the fix. A combined-suite run also exposed and fixed the unit-test session-cache isolation issue; the complete combined suite then passed in 70.238 seconds.

AAZ migration validation (2026-10-16)

  • PASS: 50 offline tests, with actual socket connections blocked. Includes all five unchanged checked-in recordings from October 13; subsequent authorized live validation and its limitations are documented above.
  • PASS: SQL AAZ command-model consistency, repository-configured flake8, and Python compilation of the generated/edited files.
  • PASS: the compatibility adapter scores 10.00/10 under repository-configured pylint. The full legacy custom.py has 27 import-analysis diagnostics in this local SDK environment, all also present in the 31-diagnostic parent baseline; the migration adds zero pylint diagnostics. A clean full-module pylint result is not claimed.
  • Additional coverage verifies MI --ids and required retention, lack of public registration for dropped-policy bases, nested systemData across all three generated resource types, and execution of all four generated examples through the public CLI wrappers.
Exact local validation commands and results

Run from the CLI worktree root. The installed SDK uses Python 3.14.6; the separate test dependencies are in E:\CLI\str-env.

# PASS: Ran 50 tests; failures=0, errors=0, skips=0.
@'
import os, sys, unittest, tempfile
from pathlib import Path
from unittest.mock import patch
root = Path.cwd()
sys.path[:0] = [str(root / 'src' / name) for name in ('azure-cli', 'azure-cli-core', 'azure-cli-testsdk')] + [r'E:\CLI\str-env\Lib\site-packages']
for key in list(os.environ):
    if key.startswith(('AZURE_CLI_TEST_SQL_', 'AZURE_TEST_')):
        os.environ.pop(key)
os.environ['AZURE_CORE_COLLECT_TELEMETRY'] = 'no'
with tempfile.TemporaryDirectory() as config, patch.dict(os.environ, {'AZURE_CONFIG_DIR': config}), patch('socket.socket.connect', side_effect=AssertionError('Offline test attempted network access')), patch('azure.cli.core.util.check_connectivity', return_value=False):
    suite = unittest.defaultTestLoader.loadTestsFromNames(['azure.cli.command_modules.sql.tests.latest.' + name for name in ('test_sql_str_policy', 'test_sql_db_str_policy', 'test_sql_mi_str_policy')])
    result = unittest.TextTestRunner(verbosity=2).run(suite)
print('SUMMARY tests={} failures={} errors={} skips={}'.format(result.testsRun, len(result.failures), len(result.errors), len(result.skipped)))
sys.exit(not result.wasSuccessful() or bool(result.skipped))
'@ | & 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe' -

# PASS: exit 0.
& 'E:\CLI\aaz-env\Scripts\aaz-dev.exe' command-model verify --aaz-path 'E:\CLI\aaz' --target sql

# PASS: exit 0.
& 'E:\CLI\aaz-env\Scripts\python.exe' -m flake8 --append-config '.flake8' 'src\azure-cli\azure\cli\command_modules\sql\_str_policy.py' 'src\azure-cli\azure\cli\command_modules\sql\custom.py' 'src\azure-cli\azure\cli\command_modules\sql\tests\latest\test_sql_str_policy.py'

# PASS: exit 0.
& 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe' -m compileall -q 'src\azure-cli\azure\cli\command_modules\sql\_str_policy.py' 'src\azure-cli\azure\cli\command_modules\sql\aaz\latest\sql\db' 'src\azure-cli\azure\cli\command_modules\sql\aaz\latest\sql\midb' 'src\azure-cli\azure\cli\command_modules\sql\custom.py' 'src\azure-cli\azure\cli\command_modules\sql\tests\latest\test_sql_str_policy.py'

# PASS: adapter pylint, exit 0, 10.00/10.
$env:PYTHONPATH='E:\CLI\azure-cli-str-immutability\src\azure-cli;E:\CLI\azure-cli-str-immutability\src\azure-cli-core;E:\CLI\azure-cli-str-immutability\src\azure-cli-testsdk;E:\CLI\str-env\Lib\site-packages;C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages'
& 'E:\CLI\aaz-env\Scripts\python.exe' -m pylint --rcfile 'pylintrc' --jobs 1 'src\azure-cli\azure\cli\command_modules\sql\_str_policy.py'

The broader AAZ repository check was stopped in favor of the focused SQL verification above. Hosted CI for this new commit is not claimed as passed; prior pipeline approvals/results do not validate this regenerated head.

SQL Database scenarios

Source: SQL Database STR tests.

Live-recorded scenario HTTP interactions Coverage and assertions
test_sql_db_short_term_retention 40 New randomized S0/Local DB after absence check. Retention/differential settings 7/24 → 6/12 → 5/12 → 5/24; explicit false sets 7 days without prompting. SET and independent GET assert retention/interval. No-wait sets 8 days, returns no output, and GET verifies 8/24. Tolerates unlocked Disabled/Enabled.
test_sql_db_short_term_retention_immutability 70 New DB; wait for Enabled; initialize 7/12. Decline/no-TTY cause no HTTP. Accepted confirmation locks at 7 days; SET returns Locked and GET independently verifies it. --yes increases to 8 days with SET and GET Locked. Explicit-lock decrease and false unlock return InvalidImmutableBackupRetentionPeriod and CannotRemoveLockedPitrImmutability. Differential-only no-wait preserves Locked at 8/24.
test_sql_db_existing_policy_immutability 14 Explicitly authorized disposable DB already Enabled. Locks with unchanged 7-day retention and preserves 24-hour differential interval; SET and GET report Locked. Rejects reduction to 6 days and unlocking. Repeated lock with no-wait has no output/prompt; independent GET remains Locked7/24. Policy-only transport guard; no create/delete.

All three refreshed SQL DB scenarios passed live on 2026-10-13 (new DBs in West US 2; policy-only in West Central US), and passed offline playback. The two new SQL DBs were retained by request. There are now eight retained SQL test databases across the earlier and current validation runs; they continue to incur charges. The policy-only run modifies only its expressly authorized retained test DB, not an original database.

SQL Managed Instance scenarios

Source: SQL MI STR tests.

Live-recorded scenario HTTP interactions Coverage and assertions
test_sql_managed_db_short_retention 46 New randomized DB on existing MI; live retention 7 → 14 → 7(false) → 10(no-wait). No confirmation for omitted/false lock. SET/GET retention checks; wait for backup readiness, then drop only its own DB. Dropped policy 10 → 7 → 6(no-wait). Both true and false lock flags with deleted-time fail locally without HTTP or prompt. Final policy GET after async SET also covers the timestamp-addressed dropped resource.
test_sql_managed_db_short_retention_immutability 53 New DB must report Enabled. Ordinary updates/false before locking; decline/no-TTY send no HTTP. Accepted lock at 7 and --yes increase to 8 both return SET Locked, with independent GET verification. Service rejects reduction/unlocking. After dropping only its own DB, policy remains Locked8; both dropped lock-flag values fail locally, and the service rejects dropped-retention reduction.

Both refreshed MI recordings passed live in West Central US on 2026-10-13 and offline playback. Earlier attempts encountered a non-JSON response and an interrupted backup-readiness wait; these are not counted as successful recordings and were superseded by complete passing captures. All MI test databases from these attempts were cleaned up, including failed-attempt databases. The seven original MI database names remain unchanged; locked dropped backups are not purged and remain for their retention period.

Offline regressions, audits and CI

  • 28 CLI/HTTP unit tests: confirmation accept/decline/no-TTY/yes; omitted/false lock; request omission and integer/Boolean serialization; dropped-MI true/false rejection, zero retention, timestamp addressing; all three statuses; flattened output and nested systemData; required wait policy-name compatibility; IDs and help. Polling coverage includes async-only/both headers for all three resource types, InProgress → Succeeded → original policy GET, synchronous200, Location-only, no-wait request counts, Failed/Canceled without final read, final-GET403/404 propagation, and actual CLI SET JSON output. Lock-only serialization coverage does not imply live support.
  • 11 scenario safety tests: six SQL DB tests for explicit targets, own-resource/policy-only write guards, keep-database behavior and bounded policy waits; five MI tests for target/write scope, signed-URL sanitization, non-JSON/non-object preservation and live/dropped policy waits.
  • Recording audit passed for all five captures: real target names/subscription IDs and credential markers absent; signed polling parameters redacted; unrelated dropped databases filtered. Every recorded completed async STR SET uses the policy GET, with five Locked SET results across the three locking scenarios. No STR OperationResults GET remains in these captures. No-wait SETs have no polling/final reads.
  • Write-scope audit and post-run inventory/policy checks passed: prior database inventories preserved, only the two approved SQL DB additions remain, other retained test policies unchanged, and all MI attempt DBs removed. No server, MI or resource-group write is permitted by the guards; original databases are untouched.
  • Targeted pycodestyle, git diff --check, and editor diagnostics passed. The shared STR operation module passed Pylint at 10.00/10. Full local SQL-module Pylint is not claimed because the installed runtime has compiled-only dependencies and previously reported unrelated findings.
  • Hosted main/full-test/breaking-change/lint/style checks passed for the prior head 3333619a.... Checks for this new head 93e16b8f2253fbdccb1a8337cc2dca763d6cae83 must run independently; earlier green checks or bot summaries are not validation of this commit.

Hosted pipeline gate (prior head): On October 13, requested /azp run for the then-current head, but the Azure Pipelines bot reported insufficient commenter privileges. A maintainer must trigger the main/full-test/breaking-change pipelines. The independent azdev-linter/style workflows are running/queued; their results are pending.

Replay with python -m unittest azure.cli.command_modules.sql.tests.latest.test_sql_str_policy azure.cli.command_modules.sql.tests.latest.test_sql_mi_str_policy azure.cli.command_modules.sql.tests.latest.test_sql_db_str_policy in a configured developer checkout.

Live recording requirements and safety guards: SQL STR recording guidance.

  • MI: AZURE_CLI_TEST_SQL_MI_RESOURCE_ID; irreversible locking additionally requires AZURE_CLI_TEST_SQL_MI_ALLOW_IMMUTABILITY_LOCK=true.
  • New SQL DB: AZURE_CLI_TEST_SQL_SERVER_RESOURCE_ID; locking additionally requires AZURE_CLI_TEST_SQL_DB_ALLOW_IMMUTABILITY_LOCK=true. These runs used AZURE_CLI_TEST_SQL_DB_KEEP_DATABASES=true.
  • Policy-only SQL DB: AZURE_CLI_TEST_SQL_DB_RESOURCE_ID plus separate irreversible-lock authorization. Requires Enabled and retention above one day; always retains the database.
  • Missing recordings do not authorize live access. Do not run the entire SQL module live on shared resources. Local diagnostic cassettes are excluded from this PR.

History Notes

[SQL] az sql db str-policy: Add short-term backup immutability locking with confirmation and expose immutability status
[SQL] az sql midb short-term-retention-policy: Add immutability locking for live databases and expose status for live and dropped databases


This checklist is used to make sure that common guidelines for a pull request are followed.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@yonzhan

Copy link
Copy Markdown
Collaborator

Please fix CI issues

@svmicano

Copy link
Copy Markdown
Member Author

Production MI validation — 2026-10-12

Validated commit 3fe125128615c9c7a7958520ccb10a970fa9be22 by running the edited CLI/core source against the public Azure production ARM endpoint in West Europe. Two new, uniquely named databases were created on an existing test MI. No MI-level write requests were made, and pre-existing databases were not modified.

Check Result
Preview STR show for both live databases Passed; retentionDays: 7, immutabilityStatus: Disabled initially
Synchronous retention update with lock flag omitted Passed on both databases: 7 → 14 days, verified by show
Explicit --lock-immutability false without --yes Passed without prompting: 14 → 7 days
Live --no-wait Passed: update returned without a result, then read-only polling/show confirmed 10 days
Declined confirmation / no-TTY safeguards Passed using a controlled prompt host: exit 1, no result, zero HTTP requests; live readback was unchanged
Authorized --lock-immutability true --yes Service rejected with LockPitrImmutabilityNotAllowed; readback remained Disabled, with retention unchanged
Dropped database STR show Passed after confirming backup readiness and dropping only a new test database
Dropped STR synchronous update Passed: 10 → 7 days, verified by show
Dropped STR --no-wait Passed: 7 → 6 days, verified by read-only polling/show
Dropped --lock-immutability true and false Both rejected locally: exit 1, no result, zero HTTP requests, no prompt
Cleanup / scope safety Both newly created databases deleted; MI configuration matched the initial baseline, and the two original databases remained online

The dropped test backups remain under their configured retention; they were not purged. Use the original textual deletionDate from az sql midb list-deleted (for example with --query ... --output tsv) so shell date conversion does not lose fractional seconds.

Still blocked: this MI reports immutability Disabled, and the service explicitly disallows locking. Successful locking, Locked readback and rejection of retention decreases after locking therefore remain unvalidated on live resources. No MI configuration was changed to work around this limitation. SQL Database (non-MI) live validation was not performed in this run.

These were live smoke checks, not newly captured/re-recorded scenario cassettes. Existing recording refresh and standard full-module validation remain outstanding; keep this PR in draft. No implementation changes were required by the supported-path checks.

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@svmicano

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Commenter does not have sufficient privileges for PR 34207 in repo Azure/azure-cli

@svmicano
svmicano marked this pull request as ready for review October 13, 2026 14:33
@svmicano
svmicano requested a review from a team as a code owner October 13, 2026 14:33
Copilot AI balanced review requested due to automatic review settings October 13, 2026 14:33
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A successful MI database creation can leak a billable resource when an immediate output check fails before cleanup is enabled.

1 open finding
What changed in this PR

Adds SQL short-term backup retention immutability support for SQL Database and Managed Instance policies.

Changes:

  • Adds locking, confirmation, status output, and preview API support.
  • Introduces shared AAZ HTTP/LRO handling with final policy reads.
  • Adds comprehensive unit, scenario, safety, and playback coverage.
File Description
sql/​_help.py Documents immutability workflows.
sql/​_params.py Defines locking and confirmation arguments.
sql/​_str_policy.py Implements shared STR HTTP operations.
sql/​commands.py Registers custom STR handlers.
sql/​custom.py Adds validation and confirmation orchestration.
tests/​latest/​README.md Documents recording and safety procedures.
recordings/​test_sql_db_existing_policy_immutability.yaml Records existing-policy locking.
recordings/​test_sql_db_short_term_retention.yaml Records standard SQL DB retention.
recordings/​test_sql_db_short_term_retention_immutability.yaml Records SQL DB locking.
recordings/​test_sql_managed_db_short_retention.yaml Records standard MI retention.
recordings/​test_sql_managed_db_short_retention_immutability.yaml Records MI locking.
tests/​latest/​test_sql_commands.py Removes superseded scenarios.
tests/​latest/​test_sql_db_str_policy.py Adds SQL DB scenarios and safeguards.
tests/​latest/​test_sql_mi_str_policy.py Adds MI scenarios and safeguards.
tests/​latest/​test_sql_str_policy.py Adds offline HTTP and CLI tests.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +135 to +137
self.cmd('sql midb create --subscription {subscription} -g {rg} --mi {mi} -n {db}', checks=[
self.check('name', '{db}'), self.check('status', 'Online')])
self.database_created = True
@teresaritorto Teresa Ritorto (teresaritorto) added the Request X Engineering Agent Ask the agent to triage this issue or review this PR label Oct 15, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

🔔 Routing this PR to @Azure/act-observability-squad.

@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

🔔 Routing this PR to @Azure/act-codegen-extensibility-squad.

@JaysonTaiMicrosoft

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

)


class _ShortTermRetentionPolicy(AAZCommand):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

svmicano

  1. Please generate this with AAZ, this should generally end up in a /aaz sub directory. You can refer to the following for more information https://azure.github.io/aaz-dev-tools/
  2. Provide the appropriate upstream PR after completing step 1

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in e2b0117fd6 (AAZ migration commit 9451dbe62e). The six STR GET/PUT bases are now generated with aaz-dev-tools 4.7.1 under the SQL aaz directory. The remaining custom layer preserves the existing public handlers and the already-tested final policy GET behavior; request/response schemas are generated.

Upstream command models: Azure/aaz#1130. Both PR descriptions are cross-linked.

Validation: 47 offline tests passed, including all five recorded scenarios and execution of all four generated public-command examples; SQL AAZ metadata verification, flake8, compilation, and adapter pylint passed. No new live resource operations were performed. Full legacy-module pylint has no new diagnostics versus its parent; local environment limitations are documented in the PR.

@x-engineering-agent

x-engineering-agent Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

X Engineering Agent Report

Review — Comments

svmicano

Reviewed head 03307dd24188df4fce24d1223c51430d19eb144f. Upstream CI completed successfully. The cleanup ordering below still needs correction.

Confirmed finding — [P2] Record database ownership before checking creation output
  • src/azure-cli/azure/cli/command_modules/sql/tests/latest/test_sql_mi_str_policy.py:135–137: creation checks execute before database_created=True. If creation succeeds but either check fails, cleanup at lines 140–144 skips deletion. Copilot feedback 4260152815 remains valid.
  • src/azure-cli/azure/cli/command_modules/sql/tests/latest/test_sql_db_str_policy.py:104–106: JSON parsing likewise precedes the ownership flag; parsing failure prevents live cleanup at lines 111–125.

The ordering is confirmed by src/azure-cli-testsdk/azure/cli/testsdk/base.py:175–177,268–287. Both paths can leave an unintended billable database.

Remediation: Separate successful cmd() execution from parsing/assertions; set ownership immediately after execution, preserving collision protection and SQL retention/playback guards.

Focused validation: Add failure-path tests where successful creation is followed by failed name/status checks or JSON parsing. Assert cleanup deletes the owned database unless explicitly retained.

Evidence and remaining validation

Tests and recordings establish Enabled-before-lock preconditions, unchanged-retention locking, SET Locked output with independent GET verification, confirmation guards, dropped-flag rejection, serialization and flattened output. All 16 completed asynchronous recorded SETs finish with the original policy GET; five return Locked.

Metadata identifies Azure/aaz#1130, generation with aaz-dev-tools 4.7.1, and pinned specification inputs; generated ownership is documented, not presumed hand-editing.

The author reports current-head 50 offline tests and eight live interval updates; these are author evidence, not Agent runs. CI succeeded, but cleanup-on-successful-create/output-validation-failure scenario coverage is not verified. Please add the focused regressions above and rerun both scenario safety suites and the five recorded scenarios.

Test validation
  • Agent live test: Not run; this is a review-only PR. Check author-provided test evidence and upstream CI separately.
  • Regression coverage: For sql: 4 focused test file(s) and 5 recording file(s) changed. Scenario coverage is unverified; review the issue's required conditions, test setup, and assertions before approval. No linked issue was found in the PR timeline; inspect the source issue and its required scenario manually.
Risk assessment — 83/100 · Critical

The Critical rating is driven by security-sensitive behavior, public CLI behavior, failure-handling behavior and 1 additional signal(s).

  • Change scope: 31 changed files, 38940 changed lines (+12734 / -26206), including 21 production files.
  • Affected components: sql
  • Risk drivers: security-sensitive behavior (+28); public CLI behavior (+18); failure-handling behavior (+12); generated output (+12)
  • Regression evidence: Changed regression tests are included, reducing risk.
  • Confidence: High because changed-line patches were available for every production file.
  • Required review: Owning-squad review is required for sql before merge.

Posted by x-engineering-agent (Reviewer)

@x-engineering-agent x-engineering-agent Bot added the X Engineering Agent Reviewed Agent reviewed this PR label Oct 16, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

🔔 Routing this PR to @Azure/act-observability-squad.

@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

🔔 Routing this PR to @Azure/act-codegen-extensibility-squad.

Generate SQL DB, live MI database, and dropped MI database STR GET/PUT command bases. Preserve existing custom command surfaces and ARM final-policy polling, and expand offline regression coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep generated examples aligned with the existing public wrappers, which supply the default policy name internally, and exercise all four examples through the CLI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

Normalize the SQL Database differential interval in the custom operation adapter without editing generated code. Add exact JSON type and fractional-input regressions. Verified 50 offline tests and eight live interval updates on retained disposable databases with settings restored.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-codegen-extensibility-squad act-observability-squad Auto-Assign Auto assign by bot Backup Request X Engineering Agent Ask the agent to triage this issue or review this PR SQL az sql X Engineering Agent Reviewed Agent reviewed this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants