Repository navigation
[Packaging] BREAKING CHANGE: Support Python 3.11-3.15 and use aware UTC datetimes - #34225
NostraDavid (NostraDavid) wants to merge 13 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Thank you for your contribution NostraDavid (@NostraDavid)! We will review the pull request and get back to you soon. |
There was a problem hiding this comment.
🔵 Needs a closer look
It introduces a repository-wide breaking runtime change across 96 files while full, live, and container test coverage was not run.
0 open findings
What changed in this PR
Raises Azure CLI’s minimum Python version to 3.11 and makes UTC date handling timezone-aware across core, command modules, packaging, and tests.
Changes:
- Updates package metadata, CI, Debian builds, and documentation for Python 3.11+.
- Replaces
datetime.utcnow()with timezone-aware UTC timestamps. - Adds regressions for timestamp parsing, validation, surveys, role audits, and Cosmos DB restores.
| File | Description |
|---|---|
src/azure-cli/pyproject.toml |
Requires Python 3.11+. |
src/azure-cli/azure/cli/command_modules/storage/util.py |
Makes generated SAS expirations UTC-aware. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_validators.py |
Tests seven-day expiry validation. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_table_scenarios.py |
Uses aware UTC SAS expiry. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_sas_tokan_url_scenarios.py |
Uses aware UTC URL expirations. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_sas_scenarios.py |
Updates SAS scenario timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_queue_scenarios.py |
Updates queue SAS timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_file_share_scenarios.py |
Updates share SAS timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_file_scenarios.py |
Updates file SAS timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_container_access_policy.py |
Updates policy timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_blob_scenarios.py |
Updates blob test timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_blob_rewrite_scenarios.py |
Updates rewrite SAS expiry. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_blob_live_scenarios.py |
Updates live SAS timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_blob_copy_scenarios.py |
Updates copy SAS timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_batch_operations.py |
Updates batch SAS timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_azcopy_scenarios.py |
Updates AzCopy SAS timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_adls_gen2_scenarios.py |
Updates ADLS timestamps. |
src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_account_scenarios.py |
Updates delegation-key expiry. |
src/azure-cli/azure/cli/command_modules/storage/operations/queue.py |
Uses aware delegation-key start time. |
src/azure-cli/azure/cli/command_modules/storage/operations/fs_file.py |
Uses aware file delegation time. |
src/azure-cli/azure/cli/command_modules/storage/operations/fs_directory.py |
Uses aware directory delegation time. |
src/azure-cli/azure/cli/command_modules/storage/operations/filesystem.py |
Uses aware filesystem delegation time. |
src/azure-cli/azure/cli/command_modules/storage/operations/fileshare.py |
Uses aware share delegation time. |
src/azure-cli/azure/cli/command_modules/storage/operations/file.py |
Updates file-copy and SAS times. |
src/azure-cli/azure/cli/command_modules/storage/operations/blob.py |
Updates blob SAS and copy times. |
src/azure-cli/azure/cli/command_modules/storage/_validators.py |
Makes validation comparisons UTC-aware. |
src/azure-cli/azure/cli/command_modules/sql/tests/latest/test_sql_commands.py |
Updates SQL restore timestamps. |
src/azure-cli/azure/cli/command_modules/role/tests/latest/test_role.py |
Updates audit query timestamps. |
src/azure-cli/azure/cli/command_modules/role/tests/latest/test_role_custom.py |
Adds audit filter regression coverage. |
src/azure-cli/azure/cli/command_modules/role/custom.py |
Normalizes role-related UTC dates. |
src/azure-cli/azure/cli/command_modules/resource/tests/latest/test_resource.py |
Updates deployment SAS expiry. |
src/azure-cli/azure/cli/command_modules/redis/tests/latest/test_redis_scenario.py |
Updates Redis import/export times. |
src/azure-cli/azure/cli/command_modules/rdbms/tests/latest/test_rdbms_commands.py |
Updates restore current time. |
src/azure-cli/azure/cli/command_modules/rdbms/flexible_server_custom_common.py |
Updates log filtering time. |
src/azure-cli/azure/cli/command_modules/rdbms/custom.py |
Updates log filtering time. |
src/azure-cli/azure/cli/command_modules/rdbms/_flexible_server_util.py |
Simplifies aware current-time formatting. |
src/azure-cli/azure/cli/command_modules/postgresql/utils/_flexible_server_util.py |
Simplifies aware current-time formatting. |
src/azure-cli/azure/cli/command_modules/postgresql/commands/server_logs_commands.py |
Updates server-log filtering time. |
src/azure-cli/azure/cli/command_modules/postgresql/commands/backup_commands.py |
Updates generated backup dates. |
src/azure-cli/azure/cli/command_modules/network/tests/latest/test_network_commands.py |
Removes obsolete timezone fallbacks. |
src/azure-cli/azure/cli/command_modules/netappfiles/tests/latest/test_bucket_commands.py |
Uses aware certificate validity dates. |
src/azure-cli/azure/cli/command_modules/mysql/tests/latest/test_mysql_scenario.py |
Updates MySQL restore timestamps. |
src/azure-cli/azure/cli/command_modules/mysql/custom.py |
Updates log filtering time. |
src/azure-cli/azure/cli/command_modules/mysql/_util.py |
Simplifies aware UTC helpers. |
src/azure-cli/azure/cli/command_modules/monitor/custom.py |
Uses aware default query times. |
src/azure-cli/azure/cli/command_modules/monitor/_legacy/custom.py |
Updates legacy query times. |
src/azure-cli/azure/cli/command_modules/marketplaceordering/tests/__init__.py |
Uses aware test timing data. |
src/azure-cli/azure/cli/command_modules/lab/validators.py |
Normalizes expiration timestamps. |
src/azure-cli/azure/cli/command_modules/lab/tests/latest/test_lab_validators.py |
Tests timezone-aware expiration validation. |
src/azure-cli/azure/cli/command_modules/keyvault/tests/latest/test_keyvault_commands.py |
Updates SAS and certificate dates. |
src/azure-cli/azure/cli/command_modules/iot/tests/latest/_test_utils.py |
Updates test certificate validity. |
src/azure-cli/azure/cli/command_modules/iot/_utils.py |
Updates certificate validity dates. |
src/azure-cli/azure/cli/command_modules/feedback/tests/latest/test_feedback.py |
Tests existing survey timestamps. |
src/azure-cli/azure/cli/command_modules/feedback/custom.py |
Normalizes stored survey timestamps. |
src/azure-cli/azure/cli/command_modules/eventgrid/tests/latest/test_eventgrid_commands.py |
Updates partner-event timestamps. |
src/azure-cli/azure/cli/command_modules/databoxedge/tests/__init__.py |
Uses aware test timing data. |
src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py |
Updates restore timestamps. |
src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_backuprestore_scenario.py |
Tests UTC restore normalization. |
src/azure-cli/azure/cli/command_modules/cosmosdb/custom.py |
Normalizes restore timestamps to UTC. |
src/azure-cli/azure/cli/command_modules/containerapp/_utils.py |
Updates workflow and polling times. |
src/azure-cli/azure/cli/command_modules/containerapp/_github_oauth.py |
Corrects token-cache epoch timestamps. |
src/azure-cli/azure/cli/command_modules/billing/tests/__init__.py |
Uses aware test timing data. |
src/azure-cli/azure/cli/command_modules/batchai/custom.py |
Updates generated SAS expiry. |
src/azure-cli/azure/cli/command_modules/backup/tests/latest/test_backup_commands.py |
Updates retention dates. |
src/azure-cli/azure/cli/command_modules/backup/tests/latest/test_afs_commands.py |
Updates file-share retention dates. |
src/azure-cli/azure/cli/command_modules/backup/tests/latest/preparers.py |
Updates preparer retention dates. |
src/azure-cli/azure/cli/command_modules/backup/custom.py |
Updates job timeout timing. |
src/azure-cli/azure/cli/command_modules/appservice/static_sites.py |
Updates Azure DevOps token expiry. |
src/azure-cli/azure/cli/command_modules/appservice/custom.py |
Updates SAS, backup, and EOL times. |
src/azure-cli/azure/cli/command_modules/appservice/appservice_domains.py |
Emits aware domain timestamps. |
src/azure-cli/azure/cli/command_modules/appservice/_github_oauth.py |
Corrects token-cache epoch timestamps. |
src/azure-cli/azure/cli/command_modules/apim/tests/__init__.py |
Uses aware test timing data. |
src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_managed_cluster_decorator.py |
Updates upgrade-window assertions. |
src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_custom.py |
Updates force-upgrade assertions. |
src/azure-cli/azure/cli/command_modules/acs/tests/latest/custom_preparers.py |
Updates resource tag timestamps. |
src/azure-cli/azure/cli/command_modules/acs/managed_cluster_decorator.py |
Uses aware upgrade deadlines. |
src/azure-cli/azure/cli/command_modules/acr/_utils.py |
Makes token expiry UTC-aware. |
src/azure-cli/azure/cli/command_modules/acr/_format.py |
Adds aware log timestamps. |
src/azure-cli-testsdk/pyproject.toml |
Requires Python 3.11+. |
src/azure-cli-testsdk/azure/cli/testsdk/preparers.py |
Updates test resource tags. |
src/azure-cli-telemetry/pyproject.toml |
Requires Python 3.11+. |
src/azure-cli-core/pyproject.toml |
Requires Python 3.11+. |
src/azure-cli-core/azure/cli/core/tests/test_intercept_survey.py |
Adds survey timestamp regressions. |
src/azure-cli-core/azure/cli/core/telemetry.py |
Uses aware telemetry timestamps. |
src/azure-cli-core/azure/cli/core/intercept_survey.py |
Normalizes survey scheduling times. |
src/azure-cli-core/azure/cli/core/commands/__init__.py |
Updates deployment event query time. |
scripts/release/debian/Dockerfile |
Builds Debian packages with Python 3.11. |
scripts/ci/codegen_report.py |
Updates report date calculation. |
scripts/ci/build.sh |
Requires Python 3.11 for full tests. |
doc/install_linux_prerequisites.md |
Documents the new Python floor. |
doc/extensions/authoring.md |
Updates extension compatibility guidance. |
doc/debug/debug_in_vs_code.md |
Updates debugging prerequisites. |
doc/command_guidelines.md |
Updates supported Python versions. |
azure-pipelines.yml |
Adds Python 3.11 CI coverage. |
.github/agents/Messaging-cmdlets-Generation/messaging-cmdlet-gen.md |
Updates agent environment guidance. |
.azure-pipelines/templates/automation_test.yml |
Defaults automation to Python 3.11. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
NostraDavid (NostraDavid)
left a comment
There was a problem hiding this comment.
I've checked and read everything; pretty sure I've got the changes nailed down and reasoned about.
| - name: pythonVersion | ||
| type: string | ||
| default: 3.10 | ||
| default: '3.11' |
There was a problem hiding this comment.
type says string, so lets make that explicit. Otherwise we're implicitly converting a float to a string, which might cause weird behaviour.
| return dt.datetime.now(dt.UTC) + dt.timedelta(days=days) | ||
| except OverflowError: | ||
| return datetime.max | ||
| return dt.datetime.max.replace(tzinfo=dt.UTC) |
There was a problem hiding this comment.
Ensure the returned value always has a timezone set.
| target_restorable_account = None | ||
| restore_timestamp_datetime_utc = _convert_to_utc_timestamp(restore_timestamp) | ||
|
|
||
| # If restore timestamp is timezone aware, get the utcnow as timezone aware as well |
There was a problem hiding this comment.
This whole check block can now collapse into one line, as Python 3.11 guarantees support.
| import dateutil.parser | ||
| timestamp_datetime = dateutil.parser.parse(timestamp_string) | ||
| timestamp_datetime_utc = None | ||
| # Convert to utc only if timezone aware |
There was a problem hiding this comment.
So eh, this could be breaking something, but appears to not have done so (according to running the tests).
Previously, this function could return a datetime object without having set the timezone object within. By adding .replace(tzinfo=dt.UTC) on line 1955, we're fixing this minor bug.
| if survey_note['last_prompt_time']: | ||
| last_prompt_time = datetime.datetime.strptime(survey_note['last_prompt_time'], '%Y-%m-%dT%H:%M:%S') | ||
| use_duration = datetime.datetime.utcnow() - last_prompt_time | ||
| last_prompt_time = dt.datetime.strptime(survey_note['last_prompt_time'], '%Y-%m-%dT%H:%M:%S').replace(tzinfo=dt.UTC) |
There was a problem hiding this comment.
By adding .replace(tzinfo=dt.UTC) we're preventing any TypeErrors from popping up.
| set_object_properties('application', body, **kwargs) | ||
|
|
||
|
|
||
| def _datetime_to_utc(dt): |
There was a problem hiding this comment.
Because we're using dt as module name, it's better to rename this to a generic value to prevent name clashes.
| .format(resource_group, server, database_name)).get_output_in_json() | ||
|
|
||
| earliest_restore_date = _get_earliest_restore_date(db) | ||
| earliest_restore_date = _get_earliest_restore_date(db).replace(tzinfo=dt.UTC) |
There was a problem hiding this comment.
Still need that tzinfo set to prevent TypeErrors.
| - template: ${{ variables.Pipeline.Workspace }}/.azure-pipelines/templates/variables.yml | ||
|
|
||
| jobs: | ||
| - job: AutomationFullTestPython311ProfileLatest |
There was a problem hiding this comment.
Add oldest supported version (3.11), instead of one newer.
|
|
||
| - job: AutomationFullTestPython314ProfileLatest | ||
| displayName: Automation Full Test Python314 Profile Latest | ||
| - job: AutomationFullTestPython315ProfileLatest |
There was a problem hiding this comment.
Add newest (3.15) here.
| displayName: Test Python Wheels | ||
| strategy: | ||
| matrix: | ||
| Python312: |
There was a problem hiding this comment.
I presume we're not running all versions here as a cost-saving measure. Under that presumption it would make sense to run the oldest and newest versions of Python that are supported, no?
|
@microsoft-github-policy-service agree |
|
I made this PR because I got a bunch of annoying warnings when installing the CLI (it's on my nix configuration; quite the custom setup), and since 3.10 is finally not supported anymore, it's time to fix it. |
|
/azp run |
|
Azure Pipelines: Successfully started running 3 pipeline(s). |
|
🔔 Routing this PR to @Azure/act-codegen-extensibility-squad. |
|
🔔 Routing this PR to @Azure/act-platform-engineering-squad. |
|
🔔 Routing this PR to @Azure/act-codegen-extensibility-squad. |
|
🔔 Routing this PR to @Azure/act-platform-engineering-squad. |
X Engineering Agent ReportLive test — FailedFAIL (exit 1) Selectors: Changed test files runWorkflow run: https://github.com/Azure/issue-sentinel/actions/runs/38655915254 Live-test recordings: 4 regenerated — archived in workflow artifact
Recording filesLast 80 lines of azdev outputPosted by x-engineering-agent (Tester) |
🤖 PR Validation — ️✔️ All clear
Related command
This affects shared date handling across command modules, CLI core, tests, and CI.
Description
Replace all 218 Python
datetime.utcnow()calls with timezone-awaredt.datetime.now(dt.UTC). Remove redundant timezone assignments and obsolete fallbacks.Use
import datetime as dtconsistently in the touched Python files:dtidentifies the module anddt.datetimethe class. This removes the ambiguity created whendatetimedenotes either one. Azure CLI already uses this alias, and Python's datetime documentation recommends it for the same reason.Require Python >=3.11.0 in all four distributions, remove Python 3.10 classifiers, and advertise Python 3.15 support. Update contributor documentation, generated test-package classifiers, and tox environments to cover Python 3.11 through 3.15. Shift the existing wheel installation, core, telemetry, packaging, and profile-integration matrices to the oldest and newest supported versions, Python 3.11 and 3.15. Run extension loading on Python 3.15. Run load-performance checks on both Python 3.11 and 3.15 to cover the supported endpoints. Shift the full-test pipeline from Python 3.12/3.14 to Python 3.11/3.15, preserving eight shards per version. The Debian build default uses Python 3.11.
Normalize UTC timestamps read from existing survey notes and command inputs before comparisons. Add regressions for surveys, SAS expiry validation, role assignment audit dates, lab expiration, Cosmos DB restore timestamps, and ACR expiration overflow. The tests verify the exact seven-day SAS limit, preservation of lab timezone offsets near the current time, explicit role audit end times, and zero UTC offsets in converted Cosmos DB timestamps. GitHub token cache timestamps now remain correct when the local timezone is not UTC.
Testing Guide
Latest local guideline validation after the feedback formatting fix:
azdev style CLI --pep8.azdev style CLI --pylintafter the formatting fix and local SDK correction.azdev linter --ci-exclusions --min-severity medium --repo . --src HEAD --tgt 5f64a805d897437f4d10e35e829968d274277386completed with exit code 0. All 82 medium-severity findings also reproduce on the original base source. Custom pylint rules and command/parameter test-coverage rules passed.azure-mgmt-monitor==7.0.0b1; the previously installed stable 7.0.0 caused four import diagnostics on both base and PR code.Regression validation at commit
e0dd574681:PerformanceCheckruns on exactly Python 3.11 and 3.15.git diff --checkpassed.Earlier broad local validation after the
dtalias refactor, before the additional tests above:Each row also has 247 CLI-unit skips, 2 core skips, and 8 recording skips. Scenario classes were excluded from the unit collection. RDBMS was collected separately because its pytest options conflict with PostgreSQL; it contains no plain unit tests. Tests without existing recordings were skipped to avoid creating cloud resources.
The same 36 core failures reproduce on the base commit, and their test IDs match on all five Python versions. A fork-isolated run also reproduced its nine failures on the base. The intermittent Key Vault test patches
azure.cli.core.util.sdk_no_waitafterkeyvault.customhas imported that function. Preloading the module reproduces that failure on both the PR and base. No new failing test IDs were observed.Additional checks:
az self-testpassed on Python 3.15. Python 3.15 support metadata validated in all four rebuilt wheels. Earlier YAML checks validated the compatibility matrices while preserving job steps, triggers, and pools. The latest YAML check also validates the updated PerformanceCheck matrix. The full-test pipeline now uses only Python 3.11 and 3.15, with eight shards per version. Packaging automation tests, source-version checks, shell syntax, andgit diff --checkpassed.dtparameter rename that avoids shadowing the module alias.git diff --checkpassed.az self-teston Python 3.11 and 3.15 after the alias change. Wheel metadata requires Python >=3.11.0.az self-testpassed on official Python 3.11, 3.14, and 3.15 images.Run the focused regression tests with:
python -m pytest -q \ src/azure-cli-core/azure/cli/core/tests/test_intercept_survey.py \ src/azure-cli/azure/cli/command_modules/acr/tests/latest/test_acr_commands_mock.py \ src/azure-cli/azure/cli/command_modules/lab/tests/latest/test_lab_validators.py \ src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_backuprestore_scenario.py::CosmosDBRestoreUnitTests \ src/azure-cli/azure/cli/command_modules/role/tests/latest/test_role_custom.py \ src/azure-cli/azure/cli/command_modules/storage/tests/latest/test_storage_validators.py \ src/azure-cli/azure/cli/command_modules/feedback/tests/latest/test_feedback.py::TestSurvey python -B -m unittest discover -s tools/automation/tests -t tools -p 'test_*.py'Live Azure tests, the complete repository suite, and official Debian/RPM release-image builds were not run.
History Notes
[Packaging] BREAKING CHANGE: Drop Python 3.10 support; require Python 3.11 or later
[Packaging] Add Python 3.15 support
This checklist is used to make sure that common guidelines for a pull request are followed.
The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
Pending upstream CI and maintainer acceptance of the Python support-range change. Local style/linter results and existing baseline findings are documented in the Testing Guide above.
I adhere to the Error Handling Guidelines.