Repository navigation
Add STG105 Storage features - #7448
Merged
JinmingHu (Jinming-Hu) merged 12 commits intoOct 16, 2026
Merged
Conversation
|
Azure Pipelines: Successfully started running 2 pipeline(s). 8 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
JinmingHu (Jinming-Hu)
force-pushed
the
stg105-all-features
branch
from
October 6, 2026 02:37
d313453 to
664ec84
Compare
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Listing metadata conversion and mixed-case session endpoint parsing still have correctness issues.
Review effort: Balanced
Findings: 4
Open (4)
What changed in this PR
Adds STG105 capabilities across the Storage SDK, extending shared infrastructure, Blob and Data Lake downloads, and Files Shares APIs.
Changes:
- Adds opt-in session authentication and locality-aware downloads.
- Changes transfer defaults to 4 MiB and Blob listing defaults to Arrow.
- Adds Files Shares metadata, NFS listing types, change feed configuration, and file-ID addressing, with tests and documentation.
| File | Description |
|---|---|
| sdk/storage/azure-storage-files-shares/test/ut/share_service_client_test.cpp | Checks listed share creation times. |
| sdk/storage/azure-storage-files-shares/test/ut/share_file_client_test.cpp | Tests file-ID addressing and hard links. |
| sdk/storage/azure-storage-files-shares/test/ut/share_directory_client_test.cpp | Tests NFS entry types and properties. |
| sdk/storage/azure-storage-files-shares/test/ut/share_client_test.cpp | Tests addressing restrictions and change feed. |
| sdk/storage/azure-storage-files-shares/src/share_options.cpp | Adds file-ID parsing and operation guards. |
| sdk/storage/azure-storage-files-shares/src/share_lease_client.cpp | Guards unsupported file-ID lease operations. |
| sdk/storage/azure-storage-files-shares/src/share_directory_client.cpp | Maps NFS entries and supports ID-addressed properties. |
| sdk/storage/azure-storage-files-shares/src/share_client.cpp | Adds ID-based factories and change feed options. |
| sdk/storage/azure-storage-files-shares/inc/azure/storage/files/shares/share_responses.hpp | Exposes NFS entries and hard-link results. |
| sdk/storage/azure-storage-files-shares/inc/azure/storage/files/shares/share_options.hpp | Adds change feed and hard-link options. |
| sdk/storage/azure-storage-files-shares/inc/azure/storage/files/shares/share_file_client.hpp | Declares file-ID and hard-link APIs. |
| sdk/storage/azure-storage-files-shares/inc/azure/storage/files/shares/share_directory_client.hpp | Adds directory file-ID state and accessor. |
| sdk/storage/azure-storage-files-shares/inc/azure/storage/files/shares/share_constants.hpp | Defines the file-ID query parameter. |
| sdk/storage/azure-storage-files-shares/inc/azure/storage/files/shares/share_client.hpp | Declares ID-addressed client factories. |
| sdk/storage/azure-storage-files-shares/inc/azure/storage/files/shares/rest_client.hpp | Updates service version and protocol models. |
| sdk/storage/azure-storage-files-shares/CHANGELOG.md | Records new Files Shares capabilities. |
| sdk/storage/azure-storage-files-datalake/test/ut/session_authentication_test.cpp | Tests session authentication and pipeline retention. |
| sdk/storage/azure-storage-files-datalake/test/ut/datalake_file_client_test.cpp | Checks transfer defaults and explicit overrides. |
| sdk/storage/azure-storage-files-datalake/test/ut/data_locality_test.cpp | Tests layouts and routed downloads. |
| sdk/storage/azure-storage-files-datalake/test/ut/CMakeLists.txt | Registers locality and session tests. |
| sdk/storage/azure-storage-files-datalake/src/private/datalake_utilities.hpp | Declares shared property conversion. |
| sdk/storage/azure-storage-files-datalake/src/datalake_utilities.cpp | Maps Blob properties and session options. |
| sdk/storage/azure-storage-files-datalake/src/datalake_responses.cpp | Implements layout pagination. |
| sdk/storage/azure-storage-files-datalake/src/datalake_path_client.cpp | Reuses shared property conversion. |
| sdk/storage/azure-storage-files-datalake/src/datalake_file_system_client.cpp | Preserves Blob configuration after renames. |
| sdk/storage/azure-storage-files-datalake/src/datalake_file_client.cpp | Implements layouts and forwards download hints. |
| sdk/storage/azure-storage-files-datalake/src/datalake_directory_client.cpp | Preserves Blob pipelines after renames. |
| sdk/storage/azure-storage-files-datalake/README.md | Documents locality APIs and routing. |
| sdk/storage/azure-storage-files-datalake/inc/azure/storage/files/datalake/datalake_responses.hpp | Adds layout responses and download hints. |
| sdk/storage/azure-storage-files-datalake/inc/azure/storage/files/datalake/datalake_options.hpp | Adds session/locality options and transfer defaults. |
| sdk/storage/azure-storage-files-datalake/inc/azure/storage/files/datalake/datalake_file_system_client.hpp | Documents deletion timing. |
| sdk/storage/azure-storage-files-datalake/inc/azure/storage/files/datalake/datalake_file_client.hpp | Declares GetLayout. |
| sdk/storage/azure-storage-files-datalake/CHANGELOG.md | Records features and changed defaults. |
| sdk/storage/azure-storage-common/test/ut/test_base.hpp | Declares shared request inspection and sanitizers. |
| sdk/storage/azure-storage-common/test/ut/test_base.cpp | Sanitizes recorded session credentials. |
| sdk/storage/azure-storage-common/test/ut/storage_url_test.cpp | Tests Storage endpoint parsing. |
| sdk/storage/azure-storage-common/test/ut/CMakeLists.txt | Registers URL parsing tests. |
| sdk/storage/azure-storage-common/src/storage_url.cpp | Parses service, account, and container URLs. |
| sdk/storage/azure-storage-common/src/storage_pipeline.cpp | Installs optional locality routing. |
| sdk/storage/azure-storage-common/src/storage_data_locality_policy.cpp | Routes requests while preserving Host. |
| sdk/storage/azure-storage-common/src/storage_bearer_token_authentication_policy.cpp | Separates authorization from sending. |
| sdk/storage/azure-storage-common/src/shared_key_policy.cpp | Generalizes request signing inputs. |
| sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_url.hpp | Declares Storage URL parsing. |
| sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_pipeline.hpp | Adds locality policy configuration. |
| sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_data_locality_policy.hpp | Declares routing policy and context helpers. |
| sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_bearer_token_auth.hpp | Enables session-policy specialization. |
| sdk/storage/azure-storage-common/inc/azure/storage/common/internal/shared_key_policy.hpp | Exposes reusable signature generation. |
| sdk/storage/azure-storage-common/CMakeLists.txt | Registers shared routing and URL sources. |
| sdk/storage/azure-storage-blobs/test/ut/storage_timeout_test.cpp | Uses the shared request inspection helper. |
| sdk/storage/azure-storage-blobs/test/ut/storage_retry_policy_test.cpp | Disables locality in an existing retry test. |
| sdk/storage/azure-storage-blobs/test/ut/CMakeLists.txt | Registers locality and session tests. |
| sdk/storage/azure-storage-blobs/test/ut/block_blob_client_test.cpp | Checks transfer defaults. |
| sdk/storage/azure-storage-blobs/test/ut/blob_container_client_test.cpp | Checks default Arrow and explicit XML responses. |
| sdk/storage/azure-storage-blobs/swagger/README.md | Updates generation input and layout customizations. |
| sdk/storage/azure-storage-blobs/src/private/token_credential_session_provider.hpp | Declares session caching and refresh state. |
| sdk/storage/azure-storage-blobs/src/private/token_credential_session_provider.cpp | Implements session acquisition and caching. |
| sdk/storage/azure-storage-blobs/src/private/session_authentication_policy.hpp | Declares authentication policy construction. |
| sdk/storage/azure-storage-blobs/src/private/session_authentication_policy.cpp | Implements session signing and Bearer fallback. |
| sdk/storage/azure-storage-blobs/src/private/data_locality.hpp | Declares layout routing state. |
| sdk/storage/azure-storage-blobs/src/private/data_locality.cpp | Selects endpoints and refreshes layouts. |
| sdk/storage/azure-storage-blobs/src/blob_service_client.cpp | Integrates session and locality policies. |
| sdk/storage/azure-storage-blobs/src/blob_responses.cpp | Implements layout pagination. |
| sdk/storage/azure-storage-blobs/src/blob_options.cpp | Constructs shared session providers. |
| sdk/storage/azure-storage-blobs/src/blob_container_client.cpp | Integrates policies and defaults listings to Arrow. |
| sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/rest_client.hpp | Adds layout protocol types and download hints. |
| sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/blob_service_client.hpp | Documents container deletion timing. |
| sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/blob_responses.hpp | Exposes layout models and paged responses. |
| sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/blob_options.hpp | Adds session/locality options and new defaults. |
| sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/blob_container_client.hpp | Documents container deletion timing. |
| sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/blob_client.hpp | Declares GetLayout. |
| sdk/storage/azure-storage-blobs/CMakeLists.txt | Registers session and locality implementations. |
| sdk/storage/azure-storage-blobs/CHANGELOG.md | Records features and default-behavior changes. |
| sdk/storage/assets.json | Pins updated test recordings. |
12 tasks
JinmingHu (Jinming-Hu)
force-pushed
the
stg105-all-features
branch
from
October 6, 2026 06:50
1206a84 to
d20342d
Compare
JinmingHu (Jinming-Hu)
requested review from
Rick Winter (RickWinter) and
Jeff Fisher (xirzec)
as code owners
October 6, 2026 08:54
Member
Author
|
/azp run cpp - storage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
JinmingHu (Jinming-Hu)
force-pushed
the
stg105-all-features
branch
from
October 7, 2026 00:18
4439fba to
3c1aba7
Compare
Member
Author
|
/azp run cpp - storage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
JinmingHu (Jinming-Hu)
force-pushed
the
stg105-all-features
branch
3 times, most recently
from
October 7, 2026 04:00
e7b57c0 to
28da08e
Compare
Member
Author
|
/azp run cpp - storage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
JinmingHu (Jinming-Hu)
force-pushed
the
stg105-all-features
branch
from
October 7, 2026 05:22
28da08e to
c0b6cd8
Compare
Anton Kolesnyk (antkmsft)
approved these changes
Oct 8, 2026
Preserve the requested Prefix in Arrow flat-list responses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Normalize hostname casing for service/account inference without changing URL casing. Use CMake automatic DLL exports for session providers without exporting STL members. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reduce default initial transfer sizes to 4 MiB and overlap remaining downloads with consumption of the initial response for Blob and Files transfers. Add Blob and DataLake layout APIs, download hints, and locality routing. Keep managed locality routing disabled by default while supporting explicit Auto and Enabled selection. Include both features' changelogs and layout member documentation. Expose the layout ranges directly as vectors in BlobLayoutInfo and FileLayoutInfo, without redundant range-container wrapper types. Preserve the explicit false legal-hold initializer through the generation customization and regenerated BlobProperties declaration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collect standalone deletion remarks, existing API descriptions, and generated-model documentation without changing executable behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Recognize the sovereign-cloud endpoint suffixes and service-defined xfiles header. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
JinmingHu (Jinming-Hu)
force-pushed
the
stg105-all-features
branch
from
October 14, 2026 06:38
4b8f586 to
d940bc5
Compare
Member
Author
|
/azp run cpp - storage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Member
Author
|
/azp run cpp - storage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Collect feature tests, fixture migrations, regression assertions, shared test helpers, test build definitions, and the published recording pin. Include default-disabled locality coverage and playback-only Files cases pending production enablement. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Initialize the nullable policy before assigning either Arrow immutability column without clearing a value parsed from the other column. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
JinmingHu (Jinming-Hu)
force-pushed
the
stg105-all-features
branch
from
October 14, 2026 07:44
1e5fb14 to
3ecb51a
Compare
Member
Author
|
/azp run cpp - storage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Move the default XML-to-Arrow listing response format note out of Breaking Changes because the public blob listing behavior is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
/azp run cpp - storage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Anton Kolesnyk (antkmsft)
approved these changes
Oct 14, 2026
7 of 14 tasks
Jeff Fisher (xirzec)
approved these changes
Oct 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the STG105 Storage SDK features across Blobs, Data Lake, and Files Shares:
2027-03-07and incorporate reproducible generation customizations.Default behavior
InitialChunkSizeandSingleUploadThresholdvalues remain respected.ResponseFormattoStorageResponseFormat::Xmlto retain XML responses.AutoandEnabledremain supported.Commit organization
The ten-commit history is rebuilt on latest upstream main (
443592ef3), with no NFS List V2 feature or revert commits. Seven feature commits each include their owning changelog changes. Partitioned transfers and data locality are combined in commitc2cbcff4c. Standalone comments/documentation and spelling allowlist updates have separate commits. All new and modified test sources, shared test helpers, test build definitions, and the published recordings pin are collected in dedicated commitd940bc507.Test results and rollout gates
Earlier feature-snapshot runs produced the following historical results, before the subsequent CI fixes and routing-default changes:
Subsequent targeted runs exercised the four rollout-dependent Files cases LIVE against the enabled account, regenerated their playback-only recordings, and confirmed PLAYBACK passes and LIVE skips. The default-disabled locality changes passed 24 targeted Blob/Data Lake cases.
The complete Files
ShareProperties,ListShares,ChangeFeed, andFileIdAddressingcases are temporarily playback-only pending CI-account feature availability. Their assertions remain intact, including existing quota and listing coverage. Restore LIVE execution after the relevant rollout. Other session/locality gates are tracked in #7449.Recordings are published and pinned to
cpp/storage_01f138ad9c. Session recordings redact tokens/keys and replace expiration with a fixed future date. Future RECORD runs must register the session sanitizers on test-proxy before recording and verify the saved recordings afterward; sanitizer setup is handled by the recording workflow, not C++ test helpers.Review notes
Jinming-Hu/azure-rest-api-specs.