Add opt-in cross-source package deduplication

[AmirMS (AmelBawa-msft)]

📖 Description

Add opt-in cross-source package deduplication for single-package install, download, and ordinary show commands.

Enable it with experimentalFeatures.packageDeduplication: true. It is disabled by default and independent of interactive package selection.

• Compare two or three complete candidates from distinct sources, after source priority and before interactive selection.
• Require matching resolved versions/channels and installer SHA256 hashes, plus compatible installer families, scope, operation restrictions, and relevant dependency behavior.
• Select the earliest participating source in the opened source order and reuse its exact version, manifest, and installer without merging metadata or probing older releases.
• Preserve existing prompt/error behavior when equivalence cannot be established. Also propagate source-reported truncation so incomplete candidate sets are not treated as complete.

This does not merge search rows or apply to show --versions, upgrade flows, multi-package operations, or COM/PowerShell/DSC callers. Payload equivalence does not guarantee identical installer behavior; the selected source's instructions remain authoritative.

🎞️ Demo

Installed winget v1.30.180-preview before, and the PR's wingetdev build with deduplication enabled after. Actual read-only CLI output is replayed with excerpts and condensed timing.

1. PuTTY: equivalent installers resolve automatically. Both sources report version 0.85.0.0 and the same full installer SHA256, with compatible MSI/WiX installer types. winget reports ambiguity; wingetdev selects one candidate without prompting. The Store candidate wins because msstore came first in the opened source order at equal priority, not because the heuristic prefers the Store. Result-table order does not determine the selection.

2. OBS Studio: different versions remain separate. The sources report 32.2.2 from winget and 32.0.1 from msstore, with different installer hashes. Both builds retain ambiguity. Deduplication does not choose an older release just to obtain a match.

🔗 References

Related to #5345.

🔍 Validation

From src\x64\Debug\AppInstallerCLITests:

.\AppInstallerCLITests.exe '[PackageDeduplication]' -r compact

Passed: 43 test cases, 7,718 assertions.

Live comparisons with the deployed PR build:

Command	Result
winget show PuTTY --disable-interactivity	Expected ambiguity
wingetdev show PuTTY --disable-interactivity	Success; equivalent installers detected and Store candidate selected
winget show "OBS Studio" --disable-interactivity	Expected ambiguity
wingetdev show "OBS Studio" --disable-interactivity	Expected ambiguity retained

Also verified both scenarios on the same PR build with deduplication disabled and enabled to isolate this feature's effect. Source-specific metadata confirmed PuTTY's matching version/hash and OBS Studio's differing versions/hashes.

Development settings were restored after capture. No packages were installed by the demo.

The full native suite and installation/download E2E tests were not run.

✅ Checklist

• Signed the Contributor License Agreement
• Linked to an issue
• Updated Release Notes (if applicable)
• Updated documentation (if applicable)
• Added or updated tests (or noted why not applicable)
• Updated Copilot instructions (if build, architecture, or conventions changed)

🤖 AI Assistance

• AI assistance was used and has been disclosed in this PR
• No AI assistance was used

AI assisted with code refactoring, validation, spelling fixes, and demo/PR preparation.

📋 Issue Type

• Bug fix
• Feature
• Task

Microsoft Reviewers: Open in CodeFlow

[Copilot]

🔵 Needs a closer look

Automatic package-source selection affects install security and reliability across several core workflows and warrants final human validation.

0 open findings

What changed in this PR

Adds opt-in cross-source package deduplication while preserving source priority, trust, agreements, and installer verification.

Changes:

• Adds the packageDeduplication experimental setting.
• Compares versions, hashes, applicability, and dependencies before automatic selection.
• Adds extensive workflow tests, documentation, resources, and truncation handling.

File	Description
src/​AppInstallerRepositoryCore/​CompositeSource.cpp	Propagates child-source truncation.
src/​AppInstallerCommonCore/​UserSettings.cpp	Validates the new setting.
src/​AppInstallerCommonCore/​Public/​winget/​UserSettings.h	Defines the setting mapping.
src/​AppInstallerCommonCore/​Public/​winget/​ExperimentalFeature.h	Adds the feature flag.
src/​AppInstallerCommonCore/​ExperimentalFeature.cpp	Implements feature discovery and enablement.
src/​AppInstallerCLITests/​SourceFlow.cpp	Adds deduplication workflow coverage.
src/​AppInstallerCLITests/​PromptFlow.cpp	Tests truncated composite results.
src/​AppInstallerCLITests/​ExperimentalFeature.cpp	Tests setting and policy behavior.
src/​AppInstallerCLITests/​CompositeSource.cpp	Tests truncation propagation.
src/​AppInstallerCLIPackage/​Shared/​Strings/​en-us/​winget.resw	Adds user-facing messages.
src/​AppInstallerCLICore/​Workflows/​WorkflowBase.h	Declares selection flags and prepared matches.
src/​AppInstallerCLICore/​Workflows/​WorkflowBase.cpp	Implements comparison and selection.
src/​AppInstallerCLICore/​Workflows/​UpdateFlow.cpp	Handles prepared install matches.
src/​AppInstallerCLICore/​Workflows/​ShowFlow.cpp	Enables prepared show matches.
src/​AppInstallerCLICore/​Workflows/​InstallFlow.h	Exposes argument compatibility checking.
src/​AppInstallerCLICore/​Workflows/​InstallFlow.cpp	Implements non-mutating argument checks.
src/​AppInstallerCLICore/​Resources.h	Registers new resources.
src/​AppInstallerCLICore/​ExecutionContextData.h	Stores prepared package matches.
src/​AppInstallerCLICore/​Commands/​ShowCommand.cpp	Opts show into deduplication.
src/​AppInstallerCLICore/​Commands/​InstallCommand.cpp	Opts install into deduplication.
src/​AppInstallerCLICore/​Commands/​DownloadCommand.cpp	Opts download into deduplication.
schemas/​JSON/​settings/​settings.schema.0.2.json	Documents the setting schema.
doc/​windows/​package-manager/​winget/​source.md	Documents source behavior.
doc/​specs/​#5345 - Interactive package selection.md	Describes interaction with prompting.
doc/​specs/​#5345 - Cross-source package deduplication.md	Adds the feature specification.
doc/​Settings.md	Documents configuration.
doc/​ReleaseNotes.md	Announces the experimental feature.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[ranm-msft]

Thanks for this - the implementation is careful and the opt-in is genuinely surgical. I read WorkflowBase.cpp/.h, the three command opt-in sites, and the InstallFlow/ShowFlow/UpdateFlow/PromptFlow hunks. The THROW_HR_IF identity/operation/version/manifest asserts, the single-use reset of Data::PreparedPackageMatch in SelectInstaller, the GetDetails().Priority skip that preserves source precedence, and the fact that OpenSource and InstallSinglePackage are untouched (so no source or package agreement is bypassed) all look right to me.

One question is holding me back from approving.

Is byte-identical SHA-256 sufficient to call two installers equivalent?

GetEquivalentInstallers requires the preferred installer hash to be present, correctly sized, non-zero and identical across candidates, plus matching version/channel, an allowlisted installer type, and non-conflicting explicit scope. That proves the two candidates point at the same payload. It does not appear to prove they would execute the same way.

A single bootstrapper or unified installer can install different products, editions, workloads or channels purely as a function of its command line. Two manifests from different sources can legitimately share a hash and a nominal version while differing in InstallerSwitches, generated arguments, expected return codes, or ExpectedReturnCodes interpretation. In that case the dedup path would silently pick one source's execution contract and suppress the disambiguation prompt that exists precisely to let the user choose.

Concretely, I did not find a comparison of the fields that determine what the installer actually does - switches and generated arguments in particular - only of the payload identity and the surrounding eligibility gates.

Two ways this could be closed out, either of which would let me approve:

1. Extend the equivalence predicate to cover the effective execution contract (switches / generated arguments / scope and elevation behaviour), and add a negative test using the same hash and version but product-selecting switches that differ, asserting fallback to the normal prompt.
2. If the shared-bootstrapper false positive is considered acceptable in practice, say so explicitly in the spec and I will take that as the answer - it is a product call, not a code defect.

I appreciate that the feature is experimental and off by default, which keeps the blast radius small. I did not want to lean on that to wave through a predicate I could not evidence, since the user-facing message describes the results as equivalent.

Everything else I read I am comfortable with, and the test coverage across SourceFlow.cpp, CompositeSource.cpp, PromptFlow.cpp and ExperimentalFeature.cpp is substantial.