Skip to content

feat(build): filter unchanged APKs using content-aware hashing before release - #171

Closed
Smart123s wants to merge 1 commit into
nullcpy:mainfrom
Smart123s:feat/filter-unchanged-builds
Closed

Smart123s wants to merge 1 commit into
nullcpy:mainfrom
Smart123s:feat/filter-unchanged-builds

Conversation

@Smart123s

Copy link
Copy Markdown
Contributor

Fixes #165. When upstream patch sources (such as hxreborn/morphe-patches) update shared patch files, the CI builder triggers builds for all apps in that source. Even if the shared changes do not affect a specific app (e.g. Pocket Whip), rebuilding produces non-identical raw checksums due to APK timestamps and v2/v3 signing blocks. However, the rest of the files inside the APK (classes, assets) remain unchanged, if the shared patches didn't affect the specific app. Calculating the hash for all of the non-signature related files in the APK can be used to prevent unchanged apps from being re-released with new build numbers, triggering false update notifications in Obtainium.

Code was generated using Google Antigravity 2 (model: gemini-3.8-flash).

… release

Fixes #165. When upstream patch sources (such as `hxreborn/morphe-patches`) update shared patch files, the CI builder triggers builds for all apps in that source. Even if the shared changes do not affect a specific app (e.g. Pocket Whip), rebuilding produces non-identical raw checksums due to APK timestamps and v2/v3 signing blocks. However, the rest of the files inside the APK (classes, assets) remain unchanged, if the shared patches didn't affect the specific app. Calculating the hash for all of the non-signature related files in the APK can be used to prevent unchanged apps from being re-released with new build numbers, triggering false update notifications in Obtainium.

Code was generated using Google Antigravity 2 (model: gemini-3.8-flash).
@nullcpy

nullcpy commented Oct 10, 2026

Copy link
Copy Markdown
Owner

Still the issue remains. If something in the shared changes for the app it'll be skipped

@Smart123s

Copy link
Copy Markdown
Contributor Author

This check runs post-build on the compiled APK. So it will still be built, but it won't be published, which is enough to prevent Obtanium notification spam.

This builds on the idea that if the updated shared patches don't affect a specific app, the resulting patched APK will be the same as with the previous patch version (except for timestamps and meta-inf). I tested this with PocketWhip, and the APKs were identical, even tho there were changes in the shared patches that affected other apps only.

So if the APK contents are identical, this PR deletes them from the build directory right after the patching completes, which prevents it from being published as an update.

@fahim-ahmed05

Copy link
Copy Markdown
Collaborator

I'm not merging this but will implement something similar directly to the build engine. looks like the code produces empty release

@Smart123s

Copy link
Copy Markdown
Contributor Author

That's all right! I'm pretty sure your implementation will be far superior to mine.
Thanks for taking the time to work on this!

fahim-ahmed05 added a commit that referenced this pull request Oct 10, 2026
check_duplicate_build runs in build_rv the moment a mode's patch_apk
returns — before the copy to build/ and before module packing — and
ends the whole table build when the fresh APK's content fingerprint
equals the published one in state/build_content_hashes.json. The
verdict is taken on the first patched APK of the run (apk-mode for
apk/both, module APK when module is the only mode): apk and module
patching differ only in microG handling, so a match means the module
zip would repeat too, and skipping its patch+pack is the point —
PR #171 deleted finished files after the fact instead.

Modes off|log|enforce (RVB_DEDUP_MODE env or the dedup-mode file-level
key, defaulting to log so the measurement phase cannot skip anything by
surprise). log compares and prints but skips nothing and mutates no
reference; enforce updates the local state so both arches of one run
agree. The published copy on data is written only by CI after the
upload chain succeeded — a hash becomes authoritative exactly when its
artifact is live, so a built-but-unpublished run can never suppress the
real file later.

The check is CI-only (local builds always produce artifacts, the
module-update precedent), tool-scoped via PATCHER_HASH_DEDUP_ELIGIBLE,
and fails toward publishing on every unknown (no state file, no
python3, odd channel): an extra redundant release is survivable, a
wrongly suppressed one is not. Fingerprints ride in tab-separated
append.<pid>.tsv lines under temp/hashes (one file, named by the parent
run; short lines keep concurrent appends intact) and skips in
temp/unchanged/<slug>.json, both surviving the
end sweep like temp/failures; an all-duplicate run exits 0 with an
all.txt marker instead of aborting "All builds failed.", which is how
the CI guard tells "nothing new" from "everything broke". build.json
and the release manifest gain no keys — dedup is bookkeeping, not
metadata.

Co-authored-by: Péter Tombor <public@tmbpeter.com>
fahim-ahmed05 added a commit that referenced this pull request Oct 10, 2026
build_check_no_change.sh turns the engine's temp/unchanged/all.txt
into HAS_NEW_FILES=false, and every publishing step of build.yml —
numbered release, manifest, changelog/update pointers, archive upload,
website merge, Telegram — is skipped for that run. An all-duplicate
pool must end as a successful no-op: PR #171 proved the concept and
still created an asset-less numbered release, which is exactly the
failure this guard exists to prevent. The guard answers true for any
state it does not recognise, so nothing can be lost silently; the
reserved build number goes simply unused (the counter reads existing
releases/tags, so no gap appears).

build_merge_hashes.sh folds the fingerprints the engine recorded into
state/build_content_hashes.json, and runs only when the upload chain
got through — including the continue-on-error archive upload, whose
failure must not publish hashes for files that never landed.
commit_data_branch.sh then pushes the file to data only when the merge
actually changed it; a failed data push is a warning, not a failure:
the next run's fetch converges and the worst case is one redundant
republish, never a wrong skip.

RVB_DEDUP_MODE lives in the workflow env per the tuning-knob decision
and starts at "log": the enforcement flip is a separate, deliberate
edit after the measurement runs show zero false duplicates.
Regression harness: temp/_deduptest/run.sh (fixture APKs differing
only in timestamps/signature entries; all three CI surfaces plus the
engine gate across off/log/enforce, with negative controls).

Co-authored-by: Péter Tombor <public@tmbpeter.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

builder issue: APK update is released when only other apps in the patch sources were updated

3 participants