Repository navigation
Replace deprecated dependencies and modernise test tooling - #415
Merged
Merged
Conversation
Move off the 8.6.0 preview now that the final release is out. Arsenal no longer pulls level-sublevel and its old leveldown stack, which removes most of the advisories reported on production dependencies. Issue: S3UTILS-249
The previous majors still run on the Node 20 actions runtime, which GitHub is phasing out. The new ones move to Node 24 and keep the inputs used here unchanged. Issue: S3UTILS-249
These pins dated back to 2022 and carried several known CVEs in requests, urllib3, certifi and idna. They are installed in the base image, so the whole image was flagged. service-report.py only uses plain requests calls, which behave the same with requests 2.34 and urllib3 2.x. Issue: S3UTILS-249
replicationAudit/ was written for a one-shot migration: finding CRR roles missing s3:ReplicateObject and fixing them before that permission got enforced. That migration is done, so the list, check and fix scripts are dropped along with their tests. The fix script was the only user of aws-sdk v2, which is end-of-life and flagged by dependency review, so that devDependency and its allowlist entry go too. Issue: S3UTILS-249
@aws-sdk/node-http-handler is deprecated and only re-exports the 1.x @smithy handler, while the S3 client itself runs on 4.x. Depend on the @smithy package so the scripts use the same handler as the SDK. In 4.x, requestTimeout became an overall request deadline which only logs a warning by default, and the socket idle timeout it used to configure is now socketTimeout. Switch to socketTimeout so stalled connections still fail and get retried instead of hanging forever. The cloudserver client was already affected, as it lets the SDK build a 4.x handler from its options, so it gets the same fix. Issue: S3UTILS-249
heapdump is an unmaintained native addon that has to be rebuilt for each Node ABI. Node has shipped v8.writeHeapSnapshot since 11, which writes the same .heapsnapshot files, so the profiler keeps working as before without the extra native build. Issue: S3UTILS-249
randomatic was only used by two tests to produce random sproxyd keys and version suffixes, and lodash/range was pulled in there without being declared at all. Node's crypto and Array.from cover both. Issue: S3UTILS-249
Nothing requires needle, strip-ansi or typescript: needle was left over from the service level sidecar, and the other two came in with an earlier tooling bump. eslint-plugin-import and eslint-plugin-jest are not loaded by the flat ESLint config either. eslint.config.mjs imports @eslint/js and @eslint/eslintrc directly, but they were only reachable as transitive dependencies of eslint, so declare them. Issue: S3UTILS-249
Move to Jest 30, sinon 22 and fake-timers 15, and bring eslint to the latest 9.x. Jest 30 renamed --testPathPattern to --testPathPatterns, and objectContaining no longer matches functions, so the replication updater tests now compare the command class directly instead of its name. mongodb-memory-server stays on 10.x: moving to 11 would change the MongoDB version the tests run against. Issue: S3UTILS-249
Contributor
Hello francoisferrand,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## development/1 #415 +/- ##
=================================================
+ Coverage 45.78% 47.65% +1.87%
=================================================
Files 88 87 -1
Lines 6555 6249 -306
Branches 1383 1334 -49
=================================================
- Hits 3001 2978 -23
+ Misses 3504 3223 -281
+ Partials 50 48 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
francoisferrand
requested review from
a team,
DarkIsDude,
SylvainSenechal,
delthas,
maeldonn and
nicolas2bert
and removed request for
a team,
SylvainSenechal and
delthas
October 15, 2026 14:20
maeldonn
approved these changes
Oct 15, 2026
DarkIsDude
approved these changes
Oct 15, 2026
DarkIsDude
left a comment
DarkIsDude
left a comment
Contributor
There was a problem hiding this comment.
Maybe should we check the codecov ?
Contributor
Waiting for approvalThe following approvals are needed before I can proceed with the merge:
|
francoisferrand
force-pushed
the
improvement/S3UTILS-249-deprecated-deps
branch
from
October 15, 2026 16:15
5671014 to
25ecad8
Compare
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
francoisferrand
force-pushed
the
improvement/S3UTILS-249-deprecated-deps
branch
from
October 15, 2026 17:37
25ecad8 to
22a2375
Compare
francoisferrand
force-pushed
the
improvement/S3UTILS-249-deprecated-deps
branch
from
October 15, 2026 17:42
22a2375 to
bf04b9f
Compare
followerDiff, compareFollowerDbs and the listing digests storage still
used the level 7 API, which went away when level was bumped to 9:
`new Level()` throws "Level is not a constructor" and createReadStream()
no longer exists, so these tools could not run at all. The unit tests
did not notice, as they only used mocked databases.
Depend on classic-level directly, like arsenal does, which also avoids
building a second copy of it. Range options and encodings are unchanged,
so existing databases read the same as with level 7.
classic-level only provides iterators, so a small read stream wraps them
into the { key, value } objects the comparison streams expect. It does
not use Readable.from(), which still emits 'end' when destroyed from a
'data' handler, as DiffStream does when it drops a digests stream.
classic-level also opens databases lazily, so the scanned databases and
the digests database are opened upfront. They are opened without
creating them, so that a wrong path fails the scan instead of silently
comparing against a new empty database.
Issue: S3UTILS-249
Each scanned database was left open until the process exited. With the legacy layout there is one database per bucket, so file descriptors and LevelDB caches piled up over a scan: close each one when its scan is done. Errors from the database read stream were not handled either, since .pipe() does not forward them: a read error crashed the process without logging anything. Errors from any stage now go through a single callback, which logs them, closes the databases and fails the scan, keeping the original error if closing fails too. Issue: S3UTILS-249
Issue: S3UTILS-249
francoisferrand
force-pushed
the
improvement/S3UTILS-249-deprecated-deps
branch
from
October 15, 2026 21:18
bf04b9f to
e287a46
Compare
Contributor
Author
|
/approve |
Contributor
|
I have successfully merged the changeset of this pull request
The following branches have NOT changed:
This pull request did not target the following hotfix branch(es) so they
Please check the status of the associated issue S3UTILS-249. Goodbye francoisferrand. The following options are set: approve |
eve-ci-cd
Bot
temporarily deployed
to
zenko/improvement/ZENKO-5376@2.16
October 15, 2026 22:03
Destroyed
eve-ci-cd
Bot
had a problem deploying
to
zenko/improvement/ZENKO-5376@2.16
October 16, 2026 17:39
Failure
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This cleans up s3utils' dependencies after the move to Node 24, and releases 1.20.0.
Most of it is about getting rid of deprecated, unmaintained or unused packages, which were also behind most of the security advisories reported on the image. Arsenal moves to the final 8.6.0, deprecated or abandoned packages are replaced by their maintained successors or by what Node now provides, unused ones are dropped, and the test tooling and GitHub Actions are brought up to date. The Python packages used by the utapi report are refreshed too, as they carried several CVEs into the image.
The replication audit scripts are removed: they were written for a one-shot migration which is now done, and they were the last reason to keep the end-of-life aws-sdk v2.
Two existing problems surfaced along the way and are fixed here:
followerDiff,compareFollowerDbs) could not run at all since an earlier upgrade of their database library; this went unnoticed because the tests only used mocks, so they now also run against a real databaseJSONStream stays for now, since arsenal still depends on it; this is followed up in ARSN-663 and S3UTILS-251.
The functional tests were not run locally, and the Raft tools still deserve a manual run on real databases.
Issue: S3UTILS-249