Skip to content

Add mirror pipeline to collect Photon advisories - #1

Merged
keshav-space merged 3 commits into
aboutcode-org:mainfrom
Samk1710:add-photon-mirror
Oct 19, 2026
Merged

keshav-space merged 3 commits into
aboutcode-org:mainfrom
Samk1710:add-photon-mirror

Conversation

@Samk1710

@Samk1710 Samk1710 commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Mirror for https://packages.broadcom.com/photon/photon_cve_metadata/cve/

Ref PR: aboutcode-org/vulnerablecode#2198

Local run Logs:

2026-10-07 18:56:18.303 Pipeline [PhotonAdvisoryMirror] starting
2026-10-07 18:56:18.303 Step [load_timestamps] starting
2026-10-07 18:56:18.303 Step [load_timestamps] completed in 0 seconds
2026-10-07 18:56:18.303 Step [create_session] starting
2026-10-07 18:56:18.303 Step [create_session] completed in 0 seconds
2026-10-07 18:56:18.303 Step [collect_advisories] starting
2026-10-07 18:56:18.303 Fetching root index from https://packages.broadcom.com/photon/photon_cve_metadata/cve/
2026-10-07 18:56:36.341 Found 14102 files to download or update.
2026-10-07 19:05:49.036 Progress: 10% (1411/14102) ETA: 4974 seconds (1.4 hours)
2026-10-07 19:15:36.803 Progress: 20% (2821/14102) ETA: 4562 seconds (1.3 hours)
2026-10-07 19:55:23.205 Progress: 30% (4231/14102) ETA: 3914 seconds (1.1 hours)
2026-10-07 20:04:15.680 Progress: 40% (5641/14102) ETA: 3315 seconds (55.2 minutes)
2026-10-07 20:13:08.484 Progress: 50% (7051/14102) ETA: 2743 seconds (45.7 minutes)
2026-10-07 20:21:27.165 Progress: 60% (8462/14102) ETA: 2161 seconds (36.0 minutes)
2026-10-07 20:31:26.448 Progress: 70% (9872/14102) ETA: 1606 seconds (26.8 minutes)
2026-10-07 20:39:27.353 Progress: 80% (11282/14102) ETA: 1057 seconds (17.6 minutes)
2026-10-07 20:47:52.008 Progress: 90% (12692/14102) ETA: 526 seconds (8.8 minutes)
2026-10-07 20:56:21.794 Progress: 100% (14102/14102)
2026-10-07 20:56:22.180 Sync complete. New: 14102 | Updated: 0
2026-10-07 20:56:22.180 Step [collect_advisories] completed in 5260 seconds (1.5 hours)
2026-10-07 20:56:22.180 Step [save_timestamps] starting
2026-10-07 20:56:22.197 Step [save_timestamps] completed in 0 seconds
2026-10-07 20:56:22.197 Pipeline completed in 5260 seconds (1.5 hours)

Signed-off-by: Samk <sampurnapyne1710@gmail.com>

@ziadhany ziadhany left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you try using this command to retrieve the data?

wget --mirror --no-parent --reject "index.html*" --no-host-directories -P photon_cve_metadata https://packages.broadcom.com/photon/photon_cve_metadata/cve/

see https://github.com/aboutcode-org/aboutcode-mirror-alpine-secdb/blob/main/.github/workflows/sync.yml#L20

Comment thread sync.py Outdated
@Samk1710

Copy link
Copy Markdown
Contributor Author

Did you try using this command to retrieve the data?

wget --mirror --no-parent --reject "index.html*" --no-host-directories -P photon_cve_metadata https://packages.broadcom.com/photon/photon_cve_metadata/cve/

see https://github.com/aboutcode-org/aboutcode-mirror-alpine-secdb/blob/main/.github/workflows/sync.yml#L20

Thanks for the recommendation @ziadhany. I'm running this locally and will tweak the workflow in a while.

Signed-off-by: Samk <sampurnapyne1710@gmail.com>
@Samk1710

Copy link
Copy Markdown
Contributor Author

@ziadhany I have updated the workflow. Thanks for your suggestion and please do review when you get a moment.

I researched and added --cut-dirs=3 so that we can cleanly store them in advisories.

@Samk1710
Samk1710 requested a review from ziadhany October 15, 2026 13:11

@ziadhany ziadhany left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Samk1710 , LGTM!

@Samk1710

Copy link
Copy Markdown
Contributor Author

@keshav-space Could you kindly have a look?

@keshav-space keshav-space left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Samk1710, see some nits below.

Comment thread .github/workflows/sync.yml Outdated
Comment thread .github/workflows/sync.yml Outdated
Signed-off-by: Samk <sampurnapyne1710@gmail.com>
@Samk1710

Copy link
Copy Markdown
Contributor Author

Thanks @Samk1710, see some nits below.

@keshav-space Thanks, I have made the changes.

@Samk1710
Samk1710 requested a review from keshav-space October 19, 2026 09:30

@keshav-space keshav-space left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Samk1710, LGTM!

@keshav-space
keshav-space merged commit dd13786 into aboutcode-org:main Oct 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants