Repository navigation
Send who requested the build in the metrics payload - #4574
migueldalberto wants to merge 8 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds support for capturing build requester information by introducing --buildUserEmail and --changeAuthorEmail command-line arguments to uploadMetadata.py. It includes a helper function resolve_requester to parse and clean these fields, which are then added to the pipeline and per-design payloads. The review feedback suggests two valuable improvements: simplifying the resolve_requester function using a dictionary comprehension for cleaner code, and hoisting the resolve_requester call outside the loop in publish_v1_per_design to prevent redundant executions and improve performance.
Adds --buildUserEmail and --changeAuthorEmail to uploadMetadata.py, sent as the top-level build_user_email and change_author_email keys on every schema version, v1 per-design fallback included. Blank values are left out. The dashboard stores them on the build and attributes the build to the user who started it, so a "my branches" view can include builds a user started by hand. They describe the build rather than the payload's shape, so they do not bump payload_schema_version. The backend must already register them as metadata before this lands: a v1 message carries metrics at its root, and an older backend would ingest the keys as metrics. Signed-off-by: Miguel Dalberto Pedro <miguel.pedro@precisioninno.com>
fcd9748 to
23a2a2c
Compare
🔍 QoR checkMetrics reflect the PR merge build — i.e. what will land on the target branch. Commit 62 design(s) checked — 0 with regression(s), 0 without a comparable baseline. |
Signed-off-by: Miguel Dalberto Pedro <miguel.pedro@precisioninno.com> # Conflicts: # flow/util/uploadMetadata.py
vvbandeira
left a comment
There was a problem hiding this comment.
Thanks. With no flags the payload does not change, and the code looks correct. Some items before merge:
- Tests. Please add tests through
build_pipeline_payloadandpublish_v1_per_design: both flags set, one blank or whitespace-only, and both absent. The last case must prove that the payload is the same as before this PR. - Linked changes. Please link the backend change that reads
build_user_email/change_author_emailand the jenkins-ciorfsUploadMetadatachange, so we can check the key names and the case where a key is absent. - Value check.
resolve_requester(line 224) accepts any non-blank string. An unset Groovy variable gives the string"null", which would be stored as an email. Please treatnull/Noneas blank, or require an@. - Console log. A Jenkins
shstep prints its command with-x, so--buildUserEmail <addr>puts the Jenkins user's email in the build log. Could the script readBUILD_USER_EMAIL/CHANGE_AUTHOR_EMAILfrom the environment instead, or could the caller useset +x? - Data access.
BUILD_USER_EMAILcan be a private address. Who can read the Pub/Sub topic and the dashboard? - Nit: in
publish_v1_per_design,payload.update(requester)runs beforepayload.update(d["metrics"]), so a metric with the same name would replace it. A comment, or applying the requester keys last, would make this clear.
Heads-up: #4646 conflicts in two places with this PR, so the second one to merge will need a rebase.
uploadMetadata.py does its work at import time, so the tests run it as a subprocess against a temporary reports/ tree, with stub google.cloud modules recording each published message. They cover both requester flags set, one blank, and both absent, on the pipeline payload and on the per-design v1 fallback messages. The absent case compares the whole payload, which proves that a build without the flags publishes exactly what it did before the flags existed. Signed-off-by: Miguel Dalberto Pedro <miguel.pedro@precisioninno.com>
An unset Groovy variable interpolates as the string "null", which resolve_requester used to accept as an email because it only rejected blank values. A value without an "@" is now dropped with a warning. The requester is resolved once in the main flow and passed to both payload builders. Resolving it inside each builder made the warning print twice when the pipeline payload exceeded the size cap and the script fell back to per-design v1 messages. Signed-off-by: Miguel Dalberto Pedro <miguel.pedro@precisioninno.com>
A Jenkins sh step echoes its command line under sh -x, so passing --buildUserEmail puts the user's address in the console log. When a requester flag is absent, the value now comes from UPLOAD_BUILD_USER_EMAIL or UPLOAD_CHANGE_AUTHOR_EMAIL, so the caller can keep the address out of the command line. An empty flag counts as given and suppresses the variable. Values from the environment go through the same blank and "@" checks as the flags. Signed-off-by: Miguel Dalberto Pedro <miguel.pedro@precisioninno.com>
A v1 message flattens the metrics onto the message root, so a metric named like a requester key replaced who asked for the build. The requester keys now go on last. The pipeline payload nests the metrics under each design and cannot collide. Signed-off-by: Miguel Dalberto Pedro <miguel.pedro@precisioninno.com>
Signed-off-by: Miguel Dalberto Pedro <miguel.pedro@precisioninno.com>
Merging master put a REQUESTER_ENV assignment in the middle of test_v1_fallback_warns_that_design_count_is_dropped and left its comment unindented, so the black check failed. The test is master's again, and black also drops the extra blank lines between the new tests. Signed-off-by: Miguel Dalberto Pedro <miguel.pedro@precisioninno.com>
|
Thanks for the review. I pushed fixes in four commits.
Also fixed merge conflicts. |
vvbandeira
left a comment
There was a problem hiding this comment.
approved from devops team side
Why
The QoR dashboard wants a "my branches / my builds" view. The only identity it has today is the commit's git author, fetched best-effort from GitHub. That misses builds a user started by hand on someone else's commit, and commits enrichment never reaches.
What
Two optional flags on
flow/util/uploadMetadata.py:--buildUserEmailbuild_user_emailBUILD_USER_EMAIL— user who started the build by hand--changeAuthorEmailchange_author_emailCHANGE_AUTHOR_EMAIL— PR author on PR buildspayload_schema_versionis unchanged.jenkins-ci passes them from
orfsUploadMetadata, probing for--buildUserEmailsupport like it does for--provenanceFile, so older ORFS branches keep uploading.