Skip to content

Include failed QoR metrics in flow reports - #4676

Merged
vvbandeira merged 4 commits into
The-OpenROAD-Project:masterfrom
vvbandeira:qor-failure-details-20261014
Oct 14, 2026
Merged

vvbandeira merged 4 commits into
The-OpenROAD-Project:masterfrom
vvbandeira:qor-failure-details-20261014

Conversation

@vvbandeira

@vvbandeira vvbandeira commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

genReport.py now places each failed QoR metric's value, baseline, and limit directly under its matching failing run. It omits the optional QoR heading. The parser reads UTF-8 logs, with a regression test for non-ASCII log text.

PR #4675 is the test case. Build #3 showed that its actual run label differed from the identity used by the parser, so the metrics were missing from the summary. The matching fix is in this PR.

The latest PR #4675 commit is 94e0bba3c4f21adc0db77761ec4ae942420a79a3. Merge build #5 has recorded the six expected Nangate45 metrics and the expected QoR gate failure. Jenkins has not published the final report summary yet.

Signed-off-by: Vitor Bandeira <vvbandeira@precisioninno.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances QoR reporting by formatting and extracting detailed regression metrics. It introduces a helper to format exact values, logs failed metrics in checkQorMetrics.py, and updates genReport.py to parse and append these regressions to the final report. Feedback suggests specifying an explicit UTF-8 encoding when opening the log file in parse_qor_regressions to avoid potential UnicodeDecodeError crashes on systems with different default encodings.

Comment thread flow/util/genReport.py Outdated
regressions = []
in_regressions = False
try:
with open(filename, "r") as file:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Opening the file without specifying an explicit encoding uses the platform's default encoding (e.g., cp1252 on Windows). If the log file contains UTF-8 characters, this can raise a UnicodeDecodeError.\n\nSince UnicodeDecodeError inherits from ValueError (and not OSError), the except OSError: block will not catch it, causing genReport.py to crash. Specifying encoding="utf-8" ensures cross-platform compatibility and prevents unhandled exceptions.

Suggested change
with open(filename, "r") as file:
with open(filename, "r", encoding="utf-8") as file:

Signed-off-by: Vitor Bandeira <vvbandeira@precisioninno.com>
Signed-off-by: Vitor Bandeira <vvbandeira@precisioninno.com>
Signed-off-by: Vitor Bandeira <vvbandeira@precisioninno.com>
@openroad-ci

openroad-ci commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

🔍 QoR check

Metrics reflect the PR merge build — i.e. what will land on the target branch.

Commit b733ba0 · Jenkins build #4 · Baseline: build · View build on dashboard

62 design(s) checked — 0 with regression(s), 0 without a comparable baseline.

@vvbandeira
vvbandeira merged commit 9c43140 into The-OpenROAD-Project:master Oct 14, 2026
13 checks passed
@vvbandeira
vvbandeira deleted the qor-failure-details-20261014 branch October 15, 2026 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants