Skip to content

Add bond-length and well-depth metrics to existing diatomic references - #463

Open
anyangml wants to merge 2 commits into
deepmodeling:mainfrom
anyangml2nd:codex/diatomic-bond-metrics
Open

anyangml wants to merge 2 commits into
deepmodeling:mainfrom
anyangml2nd:codex/diatomic-bond-metrics

Conversation

@anyangml

@anyangml anyangml commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Changes

Add equilibrium bond-length and well-depth errors to the existing 14 MLIP Arena diatomic references. The reference JSON, original finite-difference roughness formula, and removal of the last scan point remain unchanged. All three metrics use the same trimmed distances.

Bond length uses a local quadratic fit around the sampled minimum, with a midpoint guess for flat predictions. References with depth below 0.05 eV skip bond position but still score well depth. Well depth is the energy difference between the last retained point and the sampled minimum, not an infinite-separation dissociation energy.

Normalize and cap each molecule's errors against a midpoint-position dummy and a constant-energy dummy, then average across molecules. The leaderboard reports the equal-weight average of roughness, bond-length score, and well-depth score in [0, 1], lower being better. Any missing/failed molecule prevents a combined score.

Inference saves its exact reference molecule list and per-molecule dummy errors, so custom references aggregate against their own test set. Legacy roughness-only records remain readable but require recomputation for the combined score. The leaderboard column becomes Applicability-Diatomics.

Validation

  • Full test suite: 71 tests passed.
  • 56 roughness regression cases match the original implementation.
  • Tests cover reference self-matches, the constant dummy, shifted/scaled wells, weak references, per-molecule normalization, incomplete results, and custom-reference JSON roundtrips.
  • Ruff and diff whitespace checks passed.

The repository-wide pre-commit check also reports existing baseline violations in database setup and unrelated portions of metrics/utils.py (including UP031, SIM115, C403, SIM118 and PERF102). These are outside this PR's scope; the changed-code local Ruff checks pass.

Replaces the approach in closed #456 without replacing the reference data.

@codecov

codecov Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.74214% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.29%. Comparing base (e137fd9) to head (0b4ef30).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
lambench/metrics/utils.py 94.44% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #463      +/-   ##
==========================================
+ Coverage   65.19%   68.29%   +3.10%     
==========================================
  Files          37       37              
  Lines        1879     2022     +143     
  Branches      230      247      +17     
==========================================
+ Hits         1225     1381     +156     
+ Misses        602      588      -14     
- Partials       52       53       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant