You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
16-bit float segments round half up instead of to nearest even #2447
Found while reviewing #2357; pre-existing (same code on release-0.7 at 18b3587, #2357 only moves it). Building a :16/float segment from a double that is not representable in half precision does not round the way IEEE 754 and BEAM do, so the bytes and the value read back differ:
6546.0 and -4474.0 sit exactly halfway between two representable values (the spacing is 4 in [4096, 8192)); BEAM rounds the tie to the even mantissa, AtomVM always rounds it away from zero. Subnormals are not rounded at all.
Cause: in bitstring_insert_f16 (src/libAtomVM/bitstring.c:764), the normal path (:808) computes (mantissa + 0x1000) >> 13, which is round-half-up despite the "round to nearest even" comment, and the subnormal path (:801-802) shifts the mantissa right and truncates. Extraction (bitstring_extract_f16) is exact, so the error is only on construction.
Fix: round the discarded bits to nearest, ties to even, in both paths, for instance (mantissa + 0xFFF + ((mantissa >> 13) & 1)) >> 13 for normal numbers and the same scheme with the subnormal shift amount, keeping the existing carry handling into the exponent. Tests should compare bytes with BEAM for a tie (6546.0), a value just below and above a tie, and a subnormal.
Activity
bitstring: Fix f16 rounding to nearest even
bitstring: Fix f16 rounding to nearest even
bitstring: Fix f16 rounding to nearest even
bitstring: Fix f16 rounding to nearest even
bitstring: Fix f16 rounding to nearest even