Skip to content

A fixed length string could overflow its declared field length - #16

Merged
holonsoftFounder merged 1 commit into
mainfrom
perf/v4-string-write
Sep 13, 2026
Merged

holonsoftFounder merged 1 commit into
mainfrom
perf/v4-string-write

Conversation

@holonsoftFounder

Copy link
Copy Markdown
Member

The bug

The padding loop appended whole fill characters until the field was full, with no upper bound:

while (effectiveLength < max)
{
    stringBuffer.AddRange(fillupChar);
    effectiveLength += fillupChar.Length;
}

A fill character that costs more than one byte, which is any character under the unicode encoder, overshoots whenever the declared length is not a multiple of that width.

An eleven byte unicode field padded with '\0' was written as twelve bytes. The buffer it went into was a List<byte> that could grow, so nothing noticed: the frame came out one byte too long, every field behind the string moved along by one, and the converter could not read its own output back.

Measured, not inferred: for a POCO of int + 11 byte unicode string + int, the writer produces 20 bytes for a 19 byte protocol.

How it surfaced

The size check added in #15 caught it on the very first new test. Instead of silently emitting a broken frame:

ProtocolConverterException: Internal size mismatch while writing UnicodeOddLengthStringPoco:
19 bytes were calculated but the writer produced more.

That check went in as a guard against something that "could only ever mean a bug in this library". It found one immediately.

The fix

Both branches truncate to the declared length, which is what the existing truncation branch already did for values that were too long. A fixed length field is exactly its declared length, always.

Tests

TestFixedLengthStringWriting pins the whole matrix, because the golden vectors pin one frame per protocol and cannot enumerate the interesting values of a field:

  • padding, truncation, empty and null values, an exactly fitting value
  • both encoders (ASCII one byte per char, Unicode two)
  • an odd declared length in both the padding and the truncating branch, plus a round trip proving the field behind the string is still where the protocol says it is
  • a surrogate pair cut through the middle, which stays byte level truncation as it always was, so frames that existing systems exchange do not change
  • agreement between ConvertToByteArray and TryConvertToByteArray for every value length

130/130 green.

Who is affected

Only protocols whose fill character is wider than one byte and whose declared length is not a multiple of that width. A one byte fill character, or a declared length that divides evenly, was always correct. Frames of an affected protocol written by 3.x are malformed and will not match what 4.0 writes, which is noted in the upgrade section of the README.

🤖 Generated with Claude Code

The padding loop appended whole fill characters until the field was full, with
no upper bound. A fill character that costs more than one byte, which is any
character under the unicode encoder, then overshoots whenever the declared
length is not a multiple of that width.

An eleven byte unicode field padded with '\0' was written as twelve bytes. The
buffer it went into was a List<byte> that could grow, so nothing noticed: the
frame came out one byte too long, every field behind the string moved along by
one, and the converter could not read its own output back.

Both branches now truncate to the declared length, which is what the existing
truncation branch already did for values that were too long. A fixed length
field is exactly its declared length, always.

The size check added with the caller supplied buffer is what surfaced this: the
first new test with an odd declared length failed with "19 bytes were calculated
but the writer produced 20" rather than silently emitting a broken frame.

TestFixedLengthStringWriting pins the whole matrix around it: padding,
truncation, empty and null values, an exactly fitting value, both encoders, an
odd declared length in both the padding and the truncating branch, a surrogate
pair cut through the middle, and agreement between the two write overloads for
every value length.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@holonsoftFounder
holonsoftFounder merged commit aeb94ce into main Sep 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant