Skip to content

Render OBR without allocating on the render thread - #55

Open
trsonic wants to merge 3 commits into
mainfrom
fix/obr-process-no-alloc
Open

trsonic wants to merge 3 commits into
mainfrom
fix/obr-process-no-alloc

Conversation

@trsonic

@trsonic trsonic commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

ObrImpl::Process() and obr_process() allocate and free heap memory on every call (#28). Both run on a host's render thread. After this change neither allocates once the renderer is configured, and the output is bit-identical to main.

Process OBR audio without allocating

#28 measured 4 malloc and 4 free per Process() call with one 3OA element: the per-group output AudioBuffer and PeakLimiter::Process(). With loudspeaker layout or object elements there are more, in AmbisonicEncoder::ProcessPlanarAudioData(): the product assigned to each output frame builds a temporary Eigen vector, and with a moving source each frame also builds a new encoding matrix and an interpolated column per source, plus the spherical harmonic and Legendre vectors of each source and the source index lists of each processing group.

  • AmbisonicEncoder keeps its working storage as members sized in the constructor (the start and target column of each source, the per-frame encoding matrix and interpolated column, the spherical harmonic coefficients and the Legendre values) and writes the per-frame products with noalias(). Its documentation states that the output buffer must not be the input buffer, which every caller already ensures. The block's static encoding matrix is built in encoding_matrix_, which held a copy of it before.
  • AssociatedLegendrePolynomialsGenerator::Generate(x, &values_out) writes into a caller's vector; the returning overload calls it.
  • ProcessingGroup fills its world-locked and head-locked source index lists into members whose capacity UpdateAmbisonicEncoder() reserves, and UpdateAmbisonicEncoder() counts the sources instead of building their index list.
  • ObrImpl keeps the per-group output buffer as a member, allocated in the constructor.
  • PeakLimiter::Process() makes one pass per frame (the maximum sample across channels, the envelope, the gain applied to every channel) instead of filling two vectors. The arithmetic and its order are unchanged, and each frame reads its input before writing its output, so ObrImpl's in-place call gives the same result.

Render obr_process() through buffers ObrImpl keeps

obr_process() copied the planar input into a vector of vectors, then into a new input AudioBuffer, built a new output AudioBuffer, called Process() and copied the output back: several allocations per call through the C API. It also took ObrImpl's mutex twice per call, once to read the input channel count and again in Process().

  • ObrImpl::ProcessPlanar(input, output, num_frames) copies flat planar arrays into an input buffer ObrImpl keeps, renders it as Process() does and copies the two binaural channels out, under mutex_ taken once.
  • AddAudioElement() and RemoveLastAudioElement() size that input buffer in the same critical section that changes the element list, and the output buffer is sized in the constructor, so ProcessPlanar() checks that the channel counts match instead of resizing.
  • Process() checks its arguments, takes the lock and runs the shared body, ProcessLocked(), so its callers are unchanged. obr_process() calls ProcessPlanar() and keeps no buffers of its own.
  • New test ObrImplTest.TestProcessPlanarMatchesProcess: the same input through Process() on one renderer and ProcessPlanar() on another gives identical output with one element, after a second is added and after it is removed.

Verification

On macOS arm64 (Apple clang 21):

  • cmake -S . -B build -DOAR_BUILD_EXAMPLES=ON && cmake --build build && ctest --test-dir build -L 'oar|obr' --output-on-failure: 31 of 31 pass, in Release and with -fsanitize=address.

  • Standalone obr with Bazel 8.5.1, bazel test //...: 26 of 26 pass. As in Pin Bazel 8.5.1 and build the standalone obr module without WORKSPACE #47, the macOS run used a local apple_support toolchain dependency that is not part of this change.

  • RealtimeSanitizer, reproducing ObrImpl::Process() allocates and frees on the audio thread on every render call #28's measurement (Homebrew LLVM 22, -fsanitize=realtime, the render call inside a [[clang::nonblocking]] function, 128 frames, after one warm-up call, halt_on_error=false; distinct reports):

    Call main this branch
    ObrImpl::Process(), one 3OA element, limiter on 4 malloc, 4 free none
    ObrImpl::Process(), one moving mono object 20 malloc, 20 free none
    obr_process(), 3OA and a moving mono object 27 malloc, 27 free none
  • Output: a scratch program renders 40 blocks of 256 frames through ObrImpl::Process() and through obr_process(), built once against main and once against each commit of this branch, and compares an FNV-1a digest of every output sample. All 8 scenarios match: a 3OA element with the limiter on and off; a 7.1.4 layout; two moving mono objects; 3OA, 7.1.4, a moving dual object and a moving mono object together with head tracking and a changing head rotation, at an input level that engages the limiter, with the limiter on and off; and through the C API, a 3OA element and a moving object with a 7.1.4 layout added after 10 blocks and removed after 25, with the limiter on and off.

Not run here: the CMake and Bazel builds on Linux and Windows, which the CI workflows cover, and the oar root Bazel build.

ObrImpl still takes mutex_ in Process(), and SetHeadRotation(), EnableHeadTracking() and EnableLimiter() still take it too; #46 tracks that separately.

Closes #28

ObrImpl::Process() allocated and freed heap memory on every call (#28):
a temporary AudioBuffer for each processing group's output, the two
vectors PeakLimiter::Process() filled for the block, and, with
loudspeaker layout or object elements, Eigen temporaries in
AmbisonicEncoder::ProcessPlanarAudioData() for every output frame (the
product assigned to a frame built a temporary vector, and with a
moving source each frame also built a new encoding matrix and an
interpolated column per source), the spherical harmonic and Legendre
vectors of each source, and the source index lists of each processing
group. Process() runs on the render thread of a live host.

- AmbisonicEncoder keeps its working storage as members sized in the
  constructor (the start and target column of each source, the
  per-frame encoding matrix and interpolated column, the spherical
  harmonic coefficients and the Legendre values) and writes the
  per-frame products with noalias(). Its documentation states that the
  output buffer must not be the input buffer, which every caller
  already ensures. The block's static encoding matrix is built in
  encoding_matrix_, which held a copy of it before.
- AssociatedLegendrePolynomialsGenerator::Generate(x, &values_out)
  writes into a caller's vector; the returning overload calls it.
- ProcessingGroup fills its world-locked and head-locked source index
  lists into members whose capacity UpdateAmbisonicEncoder() reserves,
  and UpdateAmbisonicEncoder() counts the sources instead of building
  their index list.
- ObrImpl keeps the per-group output buffer as a member, allocated in
  the constructor.
- PeakLimiter::Process() makes one pass per frame (the maximum sample
  across channels, the envelope, the gain applied to every channel)
  instead of filling two vectors. The arithmetic and its order are
  unchanged, and each frame reads its input before writing its output,
  so ObrImpl's in-place call gives the same result.

Output is bit-identical to main for the scenarios in the pull request.
obr_process() copied the planar input into a vector of vectors, then
into a new input AudioBuffer, built a new output AudioBuffer, called
Process() and copied the output back, so every call through the C API
allocated and freed several buffers (#28). It also took ObrImpl's
mutex twice per call: once to read the input channel count, and again
in Process().

ObrImpl::ProcessPlanar(input, output, num_frames) copies flat planar
arrays into an input buffer ObrImpl keeps, renders it as Process()
does and copies the two binaural channels out, under mutex_ taken once.
AddAudioElement() and RemoveLastAudioElement() size the input buffer in
the same critical section that changes the element list, and the
output buffer is sized in the constructor, so ProcessPlanar() checks
that the channel counts match instead of resizing. Process() checks its
arguments, takes the lock and runs the shared body, ProcessLocked(), so
its callers are unchanged. obr_process() calls ProcessPlanar() and
keeps no buffers of its own.

ObrImplTest.TestProcessPlanarMatchesProcess renders the same input
through Process() on one renderer and ProcessPlanar() on another with
the same elements, and requires identical output with one element,
after a second is added and after it is removed.

@yeroro yeroro left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % nits

planar.ProcessPlanar(input.data(), output.data(), kNumFrames);
for (size_t ch = 0; ch < 2; ++ch) {
for (size_t frame = 0; frame < kNumFrames; ++frame) {
ASSERT_EQ(output[ch * kNumFrames + frame], output_buffer[ch][frame])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This line should be EXPECT_EQ.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in 387f63f: each block's channel is now compared with one EXPECT_THAT(actual, Pointwise(Eq(), expected)). A plain EXPECT_EQ per sample would log up to 1536 failures on a mismatch, and EXPECT_EQ on the two vectors prints only their first 32 elements; Pointwise reports the index and both values of the first differing sample. The ASSERT_EQ on the channel count at the top of the lambda stays, because the input buffers are sized from it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this look fine?

TestProcessPlanarMatchesProcess compared the output of ProcessPlanar()
with the output of Process() with one ASSERT_EQ per sample. It now
compares each channel of each block with one
EXPECT_THAT(actual, Pointwise(Eq(), expected)), a non-fatal check like
the rest of the file uses. A mismatch is reported once per channel, with
the index and both values of the first differing sample, and the test
goes on to the next channel and block. The ASSERT_EQ on the input
channel count stays, because the input buffers are sized from it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ObrImpl::Process() allocates and frees on the audio thread on every render call

4 participants