Skip to content

Speed up common median reference on channel-sized axes - #4836

Open
JESUSROYETH wants to merge 1 commit into
SpikeInterface:mainfrom
JESUSROYETH:radar/perf-common-median-reference
Open

JESUSROYETH wants to merge 1 commit into
SpikeInterface:mainfrom
JESUSROYETH:radar/perf-common-median-reference

Conversation

@JESUSROYETH

Copy link
Copy Markdown
Contributor

Summary

Common median reference currently calls np.median for each trace chunk. For the short channel axis used here, NumPy's full sort path turns out faster than its partition-based median path.

This adds a private reduction helper: one or two numeric channels use their equivalent mean, finite numeric arrays with at least four channels sort and average the middle value(s), and three-channel, NaN, empty, or non-numeric inputs keep np.median. No public API changes.

Benchmark

I ran a complete cold-process run_sorter("tridesclous2"), 20-minute/32-channel recording, n_jobs=1, on an Intel Sapphire Rapids box, five alternating pairs:

before after change
full sorter 170.2 s 150.5 s −11.6% (19.7 s)

Each of the five paired runs improved, 10.02–18.97%. On SpikeInterface's public 384-channel NHP snippet, the CMR reduction itself comes out 3.89× faster, with identical output hashes.

I ran the common-reference tests too: 16 passed, 1 skipped, Black/style clean. This is complementary to #4564 — it cuts default single-thread work without adding threading.

@alejoe91 alejoe91 added this to the 0.106.0 milestone Oct 7, 2026
@alejoe91 alejoe91 added the preprocessing Related to preprocessing module label Oct 7, 2026
@alejoe91 alejoe91 modified the milestones: 0.106.0, 0.105.1 Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

preprocessing Related to preprocessing module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants