Repository navigation
perf(station): resolve each listed coin's market once per catalog revision - #1143
Merged
Merged
Conversation
This was referenced Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The tape catch-up listing (
SessionManager::list_tape_catchup) walks every closed trade of a core and calledMarketDataSource::resolve_marketfor each row, which searches the provider's whole catalog. A 70-core station report (#1081) counted 427k searches over ~10k distinct coins per pass: the same answer about forty times. On our station the hourly pass held the machine at 100 % for 20–27 s every hour (cpu_watch).The listing now keeps, per listing, the market each coin resolved to (
session/lifecycle/catchup_markets.rs), and drops all of it whenever any other input of the search moves: the core's catalog revision, its account currency, its exchange. Thetape catch-up … listedline gainsN market searches, X s resolving, so a station reports both figures itself.Measured on a local station against our 27 cores (read-only, the same database copy restored before each run, one binary with the cache switched off for "before"), first listing of every core:
(The row count moves by a few between runs: the replica receives new closed trades from the live cores.) In a pair of runs with every row's answer dumped, each row looked up in both runs resolved to the same market or the same "not found": zero differences.
Live on our station with this change: the hourly listing at 11:15 UTC (27 listings, 138,764 rows, 3,759 searches) raised no
cpu_watchepisode; the release's hourly pass reached 100 % of the machine for 20–27 s.Notable decisions
Known limitations
tradable_coinsper row, as before this change.Issues
Refs #1081 — CPU of the hourly listing; the out-of-memory cause in #1081 is still open.
How to verify
fmt exit 0, build exit 0, clippy zero warnings, 8559 tests green (three new in
catchup_markets/tests.rs).