Skip to content

perf: FindAll/Count O(n²) for $-anchored patterns (UseReverseAnchored bypassed) #183

Closed

Description

@kolkov

FindAll/Count are O(n²) for $-anchored patterns — UseReverseAnchored bypassed, PikeVM re-run per position

Repro

ERROR.*$ on an ASCII log (two alternating lines, no match):

Lines Match Find FindAll Count stdlib FindAll
600 20 µs ~0 220 ms 218 ms 1.4 ms
1200 ~0 ~0 829 ms 886 ms 2.5 ms
2400 ~0 ~0 3.30 s 3.72 s —

×4 per doubling = O(n²). (?i)error.*$ behaves the same (109 / 331 / 1370 ms at 300 / 600 / 1200 lines).

Analysis

Match and Find go through ReverseAnchoredSearcher.Find — a single reverse DFA scan from the end, linear, correct.

FindAll and Count do not: FindAllIndicesStreaming → findAllIndicesLoop → findIndicesAtWithState → findIndicesNFAAtWithState → PikeVM.SearchWithSlotTableAt. CPU profile: 99.6 % in the PikeVM. The loop restarts the PikeVM from successive positions, each run scanning towards the end: O(n) calls × O(n) each.

Fix

A $-anchored pattern without (?m) matches at most once, at the end of the haystack. For UseReverseAnchored, FindAll/Count should call the reverse searcher once and stop: return the single match if start >= at, otherwise nothing. No PikeVM, no loop.

Impact

word.*$ is one of the most common log-search shapes; FindAll on a 1 MB log takes ~30 s vs stdlib's ~15 ms. Violates the linear-time search guarantee. Pre-existing (reproduced on 273ff75 and v0.12.27, unrelated to #182).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: metaStrategy selection, engine orchestrationpriority: criticalRelease blocker, correctness regressiontype: performanceSpeed/memory improvement or regression

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions