Repository navigation
Aragorn pathfinder update - #175
Merged
Merged
Conversation
- Format workers/score_paths/scorer.py with black. - Stand in for the pathfinder's process pool in the tests that call shadowfax directly; only poll_for_tasks creates it. - The edge lookups send the worker's own retrieval parameters, and merge_message no longer assembles pathfinder results, so the tests now check that contract instead of the old 3-hop behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0199gZCRjpeS9RvKBMnY7RjJ
Codecov Report❌ Patch coverage is
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Retriever is removing its batch_size_limit, so the interior hops no longer need splitting into blocks of 300 ids a side: they go out as one lookup pinning every near node against every far node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0199gZCRjpeS9RvKBMnY7RjJ
- search_paths is checked against a brute-force enumeration on random graphs: every centre's cheapest loopless path, cheapest first. - edge_lookups, build_paths and build_paths_task, plus a shadowfax run over a small backbone with the search done for real. - scorer: path reading (phrases, qualifiers, negation, categories), features, and score_message with a fake model; the score_paths worker's save and failure handling. torch is not a test requirement, so a small stand-in is installed when it is missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0199gZCRjpeS9RvKBMnY7RjJ
- build_paths takes gandalf's inferred edge on an endpoint as the hop, not the child's own edge, and the orphan filter keeps its support graph. - A dropped path is counted by where its empty hop is, and the first such hop is named with whether its nodes came back. - On the backbone every lookup (the open endpoint ones and the many-id interior one) is a valid TRAPI 2.0 query with the worker's parameters and its own callback, and the paths are built and saved; a required_intermediate_categories constraint reaches the build and is kept to. test_shadowfax_searches_the_backbone shares their setup. - merge_message accumulates several lookups' knowledge graphs under the pathfinder's own query graph. - The search graph download extracts its CSR and node list, and refuses an archive without nodes.txt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uhbrar
force-pushed
the
aragorn_pathfinder_update
branch
from
October 16, 2026 03:20
2536ac0 to
a990f94
Compare
The callback's debug line printed the lookup's whole query graph, and a pathfinder lookup pins thousands of curies. A node with more than ten ids now logs as "N curies". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0199gZCRjpeS9RvKBMnY7RjJ
Parsing a large callback holds the GIL for seconds, long enough for uvicorn's supervisor to take the worker for hung and kill it, which lost the callback and left its query waiting out its timeout. The callback route now stores the body as it came: a zstd frame that carries its content size as is, anything else (re)compressed into one, all in a thread since zstd releases the GIL. A body that doesn't start like a JSON object is still refused with a 422. merge_message parses it instead, fills in missing results and knowledge graph, logs the result counts and the query graph, and skips a callback it can't read rather than failing the batch. decode_message now reads streaming frames too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0199gZCRjpeS9RvKBMnY7RjJ
maximusunc
pushed a commit
that referenced
this pull request
Oct 16, 2026
Brings in the ARAGORN pathfinder backbone search and the new path scorer (#175), and callbacks stored without parsing them in the server, ported to TRAPI 1.x: - aragorn.pathfinder builds paths with 1.x bindings ([{"id", "attributes"}] node bindings, [{"id"}] path bindings, aux graphs with attributes), reads the path constraint's intermediate_categories, and saves with save_message. - score_paths reads 1.x node and path bindings and saves with save_message_sync. - summarize_query_graph lives in shepherd_utils/query_graph_log.py, since shepherd_utils/trapi.py (2.0 helpers) stays deleted on this branch. - Tests ported to 1.x shapes; checks that lookups validate as TRAPI 2.0 become checks of the one-hop lookup's shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TtsBikU9y9AFQ5uu9tcioz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates ARAGORN pathfinder to use a middle-first search on a sparse backbone generated from the graph to find candidate paths, then scores those paths using a transformer model that is path-length-agnostic, allowing for any-length paths to be scored and ranked together.