[kixelated]

Works on quest/m2/cluster-routes/acceptance.md: the route layer is proven on the implementation, and the Babel-era reference model is deleted. The fixtures found four flaws. Each one is pinned by a test and becomes a follow-up quest. The quest is rewritten to what remains: retiring the old mitigations once those follow-ups land.

Summary

• The reference model is gone. rs/moq-net/tests/integration/routing_contracts/ (5,100 lines) is deleted.
  • Its forwarding, export and qualified-identity fixtures now run on the path router in route/tests/forward.rs.
  • Its pool-tie fixture now runs on real sessions.
  • Its Babel-only fixtures (feasibility, seqno requests, the distance a node keeps) are dropped with Babel.
• Every fixture the quest named now runs on the router or on real lite-07 sessions with mocked time. That covers topology, prior art, failover liveness, moq-dev/moq#4644's live-graph tests, the tiered cluster, partition and scoped interest, mount continuity, and operation-count regressions.
• Three bugs found along the way are fixed.
• Four flaws are pinned by tests, each documented in its test, and split into follow-up quests.

Changes

Forwarding, on the router (rs/moq-net/src/route/tests/forward.rs). A lite-07 request names its publisher, so the model's path-only counterexamples become "a named request arrives":

• local_priorities_never_turn_a_named_request: the model's priority ring, alone and with every relay's subscription joined.
• a_transit_relay_never_swaps_the_named_publisher
• nested_claims_never_loop_a_named_request
• sweep_named_requests_arrive: 150 random meshes. Every request arrives, both alone and joined. After a ring link is cut, no request loops at any step except through a stale route.
• a_forged_identity_is_contained_by_a_scoped_admission
• scoped_delivery_through_two_uplinks_reaches_one_key
• a_qualified_identity_is_never_qualified_again
• redundant_cores_fail_over_at_once
• a_lost_p2p_session_falls_back_to_the_cdn
• a_dead_uplink_consumes_and_publishes_through_a_lan_peer
• an_upstream_request_stops_once_the_route_turns_down_only
• transit_selection_does_not_depend_on_arrival_order

Topology and prior art, on sessions (tests/integration/route_acceptance.rs):

• a_link_flap_costs_routes_not_broadcasts: 2 announce messages per session going down and 1 coming back, the same for 1 broadcast as for 64.
• a_stale_neighbour_cannot_revive_an_ended_broadcast
• a_ring_moves_at_once_to_a_backup_avoiding_the_far_end
• a_ring_waits_out_the_doubt_for_a_backup_through_the_far_end
• two_relays_routing_through_each_other_both_let_go: both let go one link delay after the cut, not at the 10 s hold deadline.
• a_vanished_relay_is_noticed_a_keep_alive_apart
• a_dispute_wheel_settles
• a_primary_that_fails_and_returns_takes_every_relay_back
• pool_ties_agree_on_every_edge
• lite/subscriber.rs::an_unassigned_or_retired_route_id_closes_the_session is the unknown Route ID check. It covers ROUTE_UPDATE, ROUTE_HOLD, ROUTE_END, ANNOUNCE_FLOOR, ANNOUNCE_START and DEAD.

Failover (route_failover.rs):

• next_hop_moves_keep_a_live_reader: four next-hop moves. The reader lost 0 frames and saw 0 retractions, 0 re-announcements and 4 in-place updates. Exactly one announce message was sent per move.
• a_narrower_next_hop_fails_over_to_a_wider_route
• live_mesh_withdraw_retracts_once_without_latency and live_mesh_link_failure_keeps_presence, ported from fix(net): replace the cursor hold with a cluster-link hold-down #4644's scenarios.
• A ROUTE cannot reach the wire ahead of its ANNOUNCE set, because the publisher flushes ROUTE_START with its first claims. The narrower-next-hop test covers the closest real case.

Cluster (route_cluster.rs):

• Two regions with redundant cores:
  • a_far_edge_reads_a_claim_and_sees_its_withdrawal_once
  • losing_a_core_keeps_the_far_edge_reading
  • a_matching_epoch_takes_over_at_the_far_edge
• a_partitioned_lan_reaches_itself_through_the_cdn_and_heals
• the_cdn_never_reaches_itself_through_the_lan
• overlapping_wildcards_resolve_to_the_most_specific_claim_everywhere
• a_preferred_down_only_route_hides_the_exportable_alternative
• an_epochless_replacement_restarts_everywhere
• a_grant_union_crosses_the_scoped_boundary_each_way
• Plan item 15 on sessions:
  • a_client_watching_one_prefix_holds_only_the_origins_meeting_it
  • an_origin_leaving_the_interest_ends_its_route_by_its_floor

Mount continuity (mount.rs):

• claimed_beneath_the_mount_lite06 and _lite07, against the unmounted baselines claimed_unmounted_lite06 and _lite07.

Cost (route/tests/cost.rs, route/node/tests.rs, and the benches routes/path/repath and routes/path/relink):

• a_repath_costs_the_streams_it_reaches_and_nothing_per_claim
• a_link_update_costs_the_routes_through_it_and_nothing_per_claim
• The pinned residual_* tests listed under Follow-ups.

Bugs fixed:

• model/origin.rs: a reader followed only the narrowest lite-07 route to a broadcast.
  • When the session behind that route closed, the reader dropped even though a wider route still covered the broadcast. The broadcast went Offline, started again 2 s later, and the reader lost its subscription.
  • A wider route now ranks after the narrowest one instead of being excluded, so the broadcast updates in place.
• lite/subscriber.rs: a reopened announce request let go of the old request's claims before the new one served them.
  • So widening a lite-07 session's scope to the root (a ** token) showed every claim inside the old scope as Offline, then Start.
  • The old request's served claims now stay until the replacement serves them.
  • Regression test: auth::lite_07_a_scope_widened_to_the_root_reopens_without_letting_go.
• route/path.rs: moving a heard route to a new path copied every claim behind it.
  • The claims now move instead.
  • A DEAD now visits only the neighbours sent a route through the closed hop.
  • work() now counts each stream it visits, so the regressions see that cost.
  • A repath with 10k claims went from 81 µs to 9.2 µs.

The epic README and quest/m3/cluster-routing/federation.md now name the ported tests.

Caused by

• moq-dev/moq#8731: its origin filter admitted only the primary lane, so a reader could not fall back to a wider covering route (found by check).
• moq-dev/moq#8375: the reopen handover dropped the old request at SYNCED. That was safe only while a reopened request filed its claims under the same identity and route prefix, which per-prefix routes from lite-07 sessions run the path router #8731 broke. I did not confirm which of the two introduced it (found by check).

Decisions

The agent took each of these under /quest-afk, for the maintainer's review.

1. Split the four flaws into follow-up quests and pin each with a test, rather than fixing them here.
   • Each one changes the route rule, the node, or both. Each needs a verify-route model check, and three of them touch path.rs or doubt, which moq-dev/moq#8734 is changing right now.
   • Rejected: fixing them in this PR, which would have made it a second route-rule change running in parallel with feat(route): a dual-homed publisher's other links survive doubt about it #8734.
2. Keep the mitigations for now: moq-dev/moq#5375's quarantine, moq-dev/moq#5391's dedup, and the 300 ms hold.
   • The quest said to retire them only once the fixtures show the replacement works. The fixtures instead show the held-claim flaws, and the quarantine is what doubles the gap those flaws leave.
   • Their retirement stays in the rewritten quest, which now depends on the follow-ups.
3. Delete the model's Babel-only fixtures without porting them.
   • They tested feasibility, seqno requests and distance retention, none of which exists under the path rule.
   • Its handoff fixtures were already ported to route/tests/handoff.rs.
4. In the sweep, a request reaches its publisher whatever path it asks for.
   • A route covers the prefix the relay asked for, not just the publisher's claims, and the publisher answers for anything it does not claim.
   • The model's version instead expected a request to stall where the claim was not heard.
5. Keep the epic's export bullet as it was: an upstream-accepted request's subscription ends when its route turns down-only.
   • The router test shows the relay stops forwarding such a request.
   • Whether a subscription already in flight ends on real sessions has no test yet. The bullet was left unchanged rather than weakened.

Follow-ups

• Held after close (M).
  • The flaw: a relay whose own session closed shows its held claims Offline instead of unavailable, and they come back as Start about 2 s later. Three fixtures pin it.
  • Recommended fix: move the held claim, unavailable, onto a stream that still hears the route.
• Doubt holds (M).
  • The flaw: a relay whose only routes are blocked by doubt lets go before the doubt lifts.
  • Measured on a 34-relay mesh across 5 publishers: 98 readers lost their subscription, and the longest gap was 2.0 s.
• Route fan-out (L).
  • The flaw: a route change visits every answering stream, rewrites every broadcast behind the route, and keeps per-session state for sessions nothing was learned on.
  • Measured: one RTT change with 1,000 origins and 1,000 unrelated streams takes 847 ms.
• Doubt and keep-alive (S). It has one open decision.
  • The flaw: a 1 s doubt does not cover a 3 s spread in when neighbours notice a silent death. Relays briefly move onto routes through the dead relay.
  • Open decision, recommended (a): make the doubt last at least the idle timeout. The alternatives are (b) shorter relay keep-alives and (c) accepting the window.

Public API and wire impact

• Wire: none.
• Public API: none.
• fuzz::PathRouteChange is new, behind the fuzz feature, for the benches.
• The origin ranks a wider lite-07 route last instead of excluding it. That changes behaviour (failover where there used to be an end) but not the API.

Validation

• quest gate just check: all 12,803 tests pass, and every check module is clean.
• cargo nextest run -p moq-net --features fuzz,async: 3,983 tests pass.
• cargo clippy -p moq-net --all-targets --features fuzz,async -D warnings: clean.
• A second, independent check:
  • The forwarding sweep runs every request two ways, alone and joined through each relay's own subscription, and checks assert_loop_free at every step after a cut.
  • The new session acceptance tests were rerun 40 times with no failures.
  • Each pinned flaw is reproduced on real sessions, and two of them independently on the router.
• I did not run just test interop --all. Nothing on the wire, in moq-ffi, or in a gateway changed.

Models

Size: L
Implementer: Claude, claude-opus-5-5, unknown effort

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

• Head: eec5830

• Base: origin/main at f8c7df9

• Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

• Classified: medium by default

• Reviewer: Codex, gpt-6.1-sol, xhigh effort

• Session: 01a1dfdd-da5a-7d81-afad-1c7c2f11fb6b, round 1

• Score: 9/10, No actionable regressions were found in the handoff fixes or routing optimizations during source review; builds and tests were not run, as instructed.

• Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

• Findings: none

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 18 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 3f2ca655-dc9d-4645-8196-4a21a2697ac3

Commits

Reviewing files that changed from the base of the PR and between 250a13a and eec5830.

Files selected for processing: 29

• quest/m2/cluster-routes/README.md
• quest/m2/cluster-routes/acceptance.md
• quest/m2/cluster-routes/doubt-holds.md
• quest/m2/cluster-routes/doubt-keepalive.md
• quest/m2/cluster-routes/held-after-close.md
• quest/m2/cluster-routes/route-fanout.md
• quest/m3/cluster-routing/federation.md
• rs/moq-net/benches/routes.rs
• rs/moq-net/src/fuzz.rs
• rs/moq-net/src/lite/subscriber.rs
• rs/moq-net/src/lite/subscriber/routed.rs
• rs/moq-net/src/model/origin.rs
• rs/moq-net/src/route/node/tests.rs
• rs/moq-net/src/route/path.rs
• rs/moq-net/src/route/tests/cost.rs
• rs/moq-net/src/route/tests/forward.rs
• rs/moq-net/src/route/tests/mod.rs
• rs/moq-net/tests/integration/auth.rs
• rs/moq-net/tests/integration/claim_continues.rs
• rs/moq-net/tests/integration/main.rs
• rs/moq-net/tests/integration/mesh_withdraw.rs
• rs/moq-net/tests/integration/mount.rs
• rs/moq-net/tests/integration/path_routes.rs
• rs/moq-net/tests/integration/route_acceptance.rs
• rs/moq-net/tests/integration/route_cluster.rs
• rs/moq-net/tests/integration/route_failover.rs
• rs/moq-net/tests/integration/routing_contracts/handoff.rs
• rs/moq-net/tests/integration/routing_contracts/mod.rs
• rs/moq-net/tests/integration/routing_contracts/model.rs

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[pullfrog]

ℹ️ No critical issues. Two small cleanups are inline.

Reviewed changes

I reviewed the full PR at eec5830. Most of the diff is new acceptance tests plus the deletion of routing_contracts/. The review focused on the four source changes that alter behaviour.

• Wider lite-07 lane as fallback (model/origin.rs): NodeFilter::Any now accepts every route-layer lane, and route_order puts !lane.primary first. A wider covering route serves only while no primary exists. The default Lane.primary is true, so routes from lite-06, IETF and local publishers rank exactly as before.
• Reopen handover (lite/subscriber.rs, routed.rs): at SYNCED, the old request is retired into a Retired holder before the new request's outputs run, and the holder is dropped after them. The node hands the claims over when the old stream drops, and the old origin entries leave only once the new ones are in place. Readers see an update, not Offline then Start.
• path::Router claim moves (route/path.rs): forget_heard returns the Heard it removes, so route and repath move the view instead of cloning it.
• Visiting fewer streams (route/path.rs): report visits only the peers indexed under the dead hop. Dropping the through.is_empty() guard is safe because each peer's pass touches only its own tx_hops entries. advertise borrows answering instead of cloning every prefix set, and it and run now count a step() for each stream.
• New tests and benches: forwarding, cost and residual fixtures on the router; session fixtures for topology, failover, cluster and mount; the routes/path/repath and routes/path/relink benches on fuzz::PathRouteChange. The pinned and residual_* tests capture known flaws on purpose, and each has a follow-up quest.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}

[pullfrog]

The route_order doc comment above (lines 860–870) still says the order is "Lower wins: first a route that is neither draining nor unavailable". Primary-lane membership is now the first key, so a wider lite-07 lane ranks below even a draining or unavailable route. Please add a sentence there that names this new first key.

[pullfrog]

This PR makes path_routes' DOUBT, HOLD and lite07() pub(super) so route_acceptance.rs can share them. route_failover.rs (lines 27, 42 and 45) and route_cluster.rs (lines 26, 29 and 34) still define their own copies. These constants mirror the router's timers, so changing the hold or doubt would mean editing three files that can drift apart. Importing them from crate::path_routes would keep one source.