Repository navigation
feat: FTRL on the unit ball with the potential -log(1 - ‖a‖) - ‖a‖ [LS, Section 28.4] - #153
Merged
lmlexplore-automation[bot] merged 4 commits intoNov 5, 2026
Conversation
…S, Section 28.4] Define the potential of [LS, Section 28.4], prove its derivative on the open unit ball, the lower bound D_F(b, a) ≥ ‖b - a‖² / (2 (1 - ‖a‖)) on its Bregman divergence, and the full-information FTRL regret bound -log(1 - r) / η + η / 2 ∑ (1 - ‖a t‖) ‖y t‖² used in the proof of [LS, Theorem 28.11]. Co-Authored-By: Claude Code <noreply@github.com>
RemyDegenne
commented
Nov 5, 2026
RemyDegenne
commented
Nov 5, 2026
RemyDegenne
commented
Nov 5, 2026
RemyDegenne
commented
Nov 5, 2026
Contributor
Author
AI review — approvedEach rubric is judged independently by multiple review agents; the PR merges only once every rubric is green — any rubric that is not green (changes requested, blocked, errored, stale, or not yet run) blocks the merge. See the rubrics.
♻️ = approved on an earlier commit, re-run before merge. Reviewing this diff at head |
…grand bound Factor the square completion of a stability term into `apply_sub_sub_div_le_of_mul_norm_sq_le` and its summed form with a per-round modulus, `sum_apply_sub_sub_bregDiv_le_of_mul_norm_sq_le`, and derive both the strongly convex case and the unit-ball case from them. Expose the bound on the right-hand side of `ftrl_regret_le` for the unit-ball potential, and rename the regret theorem to `ftrl_regret_le_of_unitBallPotential`. Co-Authored-By: Claude Code <noreply@github.com>
The sum theorem now sums apply_sub_sub_bregDiv_unitBallPotential_div_le instead of re-deriving it, and the per-round modulus conversion uses field_simp instead of a named-lemma rewrite chain. Co-Authored-By: Claude Code <noreply@github.com>
RemyDegenne
commented
Nov 5, 2026
…emma Co-Authored-By: Claude Code <noreply@github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the full-information part of [LS, Theorem 28.11] (adversarial linear bandits on the unit ball): the potential
unitBallPotential a = -log (1 - ‖a‖) - ‖a‖of [LS, Section 28.4] on a real inner product space, its derivative(1 - ‖a‖)⁻¹ • innerSL ℝ aon the open unit ball (including at0), its nonnegativity and the diameter bound-log (1 - r)on the closed ball of radiusr < 1, and the regret bound for FTRL with this potential on a convex subset of that ball:∑ t < n, y t (a t - u) ≤ -log (1 - r) / η + η / 2 * ∑ t < n, (1 - ‖a t‖) * ‖y t‖ ^ 2(ftrl_regret_le_unitBallPotential), plus existence of the plays in a proper space (the "well defined" clause of Theorem 28.11).The key step is
norm_sub_sq_div_le_bregDiv_unitBallPotential:‖b - a‖ ^ 2 / (2 (1 - ‖a‖)) ≤ D_F(b, a)for alla,bin the open ball. It follows from splitting the divergence into the divergence of the radial profileφ s = -log (1 - s) - splus(‖b - a‖² - (‖b‖ - ‖a‖)²) / (2 (1 - ‖a‖)), and a one-variable monotonicity argument forφ. This replaces the book's route (Hessian at a pointZ_tof the segment, Theorem 26.13, and the stability estimate(1 - ‖Z_t‖) ≤ 2 (1 - ‖Ā_t‖)derived from the closed form (28.13)). So the bound holds for all losses, with no conditionη ‖Ŷ_t‖ ≤ 1/2, and it saves a factor2in the stability term. The losses are continuous linear functionals with their operator norm, as inftrl_regret_le_of_strongConvexOn, so the bound plugs directly intoftrl_integral_regret_lefromBandits/StochasticMirrorDescent.lean.Milestone: Bandits Layer 10.2, [LS, Theorem 28.11] (unit ball,
‖y_t‖ ≤ 1,r = 1 - 2ηd,η = √(log n / (3dn)),R_n ≤ 2√(3nd log n)). Layer 10.2's first half, Theorem 28.10, landed in #147. What remains after this PR: define the sampling rule (exploration with probability1 - ‖Ā_t‖alongU_tuniform on{±e_i}) and the estimator (28.12), prove that they make each round anIsUnbiasedRoundwithE[(1 - ‖Ā_t‖) ‖Ŷ_t‖²] ≤ d ‖y_t‖², and combine them with this bound throughftrl_integral_regret_leand the tuning ofrandη.New file:
LMLExplore/OnlineLearning/UnitBallPotential.lean(289 lines). Nothing is vendored. The development follows [LS, Section 28.4], which the file cites.Roadmap: Bandits
🤖 Prepared with Claude Code