[RemyDegenne]

WIP: I tried to ask Claude to identify an important definition from LMLExplore that could be upstreamed, to prepare material for a PR and to use a review skill I have to improve it. This is the result. I'll review it before marking this PR as ready (or closing it).

[RemyDegenne]

A design issue: the current MDP definitions imposes independence of the reward r and the next state s' given the state-action pair (s,a). Some texts allow the reward to depend on (s, a, s'), which is not compatible with our definition. If we want to include this, we need a single kernel which describes the product (r, s') (and hypotheses on the spaces to be able to disintegrate it) or we need to replace the reward kernel by something that also looks at the next state. Possible complication: in the trajectory described in LML the reward comes before the next state.