Methods causing cascade of invalidations

[MilesCranmer]

It seems like the following methods are causing a large fraction of SymbolicRegression.jl's precompiled methods to get invalidated, when loading it inside PySR:

Base.:(!=)(x::PythonCall.Py, y::PythonCall.Py)
Base.:(>>)(x::PythonCall.Py, y::Number)

I think != is not actually needed, Julia uses existing definitions for it.

The second one might simply be overly broad, I am not sure.

Another major one is from JSON.jl which I have since reported there.

[cjdoris]

I think #702 and #709 fix this, but they are into the v1 branch because it's a breaking change.

[MilesCranmer]

Quick update, it seems like JSON.jl just released a patch for some of these: https://github.com/JuliaIO/JSON.jl/releases/tag/v1.10.0

Regarding the other ones that live here, maybe there is a non-breaking way to get rid of the invalidations? Usually it is because an extra method was defined that wasn't needed. And therefore can be excised without any breakage. But Sorry if you are already very familiar with this issue!

[MilesCranmer]

Away from laptop; @MilesCranmerBot can you investigate the above?

[MilesCranmerBot]

I measured this with SnoopCompile on Julia 1.12.7, loading using PythonCall; using SymbolicRegression inside @snoop_invalidations (SymbolicRegression v2.4.2, PythonCall 0.9.36). The metric is the number of unique method instances in invalidation trees whose triggering method is defined in PythonCall, so JSON.jl's share is excluded.

PythonCall	invalidated method instances	of which in SymbolicRegression.*
0.9.36 (current release)	6830	1595
0.9.36 with the > and >= methods deleted	2532	881
v1 branch (#702 + #709)	883	227

Which methods cause it. The biggest trees in 0.9.36 come from the mixed Py/Number comparisons, with roots like >=(::Any, ::Int64), >(::Int64, ::Any) and !=(::Any, ::Int64):

2675  >=(x::Py, y::Number)
2421  >(x::Number, y::Py)
1418  !=(x::Py, y::Number)
 681  >(x::Py, y::Number)
 289  &(x::Number, y::Py)
 123  >(x::Py, y::Py)
 108  !=(x::Py, y::Py)
  97  !=(x::Number, y::Py)
  77  >=(x::Py, y::Py)
  70  >>(x::Py, y::Number)

(These per-method counts overlap, so they add up to more than the unique total.) < and <= do not show up: Base already has many methods for them, so inference never assumed a concrete return type for calls like x < 1 with x::Any. Base defines >, >= and != only through generic fallbacks, so any a > 1 with an abstractly typed a got compiled against that one method, and adding a Py method breaks it.

!= cannot be removed on 0.9. The Base fallback is !=(x, y) = !(x == y), and on 0.9 x == y returns a Py. There is no !(::Py) method, so Py(1) != Py(2) would throw a MethodError. Adding !(::Py) would be worse: ! has far more call sites than !=, and not (a == b) is not the same as a != b in Python (NumPy arrays, for example). On v1, #702 makes != return Bool, and it still invalidates (788), so != remains the main cost there.

> and >= could be removed on 0.9. Base's fallbacks are >(x, y) = y < x and >=(x, y) = y <= x, which route to PythonCall's own < and <= methods. The return type stays Py, and results are identical for ordinary objects; I checked Py(3) > 2, 2 > Py(3), Py(3) >= 3, 4 >= Py(3), Py(3) > Py(1), Py(1) >= Py(3), and Py(3) > 2.5. That removes about 63% of the invalidations (6830 to 2532). One behavior change: x > y would evaluate as Python y < x, so Python tries y.__lt__ before x.__gt__. This matters only for types whose __gt__ and __lt__ disagree, which is arguably a bug in those types, but it is a change, so it is your call whether it fits a 0.9 patch release. The diff:

 Base.:(<)(x::Py, y::Py) = pylt(x, y)
-Base.:(>=)(x::Py, y::Py) = pyge(x, y)
-Base.:(>)(x::Py, y::Py) = pygt(x, y)
@@
 Base.:(<)(x::Py, y::Number) = pylt(x, y)
-Base.:(>=)(x::Py, y::Number) = pyge(x, y)
-Base.:(>)(x::Py, y::Number) = pygt(x, y)
@@
 Base.:(<)(x::Number, y::Py) = pylt(x, y)
-Base.:(>=)(x::Number, y::Py) = pyge(x, y)
-Base.:(>)(x::Number, y::Py) = pygt(x, y)

The remaining mixed-arithmetic methods (&, >>, <<, xor with Number) cannot be removed without breaking changes, and #709 already handles them on v1.

Summary.

• The main fix is releasing v1, which cuts these invalidations by about 87%.
• Before that, removing the > and >= methods would cut them by about 63%. The cost is the reflected-operator ordering change described above.
• != has to stay on 0.9.

[MilesCranmer]

@MilesCranmerBot ok sounds reasonable, can you please take that diff and create a PR against master?

[JamesWrigley]

Prior PR: #698
Which ended up being reverted for correctness reasons. This really needs to be fixed in Base, I made an attempt at it here but it needs more work: JuliaLang/julia#62658

[MilesCranmer]

Prior PR: #698

Thanks, I didn't see.

Which ended up being reverted for correctness reasons.

Can you say more? Is it violating some interface on the Julia side or the Python side?

If on the Python side (maybe __gt__ must be checked first or something, I don't know) then could we just handle it internally so that the Python contract is followed?