Skip to content

pyop3: Support MIN/MAX reductions in kernel calls and assignments - #5543

Open
pbrubeck wants to merge 4 commits into
mainfrom
pbrubeck/fix-minmax-write
Open

pbrubeck wants to merge 4 commits into
mainfrom
pbrubeck/fix-minmax-write

Conversation

@pbrubeck

Copy link
Copy Markdown
Contributor

Fixes #5524.

Summary

Kernel calls (pyop3/insn/visitors.py, expand_transforms for CalledFunction):

 MAX_RW:     pack  t = x               unpack  x = max(x, t)
 MIN_RW:     pack  t = x               unpack  x = min(x, t)
+MAX_WRITE:  pack  t = dtype min       unpack  x = max(x, t)
+MIN_WRITE:  pack  t = dtype max       unpack  x = min(x, t)

The temporary starts from the identity of the reduction, so the entries that the kernel does not write leave x unchanged. exec.py uses the same identities on the leaves.

Assignments (pyop3/insn/base.py, AbstractAssignment):

 argument_intents
-  WRITE -> WRITE, else assert INC -> INC
+  assignment_type_as_intent(assignment_type)   # MAX -> MAX_WRITE, MIN -> MIN_WRITE
 __str__
+  MAX -> "max=", MIN -> "min="

Evidence

  • Before:

    assemble(interpolate(x + y, V, access=op3.MIN_WRITE))
      NotImplementedError: Intent Intent.MIN_WRITE not handled
    op3.loop(c := mesh.iter("cell"), f.dat[mesh.closure(c)].assign(1.0, "max"), eager=True)
      AssertionError  (argument_intents)
    

    After: both reproducers from pyop3: MIN_WRITE/MAX_WRITE kernel calls and max/min assignments in loops fail #5524 give the correct values on 1 and 3 ranks.

  • New tests, all fail without the fix (except the existing MAX case) and pass with it:

    • tests/pyop3/insn/test_loop.py::test_reduction_assignment[max/min]
    • tests/firedrake/regression/test_interpolate.py::test_interpolate_periodic_coords_reduction[MAX/MAX_WRITE/MIN_WRITE], which replaces test_interpolate_periodic_coords_max and runs on nprocs=1 and nprocs=3. With MIN_WRITE the nodes on the periodic seam take the value 0, with MAX/MAX_WRITE they take 1.

Merge Danger

Door: two-way

Blast Radius: small

Only the MIN_WRITE/MAX_WRITE intents and the "min"/"max" assignments change; these paths raised an error before.


AI assistance: this PR was written with Claude Code (Claude Opus 5.5).

🤖 Generated with Claude Code

@pbrubeck pbrubeck added LLM used An LLM was used in the production of this PR pyop3 labels Oct 22, 2026
Comment thread pyop3/insn/visitors.py Outdated
Comment thread pyop3/insn/visitors.py Outdated
Comment thread tests/pyop3/insn/test_loop.py Outdated
m = 10
axis = op3.Axis(m)
dat0 = op3.Dat(axis, data=np.arange(m, dtype=op3.ScalarType))
dat1 = op3.Dat(axis, data=np.full(m, (m - 1) / 2, dtype=op3.ScalarType))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems weird to use (m - 1) / 2. It is confusing to read because it makes me think that there's something special about that value.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed to an explicit 4.5

@pytest.mark.skipcomplex # complex numbers are not orderable
def test_interpolate_periodic_coords_max():
@pytest.mark.parallel([1, 3])
@pytest.mark.parametrize("access,seam", [(MAX, 1), (op3.MAX_WRITE, 1), (op3.MIN_WRITE, 0)])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MIN and MAX_RW and MIN_RW?

Definitely should check MIN but not sure about the others. I had to introduce RW and WRITE variants in pyop3 but the distinction isn't important at the Firedrake level.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MIN, MAX are just the same as MIN_RW, MAX_RW

pbrubeck added a commit that referenced this pull request Oct 22, 2026
Stack the feature on #5543, which supports MIN and MAX reductions in pyop3.

firedrake/interpolation.py keeps the feature's version. #5543 changes
SameMeshInterpolator._get_tensor to accept MIN_RW and MAX_RW. The feature
removes _get_tensor, and its Interpolator.__init__ already converts MIN_RW
and MAX_RW to MIN_WRITE and MAX_WRITE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pbrubeck
pbrubeck added this pull request to stack #5550 October 22, 2026 13:57
pbrubeck and others added 3 commits October 23, 2026 12:08
Kernel calls with MIN_WRITE or MAX_WRITE access now pack a temporary
that starts from the identity of the reduction, and unpack it with a
"min" or "max" assignment. Assignments of type "min" and "max" now get
their intents from assignment_type_as_intent, so they can be put in
a loop.

Fixes #5524

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TSFC kernels add their result to the output argument. A MIN_WRITE or
MAX_WRITE temporary that starts from the identity of the reduction
therefore stays at the identity. op3.Function now takes
requires_zeroed_output_arguments, like the PyOP2 Kernel did. When it is
set, these temporaries start from zero. as_pyop3_local_kernel sets it
for TSFC kernels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pbrubeck
pbrubeck force-pushed the pbrubeck/fix-minmax-write branch from 4850bd3 to 455a844 Compare October 23, 2026 11:08
@pbrubeck
pbrubeck requested a review from connorjward October 23, 2026 12:11
Comment thread pyop3/insn/base.py Outdated
The library directories that are needed to link the kernel.
libs
The libraries that are needed to link the kernel.
requires_zeroed_output_arguments

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. This is exactly the difference between MAX_RW and MAX_WRITE.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate please?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entire difference between these operations is what we pack on the way into the local kernel. Is it the actual data or is it something null.

t <- -inf
kernel(t)
dat <- max(dat, t)

vs

t <- dat
kernel(t)
dat <- max(dat, t)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The standard Interpolate(expr, V, access=WRITE) generates a kernel whose output tensor should be zeroed at the start. For each degree of freedom we compute a weighted sum of the expr at the element's quadrature points.

tensor[i] = 0
for j in range(dual_mat.shape[1]):
    tensor[i] += dual_mat[i, j] * expr(pts[j])

For Interpolate(expr, V, access=MAX) one would like to set -inf on the dat first, which means that we are no longer able to accumulate on the output. Instead we need to call the WRITE kernel on a temporary, on which we still acumulate the weighted sum.

tensor[i] = -inf

def kernel(temp)
    temp[i] = 0
    for j in range(dual_mat.shape[1]):
        temp[i] += dual_mat[i, j] * expr(pts[j])
    
 tensor[i] = max(tensor[i], temp[i])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just played around with this. If you delete requires_zeroed_output_arguments then the interpolate tests still pass

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flag is here for #5258, it is not needed for the bespoke Interpolate codegen, only for the unified TSFC approach.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think #5258 should use generate_loopy(..., return_increments=(acces == INC))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 276218a

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

LLM used An LLM was used in the production of this PR pyop3

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pyop3: MIN_WRITE/MAX_WRITE kernel calls and max/min assignments in loops fail

2 participants