Skip to content

fix: handle monitor event data in normalization - #887

Open
jokasimr wants to merge 8 commits into
mainfrom
832
Open

jokasimr wants to merge 8 commits into
mainfrom
832

Conversation

@jokasimr

Copy link
Copy Markdown
Contributor

Fixes #832

@jokasimr
jokasimr requested a review from jl-wynen November 17, 2026 14:55

@SimonHeybrock SimonHeybrock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Can you explain why this fix is instrument specific? Does it work already for the generic workflow, i.e., is it due to a provider this workflow overrides?
  • Is NormalizationMonitorWavelengthBins to specific? Should there be a single WavelengthBins for all wavelength bins in the workflow? Looking at SANS for example, we want to use the same for monitor and other terms, as they all get combined.

@jokasimr

Copy link
Copy Markdown
Contributor Author
  1. I didn't consider making it generic. Do you have a specific idea in mind for what that should look like?
  2. Having one wavelength binning for all wavelength histograms in a workflow seems a bit rigid, especially if the rebinning is applied automatically, seems there would have to be some mechanism to override it in specific cases.

# We should make this function public or move the normalization to ESSreduce.
from ess.reduce.normalization import _mask_detector_for_norm

if monitor_wavelength_bins is not None:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use .is_binned because .bins is None is supposed to get deprecated.

@jl-wynen

Copy link
Copy Markdown
Member
  1. I think it should be generic. Any workflow might encounter this issue.
  2. You could use a generic type WavelengthBins[Component]. But why do you need it to begin with? Can't the normalsation bin with the detector's edges?

@jokasimr

Copy link
Copy Markdown
Contributor Author

You could use a generic type WavelengthBins[Component]. But why do you need it to begin with? Can't the normalsation bin with the detector's edges?

But the detectors being normalized doesn't typically have wavelength bins? I mean, the monitor normalization is applied using lookup.

@jl-wynen

Copy link
Copy Markdown
Member

You could use a generic type WavelengthBins[Component]. But why do you need it to begin with? Can't the normalsation bin with the detector's edges?

But the detectors being normalized doesn't typically have wavelength bins? I mean, the monitor normalization is applied using lookup.

Right. Then I vote for a generic type like in your latest commit.

Comment on lines +118 to +119
class WavelengthBins(sl.Scope[MonitorType, sc.Variable], sc.Variable):
"""Optional wavelength bin edges for a monitor, shared across run types.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you checked if this is interoperable with the WavelengthBins used in ess.sans? We should not have two.

@jokasimr jokasimr Nov 18, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not interoperable, as the failing CI showed. I've renamed the generic bins type to MonitorWavelengthBins. It's a bit of a mouthful, but I weren't able to find a good way to re-use the generic WavelengthBins in esssans without it feeling like a hack.

One option would be to rename the sans local type to NormalizationWavelengthBins or similar, that would be more precise anyway, and then we could keep the less precise WavelengthBins name for the generic type. But it would be a breaking change in the esssans package.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I'd think the same binning should be used for both, right? If the workflow support using two different binnings then things may go awry.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why, can you clarify?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because otherwise the monitor would need rebinning, which creates all kinds of weird effects, so unless there is a reason for it, we'd rather avoid it.

@jokasimr
jokasimr changed the base branch from main to ci-reduce-memory-usage November 18, 2026 15:27
"""Monitor data with wavelength coordinate."""


class MonitorWavelengthBins(sl.Scope[MonitorType, sc.Variable], sc.Variable):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could also use

Suggested change
class MonitorWavelengthBins(sl.Scope[MonitorType, sc.Variable], sc.Variable):
class WavelengthBins(sl.Scope[Component, sc.Variable], sc.Variable):

That would allow any monitor type and detector.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WavelengthBins collides with a type already defined in esssans, but we can use Component instead of MonitorType

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather change the more specialised type to use a longer name as that has a smaller impact than the generic name.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically I agree about that. But that would be a breaking change.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above, I don't think we should have different wavelength binnings in a workflow, unless there is a clear motivation for it.

Base automatically changed from ci-reduce-memory-usage to main November 19, 2026 08:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ESSSPECTROSCOPY] BifrostWorkflow fails on event-mode normalization monitor

3 participants