You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Compute score bounds based on grading conf and warn about discrepancies #439
Check that no test case gets score.txt and a verdict other than accepted
Warn if some test data node has infinite max, but we can prove from children's ranges with aggregation that a finite bound can be found
Warn if any test data node has negative range lower bound. In particular, this will give a warning for every node without an explicitly set range, which I think is good?
Check that score > max_score never happens at any node in the tree
What check do you think is missing? There is already a check on max (and min) score for every group (which is, AFAIK, the only level which has max scores in legacy).
Warn if some test data node has infinite max, but we can prove from children's ranges with aggregation that a finite bound can be found
Why should this be a warning? It's a reasonable thing to compute and make use of (i.e., if a problem has no limit on the root group, but obviously can't go above 100 due to childrens' max scores and aggregation, we would treat it as having 100 on the root when installing), but why do we want to force a problem setter to manually write files? The default is also +/-inf, so this can get very trigger happy.
I'd be more inclined to give a warning if there is a finite bound we can prove from aggregation rules cannot be reached.
Warn if any test data node has negative range lower bound. In particular, this will give a warning for every node without an explicitly set range, which I think is good?
In the new "Kattis" mode, error on objective: min
If we want to drop minimization and negative scores, why don't we just update the spec? Do we have any reason to believe there are problems out there which people care about that use these? I particularly don't want to add a warning which would be triggered by the default value for a field.
Why should this be a warning? It's a reasonable thing to compute and make use of (i.e., if a problem has no limit on the root group, but obviously can't go above 100 due to childrens' max scores and aggregation, we would treat it as having 100 on the root when installing), but why do we want to force a problem setter to manually write files? The default is also +/-inf, so this can get very trigger happy.
I hold a perhaps extreme position on this. Every single time I've seen a problem where range was omitted, it was unintentional and buggy. In the edge-case where you do want infinite max_score, it's trivial to simply specify this explicitly in the group.
IMO, one should therefore always specify the range in every node. testdata_tools already does this, and given the ambiguities in the spec (that I tigtened maybe 6 months ago), any sane generator ought to do the same. If you are hand-editing testdata.yaml files, it is very easy to get it wrong, so I prefer forcing authors to spend 2 minutes deciding this explicitly.
If we want to drop minimization and negative scores, why don't we just update the spec? Do we have any reason to believe there are problems out there which people care about that use these? I particularly don't want to add a warning which would be triggered by the default value for a field.
There are mixed opinions on backporting. Perhaps I should make an attempt. But Kattis has in fact killed off support for minimization and negative scores IIRC, and I would therefore be very happy to explicitly forbid it, so authors discover it using problemtools and not during a running contest.
Check that no test case gets score.txt and a verdict other than accepted
Lol, this actually gets a bit annoying with the mulitpass test problem I built. I was accumulating score between passes in score.txt. Now I suddenly need to rm that file on a WA. I guess the recommended approach would be to accumulate score in a different file and rename to score.txt on the final verdict (if AC).
There are mixed opinions on backporting. Perhaps I should make an attempt. But Kattis has in fact killed off support for minimization and negative scores IIRC, and I would therefore be very happy to explicitly forbid it, so authors discover it using problemtools and not during a running contest.
Well, "legacy" is the current version. I strongly dislike that we are in a position where making changes to the current version of the spec is painful (cf. Kattis/problem-package-format#585)
I added a check for score.txt in #441. Good suggestion, thanks!
IMO, one should therefore always specify the range in every node.
I don't want to require that. Your feature request points in a good direction for computing bounds based on given max scores and aggregation rules. I'll think a bit about it a bit more, but my initial take is that I want to:
Propagate given score constraints based on aggregation (for non-custom graders)
Warn if an unreachable bound has been set
Warn if the root group (after propagation) has unbounded score, and is not explicitly setting range
I think that solves several issues while still being user friendly for problem setters who do not use tools.
Regarding minimization aspects of this issue, I feel like those are sort of wontfix until we've tried dropping min from the spec. If not, they are duplicate of your earlier #345.
Activity
gkreitz commentedon Jul 15, 2026
What check do you think is missing? There is already a check on max (and min) score for every group (which is, AFAIK, the only level which has max scores in legacy).
Matistjati commentedon Jul 15, 2026
I had 3 bullet points and deleted the wrong one, oops. Updated
gkreitz commentedon Jul 15, 2026
Why should this be a warning? It's a reasonable thing to compute and make use of (i.e., if a problem has no limit on the root group, but obviously can't go above 100 due to childrens' max scores and aggregation, we would treat it as having 100 on the root when installing), but why do we want to force a problem setter to manually write files? The default is also +/-inf, so this can get very trigger happy.
I'd be more inclined to give a warning if there is a finite bound we can prove from aggregation rules cannot be reached.
If we want to drop minimization and negative scores, why don't we just update the spec? Do we have any reason to believe there are problems out there which people care about that use these? I particularly don't want to add a warning which would be triggered by the default value for a field.
Matistjati commentedon Jul 15, 2026
I hold a perhaps extreme position on this. Every single time I've seen a problem where range was omitted, it was unintentional and buggy. In the edge-case where you do want infinite max_score, it's trivial to simply specify this explicitly in the group.
IMO, one should therefore always specify the range in every node.
testdata_toolsalready does this, and given the ambiguities in the spec (that I tigtened maybe 6 months ago), any sane generator ought to do the same. If you are hand-editing testdata.yaml files, it is very easy to get it wrong, so I prefer forcing authors to spend 2 minutes deciding this explicitly.There are mixed opinions on backporting. Perhaps I should make an attempt. But Kattis has in fact killed off support for minimization and negative scores IIRC, and I would therefore be very happy to explicitly forbid it, so authors discover it using problemtools and not during a running contest.
gkreitz commentedon Jul 15, 2026
Lol, this actually gets a bit annoying with the mulitpass test problem I built. I was accumulating score between passes in
score.txt. Now I suddenly need tormthat file on a WA. I guess the recommended approach would be to accumulate score in a different file and rename to score.txt on the final verdict (if AC).Matistjati commentedon Jul 15, 2026
Well, score.txt and non-AC verdict is broken in Kattis as far as I'm aware 🙂
gkreitz commentedon Jul 15, 2026
Well, "legacy" is the current version. I strongly dislike that we are in a position where making changes to the current version of the spec is painful (cf. Kattis/problem-package-format#585)
gkreitz commentedon Jul 15, 2026
I'll be happy to fix that once all installed problems behave properly. :) Adding a fix that causes Judge Errors feels bad.
gkreitz commentedon Jul 15, 2026
I added a check for score.txt in #441. Good suggestion, thanks!
I don't want to require that. Your feature request points in a good direction for computing bounds based on given max scores and aggregation rules. I'll think a bit about it a bit more, but my initial take is that I want to:
I think that solves several issues while still being user friendly for problem setters who do not use tools.
Regarding minimization aspects of this issue, I feel like those are sort of wontfix until we've tried dropping min from the spec. If not, they are duplicate of your earlier #345.
[-]Sanity checks[/-][+]Compute score bounds based on grading conf and warn about discrepancies[/+]gkreitz commentedon Jul 15, 2026
I'll take the liberty to edit your issue description to something more descriptive for what remains.