Repository navigation
fix(harness): state the counts a plan's fetch reads - #81
Merged
Merged
Conversation
…s plan "What share of ODA to Kenya came from the United States?" divides by official donors (DAC) in some runs and by all providers in others, and neither the card nor the approved plan said which. make_charts now writes the whole into a share card's title from the denominator's selected source, as it writes the period, and the plan's fetch label and share sentence name the group or member count that source holds in place of "every donor".
jm-rivera
marked this pull request as draft
October 25, 2026 14:58
A denominator filtered by value or cut to its top members before its total still read "of the total from all 140 donors"; a per-period rank writing positions carried the share's whole; and the whole a numerator also holds was matched by wording. share_wholes now names no whole past a narrowing step, stops at a step whose output is not a ratio, and compares populations by group ref or axis and member count. The plan's share sentence names the narrowing step in place of the parties.
A plan said "Uganda and all 108 donors" and "summed over its 288 indicators" while the answer said 92 providers with data in 2022 and 13 indicators, with nothing to tie the two. The plan now says "the 108 donors the source holds in any year" and "summed over whichever of its 288 indicators hold data", a share card's whole names an open axis with no count, and the orchestrator instruction quotes the new phrases.
…guess
The plan's donor count is already the providers with data in the plan's
years (and, with dfo-apis counting parties only, the donors the frame
holds), so "the N donors the source holds in any year" was false; it
reads "all N donors" again. The indicator clause states Search's exact
count of indicators with data where every entity axis is named
("summed over its 13 indicators"), and the family size otherwise. A
flow side's count comes from role_axis_facts over the scope list's
length, and the orchestrator instruction says what each count counts.
jm-rivera
force-pushed
the
fix/plan-counts-say-what-they-count
branch
from
October 25, 2026 16:22
e2e8bad to
1ba614b
Compare
jm-rivera
changed the title
jm-rivera
marked this pull request as ready for review
October 25, 2026 16:35
Two open populations are the same only when their sources are one dataset, since sources record counts and no members; a different dataset of as many donors keeps its clause. A denominator reduced by a mean, maximum or minimum names no total, and neither does one narrowed to one entity. A card title keeps its whole only where the whole subject fits beside it in 120 characters; otherwise the whole goes in the card's note.
… fix/plan-counts-say-what-they-count # Conflicts: # services/harness/src/harness/plan_display.py # services/harness/tests/test_plan_display.py
…at-they-count # Conflicts: # services/harness/src/harness/plan_display.py # services/harness/tests/test_graph.py # services/harness/tests/test_orchestrator_tools.py # services/harness/tests/test_plan_display.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #79; merge #79 first. The donor counts this PR states are true once DevelopmentFinanceObservatory/apis#151 is deployed on the same environment, so release apis#151 first.
The plan and the answer for "What share of ODA to Uganda came from each donor in 2022?" gave two donor counts with nothing to tie them: "Fetch gross disbursements (official development assistance) for Uganda and all 108 donors" against "92" on the card and in the answer. The 108 is already limited to 2022; 16 of the 108 are group totals, such as DAC countries and the Group of Seven, which every flow retrieval leaves out. apis#151 makes an open role's count leave them out, so the plan reads "all 92 donors". The US health share of Kenya had the same kind of gap with indicators: the plan said "summed over its 288 indicators", the size of the measure's family (
served_count), while the answer said 13, the indicators with a value for the US to Kenya in 2022 (stat_vars.member_count).The plan now states the counts its fetch reads.
compiler/sources.pyopen_entity_axes_oftakes an origin or destination axis's count fromrole_axis_facts, the parties-only count, before the length of a discovered scope's member list, which lists the totals; a place axis keeps the scope's count, and a flow side whose fact states no count keeps the scope's. The fetch label and rank sentence keep dev's "all 92 donors" and "ranks all 55 donors the source holds" wording, and a count below two is left unstated, so a one-member word reads "every donor", not "all 1 donors".plan_display._reading_clausestates Search's exact windowed count of indicators with data when the source leaves no entity axis open: "summed over its 13 indicators". With an axis open, the count Search confirms at its probe entities can differ from what the fetch retrieves, so the clause names the family fromserved_count, "summed over whichever of its 288 indicators hold data", and names it without a count when Search states no family size above one.stat_var_countstill prefersserved_countfor deciding whether a source is one series or a family, since Search states that count only once membership is settled.A share card's title states a period, so
_populationsnames an open axis with no count: the Kenya card from #79 reads "US share of aid to Kenya, of the total from every donor, 2023 to 2025", and a group the source binds keeps its label, "of the total from official donors (DAC)". Populations are still compared by #79's identity (an axis with its member count), so a 120-donor numerator never matches a 140-donor whole. The orchestrator instruction inorchestrator/agent.pytells the model what each count counts, and now ties "all N donors" and "summed over its N indicators" to counts with data in the plan's years and "whichever of its N indicators hold data" to the family. The_open_axis_issuesdocstring incompiler/evidence.pysays the same.tests/context_baseline.jsonis unchanged.Testing: pinned plan texts across
test_plan_display.py,compiler/test_family_reading.py,compiler/test_sources.py,planner/test_phrasing.py,test_turn_records.pyandtest_studio_orchestrator_eval.pycover 13 indicators with data in a family of 288, the same family with donors open, a place-open source with no family size, flow and place axes' counts, and a one-member count. The changed tests fail without the source change. The full harness suite passes (7194 passed, 1 skipped, 1 xfailed). ruff and ty are clean.