Skip to content

fix(translator): keep a pin a stated source names, and record its identity - #82

Merged
jm-rivera merged 3 commits into
devfrom
fix/user-pin-by-dataset
Oct 26, 2026
Merged

jm-rivera merged 3 commits into
devfrom
fix/user-pin-by-dataset

Conversation

@jm-rivera

Copy link
Copy Markdown
Contributor

On dev, "What share of ODA to Kenya came from the United States each year since 2015?" sometimes asked the user the same single-option question about an "updated or revised dataset definition" twice and reworked its draft four times. Two of seven recorded turns in Langfuse (environment dfo-dev-ai) repeat it. The planner writes both questions through adk_request_input, prompted by user_decision_dropped and user_decision_contradicted refusals, and the four reworks are those two refusals, an invalid_plan (rolling_needs_open_period, no_source_on_record) and a second user_decision_contradicted.

Two faults start it. First, compiler.evidence._tokens split Search's provenance label "OECD DAC2A" into dac2, a, while a requirement stating "OECD DAC2a" gives dac2a. source_requirement_matches then failed, so _pin_source (translator/discovery.py) treated the orchestrator's pin as an unconfirmed source and searched again, and the pin came back "not found". A turn that pinned the same series with no stated source translated with no search. _CAMEL_BOUNDARY now splits a digit from a capital only where the capital starts a word: "OECD DAC2A" and "OECD DAC2a" both give oecd, dac2a, while "Covid19Deaths", "UNICEFData" and "WorldDevelopmentIndicators" tokenize as before. Over every provenance label and ref in evals/, services/ and packages/, tokens change only for the DAC2A and DAC2B names. The matches gained are all DAC2a against DAC2A and DAC2b against DAC2B. The matches lost are a bare "DAC2", "DAC2 A", a single "A" and "G20" against "G20GDP", none of which occurs in the recordings.

Second, DiscoveryLog.recorded_alternative answered the identities of a logged source's sibling measures, never of the source itself. The candidate a pinned_source_not_found referral offered therefore had no identity on record, so after the user accepted it identity_on_record refused its open period, the model closed the period, discovery minted another spec_id, and the user's decision read as contradicted, which asked the question again. The method is now recorded_definition and also answers each logged source's own verified identity. An identity must hash to the spec_id it is looked up by, so it cannot answer a stale one. admit passes it to identity_on_record, and _recorded_source and _first_year_identity read it, so a pin held only in the log gets its first-year read. Over 2015 to 2024, windowed_identity copies the accepted candidate to exactly the spec_id discovery had re-minted, so _window_pins reaches it with no search and the user's decision stays on the accepted id.

A pin of a source only the translator found now moves to its copy over the requirement's years, the rule a pin of a searched source already followed. The user-decision guard is unchanged, so a different dataset is still refused as a contradiction.

admit gains a one-token rename that touches a line #80 also edits, so whichever merges second needs a one-line rebase.

Testing: new rows cover a label with a digit before a capital, a shown block labelled "OECD DAC2A" honouring a requirement stating "OECD DAC2a" with no search, a referral candidate held only in the log carrying an open period, and a first-year read for a log-held pin with no history saved. Each fails without the change. The full harness suite passes (7169 passed, 1 skipped, 1 xfailed). ruff and ty are clean.

…ntity

"What share of ODA to Kenya came from the United States?" could ask the
same single-option dataset question twice and rework its draft four
times. The tokenizer split Search's label "OECD DAC2A" into "dac2, a",
so a requirement stating "OECD DAC2a" re-searched a pin it named, and
the discovery log answered identities only for a logged source's
alternatives, so a candidate the user accepted had none on record and
its open period was refused. A digit now splits from a capital only
where the capital starts a word, and the log's recorded_definition
answers each logged source's own identity for admission, windowing and
the first-year read.
@jm-rivera
jm-rivera merged commit 5579f10 into dev Oct 26, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant