Skip to content

ADR 0013: Follow adds the rows that a batch has not considered - #238

Merged
SimonHeybrock merged 1 commit into
mainfrom
236-follow-adds-rows
Nov 19, 2026
Merged

SimonHeybrock merged 1 commit into
mainfrom
236-follow-adds-rows

Conversation

@SimonHeybrock

Copy link
Copy Markdown
Member

ADR 0013, proposed: Follow adds the rows that a batch has not considered. A decision to review; no code. On top of #237 (ADR 0012), which left open how a batch is amended from the files. Part of #236.

The decision proposed

A batch made from a source keeps the name of the source and the keys it has considered, next to its table. A step of Follow adds the rows of the source whose keys are not among them, once the source has filled them, and changes no row that the table has.

def step(batch, source, files, considered):
    given = table_of(source.rows(files))
    table = batch.frame()
    filled = [c for c in source.fills if c in table.columns]
    is_new = ~given.index.isin(considered) & ~given.index.isin(table.index)
    is_ready = given[filled].notna().all(axis=1)
    new = given[is_new & is_ready].reindex(columns=table.columns)
    batch.put(pd.concat([table, new]))
    considered |= set(new.index)
    batch.reduce(ready_without_result(batch.state()))
  • A row that a person removes stays away: its key is considered. The table keeps no mark.
  • A restart, or Follow turned on again, adds the rows that arrived in between: considered is kept.
  • A row waits until the source has filled it. A sample run measured before its transmission run is added when the transmission run is listed. The page says how many rows wait, and a person can add them as they are.
  • No scope. "From run 60339 on" is the choice of rows when the batch is made. "Start the next batch here" splits the table and moves the source to the next batch.
  • A step reduces the ready rows without a record. It never reduces a row that has a result.

What it gives up

  • A transmission run measured again reaches no earlier row, and a correction of a run changes no row that the table has. A person sets the cell.
  • A row that waits is not in the table.
  • "Start the next batch here" ends Follow, where ADR 0011 leaves it on the batch that ends.

What goes when it is built

Scope, Row.index, Source.index_type, Header.scope, Header.source, manager.change(scope=...), RowState.changed_by_hand, and the columns at of Automatic: most of #229 and part of #235.

To decide in review

  • A row is added "once the source has filled the columns it fills", not "once it can be reduced". With the second, a column that the source does not fill would hold a row back for ever, and a parameter that takes None would let a row through before its file is listed ("Alternatives considered").
  • In a batch that follows, a row that a person adds is reduced as soon as it is complete.

🤖 Generated with Claude Code

Proposed, on top of ADR 0012. A batch made from a source keeps the name
of the source and the keys it has considered. A step of Follow adds the
rows of the source whose keys are not among them, once the source has
filled the columns it fills, and changes no row that the table has. So
a row that a person removed stays away without a mark in the table, and
a restart adds the rows that arrived in between.

No scope is kept: its start is the choice of rows at making, and its end
is "Start the next batch here", which moves the source to the next
batch. A step reduces the ready rows without a record.

ADR 0012 and ADR 0011 point to it for the parts it decides or replaces.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@SimonHeybrock SimonHeybrock self-assigned this Nov 19, 2026
Base automatically changed from 236-a-batch-keeps-its-rows to main November 19, 2026 11:14
@SimonHeybrock
SimonHeybrock merged commit 1d71003 into main Nov 19, 2026
2 checks passed
@SimonHeybrock
SimonHeybrock deleted the 236-follow-adds-rows branch November 19, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant