Repository navigation
A batch keeps its table; Follow adds the rows not considered (ADR 0012, ADR 0013) - #241
Draft
SimonHeybrock wants to merge 10 commits into
Draft
SimonHeybrock wants to merge 10 commits into
SimonHeybrock wants to merge 10 commits into
Conversation
A batch is a template and a table. The table is one DataFrame that only `Batch.put` changes; its parameter columns are the blanks of the template, and it is kept as rows.json. A batch is shown without its source and without a scan, so the files no longer decide what rows it has. - `Batch(client, spec, label=, settings=, rows=, definitions=)` with `frame()` and `put(frame)`. A put of an equal table writes nothing; a put with other columns rewrites the settings. The request of a row is the template filled with its cells, an empty cell giving the default, or None if the parameter takes None. Resolved cells are forgotten at each put. The Batch writes its definitions and never reads them. - `source.py`: a source makes a frame (`rows_of`), with an empty column for each column it fills. - Follow adds the rows of the source that are new and filled (`auto.added`), never changes a row the table has, and reduces the rows without a result. The keys considered are kept in source.json by the manager alone. `manager.step()`, `offered(label)` and `take(label, keys)` are what the page calls. - The header is title and description. `create` takes the first rows and, with a source, considers every key it gives now. - Gone: `RowTable`, `Row`, `EMPTY`, `Scope`, `Words`, the scan, edits.json, `start_next`, `check_fills`, `auto.run`, the problem "not a column". The command, the page and the applications are adapted after this, so their tests fail until then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pair_by_sample returns a frame: the key as index, sample and sample_transmission as the text of the file, and the notes run (Int64, empty for a run that is not a number) and name. SAMPLE_RUNS is a plain Source of these fills; the words, the row name and the index title go with the Source fields that held them. iofq_batch and COLUMNS go: only a test used them. A batch of I(Q) without a manager is Batch(client, IOFQ, rows=rows_of(client, SAMPLE_RUNS), staged=True), and the module docstring shows it. A test checks that such a batch has its columns on a folder without files. The stories of section E make their batch with rows=rows_of(...) and pass the source to Automatic; E4 no longer scans. user-stories.md shows the same code. E1 stays xfail for the reason of ADR 0013. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A table file (.csv, .tsv, .xlsx) is the frame of a batch as text, so it is read and written in table.py with the dtypes of the parameters: read_table needs a column key, reads each parameter column with its dtype, a list or a group as JSON text, and drops status, record and failure, so a table downloaded and loaded again unchanged changes nothing. read_text reads a table that is not of a batch (a run table, a reduced table to export) as text. read_cell and write_cell move from text.py to table.py. frames.py keeps what the command runs. reduce_table makes a Batch from the template's settings and the table, whose parameter columns are the blanks. reduce_batch reduces rows of a batch that is already set up, and with_results adds what came of each row to a frame indexed by key. rows_of, table_of and batch_table go: a source and a file give the frame that put takes. text.py keeps templates. checked_settings no longer compares the blanks with the columns, leaves out a setting at a column, and names files with table.as_request; named_file and with_named_files go. The command (ADR 0012, 0013): reduce on a new label makes the batch from the table, or from every row of --source, or from one row keyed by the label, and checks the template before anything is written. On a label that exists, the template becomes the settings and the table replaces the table; --source is refused. The clean-up of a batch made by a failed run goes. export_table takes the key from the index. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p kind - A Follow step that raises after adding rows still keeps the keys it considered, so a row a person removes stays away. - The table owns the lists and groups in its cells: typed and Batch.frame copy them, so only put changes a batch. - A note with no value keeps its dtype, or is text where pandas made it float or object, so text can be set into it after a step or a take. - create deletes what it wrote if a write is refused, so the label stays free; the header is written last. Tests of both write orders. - Batch holds one Template; create makes its first table and template with a Batch. - A step names the source of a row the batch refuses; take says which rows are in the batch already. FromSource is not exported. - ADR 0012 and BatchManager.batch name the second case in which a batch cannot be shown: a version of the spec that refuses the cells. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # packages/essapps/src/ess/apps/batch/table.py
frames.table_of is gone, and a source gives the frame that a batch takes, keyed by its index. The notebook takes the table from rows_of, edits a cell by its key, names a run in a file column as 'run:60394' (a cell of a file column is text), selects the key as the index, and reads a run table from a file with table.read_text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each action that changes rows builds a frame and puts it to the batch: an edit of cells, a row added or removed, and Load table, which replaces the whole table. A value chosen in a picker is kept as table.cell_of makes it. The forms of the cells and settings are built again when the columns of the batch change. The page shows the table as it is kept, without its source or files. A row is named by its note `name`, else by its key, and every other note is shown. "Rescan runs" lists the runs again and changes no row; a correction of runs changes no row either. A tick of Follow is `manager.step()`; the page of the batch that follows shows what it refused, how many rows wait, and its failure. Follow is offered on every page; the manager refuses a batch not made from a source, and the page says why. New batch makes a batch of every row that its source gives now, with an empty column for each ticked column the source lacks, or, without a source, of the ticked columns and no rows. The counts read each state without a scan; `outside` counts the keys of each source that no table has. Gone from the page: revert and the edited/scanned marks of batch cells (the Runs view keeps both), bring back, rows added by hand, strays, scope, "Start the next batch here", and the scan after a correction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build made its first batch without rows, so the batch had no columns
and no blanks (plan, G10). It is now made with
rows=manager.rows_of('sample-runs') and source='sample-runs', and so
considers every sample run listed at the start; a run that arrives
later comes as a row through a step of Follow.
"Rescan runs" lists the runs again and changes no row. The tests of the
page and of the arrivals check that: a relist leaves the rows as they
are, and a step of Follow adds and reduces the sample runs whose
transmission run arrived. The docstrings of web.py and arrivals.py say
the same, and that a newer transmission run changes no row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third pull request for #236: the code of ADR 0012 (a batch is a template and a table) and ADR 0013 (Follow adds the rows that a batch has not considered).
What a batch is after this
DataFrame(Source(rows, fills, selector)), the same thing thatputtakes.'run:60339','r17.centre'), converted to what a request takes in one place,table.as_request..csv,.tsv,.xlsx) is the frame as text; the command and the page read and write it withtable.read_tableandtable.write_table.What goes
Row,RowTable,Scope,EMPTY,edits.json,Batch.scan, the spec, columns, source and scope in the header, "set by hand", and on the page: revert of a cell, "bring back", the scope fields, the scan after a correction, and "Start the next batch here". The batch layer shrinks from 2865 to about 2480 lines in the modules of the model; the pull request as a whole removes more than it adds.Choices made here, not in the ADRs
source.json, so the web layer copies no wiring.Sourcekeeps onlyrows,fillsandselector. A row is named by its notenameif the table has one, else by its key.Not in this pull request
The page actions that the stored table makes possible ("New batch" from a range of rows or a table file, "Add rows from the runs", "Make this a column") and the rewrite of the guide
batch.mdfollow.🤖 Generated with Claude Code