server2: restore station observations from day products after a restart

docs/architecture/diagrams/server2-kma-data-design.html

@@ -5431,7 +5431,7 @@ <h1>KMA response-first contract and qualification</h1>
           <rect x="474.475" y="1693" width="163.79999999999998" height="18" rx="3" class="c-mask"/>
           <text x="556.375" y="1703" class="t-muted" font-size="11" text-anchor="middle">Read and bounded repair</text>
         </g>
-        <text data-detail="context" x="319.25" y="1731" class="t-muted" font-size="7"><tspan x="319.25" dy="0">Zone state and KASI restore from</tspan><tspan x="319.25" dy="11">one stored fold; a restore holds</tspan><tspan x="319.25" dy="11">a request at most 500 ms.</tspan></text>
+        <text data-detail="context" x="319.25" y="1731" class="t-muted" font-size="7"><tspan x="319.25" dy="0">Zone state, KASI and station</tspan><tspan x="319.25" dy="11">pages restore from stored folds;</tspan><tspan x="319.25" dy="11">a restore holds a request at most</tspan><tspan x="319.25" dy="11">500 ms.</tspan></text>
         </g>
 
         <g data-edge-from="owner" data-edge-to="provider" data-edge-label="If gap: qualified reacquisition" data-edge-key="22" data-edge-id="reacquire">
@@ -5758,7 +5758,7 @@ <h3>Canonical reuse; conditional candidates</h3>
         </div>
         <ul>
           <li>Canonical v2 raw/catalog/group controls remain the restore authority. Same selected original closure must replay exactly.</li>
-          <li>The collector stores zone-state daily, KASI capture stores kasi-fold and the image timer airkorea-forecast-hourly: folds of raw inputs, never assembled weather.</li>
+          <li>The collector stores zone-state daily, KASI capture stores kasi-fold, folding processes store station-pages per day and the image timer airkorea-forecast-hourly: folds of raw inputs, never assembled weather.</li>
           <li>Per-grid optimal combinations, source plans, day locators, heads and packs remain deferred. Single EC2 runs only: the cold p95 gate is not qualified.</li>
         </ul>
       </div>

docs/architecture/diagrams/server2-kma-data-design.json

@@ -240,7 +240,7 @@
       "to": "s3",
       "y": 1713,
       "label": "Read and bounded repair",
-      "note": "Zone state and KASI restore from one stored fold; a restore holds a request at most 500 ms."
+      "note": "Zone state, KASI and station pages restore from stored folds; a restore holds a request at most 500 ms."
     },
     {
       "id": "reacquire",
@@ -285,7 +285,7 @@
       "title": "Canonical reuse; conditional candidates",
       "items": [
         "Canonical v2 raw/catalog/group controls remain the restore authority. Same selected original closure must replay exactly.",
-        "The collector stores zone-state daily, KASI capture stores kasi-fold and the image timer airkorea-forecast-hourly: folds of raw inputs, never assembled weather.",
+        "The collector stores zone-state daily, KASI capture stores kasi-fold, folding processes store station-pages per day and the image timer airkorea-forecast-hourly: folds of raw inputs, never assembled weather.",
         "Per-grid optimal combinations, source plans, day locators, heads and packs remain deferred. Single EC2 runs only: the cold p95 gate is not qualified."
       ]
     }

docs/architecture/server2-kma-data-design.md

@@ -419,6 +419,7 @@ representations/v1/{source}/{kind}/{key_hash}/{day}/{manifest_sha256}.json
 raw/v2/kma-warning/zone-state/{YYYY}/{MM}/{DD}/...
 raw/v2/data-go-kr/kasi-fold/{YYYY}/{MM}/{DD}/...
 raw/v2/airkorea-image/airkorea-forecast-hourly/{YYYY}/{MM}/{DD}/...
+raw/v2/kma-web/station-pages/{YYYY}/{MM}/{DD}/...
 
 # Deferred discovery experiments; not initial serving dependencies:
 index/v3/{source}/{kind}/{key_hash}/day/{YYYYMMDD}.json
@@ -505,7 +506,11 @@ capture pass. These hold provider items or legacy-shaped state, not assembled we
 process falls back to the original objects when no usable fold exists. The AirKorea image hourly
 forecast is the one fold requests read without that fallback: `airkorea-forecast-hourly`, the
 legacy `ArpltnHourlyForecast` rows decoded from every stored image acquisition, is made by the
-process timer (below), and no request decodes the GIFs. The observed response also
+process timer (below), and no request decodes the GIFs. The station page product
+(`station-pages`, one per KST day) holds the parsed AWS hourly and city pages any folding process
+keeps; a restarted process reads it, compares it with the day catalogs and restores only the
+pages it lacks, falling back to the raw pages when it cannot be used
+([restart read](server2.md#s15b-2-restart-read-and-station-page-products-2026-11-07)). The observed response also
 uses the release's land zones (`data/kma_forecast_land_zones.csv`) until a forecast-zone list is
 stored, so its first answer can request mid temperatures. Measurements are in the
 [speed evidence](../evidence/research/kma-speed-options-2026-11-04.md#result-of-the-change).

docs/architecture/server2.md

@@ -2270,6 +2270,89 @@ from `/gather` against the retired `KmaStnMinute`; the rules are ported but not
 decision), and the deployed `ASOS_HISTORY_STATIONS` allowlist (every registry city station is a
 candidate).
 
+### S15b-2 restart read and station page products (2026-11-07)
+
+The EC2 restart comparison of #2866 showed the first request after a restart without minute
+observations or past days: the load restored every stored hourly and city page of the window one
+day after another, parsed each again (about 1.0 to 1.4 s per day on a t3.small), and read the
+minute pages last through latest-first pages that read the descriptor of every minute group of the
+day. AK chose the day product design on 2026-11-07 ([intent](../../intent/issue-2614-kma-restart-observe.md),
+[spec](../../specs/issue-2614-kma-restart-observe.md), [plan](../../plans/issue-2614-kma-restart-observe.md)).
+
+**Targeted reads.** `CatalogStore::lookup_members` reads one catalog and then only the named
+members' groups (one descriptor and one raw body each); `lookup_acquisition` is its one-member
+case, and `ordered_candidates`/`lookup_page` are unchanged for the other cursor readers.
+`Reader::listed_acquisitions` lists a catalog's acquisitions from the control object alone and is
+shared with the image forecast. The minute catalogs of the readers' 40 minutes are listed once; of
+each minute the newest acquisition (completion, then raw digest) is restored, and when it is not
+complete or does not parse the next one of that minute (at most three per minute), at most four
+members per read.
+
+**Day product.** One product per KST day D: source `kma-web`, kind `station-pages`, the AWS page
+key, partition `YYYYMMDD`, period D/0000, fetched at its making instant, published as a one-member
+group. It is gzip JSON of the parsed inputs of the hourly fold, not its output: `schema`,
+`parseRevision` (bumped when page parsing or the reduction to registry facts changes), `made`,
+`day`, `excluded`, `knownIds`, `excludedIds`, `cityIds`, a `facts` table and `pages`. Per kind and hour of D it holds the
+newest page as the process keeps it (`kind`, `hour`, `fetchedAt`, `sha256`, `rawId`, `rows`).
+AWS rows equal to their registry facts are indices into `facts`; city rows are whole. `cityIds` is
+the city set the reduced rows are whole for (the intersection of the sets the day's pages were
+reduced with). Every listed group of the day is accounted for: its pages, `knownIds` (the raw
+identities of the other groups found complete, such as superseded acquisitions) and `excludedIds`
+(a member of each group S06 excluded; `excluded` is their number). Values keep every
+stored value (undefined, dates, negative zero, NaN) and key order. A reader refuses a product that
+expands past 16 MiB.
+
+Any process that folds writes products: after a fold that recomputed the hourly part a
+`Flight::Product` (never awaited by a request) stores the day whose made-of (sorted page raw
+identities, `cityIds`, excluded identities, `parseRevision`) differs from the last product this process read or stored.
+Only days read without failure whose every hour is in the window are stored, each at most once per
+ten minutes per process. Products are not cached on disk.
+
+**Restart read.** In parallel: the product, AWS hourly and city partition LISTs, the seed and the
+minute catalogs. Then for the due days, newest first and eight at a time, the day's page and
+product catalogs are listed (control objects only) and, the first time a process reads the day,
+its newest product (completion, then digest) is read and decoded. A product whose `cityIds` does
+not cover the current city set (seed and city pages) is not used. Product pages the catalogs do not
+list are dropped (S3 is truth). A day is then compared: every listed group that is neither held nor
+known complete (the excluded ones again, and those published since) is restored, four per catalog
+read; a complete one becomes known, an incomplete one excluded. Of each kind and hour the newest
+complete new page (latest fetch; of one instant the smaller digest, the page a read of every
+acquisition keeps) that is newer than the held one is parsed and held. An unreadable product, an
+older schema or parse revision, a grown city set, no product, or a new newest page that does not
+parse gives the day's raw read as before, which records every complete group as known and every
+excluded one. When a fold's city set has grown past the set an AWS page was reduced for (a city
+page restored after the products, or acquired since, names another station), `Flight::Expand`
+reads those pages in full again by their raw identity, four per catalog read, and keeps each read
+as it completes; no request waits for it. A page whose read was started waits a minute before it
+is tried again; pages not reached before the read's 30 s limit, and due retries, are started by the
+next fold or request without waiting for a new hourly page. The next fold reduces the pages read
+in full for the new set. `Flight::Load` ends after the products, today's
+and yesterday's comparisons, the seed and the four newest minute pages. `Flight::LoadRest`, which
+no request waits for, then compares the other days (each marked read and the pages dirty) and
+restores the rest of the minute window. Reloads every five minutes compare with the pages held.
+
+**S16.** Answers, the station page input identities and the response-first durability set are
+unchanged: a page restored from a product carries the raw identity and digest of the stored page.
+A day is partial while one of its listed groups is excluded, as a raw read with excluded groups is.
+Excluded groups are checked again at every read of the day (a restart, and each reload), so a
+repaired group completes the day and its product is stored again without it. A group published
+after a product, older than its pages, is unaccounted for and checked like any new group, so a
+restart from products counts a day complete exactly when a raw read would, given that a complete
+raw object is never deleted. `settled()` counts `LoadRest`, `Product`, `Expand` and a due product
+comparison as busy, and is false while a held AWS page is reduced for a city set that does not
+cover the last fold's (its past rows of a city station would be missing).
+Right after the first deployment the window's oldest day has no product (some of its hours were
+already out of the window) and is read from its raw pages after the first answer.
+
+Verification (local, S06 loopback S3 peer, no provider): targeted member GET counts, a restart over
+200 stored minute groups, the value codec round trip, a restart with unreachable providers answering
+168 past hourly pages and minute rows on its first request without raw page GETs, a later page
+restored alone and equal to a cold raw read, unusable products replaced by the raw pages and stored
+again, two processes' products, unsettled answers during the rest of the read, a day with an
+S06-excluded group kept partial, a repaired group completing its day (and recording summaries), a
+station joining the city set after the products (also with failing and with slow full reads), an
+incomplete newest minute acquisition, and the disk cache exclusion.
+
 ### S15b-2 s station registry seed (2026-10-15)
 
 AK authorized a one-time read-only export of the legacy `kmastninfos` on 2026-10-15. It ran with
@@ -3040,6 +3123,13 @@ S16 also changes one S15b-2 behavior: a station-page day whose `aws-hourly` or `
 is not listed now reads as empty. Before, the absent catalog's lookup answered "incomplete", so a
 day without city pages never loaded its hourly pages from S3 and was read again on every load.
 
+Since the [restart read](#s15b-2-restart-read-and-station-page-products-2026-11-07) (2026-11-07) a
+station-page day can be read from its day product and the day catalogs instead of every raw page.
+The input identities are the same, a product carries the S06 exclusions of the read it was made
+from, and a station fold is not complete while the rest of a restart read or a product comparison
+is still running.
+The first answer after a start therefore records no summary; a later request does.
+
 **Last-year lookup** (`summary::Lookup::last_year`, not wired to a route). The same date a year
 earlier (Feb 29 → Feb 28), then ±1..±3, earlier first. At each date: the town's grid summary, then
 the station's ASOS summary (when missing, one data.go.kr daily range `[base-3, base+3]` through the

docs/operations/server2-infrastructure.md

@@ -264,7 +264,9 @@ is read by the tests or measurement tool.
 The cache holds verified immutable raw objects for `data-go-kr`, `kma-web`, `vc`
 and the exact `airkorea-image` / `airkorea-forecast-image` pair. AirKorea's image
 raws are included; the decoded hourly forecast (`airkorea-forecast-hourly`, made by
-the process timer and read from S3) and packs are **not** cached on disk.
+the process timer and read from S3), the KMA station page products
+(`kma-web` / `station-pages`, made by any process that folds the station pages) and packs
+are **not** cached on disk.
 Mutable catalogs, absence proofs, state, budgets, push tokens, geocode and exact
 request coordinates are excluded. Catalog/group publication checks still use S3.
 Filename and namespace hashes do not replace the raw codec's identity validation.

intent/issue-2614-kma-restart-observe.md

@@ -0,0 +1,26 @@
+# Station observations after a restart
+
+AK requires that a request waits at most 3 s and answers from what was gathered by then. The EC2 restart comparison of #2866 showed that the first weather request after a restart misses the minute observations and the past days. The restart read restored every stored AWS hourly and city page of the 8-day window one day after another and parsed each page again: about 1.0 to 1.4 s per day on a t3.small, plus 4 to 5 s of sequential GETs. The minute pages came last, and every page call read the descriptor of every group in the day's minute catalog. The request stopped waiting at 2.5 s with neither. On 2026-11-07 AK chose the day product design ("A로 진행").
+
+## Outcomes
+
+- **MINUTE:** a restart lists the day's minute catalog once and restores only the newest acquisition of each minute within the readers' 40 minutes. The newest four come before the first answer and the rest follow.
+- **PRODUCT:** any process that folds the station pages stores one product per KST day holding the parsed inputs of the hourly fold, the newest page of each kind and hour, at most once per ten minutes per day and only when what the day is made of changed. No request waits for it.
+- **RESTART:** a restarted process reads each day's newest product and compares it with the day catalogs (S3 is truth). It restores only the pages the product lacks, parses only those, and answers past days and minute rows from its first request. Today and yesterday are compared before the answer, the other days after it.
+- **ACCOUNTED:** a repaired group completes its day after a restart, a station that joins the city set after the products has its AWS rows read in full again, and a minute whose newest acquisition is not complete is read from the next one, each as a raw read gives it.
+- **FALLBACK:** an unreadable product, an older schema or parse revision, or a city set that grew is replaced by the raw pages of that day, and the day's product is stored again. A page the catalog does not list is dropped.
+- **S16:** legacy answers, the station page input identities and the response-first durability set are unchanged. A day stays partial when its product's read had a group S06 excluded or a newer group is not complete, and answers stay unsettled until the rest of the read ends.
+- **PLACEMENT:** the placement gate and the artifact policy pass on the outgoing commits.
+
+## Decisions for AK
+
+This adds a stored object that other processes trust as a starting point, so it is not merged without AK:
+
+- Products are written to S3 under `kma-web` / `station-pages` beside the pages they are made of, never deleted, at most one per day and ten minutes per process. They are excluded from the disk cache, like the image forecast product.
+- A day product records every listed group of its day: the pages, the other complete groups and the S06-excluded ones. Each read checks the excluded groups and any group published since, so a restart counts a day complete exactly when a raw read would, as long as no complete raw object is ever deleted. While a group stays excluded, every read of its day spends one catalog read and the group's descriptor and raw GETs on it.
+- The first answer after a start is never complete for S16: the days past yesterday are compared after it, so a later request records the daily summaries. A fold that recomputes the hourly part also holds summaries back until its product comparison ends.
+- A process at hour H stores products only for days whose every hour is in the window. Right after the first deployment, the oldest day of the window therefore has no product and is read from its raw pages after the first answer.
+
+## Exclusions
+
+Not in this change: a real EC2 measurement of the new path, a product for the minute pages, the station ASOS state, and the collector's restart probing (KMA-SR-06 of #2862).