cpu: keep valid R10000-family cache lines in each CPU's own view of memory

CHANGELOG.md

@@ -13,6 +13,13 @@ hashes are given where a change is easiest to understand by reading the commit.
 
 ## November 2026
 
+### R10000-family cache lines as memory
+
+- A valid primary data or secondary cache line now holds its page in that CPU's own copy of ppmem's window, so a PROM that keeps its stack in cache lines reads and writes the lines with cached accesses while the uncached path, DMA and the other CPUs see memory. Index Store Data writes through to such a line. `l2 probe 0` lists the pages held. Nothing changes on a load, store or fetch. A valid R5000 primary data line does the same, for the O2's POST stack.
+- Primary instruction lines hold their pages too, each word written there as the instruction it predecodes, for the eleven predecode classes the model knows. Index Load Data (I) where a held secondary line would refill the set returns that line's words, predecoded, until Index Store Data (I) writes the set. Only the Fuel's and Tezro's PROMs use them.
+- jitv2 does not compile a page any CPU's cache holds lines in: that CPU runs it from its own view in the interpreter, and every other page stays compiled.
+- The O2 keeps POST's stack in those lines. CRIME's side page for the uncached path and the tag-store hook are gone, and an O2 snapshot an older iris took during POST is refused.
+
 ### R10000 family fixes, R12000 and R14000
 
 - The R10000 family has 44-bit virtual and 40-bit physical addresses, FrameMask and a 28-bit EntryLo PFN, picks the TLB refill vector by address space, invalidates overlapped TLB writes (Status.TS), and completes uncached stores nothing answers. Partial LWR keeps the upper word, and the cache diagnostics (Index Store Data, secondary tag mask, Config size) work.

clanker/INDEX.md

@@ -24,6 +24,7 @@ CPU architecture: CP0, TLB and its fast paths, caches, FPU, LL/SC, CPU models. B
 | Note | Kind | Title |
 |---|---|---|
 | [cached-flash-is-a-cache-with-two-cpus.md](cpu/cached-flash-is-a-cache-with-two-cpus.md) | rule | With two CPUs, cached flash reads must hold their lines: a fetch through to a chip in status mode executes `0x80808080` |
+| [cache-lines-live-in-the-cpus-view.md](cpu/cache-lines-live-in-the-cpus-view.md) | design | A valid R10000-family cache line, or R5000 primary data line, makes its page private in that CPU's own copy of ppmem's window; the uncached path and other masters keep memory. Limits and how to see what is held |
 | [cache-attributes-and-fetch-alignment.md](cpu/cache-attributes-and-fetch-alignment.md) | rule | Cache attributes and fetch alignment: three hardcodings, all wrong |
 | [cache-tag-index-and-check-bits.md](cpu/cache-tag-index-and-check-bits.md) | rule | Cache tags: index ops index by address; check bits are generated, not stored |
 | [cop0-requires-kernel-or-cu0.md](cpu/cop0-requires-kernel-or-cu0.md) | rule | CP0 instructions must be gated on Kernel-or-CU0 |
@@ -126,7 +127,7 @@ Non-graphics devices: MC/MEMCFG and RAM banks, HPC3, VDMA, EEPROMs and NVRAM, PS
 | [aic7880-patchsim.py](devices/aic7880-patchsim.py) | howto | Reproduces the Linux aic7xxx firmware patch selection and critical sections for a 7880 |
 | [o2-no-ecc-alias-is-memory.md](devices/o2-no-ecc-alias-is-memory.md) | rule | O2: physical 0x00_8000_0000 is CRIME's no-ECC memory alias, folded onto RAM by the CPU (netstat/Netscape hang, /dev/mem all ones) |
 | [o2-cache-handover-full-to-shadow.md](devices/o2-cache-handover-full-to-shadow.md) | history | O2 cache as it was until 2026-10-30: the full model through POST, the shadow model after, switched by PC range and a tag-store trigger |
-| [o2-prom-stack-is-a-cached-page-at-phys-0.md](devices/o2-prom-stack-is-a-cached-page-at-phys-0.md) | design | O2 PROM stack: one tag-installed page at physical 0 that the memory test overwrites through KSEG1; a 4 KB page in the bus decode answers for it uncached, so the shadow cache runs from reset |
+| [o2-prom-stack-is-a-cached-page-at-phys-0.md](devices/o2-prom-stack-is-a-cached-page-at-phys-0.md) | design | O2 PROM stack: one tag-installed page at physical 0 that the memory test overwrites through KSEG1. Retired: the CPU's cached view holds it now |
 | [memcfg-limit-is-not-the-bank-size.md](devices/memcfg-limit-is-not-the-bank-size.md) | rule | MEMCFG: `limit` is the slot, `addr_mask+1` is the mirror period — they differ |
 | [nvram-persistence.md](devices/nvram-persistence.md) | design | Battery-backed state on Stop |
 | [ps2-aux-mux-probe-and-status-bit.md](devices/ps2-aux-mux-probe-and-status-bit.md) | rule | PS/2 mouse not detected under Linux 3.2 guests (and keyboard sometimes flaky too) |

clanker/cpu/cache-lines-live-in-the-cpus-view.md

@@ -0,0 +1,104 @@
+# Cache lines live in the CPU's own view of memory
+
+The R10000-family PROMs use the caches as memory before memory works. They make
+lines valid with Index Store Tag and Index Store Data and then load, store and
+fetch through them, while uncached accesses, which bypass the caches (NEC
+VR10000 user's manual p. 75), and every other master see memory. What the shipped PROMs do on iris:
+
+| Machine | Lines | Memory behind them |
+|---|---|---|
+| O2 | secondary and primary data lines for page 0 and 0x1000 hold POST's stack for about 50-65 ms, while POST tests page 0 uncached. On the R5000, primary data lines for page 0 (TagLo 0xc0) | yes |
+| Octane | a tag and data RAM test on secondary set 0, read back only with Index Load Tag/Data | no: the patterns name no memory |
+| Fuel | primary data lines for 0x100000-0x107fff hold the stack; primary instruction lines hold PROM code | no, until memory is configured mid-phase |
+| Tezro | the same as the Fuel, on every processor at once, with different stack contents at the same addresses | the same |
+
+Every line ends by Index Store Tag. None is written back, and none is read back
+with Index Load Data while it holds a page.
+
+## How
+
+- Each CPU reads memory through a window of its own (`PpMemSpace::add_cached_view`).
+  Every mapping laid in the shared window is laid in each view too, and the view's
+  region bitmap goes to that CPU's inline fields. The shared window, which
+  `Physical`'s direct paths, CRIME's rendering engine and HEART read, never holds a
+  line.
+- When Index Store Tag makes a line valid, its page becomes private memory in that
+  view, a copy of memory, and the line's data array is written into it
+  (`ShadowCache::line_enters`). An invalid or different tag ends the line, and the
+  page goes with its last line. The shadow's tags stay the only line state, with a
+  held bit beside each. Index Store Data into a held line writes through.
+- The R5000 does the same for a valid primary data line, whose tag has the
+  R10000's PTag and PState fields (NEC VR5000 user's manual U11761EJ6V0UM p. 133).
+  It has no Index Store Data, so the page starts as memory. Its instruction lines
+  hold nothing, since one could hold only what memory holds.
+- A page with no memory behind it holds nothing: the line stays in the arrays, as
+  a tag test's patterns do. The region bitmap is all a CPU tests, so a held page's
+  whole region is the view's while it stands, reading zeros wherever nothing is
+  mapped. The O2's POST holds lines for page 0 before CRIME maps the rest of that
+  region.
+- A remap saves and restores held pages, so memory configuration mid-phase keeps
+  the stack.
+- Loads, stores, fetches, the decode cache and compiled code are untouched: they
+  read the view through the base pointer and bitmap they always had. The decode
+  cache compares every word it serves, so it sees a page change hands.
+
+## Limits
+
+Each is an approximation, stated where it is made.
+
+- Page, not line: the rest of a held page shows memory as it was when the page was
+  taken, uncached writes to it meanwhile are not seen by cached reads, cached stores
+  to it are dropped at release, and a line that ends leaves its bytes until the
+  page's last line goes. The page is the host granule: 4 KB on Linux and x86 macOS,
+  16 KB on Apple silicon, 64 KB on Windows.
+- Lines end only by a tag change the shadow already models: Index Store Tag, and
+  Index Writeback Invalidate (D) zeroing the tag from the R12000 on. Nothing is
+  written back. The hit operations and the other index invalidates leave a held
+  line as they leave every shadow tag, untouched.
+- Index Load Data reads the data array, not the page, so it does not see cached
+  stores.
+- A region holding a page reads zeros in its unmapped gaps for that CPU's cached
+  accesses, where the bus would answer.
+- A secondary doubleword puts TagLo at the lower address. The manuals say only
+  that TagHi is the high word and TagLo the low word (MIPS R10000 UM p. 201, NEC
+  VR10000 UM p. 289); which address each pairs with is inferred. PROM 6.210 (Fuel,
+  Tezro) stages code with the word at A in TagLo and the word at A + 4 in TagHi,
+  990 times in a Tezro boot, and its staged code equals its image only this way.
+  The IP35 boot-flash PROM's staging pairs the words the same way: simulated with
+  the other order, 378 of its 562 staged fetches break, every trampoline fetch
+  among them. With the other order the Tezro's PROM ran its long cache test
+  uncached, taking twice as long.
+- No replacement, parity or ECC, and no coherence with other CPUs or DMA.
+- A snapshot taken while lines hold pages is refused at restore.
+
+## Compiled code
+
+A page some CPU holds is not compiled: compiles read memory, and compiled code is
+shared. ppmem puts `GEN_HELD` on the page's generation counter at the first hold
+and takes it off at the last release, moving it with the page on a remap
+(`PpMemSpace::mark`). The compile that finds it publishes nothing and denies every
+offset until the release. ppmem's counters are the bus's for memory; a machine whose bus
+maps a device over memory, as the IP35 boards map their reset PROM, must mark that
+device's counter as well.
+
+## Primary instruction lines
+
+Only the Fuel's and the Tezro's PROMs use these. A valid primary instruction line
+writes each 36-bit predecoded word (p. 288) into its page as the instruction it
+predecodes, through an eleven-class table inferred from PROM 6.210's words
+(`l1i_decode`). A word of another class leaves memory's word, where hardware would
+run it. A page held only for code also answers cached loads with code.
+
+The model never refills on a fetch. The Tezro's PROM runs a staged secondary line
+and reads the primary words back with Index Load Data (I) to stage them itself, so
+Index Load Data (I) in a set a held secondary line maps to answers with that line's
+words predecoded (`refilled_word`), until Index Store Data (I) writes the set. When
+two held lines map one set, the later wins. An instruction no class holds comes back
+in the model's own form (bits 35:32 set, which takes classes 0x78-0x7f), which
+hardware would not produce. Index Load Tag (I) reads the stored tag.
+
+## Seeing it
+
+`l2 probe 0` in the monitor prints how many lines hold pages and which. Every
+machine should show none at the IRIX login. `log l2c mask op` logs each page held
+and released.

clanker/devices/o2-prom-stack-is-a-cached-page-at-phys-0.md

@@ -1,10 +1,11 @@
 # O2 PROM stack is one cached page at physical 0, and RAM behind it is a different thing
 
-Status: landed 2026-10-30. Both O2 CPUs run the shadow cache from reset; the handover
-([o2-cache-handover-full-to-shadow.md](o2-cache-handover-full-to-shadow.md)) is gone. Code:
-`O2Memory::stack_tag_store` in `src/dev/crime_mem.rs`, the tag-store hook in
-`src/cpu/mips_cache_shadow.rs`. Sections below are the measurements that chose the design; the
-prototype results used earlier binaries.
+Status: retired 2026-11-08. The stack now lives in the CPU's own cached view of ppmem's window:
+a valid primary data or secondary line holds its page there (`ShadowCache::line_enters` in
+`src/cpu/mips_cache_shadow.rs`, `PpMemSpace::add_cached_view`), and the uncached path, the decode
+table and the rendering engine keep seeing memory. `O2Memory::stack_tag_store`, its side page and
+the tag-store hook are gone. The sections below are the measurements that chose the old design;
+what they say about the PROM still holds.
 
 ## Where the stack is
 

src/cpu/jitv2/comp.rs

@@ -116,6 +116,8 @@ pub(crate) enum Snapshot {
     Taken(u64),
     /// A word could not be read (not RAM right now).
     Unreadable,
+    /// A CPU's cache holds lines in the page, under this generation.
+    Held(u64),
     /// The generation moved under every attempt.
     Busy,
 }
@@ -129,6 +131,10 @@ pub(crate) fn snapshot_page(page: &PhysicalCodePage, bus: &dyn BusDevice, phys_b
     const MAX_SEQLOCK_RETRIES: u32 = 8;
     for _ in 0..MAX_SEQLOCK_RETRIES {
         let gen0 = page.current_gen();
+        // A CPU's cache holds lines in the page (`GEN_HELD`).
+        if gen0 >= crate::cpu::jitv2::jitv2::GEN_HELD {
+            return Snapshot::Held(gen0);
+        }
         // A seqlock reader: the words are read after `gen0` and before
         // `gen1` (acquire), and every writer bumps the generation after its
         // bytes, with release. Then a generation that held means these are
@@ -199,11 +205,27 @@ fn prepare_multi_entry_compile(
     // §13.3 step 2: seqlock-snapshot the page bytes.
     let mut words = [0u32; ENTRIES_PER_PAGE];
     let gen_snap = match snapshot_page(page, &**bus, phys_base, &mut words) {
-        Snapshot::Taken(gen) => gen,
+        Snapshot::Taken(gen) => {
+            // The lines are gone: lift the held publish's denials (below).
+            if page.entry_gen() >= crate::cpu::jitv2::jitv2::GEN_HELD {
+                page.allow_all();
+            }
+            gen
+        }
         Snapshot::Unreadable => {
             page.mark_prepare_bounced();
             return PrepareOutcome::Done(false); // page not readable right now; retry later
         }
+        Snapshot::Held(gen) => {
+            // Publish no code for this generation and deny every offset, so
+            // arrivals interpret the page without asking again until the
+            // release moves the generation (`step_jit`'s denylist gate).
+            if page.publish(&[0; crate::cpu::jitv2::BITMAP_WORDS], std::ptr::null(), gen, 0, 0, req.compiled_for_fr1) {
+                page.denylist_all();
+            }
+            page.mark_prepare_bounced();
+            return PrepareOutcome::Done(false);
+        }
         Snapshot::Busy => {
             // A sufficiently hot self-modifying/shared-DMA page can bounce here
             // EVERY single attempt (gen keeps changing faster than
@@ -976,6 +998,26 @@ mod tests {
         assert!(!page.is_denylisted(4));
     }
 
+    /// A page whose counter carries `GEN_HELD` compiles nothing and denies
+    /// every offset, and compiles once the mark comes off.
+    #[test]
+    fn a_held_page_compiles_once_its_lines_go() {
+        let _stable = crate::cpu::jitv2::opcode_support::table_stable();
+        let bus: Arc<dyn BusDevice> = Arc::new(AddiuDevice);
+        let counter = AtomicU64::new(crate::cpu::jitv2::jitv2::GEN_HELD);
+        let mut page = PhysicalCodePage::new(0, &counter as *const AtomicU64);
+        let mut compile = |page: &mut PhysicalCodePage| {
+            page.mark_requested(4);
+            let req = CompileRequest { page: page as *mut PhysicalCodePage, compiled_for_fr1: true };
+            handle_request_for_test(&req, &bus, &mut Analyzer::new(), &mut Codegen::new());
+        };
+        compile(&mut page);
+        assert!(!page.is_runnable(4) && page.is_denylisted(4), "held: interpreted, not asked again");
+        counter.fetch_add(1u64.wrapping_sub(crate::cpu::jitv2::jitv2::GEN_HELD), std::sync::atomic::Ordering::Release);
+        compile(&mut page);
+        assert!(page.is_runnable(4) && !page.is_denylisted(4), "released: compiled");
+    }
+
     #[test]
     fn a_second_incremental_compile_does_not_drop_the_first_entry() {
         // §13's one-function-per-page model fully replaces `func` on every

src/cpu/jitv2/jitv2.rs

@@ -228,6 +228,13 @@ pub fn push_hot_request(hot: &crossbeam_queue::ArrayQueue<CompileRequest>, req:
 /// for this constant.
 pub const PAGE_SIZE: u32 = 4096;
 
+/// Carried by a page's generation counter while some CPU's cache holds
+/// lines there (`PpMemSpace::hold_line_page`): that CPU runs the page from
+/// its own view, so it is not compiled (`comp::snapshot_page`). It comes off
+/// with an ordinary bump, below the held page's last publish, so `publish`
+/// takes any change of generation as new bytes.
+pub const GEN_HELD: u64 = 1 << 48;
+
 /// Number of possible entry offsets per page: one per 4-byte-aligned word
 /// (MIPS instructions are always word-aligned), i.e. `PAGE_SIZE / 4` (§2.4:
 /// `entry_bits` 16×u64 = 1024 bits, `entry_table` 1024 entries).
@@ -1620,6 +1627,11 @@ impl PhysicalCodePage {
         for word in self.denied.iter() { word.store(0, Ordering::Relaxed); }
     }
 
+    /// Undo [`Self::denylist_all`].
+    pub fn allow_all(&self) {
+        for word in self.denied.iter() { word.store(u64::MAX, Ordering::Relaxed); }
+    }
+
     /// Clear `offset_word`'s `compiled` bit only — NOT `denied`. Two
     /// distinct callers:
     ///
@@ -1854,8 +1866,9 @@ impl PhysicalCodePage {
     /// a reader that observes a `compiled` bit set (Acquire-paired via
     /// `is_runnable`'s `entry_gen` load) never observes a stale/null
     /// `func` for it. `entry_gen` is written unconditionally on every actual
-    /// publish, but only **numerically advances** when `snap_gen >
-    /// entry_gen` (a real page-bump-triggered recompile, §13.6/§13.3's
+    /// publish, but only **changes** when `snap_gen != entry_gen` (a real
+    /// page-bump-triggered recompile, or the first after `GEN_HELD` comes
+    /// off, §13.6/§13.3's
     /// discussion) — a pure entry-coverage publish against unchanged bytes
     /// (`snap_gen == entry_gen` already) rewrites the same value. Readers
     /// never need to distinguish the two: `entry_gen == current_gen()` is
@@ -1909,7 +1922,8 @@ impl PhysicalCodePage {
         }
 
         let prev_entry_gen = self.entry_gen.load(Ordering::Relaxed);
-        let is_real_invalidation = snap_gen > prev_entry_gen;
+        // Not `>`: a page's generation drops when `GEN_HELD` comes off.
+        let is_real_invalidation = snap_gen != prev_entry_gen;
         // Code for the other FR mode replaces the old function outright, so
         // the old function's entries must go with it: its dispatch switch is
         // gone, and the new one only has cases for `new_entries`.