ip27: clean up the IP27 machines, and fixes found bringing up the Onyx2

CHANGELOG.md

@@ -109,12 +109,32 @@ hashes are given where a change is easiest to understand by reading the commit.
 - The R10000 family has 44-bit virtual and 40-bit physical addresses, FrameMask and a 28-bit EntryLo PFN, picks the TLB refill vector by address space, invalidates overlapped TLB writes (Status.TS), and completes uncached stores nothing answers. Partial LWR keeps the upper word, and the cache diagnostics (Index Store Data, secondary tag mask, Config size) work.
 - `cpu` takes the parts each machine shipped with, such as `r12000_360` on the Octane, or a family name for the machine's own. The Indy and Indigo2 R4400 is now rev 5.0 at 150 MHz, the O2 R10000 rev 3.4 at 250 MHz, and an Indy no longer takes an R10000 (the R10000 Indigo2 is the IP28). The R12000 and R14000 are new.
 
+### SN0 machines
+
+- The Origin 200, Origin 2000 and Onyx2 each read their own table
+  (`[origin200]`, `[origin2000]`, `[onyx2]`), and a table that is not the
+  profile's own is refused. The module controller logs under a new `module`
+  category (see [debug.md](debug.md)), accesses to node memory that is not
+  fitted log under `hub`, and the Origin 2000 and Onyx2 share one board type.
+- On a two-node Origin 2000 or Onyx2, both processors read both nodes' memory
+  through the ppmem window, so the second node's processor no longer takes the
+  bus path for every cached access and runs as fast as the first.
+
 ### Onyx2
 
 - `machine.profile = "onyx2_ip27"` runs the stock IP27 PROM on one or two Hub
   nodes of an Onyx2 deskside, sharing one Crossbow, with a test widget in the
   graphics slot, and boots an installed IRIX 6.5.30 to a serial login with one
-  node. See [the guide](docs/onyx2-ip27.md).
+  node. Its own keys are in `[onyx2]`, as `[origin200]` and `[origin2000]` hold
+  the other two products'; a table that is not the profile's own is refused.
+  An existing Onyx2 config file that keeps its keys under `[origin2000]` must
+  rename that table to `[onyx2]`; iris refuses it otherwise and names the
+  table to rename. See [the guide](docs/onyx2-ip27.md).
+
+- `[onyx2] io_prom` and `[origin2000] io_prom` fit the I/O PROM image in the
+  BaseIO card's boot flash. The node PROM loads it at "Loading BASEIO prom";
+  without it the PROM runs its own reduced copy, which has no graphics
+  console.
 
 ### Origin 2000
 

FEATURES.md

@@ -175,22 +175,28 @@ the first `[section]`, because TOML tables continue until the next header.
 
 | Key | Default | Values and behavior |
 |---|---|---|
-| `machine.profile` | `indy_ip24` | indy_ip24, indigo2_ip22, indigo2_ip28, o2_ip32 (configured by `[o2]`, see [the O2 guide](docs/o2-ip32.md)), octane_ip30 (configured by `[octane]`, see [the Octane guide](docs/octane-ip30.md)), origin200_ip29 (configured by `[origin200]`, see [the Origin 200 guide](docs/origin200-ip29.md)), origin2000_ip27 (configured by `[origin2000]`, see [the Origin 2000 guide](docs/origin2000-ip27.md)), onyx2_ip27 (also configured by `[origin2000]`, see [the Onyx2 guide](docs/onyx2-ip27.md)). |
+| `machine.profile` | `indy_ip24` | indy_ip24, indigo2_ip22, indigo2_ip28, o2_ip32 (configured by `[o2]`, see [the O2 guide](docs/o2-ip32.md)), octane_ip30 (configured by `[octane]`, see [the Octane guide](docs/octane-ip30.md)), origin200_ip29 (configured by `[origin200]`, see [the Origin 200 guide](docs/origin200-ip29.md)), origin2000_ip27 (configured by `[origin2000]`, see [the Origin 2000 guide](docs/origin2000-ip27.md)), onyx2_ip27 (configured by `[onyx2]`, see [the Onyx2 guide](docs/onyx2-ip27.md)). |
 | `machine.cpu` | the machine's first | A family (`r4400`, `r5000`, `r10000`, `r12000`, `r14000`) or a part (`r14000_600`); independent of build features. Indy and `indigo2_ip22`: r4400, r5000; `indigo2_ip28`: r10000; O2: r5000, r10000, r12000; Octane: r10000, r12000, r14000; Origin 200, Origin 2000 and Onyx2: r10000 (`r10000_180`), r12000 (`r12000_270`). The first listed is the default. |
 | `machine.cpus` | `1` | Processors fitted. 2 only on the Octane; see [its guide](docs/octane-ip30-multiprocessor.md). |
 
-### [origin2000]
+### [origin200], [origin2000] and [onyx2]
 
-Origin 2000 settings apply when `machine.profile = "origin2000_ip27"` or `"onyx2_ip27"`.
+The SN0 machines read one table each, named for the product and holding the
+same keys: `[origin200]` when `machine.profile = "origin200_ip29"`,
+`[origin2000]` for `"origin2000_ip27"` and `[onyx2]` for `"onyx2_ip27"`. A
+table that is not the profile's own is refused, with the name of the one the
+profile reads.
 
 | Key | Default | Values and behavior |
 |---|---|---|
-| `nodes` | `1` | Node boards: 1 or 2. Two are joined by the direct cable. Every node gets the `banks` memory. |
-| `flash` | unset | Persistent 1 MiB node boot-flash file of the first node; the second keeps its own at the name with `.1` added. Unset, the flash is volatile. |
+| `nodes` | `1` | Node boards on a deskside: 1 or 2. Two are joined by the direct cable. Every node gets the `banks` memory. The Origin 200 has one. |
+| `flash` | unset | Persistent 1 MiB node boot-flash file (of the first node on a deskside; the second keeps its own at the name with `.1` added). Unset, the flash is volatile. |
 | `controller_store` | unset | Persistent file for the module controller's configuration store. Unset, it is volatile. |
+| `io_prom` | unset | Deskside only. The BaseIO card's I/O PROM image (`io6prom.img`, beside the node PROM on the IRIX 6.5.30 media), fitted in its boot flash. Unset leaves the flash unfitted and the node PROM runs its own reduced copy, which has no graphics console. |
 
-An `[origin2000]` table is written to a saved config only when one of these
-keys is set. See the [Origin 2000 guide](docs/origin2000-ip27.md).
+A table is written to a saved config only when one of its keys is set. See the
+[Origin 200](docs/origin200-ip29.md), [Origin 2000](docs/origin2000-ip27.md) and
+[Onyx2](docs/onyx2-ip27.md) guides.
 
 ### [graphics]
 

HACKING.md

@@ -380,9 +380,11 @@ consumer thread never takes `rec`.
   per-model branch on the hot path. `Machine::new` picks one from `[machine]
   cpu`.
 - Every uncached SB/SH goes to the bus through `BusDevice::write_narrow_from`
-  with the whole source register: the byte or halfword is the low bits, the
-  other lanes are undefined by hardware, and only Bedrock's ECC view reads
-  them.
+  with the whole source register: the byte or halfword is the low bits and
+  the other lanes are undefined by hardware. The default delivers the low
+  bits, and no device in this tree overrides it; one that models a path
+  observing the other lanes (a memory ECC view) would. The call is a `dyn`
+  call in place of the `write8`/`write16` one.
 - With `opcodefusion` (implied by `lightning`), the interpreter executes
   common pairs (branch + NOP, LUI + ORI/ADDIU, address + load/store) as one
   dispatch.

README.md

@@ -148,7 +148,7 @@ Hardware, testing, and development references:
 
 | Area | Documentation |
 |---|---|
-| Machine profiles | [Indigo2 IP22](docs/indigo2-ip22.md), [Indigo2 IMPACT IP28](docs/indigo2-ip28.md), and [GR2 XZ/Extreme](docs/graphics/gr2-xz-extreme.md), the [SGI O2 (IP32)](docs/o2-ip32.md), and the [SGI Octane (IP30)](docs/octane-ip30.md) with its [address map](docs/octane-ip30-address-map.md). |
+| Machine profiles | [Indigo2 IP22](docs/indigo2-ip22.md), [Indigo2 IMPACT IP28](docs/indigo2-ip28.md), and [GR2 XZ/Extreme](docs/graphics/gr2-xz-extreme.md), the [SGI O2 (IP32)](docs/o2-ip32.md), the [SGI Octane (IP30)](docs/octane-ip30.md) with its [address map](docs/octane-ip30-address-map.md), and the SN0 machines: [Origin 200](docs/origin200-ip29.md), [Origin 2000](docs/origin2000-ip27.md) and [Onyx2](docs/onyx2-ip27.md). |
 | Graphics and peripherals | [virtio-serial](docs/dev/virtio-serial.md) (guest byte channels, Indy/Indigo2/Octane), [REX3](docs/dev/rex3.md), [HAL2 audio](docs/dev/hal2.md), [WD33C93A SCSI](docs/dev/wd33c93a.md), [QLogic ISP1040 SCSI (Octane)](docs/dev/isp1040.md), [IOC3 PS/2 keyboard and mouse (Octane)](docs/dev/ioc3-ps2.md), [interrupt map](docs/interrupt_map.md), and [Ultra64](docs/dev/ultra64.md) / [N64 GIO specifications](docs/dev/ultra64-gio.md). |
 | CPU test suite | [Suite README](cpu-tests/README.md), [current status](cpu-tests/docs/status.md), [findings](cpu-tests/docs/findings.md), [gotchas](cpu-tests/docs/gotchas.md), and [emulator support](docs/cpu-test-support.md). |
 | CPU test infrastructure | [Toolchain](cpu-tests/docs/toolchain.md), [memory map](cpu-tests/docs/memory-map.md), [oracle overview](cpu-tests/docs/oracle.md), [hardware oracle](cpu-tests/oracle/README.md), [R4600 notes](cpu-tests/docs/r4600.md), and [original test plan](cpu-tests/PLAN.md). |

clanker/INDEX.md

@@ -104,6 +104,7 @@ Non-graphics devices: MC/MEMCFG and RAM banks, HPC3, VDMA, EEPROMs and NVRAM, PS
 | Note | Kind | Title |
 |---|---|---|
 | [pci-layer-design.md](devices/pci-layer-design.md) | design | The PCI layer: headers change only through the bus, byte orders are per-window values, `pci/` imports no card or SCSI; the stale-run and delivery-order hazards, the SeqCst handshake, and what the first attempt got wrong |
+| [sn0-node-zero-and-bridge-hosts.md](devices/sn0-node-zero-and-bridge-hosts.md) | rule | SN0 machines: a renumbered node sends node zero's cached window beyond its alias to the Hub that holds zero (and loses the ppmem fast path); a Bridge behind a Crossbow reaches the host at the widget the guest programs, gated by that port's link |
 | [pci-layer-open.md](devices/pci-layer-open.md) | open | PCI layer open items: restore vs reset and the AIC-7880 chip reset, the O2's RST#, Bridge reset pins and master time-out approximations, the 2 GB window's reserved range, no external ATE SSRAM, MACE 64-bit PIO |
 | [virtio-serial-design.md](devices/virtio-serial-design.md) | design | virtio-serial: one transport-neutral core, mmio in a GIO slot (IP22/24/28) and pci (IP30); threading, byte order and open items |
 | [empty-gio-slot-reads-bus-error.md](devices/empty-gio-slot-reads-bus-error.md) | rule | Reads of an empty GIO slot end in a bus error (ABORT_EN); all ones made the PROM poll a phantom Newport 200 000 times headless |

clanker/devices/sn0-node-zero-and-bridge-hosts.md

@@ -0,0 +1,87 @@
+# SN0 machines: node zero's window and a Bridge's host (rule)
+
+Two rules about the Hub machines (Origin 200, Origin 2000, Onyx2) that are easy
+to get wrong, because each looks like something the code could simply assume.
+
+## A renumbered node reroutes its cached references
+
+Cached references use the node's `Physical` directly: RAM at node-relative
+offsets, with the ppmem window the cache and the JIT read. That is right while
+the node holds node number zero, which is the one-node Origin 200 and the first
+node of a deskside.
+
+The stock PROM numbers the second node 1 after reset. A cached reference from
+that node's processor to its own memory may still name node zero, and then the
+reference is not the node's own: it belongs to whichever Hub holds zero.
+`NodeZero` (`src/ip27.rs`) applies this when the node's number or its cached
+alias size changes:
+
+- Within the cached alias (`PI_CALIAS_SIZE`) the reference stays on the node.
+- Beyond it, node zero's window is sent through the Hub's node-qualified memory
+  to the Hub that holds zero, across the cable.
+- The node's own memory is then reached by its own number.
+
+The ppmem window, which the cache and the JIT read without the bus, must show
+the same memory at the same addresses, or the renumbered node's processor takes
+the bus path for every cached reference (about 2.2 times as long for a
+CPU-bound loop when only the first node had a window). `Windows`
+(`src/ip27.rs`) lays every node's window from the numbers the Hubs publish,
+whenever any node's number or alias changes:
+
+- node zero's window as above, from the node's own banks or the zero holder's;
+- each node's memory at its number (`n << 32`), its own and the other's;
+- only the difference, so a page that stays mapped is replaced in one host
+  mapping call and a renumbering never clears the window a running processor
+  executes from.
+
+A store to any Hub's number or alias publishes the numbers (`Routing`) and
+lays every node's window at once, from the storing thread: the hardware takes
+the new number at once, so no window trails the decode table. The other node's
+processor can be running meanwhile, and its CACHE instructions hold and release
+line pages in the same `PpMemSpace`. The space has one small mutex, taken by
+every remap (`map_bank`, `map_alias`, `map_range`, `unmap_range`,
+`clear_mappings`) and by `add_cached_view`, `hold_line_page` and
+`release_line_page`. A load, store or fetch never takes it: they read through
+the processor's inline bitmap and its view's base pointer. Without it a
+renumber rebuilds the mapping list under a `hold_line_page` that is reading it
+(`a_renumber_on_one_processor_does_not_disturb_the_others_line_pages`). Lock
+order is `Windows`, then the space.
+
+A cabled board's window spans the node numbers up to the next power of two
+above its node count (8 GiB for two nodes), reserved as inaccessible address
+space that commits nothing. A number past it stays on the bus path. Its
+one-word region bitmap, the test on every load, store and fetch, counts 64
+regions of 128 MiB, numbers 0 and 1. A region it does not hold whole, a 64 or
+256 MiB bank leaves half of each, or another number's, is found in the window's
+fine table (`PpMemSpace::fine_table`, one bit per 64 MiB) on the bitmap's miss
+branch, before the bus. The decode cache keeps window pointers for both, so it
+keys them on a value the space moves with every change of either
+(`add_key_sink`), not on the bitmap. Do not add a decision to the hit path for
+any of this: the rare state change lays a different mapping.
+
+Limit: a CPU's lines are private to it (`hold_line_page`) only for pages the
+bitmap names, its first 64 regions. A page past them returns `None`, so on a
+three- or four-node board the lines of node numbers 2 and 3 are not modelled as
+the CPU's own, and a store there is visible to the other CPUs at once. Numbers
+0 and 1, which the stock PROM gives, are covered. Nothing observed depends on
+it.
+
+## A Bridge behind a Crossbow picks its host by the widget the guest programs
+
+A Bridge sends DMA and interrupt packets to a host chip addressed by widget
+number. Which widget is the guest's to program (`DIRECT_MAP` and `INT_DEST` in
+the Bridge's registers), not the board's:
+
+- On the Onyx2 both node boards are hosts of the one Crossbow. The Bridge's
+  `Upstream` holds a `HostPort` for each, at its widget (10 and 9), and the stock
+  PROM programs the widget of the board it makes the global master. DMA and
+  interrupts go to that Hub.
+- On the Origin 2000 the Bridge sits behind the first node's own Crossbow and
+  reaches only that node's Hub, at widget 9.
+- Behind a Crossbow, `Xbow::upstream` gates each host on its port's link, so a
+  packet to a host with no live link is dropped where the Crossbow drops it. The
+  Origin 200 has no Crossbow, so its host is not gated.
+
+Build the upstream from `Hub::host_port(widget)` for every Hub the Bridge can
+reach, never with a fixed host. `Upstream::get(widget)` is one indexed load, so
+nothing is searched per transfer.

debug.md

@@ -594,7 +594,11 @@ Per-module logging; all of it needs a developer build to produce output.
 | `log <module> file <path\|off>` | Send a module's log to a file |
 
 Modules: `net hpc3 seeq hal2 mc rex3 mips ioc scsi pdma vino dcb vc2 cmap
-xmap bt445 scc ps2 rtc eeprom l1i l1d l2c bedrock pci bridge heart nvme`. `pci` logs every
+xmap bt445 scc ps2 rtc eeprom l1i l1d l2c bedrock hub module pci bridge heart nvme`. `hub`
+logs the SN0 Hub's unsupported register accesses and accesses to unfitted
+node memory. `module` logs the module controller's refused commands and
+protocol surprises (the Origin 200's ELSC and the Origin 2000 module's
+controller, with their NVRAM and I2C controller). `pci` logs every
 configuration cycle on a PCI bus (MACE's, the Octane Bridge's) and
 whether a function answered it. `bridge` logs the Octane Bridge's
 register and translation-table accesses by name, and any transfer a PCI

docs/onyx2-ip27.md

@@ -9,9 +9,14 @@ identification word and nothing else.
 
 ## Configuration
 
-Use [`origin2000.toml.example`](../origin2000.toml.example) with
-`machine.profile = "onyx2_ip27"`; `[origin2000]` (`nodes`, `flash`,
-`controller_store`) configures this machine too. Node slot 1 drives raw slot
+Copy [`onyx2.toml.example`](../onyx2.toml.example) to `onyx2.toml`, provide the
+PROM and disk named there, and run `iris --config onyx2.toml`. The sample
+selects `machine.profile = "onyx2_ip27"`; `[onyx2]` (`nodes`, `flash`,
+`controller_store`, `io_prom`) holds its own keys. `io_prom` names the I/O PROM
+image the IRIX 6.5.30 media ship beside the node PROM, fitted in the BaseIO's
+boot flash; the graphics console needs it, since the node PROM's own internal
+copy has none. A file that still names `[origin2000]`
+for an Onyx2 is refused, with the table to rename it to. Node slot 1 drives raw slot
 identity 3 into its Hub and takes widget 10; slot 2 drives 2 and takes widget 9.
 With one node, the board is in slot 1.
 
@@ -29,7 +34,10 @@ A disk installed on an Onyx2 boots; the machine cannot install without a pipe.
   it, and both Hubs are hosts, each behind its Crossbow port's link. With two
   nodes the PROM and IRIX program widget 9, the master's Hub; with one, widget
   10.
-- The BaseIO-G is its base board: no MediaIO or audio.
+- The BaseIO-G carries the MediaIO daughtercard: a second IOC3 at PCI device 6
+  and the RAD1 audio chip at device 7, with the daughtercard's own board
+  record. The MediaIO's serial ports have no host backend, and the host's keys
+  and pointer go to the board IOC3's controller only.
 - The module controller, the cable and the node windows are the
   [Origin 2000 deskside's](origin2000-ip27.md), with its limits.
 - Snapshots are not supported.

docs/origin200-ip29.md

@@ -21,7 +21,10 @@ The sample selects `machine.profile = "origin200_ip29"`.
   module's, so its part is blank.
 - `banks` are the four DIMM-pair banks in MB (64, 128, 256 or 512), filled
   from bank 0. A bank of 128 MB or less is one run at its 512 MiB stride; a
-  larger one is four runs at 128 MiB intervals.
+  larger one is four runs at 128 MiB intervals. A 512 MB bank 0 holds the boot
+  flash's reset address: cached references reach the bank, and the reset
+  fetch, which is uncached, reaches the flash. Memory that is not fitted
+  answers every access with a bus error, logged under `hub`.
 - `[origin200] flash` keeps the 1 MiB node boot flash, where the PROM stores
   its environment log. Unset, the flash starts fresh every run.
   `controller_store` does the same for the module controller's store.
@@ -35,8 +38,9 @@ boots to a serial login.
 
 ## Limits
 
-- One node and one processor. A second module, the GIGAchannel cabinet and
-  the Origin 2000 are not built.
+- One node and one processor. A second module and the GIGAchannel cabinet
+  are not built. The [Origin 2000](origin2000-ip27.md) and
+  [Onyx2](onyx2-ip27.md) desksides are separate profiles.
 - Cached accesses use node-relative addresses, which is right for the node
   number zero a one-node machine has.
 - The module controller's NMI output and the Hub's per-processor enable and