diff --git a/CHANGELOG.md b/CHANGELOG.md index ec0485dbe..432e5f1fd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,6 +33,24 @@ hashes are given where a change is easiest to understand by reading the commit. boot: the century IRIX writes while SET is up restarted the clock from the old time. The DS1685/DS1687 model now holds a century written under SET until SET clears, as the part does, and snapshots keep it. +### SGI Fuel (IP34 board, IP35 software platform) + +- Added a headless Fuel profile, `fuel_ip34`: the 500 MHz R14000, 512 MiB + of RAM, Bedrock, two XBridge PCI heads, IOC3 serial ports, PS/2 and + Ethernet, a two-channel ISP12160, a Lucent OHCI USB controller and the + L1 system controller. PROM 6.210 passes its diagnostics and IRIX 6.5.30 + boots to the login on the L1 or an IOC3 serial console, and `halt -p` + powers it off. See the [Fuel guide](docs/fuel-ip34.md). +- **The Fuel takes a VPro board**: `[graphics] board = "v12"` or `"v10"` + fits the Octane's VPro model at crossbar widget 13. IRIX 6.5.30 with the + VPro driver brings the X login screen up on it. +- **jitv2 compiles the reset PROM from the PROM.** The PROM runs from the reset + window uncached; once memory covers that window the cached view returns RAM + there, and the compile pool used to compile RAM's bytes in place of the + PROM's. The board now presents an uncached instruction fetch of that window at + an address of the PROM's own (`MipsExecutor::uncached_fetch_alias`), so its + code page is keyed, compiled and invalidated by address like any other. The + other machines are unchanged. ### Benchmark and test suites: one pinned compiler @@ -83,6 +101,7 @@ hashes are given where a change is easiest to understand by reading the commit. ### R10000-family cache lines as memory - A valid primary data or secondary cache line now holds its page in that CPU's own copy of ppmem's window, so a PROM that keeps its stack in cache lines reads and writes the lines with cached accesses while the uncached path, DMA and the other CPUs see memory. Index Store Data writes through to such a line. `l2 probe 0` lists the pages held. Nothing changes on a load, store or fetch. A valid R5000 primary data line does the same, for the O2's POST stack. +- Primary instruction lines hold their pages too, each word written there as the instruction it predecodes, for the eleven predecode classes the model knows. Index Load Data (I) where a held secondary line would refill the set returns that line's words, predecoded, until Index Store Data (I) writes the set. Only the Tezro's PROM uses them. - jitv2 does not compile a page any CPU's cache holds lines in: that CPU runs it from its own view in the interpreter, and every other page stays compiled. - The O2 keeps POST's stack in those lines. CRIME's side page for the uncached path and the tag-store hook are gone, and an O2 snapshot an older iris took during POST is refused. @@ -90,6 +109,7 @@ hashes are given where a change is easiest to understand by reading the commit. - The R10000 family has 44-bit virtual and 40-bit physical addresses, FrameMask and a 28-bit EntryLo PFN, picks the TLB refill vector by address space, invalidates overlapped TLB writes (Status.TS), and completes uncached stores nothing answers. Partial LWR keeps the upper word, and the cache diagnostics (Index Store Data, secondary tag mask, Config size) work. - `cpu` takes the parts each machine shipped with, such as `r12000_360` on the Octane, or a family name for the machine's own. The Indy and Indigo2 R4400 is now rev 5.0 at 150 MHz, the O2 R10000 rev 3.4 at 250 MHz, and an Indy no longer takes an R10000 (the R10000 Indigo2 is the IP28). The R12000 and R14000 are new. +- The R16000 is a family of its own, with four parts for the Tezro (`r16000_700` rev 2.1, `r16000_800` rev 2.2, `r16000_900` and `r16000_1000` rev 3.0) and a 4, 8 or 16 MB secondary cache picked once when a machine is built, so no run-time field or branch is added. `hinv` prints an R16000's major revision two below the PRId register's, and the part table undoes that in one place. The FIR is 0x0900, the one R14000 value on record; a model choice. No machine takes them yet. ### Onyx2 @@ -490,6 +510,14 @@ where they differed: the cost follows what the guest wrote. Reflinking is unchanged. On Windows the copy is marked sparse first. +### SGI Tezro (IP53 board, IP35 software platform) + +- Added a headless Tezro profile with the 700 MHz R16000 (`r16000_700`, the default; `cpu = "r16000"` takes it too), 4 MiB cache, 4 GiB RAM, a PIC with PXBOW, and the IO9's IOC4, ISP12160 and BCM5701. The 800, 900 and 1000 MHz chips (`r16000_800`, `r16000_900`, `r16000_1000`) are listed and take four processors; see the [Tezro guide](docs/tezro-ip53.md). +- A Tezro can fit a VPro V12 at widget 12 with `[graphics] board = "v12"`, on the VPro model the Octane and the Fuel use. +- A Tezro fits one, two or four processors, four by default, and two or four bring a second PIC at widget 11 on the PXBOW's link 11. Every fitted processor runs, each on its own host thread, so a default four-processor Tezro runs four. +- The Tezro's L1 names its node board for the processor count (`IP53_1CPU`, `IP53_2CPU`, `IP53_4CPU`), so the PROM tries only the fitted processor positions and a one- or two-processor Tezro's first boot does not power-cycle the module for its empty ones. +- The Tezro's PIC fails a configuration read of an empty slot as a master timeout, so the PROM no longer records two phantom devices per empty slot. + ### `IRIS_GUI_GL` works in iris-gui - The GUI's GL capture path (`IRIS_GUI_GL=1`, or `[debug] gui_gl_capture`) never diff --git a/FEATURES.md b/FEATURES.md index 9b7b81fdd..e16ddc441 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -175,9 +175,9 @@ the first `[section]`, because TOML tables continue until the next header. | Key | Default | Values and behavior | |---|---|---| -| `machine.profile` | `indy_ip24` | indy_ip24, indigo2_ip22, indigo2_ip28, o2_ip32 (configured by `[o2]`, see [the O2 guide](docs/o2-ip32.md)), octane_ip30 (configured by `[octane]`, see [the Octane guide](docs/octane-ip30.md)), origin200_ip29 (configured by `[origin200]`, see [the Origin 200 guide](docs/origin200-ip29.md)), origin2000_ip27 (configured by `[origin2000]`, see [the Origin 2000 guide](docs/origin2000-ip27.md)), onyx2_ip27 (also configured by `[origin2000]`, see [the Onyx2 guide](docs/onyx2-ip27.md)). | -| `machine.cpu` | the machine's first | A family (`r4400`, `r5000`, `r10000`, `r12000`, `r14000`) or a part (`r14000_600`); independent of build features. Indy and `indigo2_ip22`: r4400, r5000; `indigo2_ip28`: r10000; O2: r5000, r10000, r12000; Octane: r10000, r12000, r14000; Origin 200, Origin 2000 and Onyx2: r10000 (`r10000_180`), r12000 (`r12000_270`). The first listed is the default. | -| `machine.cpus` | `1` | Processors fitted. 2 only on the Octane; see [its guide](docs/octane-ip30-multiprocessor.md). | +| `machine.profile` | `indy_ip24` | indy_ip24, indigo2_ip22, indigo2_ip28, o2_ip32 (configured by `[o2]`, see [the O2 guide](docs/o2-ip32.md)), octane_ip30 (configured by `[octane]`, see [the Octane guide](docs/octane-ip30.md)), origin200_ip29 (configured by `[origin200]`, see [the Origin 200 guide](docs/origin200-ip29.md)), origin2000_ip27 (configured by `[origin2000]`, see [the Origin 2000 guide](docs/origin2000-ip27.md)), onyx2_ip27 (also configured by `[origin2000]`, see [the Onyx2 guide](docs/onyx2-ip27.md)), fuel_ip34 (configured by `[fuel]`, see [the Fuel guide](docs/fuel-ip34.md)), tezro_ip53 (configured by `[tezro]`, see [the Tezro guide](docs/tezro-ip53.md)). | +| `machine.cpu` | the machine's first | A family (`r4400`, `r5000`, `r10000`, `r12000`, `r14000`) or a part (`r14000_600`); independent of build features. Indy and `indigo2_ip22`: r4400, r5000; `indigo2_ip28`: r10000; O2: r5000, r10000, r12000; Octane: r10000, r12000, r14000; Origin 200, Origin 2000 and Onyx2: r10000 (`r10000_180`), r12000 (`r12000_270`); Fuel: r14000 (`r14000_500`); Tezro: r16000 (a bare `r16000` is `r16000_700`; `r16000_800`, `r16000_900` and `r16000_1000` are the other parts it ships with). The first listed is the default. | +| `machine.cpus` | `1`; `4` on the Tezro | Processors fitted. 2 on the Octane; see [its guide](docs/octane-ip30-multiprocessor.md). 1, 2 or 4 on the Tezro, each run on its own host thread; one selects the one-processor board with one PIC, two or four the board with two PICs. | ### [origin2000] @@ -192,11 +192,50 @@ Origin 2000 settings apply when `machine.profile = "origin2000_ip27"` or `"onyx2 An `[origin2000]` table is written to a saved config only when one of these keys is set. See the [Origin 2000 guide](docs/origin2000-ip27.md). +### [fuel] + +Fuel settings apply when `machine.profile = "fuel_ip34"`. + +| Key | Default | Values and behavior | +|---|---|---| +| `flash` | unset | Persistent 16 MiB boot-flash file. Unset, the flash is volatile. | +| `console` | `l1` | `l1`, `ioc3a` or `ioc3b`. Selects the guest console connection. | +| `l1_port` | `8882` | Host TCP port for the L1 console. Used when `console = "l1"`. | +| `l1_log` | unset | Optional serial log path for L1 output when an IOC3 port is selected as console. | +| `ps2_input` | `true` | Fit the IOC3's PS/2 keyboard and mouse and feed them host input. | +| `l1_revision` | `1.48.1` | L1 firmware revision as `major.minor.patch`, decimal. The guest reads it with L1 request `0x0005`. | +| `brick_serial` | `EMUN00` | L1 brick serial, at most 37 bytes. The guest reads it with L1 request `0x0004`. The default is a made-up serial, not a real one. | + +The machine has one 500 MHz R14000 and 512 MiB of RAM in two 256 MiB +banks, and runs on a serial console. A `[fuel]` table is written to a +saved config only when one of these keys is set. See the +[Fuel guide](docs/fuel-ip34.md). + +### [tezro] + +Tezro-specific settings apply when `machine.profile = "tezro_ip53"`. + +| Key | Default | Values and behavior | +|---|---|---| +| `flash` | unset | Persistent 16 MiB boot-flash file. Unset, the flash is volatile. | +| `console` | `l1` | `l1`, `ioc4_a`, `ioc4_b`, `ioc4_c` or `ioc4_d`. Selects the guest console connection. | +| `l1_port` | `8882` | Host TCP port for the L1 console. | +| `l1_log` | unset | Optional serial log path for L1 output when an IOC4 port is selected as console. | +| `serial_port_c`, `serial_port_d` | `8883`, `8884` | Host TCP ports of IOC4 ports 2 and 3; ports 0 and 1 use `serial_port_a` and `serial_port_b`. | + +The machine runs the 700 MHz R16000 (`r16000_700`, the default), or the 800, 900 +or 1000 MHz one (`r16000_800`, `r16000_900`, `r16000_1000`, with 4, 8 and 16 MiB +secondary caches; those take four processors). `banks` +takes 512 or 1024 MiB banks from bank 0; left unset, it is the 4 GiB layout of +1024, 1024 and four 512 MiB banks. A `[tezro]` table is written to a saved +config only when one of these keys is set. See the +[Tezro guide](docs/tezro-ip53.md), which lists its limits. + ### [graphics] | Key | Default | Values and behavior | |---|---|---| -| `graphics.board` | `newport`; `solidimpact` on the Octane | newport, xz, extreme, solidimpact, highimpact, maximpact, v6, v8, v10, v12, none. Extreme is Indigo2 only. The Octane takes only `solidimpact` (SI), `highimpact` (SI with texture option), `maximpact` (MXI), `v6`, `v8`, `v10`, `v12` (the VPro boards, also `vpro:v12` and so on; need an IRIX install with the VPro driver, IRIX 6.5.10 or later and PROM 4.0, or 4.5 for the V10 and V12, which shipped only in the Octane2 and so give the machine the Octane2's Crossbow 1.4 and Bridge revision) or `none` (no board: the machine runs on its serial console, with no window). | +| `graphics.board` | `newport`; `solidimpact` on the Octane | newport, xz, extreme, solidimpact, highimpact, maximpact, v6, v8, v10, v12, none. Extreme is Indigo2 only. The Octane takes only `solidimpact` (SI), `highimpact` (SI with texture option), `maximpact` (MXI), `v6`, `v8`, `v10`, `v12` (the VPro boards, also `vpro:v12` and so on; need an IRIX install with the VPro driver, IRIX 6.5.10 or later and PROM 4.0, or 4.5 for the V10 and V12, which shipped only in the Octane2 and so give the machine the Octane2's Crossbow 1.4 and Bridge revision) or `none` (no board: the machine runs on its serial console, with no window). The Fuel takes `v12` or `v10` in its graphics slot, or `none`, its default. The Tezro takes `v12` at widget 12, or `none`, its default. | | `graphics.heads` | `1` | 1 or 2; dual-head requires Newport. | | `graphics.resolution` | `guest` | guest, 1024x768, 1280x960, 1280x1024; presets require Newport. | | `graphics.threads` | `2` | Worker threads the board's fragment pipeline draws on, at least 1. VPro: the screen tiles' owners, besides its front-end and raster threads; more help on a host with cores to spare. | @@ -231,7 +270,7 @@ An explicit non-disk kind takes precedence over the legacy cdrom flag. | Key | Default | Values and behavior | |---|---|---| -| `network.mode` | `nat` | nat, pcap, stream or off (no Ethernet; O2 and Octane only). Stream needs `network.stream`. PCAP needs build support and host capture permissions. PCAP works on every Ethernet; stream on every onboard one, not on a DaynaPort. | +| `network.mode` | `nat` | nat, pcap, stream or off (no Ethernet; O2, Octane and Fuel only). Stream needs `network.stream`. PCAP needs build support and host capture permissions. PCAP works on every Ethernet; stream on every onboard one, not on a DaynaPort. | | `network.stream` | `unset` | `tcp:IP:PORT` or `unix:PATH`: the server of the private segment for `mode = "stream"` (QEMU `-netdev stream` framing). See NETWORKING.md. | | `network.pcap_interface` | `auto-pick when unset` | Interface name or numeric list index; PCAP only. | | `network.nfs_pcap_ip` | `unset` | Virtual LAN IP for the built-in NFS responder in PCAP mode. | @@ -434,8 +473,8 @@ IP22/IP24 four-bank IRIX 6.5 behavior remains separate from IP28's verified two-bank 1 GB layout. Extreme requires IP22. All non-Newport boards require `heads=1` and `resolution="guest"`. IP28 IRIX needs IMPACT; its PROM has an embedded fallback. -The config validator permits SCSI controller 1 only on IP22, the O2 and the -Octane, the machines with two controllers. +The config validator permits SCSI controller 1 only on IP22, the O2, the +Octane and the Fuel, the machines with two controllers. Standalone CLI options update the loaded config where supplied; boolean enable switches do not provide a general way to clear every saved true setting. @@ -488,7 +527,7 @@ iris-gui does not parse this standalone option set; use its named-machine UI. | `--list-net-interfaces` | Print the host network interfaces libpcap can bridge onto, then exit. Requires a build with --features pcap. | | `--no-scsi-deferred-int` | Disable deferred SCSI status interrupts (default: enabled for OpenBSD/NetBSD compatibility). | | `--cpu` | Emulated CPU: a family (`r4400`, `r5000`, `r10000`, `r12000`, `r14000`) or a part (`r14000_600`) the machine takes. Overrides `[machine] cpu`. | -| `--graphics` | Graphics board: `newport` (default), `xz`, `extreme`, `solidimpact`, `highimpact`, `maximpact` (or `impact:solid`, `impact:high`, `impact:max`), `v6`, `v8`, `v10`, `v12` (`vpro:v12` and so on, Octane only). | +| `--graphics` | Graphics board: `newport` (default), `xz`, `extreme`, `solidimpact`, `highimpact`, `maximpact` (or `impact:solid`, `impact:high`, `impact:max`), `v6`, `v8`, `v10`, `v12` (`vpro:v12` and so on, Octane; `v10` and `v12` on the Fuel). | | `--raster` | VPro and IMPACT: `cpu` (default) or `wgpu`, the raster unit on the host GPU (`[graphics] raster`; needs a build with the `wgpu` feature). | | `--renderer` | What draws the window: `gl` (default) or `wgpu` (`renderer`; needs a build with the `wgpu` feature). | | `--gdb-port` | Enable GDB stub on the given TCP port (e.g. --gdb-port 1234). Connect with: target remote localhost: | diff --git a/HACKING.md b/HACKING.md index 38b6eb5e2..51908596e 100644 --- a/HACKING.md +++ b/HACKING.md @@ -16,13 +16,15 @@ Other documents: ## 1. What IRIS is -IRIS emulates five SGI workstations: the Indy (IP24), the Indigo2 (IP22), the -Indigo2 IMPACT (IP28), the O2 (IP32) and the Octane (IP30). The CPU is an -R4400, R5000, R10000, R12000 or R14000, chosen at runtime from the parts the -machine shipped with (`src/cpu/parts.rs`). -Graphics is Newport (REX3), GR2 (XZ/Extreme) or IMPACT on the Indy and -Indigo2, GBE on the O2 and Solid IMPACT on the Octane. It boots the real PROMs -and IRIX 5.3 and 6.5 to a desktop with networking. +IRIS emulates seven SGI workstations: the Indy (IP24), the Indigo2 (IP22), +the Indigo2 IMPACT (IP28), the O2 (IP32), the Octane (IP30), the Fuel (IP34 +board, IP35 software platform) and the Tezro (IP53 board, IP35 software +platform). The CPU is an R4400, R5000, R10000, R12000, R14000 or R16000, +chosen at runtime from the parts the machine shipped with +(`src/cpu/parts.rs`). Graphics is Newport (REX3), GR2 (XZ/Extreme) or IMPACT +on the Indy and Indigo2, GBE on the O2 and Solid IMPACT on the Octane; the +Fuel and the Tezro take a VPro or run headless on their serial consoles. It +boots the real PROMs and IRIX 5.3 and 6.5 to a desktop with networking. It is **not cycle-accurate**, anywhere. IRIX is interrupt-driven and does not care how many cycles an instruction takes, so IRIS models behaviour @@ -101,6 +103,12 @@ the machine supplies only a route from controller number to bus. Flash command engines share `FlashBacking` file storage while retaining their own command state. +`ScsiTarget::bus_reset` defaults to no action. Disk targets record reset as +unit attention. Only the ISP12160 calls it; the other controllers keep +their earlier behaviour of not reporting resets to targets. +The ISP12160 releases its controller state lock before walking either bus. +Target callbacks run under that bus lock and must not reacquire it. + ## 2. Data path and endianness **Word-transparent architecture.** Host `u32`/`u64` values are bit @@ -169,6 +177,9 @@ screenshots), `CpuDevice` (what `Machine` holds for the CPU). Serial lines have two halves. A UART model owns the registers: the Indy's `Z85c30`, and `Ns16550` (`src/dev/ns16550.rs`), the 16550A core that the O2's `MaceUart` (lane decode, interrupt line, monitor) and the Octane's IOC3 wrap. +The IOC4's serial ring-DMA engine is its own module, `ioc_serial_dma.rs`, which +the chip drives through its `SerialDmaHost` trait (its 16550s, interrupt bits and +bus master); the IOC3 keeps its own copy for now. The host end is a `SerialBackend` from `src/dev/serial_host.rs`, which the machine constructors pick with `console_backends`: the CI backend, a TCP console, or a log tee. A UART's transmit is instant and its receive side @@ -305,9 +316,14 @@ Every device that has real work to do runs on its own thread: | `VNC-Grab`, `VNC-Pointer`, `VNC-Client`/`VNC-Read` | with `--vnc`: frame diffing, pointer steering, one pair per viewer | | `machine-events` | power-off and reset requests from devices | | `ISP1040-0`, `ISP1040-1` | Octane SCSI request rings and disk transfers | -| `IOC3-RX-A`, `IOC3-RX-B`, `IOC3-NVRAM` | Octane serial input per port; the DS1685 NVRAM file writer (IOC3's other deadlines are `hptimer` one-shots) | +| `ISP12160-0` | Fuel SCSI request ring for both ISP12160 channels | +| `IOC3-RX-A`, `IOC3-RX-B`, `IOC3-NVRAM` | Octane and Fuel serial input per port; the DS1685 NVRAM file writer (IOC3's other deadlines are `hptimer` one-shots) | +| `l1-serial` | Fuel and Tezro L1 console UART worker | +| `IOC4-RX-0` to `IOC4-RX-3` | Tezro IOC4 serial input | +| `bcm5701-rx`, `bcm5701-net` | Tezro BCM5701 receive pump and network backend | | `ip30-eth-rx`, `ip30-eth-net` | Octane ethernet receive pump; the network backend | | `odyssey-work`, `odyssey-raster`, `odyssey-pool-N`, `odyssey-display` | VPro: the front end decodes the queued command-FIFO stores (the processor's stores only queue); the raster unit sets each draw up and walks its primitives into 2x2 quads; a pool of workers (`[graphics] threads`, default 2) walks, shades and stores filled triangles and stores every other fragment, one screen tile at a time; and the display composes each whole frame from board memory (64-bit words, as main memory is: a field is a bit range of its word, read by shift and mask with no byte order, `vram.rs`) as the scan-out registers describe it, counts the retrace and presents the window at 60 Hz. No board-wide lock: the front end owns its state, an access that needs the drawing is answered busy until it is done, the display FIFO (PB&J's register file) has its own lock and never waits for the drawing (`display_fifo.rs`), and the display reads board memory while it is drawn, as the board's own scan-out does; nobody tracks which rows changed. The raster thread sets primitives up and writes each as one command in a ring of 64-byte slots (`pipe/ring.rs`): a triangle's record and bounding box, a quad it shaded itself (lines, points), or a clear's rectangle; the slot's index goes on the queue of every screen tile (`[graphics] tile`, default 32 pixels) it touches. A worker takes a ready tile and owns it until its queue is empty, walking each triangle within the tile and shading and storing quad by quad, so every pixel sees its fragments in command order with no other ordering, and the depth pre-test reads depth its own tile stored. A slot is retired once all its tiles have run it. The raster thread drains the ring before any other work on board memory (a texture upload waits for the shader alone). A completion (a fence's token, a query's words) does not drain: it rides the ring and runs once the ring retires past the quads queued before it. A quad's four pixels are lanes: the walk's depth, the texture unit's positions and the colour weights run on the host's SIMD unit (`fearless_simd`, `pipe/lanes.rs`, the level detected at start-up), bit for bit the scalar arithmetic. When its queue runs dry the raster thread does not drain: it keeps one count of the raster unit's pending jobs for the quads still in flight, and a front-end wait or a refused processor read asks it to store them. `IRIS_ODSY_SHADER=inline` makes the raster thread shade every quad itself, and `IRIS_ODSY_BACKEND=inline` stores on the raster thread with no pool. The texture registers a draw reads live in an array of draw state slots (`pipe/slot.rs`): the front end writes the current slot, a draw latches it and holds it until its job is given back, and a write while a queued draw holds the current slot first copies it to a free one, so the stages read their draw's registers with no lock | +| `ioc3-eth-rx`, `ioc3-eth-net` | Fuel and Origin 200 IOC3 Ethernet receive pump and network backend | Threads that have nothing to do park instead of spinning: the graphics consumers when their FIFO is empty, and the CPU thread in `idle-pause` builds @@ -362,6 +378,16 @@ When a device has several locks, document their order next to them and keep to it. Graphics recording is an example: `rec` before `trace`, and the consumer thread never takes `rec`. +Bedrock, the Fuel's node chip, shows rule 4 at the scale of a whole chip. +Each processor interface's interrupt inputs (pending banks, masks, timer and +error status) are atomics, and a store or an inbound packet changes one and +then publishes the slot's levels into Cause bits 10-14 of its CPU's interrupt +word with no lock: a writer that published a stale level re-reads the inputs +and corrects it, so the last write always matches the registers. A mask +store, an interrupt packet, a counter read, inbound DMA and PIO take no lock. +The comparators fire on host timers, as the Hub's do, not from the CPU's +batch hook. + ## 6. The CPU ### State and execution @@ -734,6 +760,17 @@ code into host code with Cranelift. The concepts: pfn map publishes a slot after its claim, and inline stores bump generations atomically. Compiled code is freed only by a full flush, and every flush parks all CPUs first (`jitv2_flush_from_here`, `run_leader_flush`). +- **Uncached fetches the board decodes apart.** A page is keyed by the + physical address of the fetch. The IP35 reset PROM answers an uncached fetch + at 0x1fc00000 whatever RAM sits behind the cached view there, so a board whose + banks reach that window presents the fetch at an address of its own + (`uncached_fetch_alias`, which only the R16000 model takes) and the JIT sees + two different frames. Compiles read + that frame through the executor's cached bus (`set_cached_bus`, given + `Ip35Bus::code_view`), and its generation counter is the PROM device's, so a + flash command invalidates it and a store to RAM never does. The alias wraps the + translation (`update_translate_fn` installs the wrapper when one is set), so + the other machines' fetch paths are the ones they always had. - Compiled pages can optionally be cached on disk across runs (`[jitv2] cache`, [docs/jitv2-persistent-cache.md](docs/jitv2-persistent-cache.md)). @@ -757,6 +794,26 @@ introspection commands are in [debug.md](debug.md#jit-v2). Crossbow and the Bridge with IOC3 and two ISP1040 SCSI controllers, an R10000, and IMPACT graphics (see the [Octane guide](docs/octane-ip30.md) for the boards). +- **IP34 board, IP35 software platform (Fuel)**: one 500 MHz R14000, 512 MiB + in two 256 MiB banks, Bedrock, an XBridge with two PCI heads (`Bridge` and + `Xbow` built as the XBridge's parts), IOC3, a two-channel ISP12160, an OHCI + USB controller and the L1. A VPro board (the Octane's Odyssey model) can sit + at crossbar widget 13; its packets and DMA go through Bedrock's inbound path, + and the PI graphics registers are stored but count no credits. Without one + the machine runs headless on the L1 or an IOC3 serial console. Bedrock is + one device (`src/dev/bedrock/`): its two PIs, memory controller, I/O, + local-bus, network and crossbar blocks are plain fields, and a block's + access returns what it changed elsewhere for the chip to act on. The node around it (the processor bus, flash, L1 and Hub-special page) + is `src/ip35/node.rs`. Snapshots are unsupported. See the + [Fuel guide](docs/fuel-ip34.md). +- **IP53 board, IP35 software platform (Tezro)**: its own board + (`src/ip53_machine.rs`) on the Bedrock node it shares with the Fuel + (`ip35::node`): one, two or four R16000s (700 MHz by default), each run on its own host + thread, memory from 512 and 1024 MiB banks (4 GiB by default), and a PIC + whose PXBOW is widget 0, carrying the IO9's IOC4, ISP12160 and BCM5701; + the two- and four-processor board adds a second PIC at widget 11. It runs + headless on the L1 or an IOC4 serial console. See the + [Tezro guide](docs/tezro-ip53.md). The Indy and Indigo2 profiles each have an embedded PROM fallback; `src/platform.rs` and `src/machine.rs` wire their differences, and diff --git a/HELP.md b/HELP.md index 7619f4939..b523a5e0e 100644 --- a/HELP.md +++ b/HELP.md @@ -1,7 +1,9 @@ -# IRIS — SGI Indy and Indigo2 emulator +# IRIS — SGI workstation emulator The complete core/GUI feature and configuration inventory is in -[FEATURES.md](FEATURES.md). This guide explains everyday use; the monitor's full command reference is in [debug.md](debug.md). +[FEATURES.md](FEATURES.md). For the Fuel, see the [Fuel guide](docs/fuel-ip34.md); for the Tezro, the [Tezro guide](docs/tezro-ip53.md). +This guide explains everyday use. The monitor's full command reference is in +[debug.md](debug.md). ## Quick start @@ -630,7 +632,7 @@ scsi_deferred_int = true # needed by OpenBSD/NetBSD; see --no-scsi-deferr [machine] profile = "indy_ip24" # "indigo2_ip22" (--ip22), "indigo2_ip28", or "o2_ip32" (an [o2] section: docs/o2-ip32.md); --machine NAME picks any -cpu = "r4400" # "r5000" (--cpu); IP28: "r10000", Octane: r10000/r12000/r14000, O2: r5000/r10000/r12000; or a part, "r14000_600" +cpu = "r4400" # "r5000" (--cpu); IP28: "r10000", Octane: r10000/r12000/r14000, O2: r5000/r10000/r12000, Tezro: r16000; or a part, "r14000_600", "r16000_800" # cpus = 2 # Octane only: a second processor (docs/octane-ip30-multiprocessor.md) [graphics] diff --git a/clanker/cpu/cache-lines-live-in-the-cpus-view.md b/clanker/cpu/cache-lines-live-in-the-cpus-view.md index 2041787f6..8f74e12d1 100644 --- a/clanker/cpu/cache-lines-live-in-the-cpus-view.md +++ b/clanker/cpu/cache-lines-live-in-the-cpus-view.md @@ -10,6 +10,7 @@ VR10000 user's manual p. 75), and every other master see memory. What the shippe | O2 | secondary and primary data lines for page 0 and 0x1000 hold POST's stack for about 50-65 ms, while POST tests page 0 uncached. On the R5000, primary data lines for page 0 (TagLo 0xc0) | yes | | Octane | a tag and data RAM test on secondary set 0, read back only with Index Load Tag/Data | no: the patterns name no memory | | IP28 | a secondary line for page 0, once while the PROM starts, then released | yes | +| Fuel | primary data lines for 0x100000-0x107fff hold the stack | no, until memory is configured mid-phase | | Origin 200 | none held in a boot to the IRIX login | - | | Indy, Indigo2 (R4400) | none: an R4400 line never holds a page | - | @@ -32,10 +33,11 @@ with Index Load Data while it holds a page. R10000's PTag and PState fields (NEC VR5000 user's manual U11761EJ6V0UM p. 133). It has no Index Store Data, so the page starts as memory. Its instruction lines hold nothing, since one could hold only what memory holds. -- A page with no memory behind it holds nothing: the line stays in the arrays, as - a tag test's patterns do. The region bitmap is all a CPU tests, so a held page's - whole region is the view's while it stands, reading zeros wherever nothing is - mapped. The O2's POST holds lines for page 0 before CRIME maps the rest of that +- A page with no memory behind it holds nothing, and the line stays in the arrays, + as a tag test's patterns do, unless the machine allows lines there + (`PpMemSpace::allow_unbacked_lines`, as the Fuel does for its bank slots). The + region bitmap is all a CPU tests, so a held page's whole region is the view's + while it stands, reading zeros wherever nothing is mapped. The O2's POST holds lines for page 0 before CRIME maps the rest of that region. - A remap saves and restores held pages, so memory configuration mid-phase keeps the stack. @@ -76,8 +78,27 @@ A page some CPU holds is not compiled: compiles read memory, and compiled code i shared. ppmem puts `GEN_HELD` on the page's generation counter at the first hold and takes it off at the last release, moving it with the page on a remap (`PpMemSpace::mark`). The compile that finds it publishes nothing and denies every -offset until the release. ppmem's counters are the bus's for memory; a machine whose bus -maps a device over memory must mark that device's counter as well. +offset until the release. ppmem's counters are the bus's for memory. Where the bus +maps a device over memory, as the IP35 boards map their reset PROM, the compile checks +that device's counter instead, so the cache puts the same mark there when a page +becomes held and takes it off with the page's last line (`mark_foreign`). That work +is at CACHE instructions only; a load, store or fetch is unchanged. + +## Primary instruction lines + +Only the Tezro's PROM uses these. A valid primary instruction line +writes each 36-bit predecoded word (p. 288) into its page as the instruction it +predecodes, through an eleven-class table inferred from PROM 6.210's words +(`l1i_decode`). A word of another class leaves memory's word, where hardware would +run it. A page held only for code also answers cached loads with code. + +The model never refills on a fetch. The PROM runs a staged secondary line +and reads the primary words back with Index Load Data (I) to stage them itself, so +Index Load Data (I) in a set a held secondary line maps to answers with that line's +words predecoded (`refilled_word`), until Index Store Data (I) writes the set. When +two held lines map one set, the later wins. An instruction no class holds comes back +in the model's own form (bits 35:32 set, which takes classes 0x78-0x7f), which +hardware would not produce. Index Load Tag (I) reads the stored tag. ## Seeing it diff --git a/debug.md b/debug.md index 47048e251..593d9e313 100644 --- a/debug.md +++ b/debug.md @@ -369,6 +369,13 @@ widget's link, refused Crossbow accesses, and accesses to physical addresses nothing is mapped at, with the PC; a device's own refusal is not listed). The `phys` commands work on the Octane as well. +On the Fuel the one controller is a two-channel ISP12160: `scsi` (or +`scsi0`) and `cow` act on channel 0, `scsi1` and `cow1` on channel 1, and +ids run 0 to 15 on each (`[scsi.17]` is id 1 on channel 1). `scsi notes` +shows the whole chip's command total and anything odd the driver did, and +has no `scsi1` form; `regs`, `wdt`, `defer`, `debug` and `dayna` are not +there. + Queue position `[0]` is the mounted disc, `[1]` the next on eject. A disc change reaches IRIX as a Unit Attention ("medium changed") on its next TEST UNIT READY; IRIX's `mediad` may still need a manual remount @@ -575,7 +582,7 @@ a forced termination; a normal stop saves them automatically. |---|---| | `phys mem [n]` / `mm` | Dump physical memory | | `phys dis [n]` / `md` | Disassemble physical memory | -| `phys map` | The decode table: pages by kind (untouched, filled but not yet mapped, shared, private), memory in use, slots faulted in, and the table as runs of devices. Every machine on `Physical` has the `phys` commands (Indy, Indigo2, O2, Octane) | +| `phys map` | The decode table: pages by kind (untouched, filled but not yet mapped, shared, private), memory in use, slots faulted in, and the table as runs of devices. Every machine on `Physical` has the `phys` commands (Indy, Indigo2, O2, Octane, Fuel) | | `phys trace ` / `trace ` | Log physical bus accesses | | `phys error ` | Stop on bus errors | | `phys hole ` | Stop on accesses to unmapped addresses | @@ -594,7 +601,7 @@ Per-module logging; all of it needs a developer build to produce output. | `log file ` | Send a module's log to a file | Modules: `net hpc3 seeq hal2 mc rex3 mips ioc scsi pdma vino dcb vc2 cmap -xmap bt445 scc ps2 rtc eeprom l1i l1d l2c bedrock pci bridge heart nvme`. `pci` logs every +xmap bt445 scc ps2 rtc eeprom l1i l1d l2c bedrock pci bridge heart nvme l1 usb`. `pci` logs every configuration cycle on a PCI bus (MACE's, the Octane Bridge's) and whether a function answered it. `bridge` logs the Octane Bridge's register and translation-table accesses by name, and any transfer a PCI @@ -605,7 +612,8 @@ interrupt vector raised or lowered. Mask categories: `pdma` — `hal enet scsi`; `mips` — `insn tlb mem fpu cp0`; `l1i`/`l1d`/`l2c` — `hit miss op`; `nvme` — `state admin regs io` ([docs/nvme.md](docs/nvme.md#tracing)); every module also takes `on`/`all`, `off`/`none` or a hex mask. -`bedrock` covers the Fuel's devices on its Bedrock hub. +`bedrock` covers the Fuel's devices on its Bedrock hub, `l1` its system +controller and `usb` its USB host controller. `IRIS_DEBUG_LOG` or `[debug] debug_log` set the same thing at startup. ### Performance and locks diff --git a/docs/fuel-ip34.md b/docs/fuel-ip34.md new file mode 100644 index 000000000..d853ff6df --- /dev/null +++ b/docs/fuel-ip34.md @@ -0,0 +1,100 @@ +# SGI Fuel (IP34 board, IP35 software platform) + +The Fuel is the IP34 board running the IP35 software platform. IRIS +emulates it with one 500 MHz R14000 and 512 MiB of RAM in two 256 MiB +banks, the Bedrock hub chip, two XBridge PCI heads, an IOC3, a +two-channel ISP12160 SCSI controller, a Lucent OHCI USB controller and +the L1 system controller, and a VPro graphics board when one is fitted. +Without one it runs headless on its L1 or IOC3 serial console. + +With PROM 6.210 and an installed IRIX 6.5.30 disk, the PROM reports +`ALL DIAGS PASSED` and IRIX boots to a serial login. `hinv` lists the 500 MHz R14000 (revision +2.3), 512 MB, both QL12160 channels with the disk at unit 1 on +controller 0, the IOC3 serial and parallel ports, `ef0` and the OHCI +controller, and `halt -p` powers the machine off. During its first boot IRIX's file indexer can keep the +guest busy while the serial console shows little. + +## Configuration + +Copy [`fuel.toml.example`](../fuel.toml.example) to `fuel.toml`, provide +the PROM and disk files named there, then run `iris --config fuel.toml` +from the repository root. The sample selects `machine.profile = +"fuel_ip34"`. The machine always runs its one R14000 (`r14000_500`); +`machine.cpu` may name it or be left out, and any other value is +refused. The sample sets `banks = [256, 256]`, `headless = true`, +`no_audio = true` and serial ports 18880 and 18881. The `[fuel]` table +selects the boot-flash backing file, the console (`l1`, `ioc3a` or +`ioc3b`) and the L1's TCP port and log; [FEATURES.md](../FEATURES.md#fuel) +lists its keys. + +The PROM decoder accepts a raw payload or a supported version-six, +single-segment stored or gzip container. The sample attaches a raw IRIX +disk at SCSI ID 1 and writes guest changes to its overlay. `[network] +mode` defaults to `nat`; `off` disconnects the link. + +The L1 reports firmware revision 1.48.1, the one a real Fuel's L1 +prints, and the made-up brick serial `EMUN00`; `[fuel] l1_revision` and +`brick_serial` replace them. Its system serial is the machine's Ethernet +address (`[network] mac`) as twelve uppercase hexadecimal digits, the +text the PROM converts back into that address. IRIX's `numastatd` reads +the revision at startup and logs `SNDRV_GET_ELSC_VERSION: I/O error` +without one. The L1 answers the requests the PROM and IRIX send and +refuses others as unknown. A power cycle the PROM asks for restarts the +machine; a power down turns it off. + +## Graphics + +`[graphics] board = "v12"` (or `"v10"`) fits a VPro board in the graphics +slot, crossbar widget 13: a Fuel IRIX inventory lists its board under +xtalk/13, and Fuels have been reported with both the V12 and the V10. It is +the Octane's VPro model. Its interrupt packets reach Bedrock as the PCI +heads' do, its DMA enters Bedrock's inbound path as widget 13, and a link +reset of port 13 resets it. The processor interfaces' graphics flow-control +registers keep what IRIX writes but count no stores, so the +credit-starvation interrupt never fires. + +With PROM 6.210 the graphics diagnostic finds the board and passes. An IRIX +6.5.30 disk whose kernel includes the VPro driver lists `Graphics board: V12` +in `hinv` and brings the X login screen up on it. A disk installed for a +machine without graphics has no VPro driver in its kernel and runs as before. + +## Input + +The IOC3 carries a PS/2 keyboard and mouse fed from host input, the +standard pair in the Fuel's manual; `[fuel] ps2_input = false` leaves +them out. Nothing is plugged into the USB controller's ports. + +## The PROM's stack in cache lines + +The PROM uses the R14000's primary data cache as memory before it +configures DRAM: it tags lines for the 32 KiB window `0x100000`-`0x107fff` +with Index Store Tag and fills them with Index Store Data, so its stack +has no memory behind it. The CPU keeps those lines in its own view of memory and serves cached +accesses that hit them, ahead of memory, until the PROM invalidates them +or the machine powers on; the machine declares every slot that can hold +a bank as a place lines may stand before memory is mapped, and the +memory controller has no special case for the window. Without them the +PROM takes a data bus error on its first stack access. The model's +approximations are in +[`clanker/cpu/cache-lines-live-in-the-cpus-view.md`](../clanker/cpu/cache-lines-live-in-the-cpus-view.md). + +## Limits + +- Machine snapshots are unsupported. +- Two values are chosen to match what the guests expect, not read from a + board: the board-id word's bits 11:0 (`0x4004`; bits 15:12 are the + Fuel's product code as OpenBSD reads it), which PROM 6.210's CPU + configuration check accepts, and NI port status bit 0, set so IRIX + 6.5.30 prints no link-reset notice on a faulting widget probe. +- The Hub-special head windows read zero and acknowledge writes with no + side effect. PROM 6.210 writes the 64-bit word `0x2000` at offsets + `0x0` and `0x40` before it finds DRAM; what those registers do is + unknown. `log bedrock` records the accesses. +- The literal `0x53474972756c6573` is named `IIO_PROTECT_OVRRD_KEY` in + public Linux (`arch/mips/include/asm/sn/sn0/hubio.h`, revision + `db2ddb87143519e20a95aa36c60b36107b736a58`). That names the value, not + the Fuel register offsets that take it, which remain approximate. + +The board composition follows the BSD-licensed QEMU SGI model in +jamesbraid/qemu-sgi-origami (`hw/mips/sgi/sn1-node.c`), an +implementation reference, not proof of hardware behaviour. diff --git a/docs/pci.md b/docs/pci.md index 9d0aecd39..7c283c578 100644 --- a/docs/pci.md +++ b/docs/pci.md @@ -15,7 +15,10 @@ header and, once the bus attaches it, its interrupt line and DMA. The bus resets a function's header; a function never resets its own, so the decode table cannot go stale behind a BAR that moved. Headers change only through the bus: a configuration write, a reset or a snapshot load, and each updates the decode -table and the BAR mappings. +table and the BAR mappings. A function whose chip +computes a configuration register (the indirect windows of a NIC) overrides +`config_dword` for the read, and `config_write` to take a whole-dword write +itself, which the header then never sees. `PciBus::attach` gives the function its DMA and, if its header names an interrupt pin, the line that pin drives, then calls `attached` for a function diff --git a/docs/tezro-ip53.md b/docs/tezro-ip53.md new file mode 100644 index 000000000..2304bd9ec --- /dev/null +++ b/docs/tezro-ip53.md @@ -0,0 +1,186 @@ +# SGI Tezro (IP53 board, IP35 software platform) + +The Tezro is the IP53 board running the IP35 software platform, the same +Bedrock node as the [Fuel](fuel-ip34.md) with a different interface board. +The emulated configuration has R16000 processors (700, 800, 900 or 1000 MHz) with +secondary caches of 4, 4, 8 or 16 MiB, a Bedrock node with 4 GiB of RAM by default, and an IO9 +board behind a PIC PCI host bridge: an IOC4 with four serial ports, a +DS1744-compatible clock and a PS/2 keyboard and mouse, an ISP12160 with two +SCSI channels, and a BCM5701 gigabit Ethernet controller. The machine runs +headless on its L1 or IOC4 serial console. + +## Configuration + +Copy [`tezro.toml.example`](../tezro.toml.example) to `tezro.toml`, provide +the PROM and disk files named there, then run `iris --config tezro.toml` from +the repository root. The sample selects `machine.profile = "tezro_ip53"` and +`machine.cpu = "r16000_700"`, the 700 MHz R16000 (revision 2.1, the part +table's `R16000_700_EARLY`), which is +also what the machine runs with no `cpu` line or with the family name +`cpu = "r16000"`. The model takes four chips: `r16000_700`, `r16000_800` +(revision 2.2), `r16000_900` (revision 3.0, 8 MiB secondary cache) and +`r16000_1000` (revision 3.0, 16 MiB). Each revision is the one inventories +print. The 1 GHz part's PRId 0x0f50 is established (a PROM banner prints +`f50` beside hinv's 3.0 on one machine); 0x0f41, 0x0f42 and the 900 MHz +part's 0x0f50 are inferred: the register's major is the printed one plus 2 +for an R16000, the rule the Linux and OpenBSD CPU probes and PROM 6.210's +printers apply, and no raw read of a 2.x part is on record. No IP53 board at +900 MHz is captured (the 900 MHz parts on record are on Fuel boards), so the +900 MHz chip rests on the PROM's speed table alone. A chip the Tezro lacks +gets one error that names the machine and these four. The processor counts +follow the boards captured with each chip. The 700 MHz chip is fitted as +one, two or four. The 800 MHz chip is fitted as four only: every captured +board with it is an `IP53_4CPU`, and the two-processor 800 MHz board +(030-2029-001) carries revision 3.0 processors, another PRId. The 900 MHz +chip is fitted as four only, a model choice, since no board at that clock +is captured. The 1 GHz chip is fitted as two or four, as the captured +`IP59_4CPU` and `IP59_2CPU` boards are. Every chip boots the PROM and IRIX +to login with four processors. The 1 GHz chip's mode word is computed, so +its clean boot shows the model is self-consistent, not that hardware uses +that word. Captured 1 GHz Tezros carry IP59 +node boards, not IP53; the model builds them as the IP53 board with a 1 GHz +chip, as QEMU does, and the PROM's socket masks for IP59 equal IP53's. An +800 MHz part with an 8 MB cache is captured in a Tezro (`IP53_4CPU` +030-1959-003, revision 2.2, PROM row 0x549caf85); it is not modelled yet. The +Tezro tower guide gives every processor 4 MB of cache; the captures, not the +manual, back the 8 and 16 MB caches. The processor clock in the flash record and the count rate (half +the clock) follow the chip; `[clock] fixed_mhz` still sets the count rate and +does not select a chip. The secondary cache clock and the cache tap (0xc, 0xa, +0xb and 0x15 from the 700 MHz chip up) are not modelled; the tap appears only +as part of the flash record's mode word. The board's table of chips, with how +well each value is established, is in `src/ip53_machine.rs`; the parts +themselves, with the PRId rule, are in `src/cpu/parts.rs`. + +`[machine] cpus` is the number of processors fitted: 1, 2 or 4, with 4 the +default, as in the Tezros whose inventories have been seen. The count selects +the interface board. The one-processor board has one PIC at XIO widget 15 +carrying the IO9. The two- and four-processor board adds a second PIC at +widget 11, with nothing on its buses. It sits on link 11 of the IO9 PIC's +PXBOW, beside the hub's link 10 and the IO9's link 15, so the PROM and IRIX +find it through that crossbar's link status. Whether it has a crossbar of +its own in front of it is not known. Either board takes a V12 at widget 12 +with `[graphics] board = "v12"`. + +Memory comes from `banks`: one 512 or 1024 MiB bank per locational bank, from +bank 0 with no gaps, up to eight. The IP53 board takes 512 MB and 1 GB DIMMs +only (Origin 350 guide 007-4566-001, Table 3-3: established for the board; +inferred for the Tezro, whose guide names no size), and each DIMM pair +provides two banks of the DIMM's size. The model also accepts unpaired and +unequal bank pairs, which the board cannot hold. Left unset, the machine fits +the 4 GiB layout of 1024, 1024 and four 512 MiB banks, which follows QEMU and +matches a captured Tezro. One processor needs at least 512 MiB, two or four at least +1 GiB. + +The `[tezro]` settings also select the flash backing file, the console and +the L1 port and log. The console options are `l1` (the default) and `ioc4_a` +to `ioc4_d`, the IOC4's ports 0 to 3. IOC4 ports 0 and 1 listen on +`serial_port_a` and `serial_port_b`, ports 2 and 3 on `tezro.serial_port_c` +and `tezro.serial_port_d`. `nvram` names the 32 KiB DS1744 store; its clock +reference is kept beside it with a `.clock` suffix. + +## Firmware inputs + +The PROM is the IP35 PROM 6.210 payload, given raw or in a supported +version-six, single-segment container, as for the Fuel. A fresh flash is +formatted with the payload in the first 8 MiB position and its configuration +block replaced by the chip's: the PROM's own 0x50-byte layout with the chip's +mode word and clock, machine type 0 (the class PROM 6.210 derives from the +modelled board word 0x2001; approximated), an edit counter of one, and the +compensation byte the PROM's editor would choose, so the block sums to what the +block it replaced did and the whole image still sums to zero. The volume label +goes in the second position. A flash store that already exists keeps the +configuration block it holds, so a block the PROM rewrote survives the next +start. The processor resets with the matching Config, `0x6c9bad85` for the 700 +MHz chip. + +The PROM brings up only the processor positions the node board's product +name allows (IP53_1CPU position A, IP53_2CPU A and C, IP53_4CPU all four), +and all four when the name matches none of its rows. The L1 names the board +for the processor count, so the PROM tries no empty position and a fresh +flash boots straight through. With a name that matches no row, such as the +Fuel's placeholder, a one- or two-processor board power-cycles the module +three times for each empty position it tries before it writes a +`Disable` record and carries on. + +The mode word is a different field from the processor's Config: its low 25 +bits are the Config reset bits, and the bits above carry, among others, the +cache tap. The mode words of the 700, 800 and 900 MHz chips (`0x589bad85`, +`0x549baf85`, `0x569cb185`) are rows of the stock PROM's CPU speed table. The +table stops at 900 MHz, so the 1 GHz word `0x4aedb385` is built from the +formulas the PROM's editor uses for the other rows and the cache clock and tap +its inventories print; it is approximated. A flash store keeps the block it was +formatted with, and the PROM checks its mode word against the processor's, so +give a different chip a fresh `[tezro] flash` file. + +## Limits + +Every fitted processor runs, each on its own host thread. The IP35 PROM +assumes its processors left reset together and run at hardware speed. Its +reset barrier on LB scratch 1 depends on that: every processor clears the +word before the boot master stores its release there. Bedrock's LB orders +those stores itself, by who stored in what order, so a late clear does not +land on the release (`src/dev/bedrock/lb.rs`); hardware drops no store, so +this is a model rule. Two later handshakes need more. In the first, each +processor of an interface with both slots fitted posts a status value in +its own PI real-time compare register and reads its partner's, at most +200,000 times, before it declares the partner dead (`peer_pled failed: Other +CPU on this node died.`). In the second, the lowest processor of each +interface publishes its interface's processors in II scratch 0 and then +polls, for a time inferred to be bounded by the PI's real-time counter, for +the other interface's. The host threads drift apart by more than either +wait allows, so the first of a pair to post is held at its batch boundary +until its partner posts, and a publisher that has published while the other +interface has not is held until that interface publishes; each hold also +ends once the processors it waits for have retired 2,000,000,000 cycles +(`src/ip35/presence.rs`). The holds change when a processor runs, never a +value it reads, and have no hardware counterpart. An NMI, and a write that clears a running processor's +enable bit, are logged by the Bedrock PI and not carried out: the CPU has no +NMI entry. + +The R16000 model leaves out three R16000-specific choices that QEMU's model +makes: the CP0 diagnostic register's mask and reset value, a trap on a +subnormal operand while FCSR.FS is clear, and the limit on CVT.L results. +Those cases follow the behaviour the rest of the R10000 family has in Iris. +No R16000 FIR was found in the retained evidence for any chip; the model +reports `0x0900`, the one R14000 value on record, as a model choice. + +Every board identity the machine reports is a software choice of the QEMU +Tezro profiles, not a value measured on a Tezro: Bedrock board word `0x2001`, +L1 brick type `0x39` with revision 1.48.1 and the router `brick` reply, the +PIC's revision 3 profile, the PCI devices (IOC4 at device 1, ISP12160 at 3, +BCM5701 at 4), their revisions and their interrupt inputs, and REQ64# on the +BCM5701's slot. No EEPROM image is fitted for the IO9 board, the node board +or the BCM5701, because no published source gives their contents. The L1 +gives the node board's record the flash block's CPU configuration, which the +PROM checks the flash against, and the identity of the IP53 board Tezro +inventories print for the processor count (IP53_4CPU 030-1868-001 or +IP53_2CPU 030-1869-001, revision C; empty for one processor, as no capture +shows that board); no real board's record has been read. +PIC DMA to any widget other than Bedrock IO is refused. Where two PCI +functions decode the same range, as a probing PROM leaves them, the lower +device answers; what silicon does there is not defined. A configuration read +of an empty PCI slot fails as a master timeout; the error enable gating and +the treatment of device 0 there are inferred. + +The IRIX 6.5.30 disks tested so far were installed on an Origin 300, not a +Tezro, so their device numbering comes from that machine: the IOC4 serial +ports appear as `tty5` and `tty6`. + +`[graphics] board = "v12"` fits a VPro V12 at widget 12 behind the PXBOW, +on either board, using the VPro model the Octane and the Fuel use. On the +two- and four-processor board this is QEMU's graphics slot. On the +one-processor board the tower guide gives one XIO connector for a VPro board +but names neither its widget nor its route, so widget 12 there is inferred. +Each PI keeps its graphics flow-control registers as written, as on the Fuel; +nothing counts a processor's stores into the graphics page or returns credits +for them. IRIX 6.5.30 with the VPro driver reaches the X desktop on the V12 +that way. +Without the V12 the machine has no graphics. +Machine snapshots are unsupported. + +## Source basis + +The board composition follows the project's BSD-licensed QEMU SGI models in +`hw/mips/sgi/sn-machine.c` and `hw/mips/sgi/sn1-node.c` and their Tezro +profile layers. That source is an implementation reference, not proof of +hardware behavior. diff --git a/fuel.toml.example b/fuel.toml.example new file mode 100644 index 000000000..b10e31456 --- /dev/null +++ b/fuel.toml.example @@ -0,0 +1,44 @@ +# SGI Fuel (IP34 board, IP35 software platform), serial-only setup. +# PROM 6.210 and IRIX 6.5.30 boot to the IRIX login. +# Copy to fuel.toml and run from the repository root: iris --config fuel.toml +# Create ignore/fuel/ and supply your decoded raw PROM payload and IRIX disk. +# PROM also accepts supported version-six single-segment stored/gzip containers. +# No PROM or disk images are included. Convert QCOW2 disks to raw first. + +prom = "ignore/fuel/ip35prom.payload.bin" +nvram = "ignore/fuel/nvram.bin" +banks = [256, 256] +headless = true +no_audio = true +monitor_port = 18888 +serial_port_a = 18880 +serial_port_b = 18881 + +[machine] +profile = "fuel_ip34" +# The Fuel's one processor, the R14000 at 500 MHz with a 2 MiB secondary +# cache (`r14000_500`); other values are refused. Unset takes the same. +cpu = "r14000" + +[graphics] +# "v12" or "v10" fits a VPro in the graphics slot; IRIX needs a kernel with +# the VPro driver. The console stays on serial either way. +board = "none" + +[fuel] +flash = "ignore/fuel/flash.bin" +console = "l1" +l1_port = 18882 + +[network] +# Disconnected cable. For NAT, use mode = "nat". +# Keep the installed disk's working ef0 address and default route. +# NAT can adopt its static gateway before routing begins. +# Initial defaults: guest 192.168.0.2/24, gateway 192.168.0.1. +mode = "off" + +[scsi.1] +path = "ignore/fuel/irix.raw" +cdrom = false +# Keep the base disk unchanged. Guest writes go to irix.raw.overlay. +overlay = true diff --git a/iris-gui/src/config_ui.rs b/iris-gui/src/config_ui.rs index 5743d842a..47675f0d2 100644 --- a/iris-gui/src/config_ui.rs +++ b/iris-gui/src/config_ui.rs @@ -2031,10 +2031,10 @@ fn show_performance(ui: &mut Ui, cfg: &mut MachineConfig) -> ConfigAction { ui.horizontal(|ui| { ui.add_enabled_ui(octane, |ui| { ComboBox::from_id_salt("machine_cpus") - .selected_text(format!("{}", cfg.machine.cpus)) + .selected_text(format!("{}", cfg.machine.cpus())) .show_ui(ui, |ui| { for n in [1u8, 2] { - ui.selectable_value(&mut cfg.machine.cpus, n, format!("{n}")); + ui.selectable_value(&mut cfg.machine.cpus, Some(n), format!("{n}")); } }); }); diff --git a/src/config.rs b/src/config.rs index 35301de16..cce13aabd 100644 --- a/src/config.rs +++ b/src/config.rs @@ -676,6 +676,13 @@ pub enum MachineProfile { /// a test widget. Configured by the shared keys and `[origin2000]`; see /// `src/ip27_onyx2.rs`. Onyx2Ip27, + /// SGI Fuel: the IP34 board on the IP35 software platform, headless or + /// with a VPro in its graphics slot. See `src/ip34_machine.rs`. + FuelIp34, + /// Serial-only Tezro: the IP53 board on the IP35 software platform, + /// configured by the shared keys and the `[tezro]` section; see + /// `src/ip53_machine.rs`. + TezroIp53, } impl MachineProfile { @@ -699,6 +706,8 @@ impl MachineProfile { Self::Origin200Ip29 | Self::Origin2000Ip27 | Self::Onyx2Ip27 => &[0, 64, 128, 256, 512], Self::O2Ip32 => &[0, 32, 64, 128, 256], Self::Indigo2Ip28 => MC_BANK_SIZES, + Self::FuelIp34 => &[0, 256], + Self::TezroIp53 => &[0, 512, 1024], _ => &[0, 8, 16, 32, 64, 128], } } @@ -723,7 +732,7 @@ impl MachineProfile { /// Every profile, in the order `workstation::build` lists them. `ALL` is /// only the ones the GUI offers. - pub const EVERY: [Self; 8] = [ + pub const EVERY: [Self; 10] = [ Self::IndyIp24, Self::Indigo2Ip22, Self::Indigo2Ip28, @@ -732,6 +741,8 @@ impl MachineProfile { Self::Origin200Ip29, Self::Origin2000Ip27, Self::Onyx2Ip27, + Self::FuelIp34, + Self::TezroIp53, ]; /// The name `[machine] profile` and `--machine` give this profile. @@ -745,6 +756,8 @@ impl MachineProfile { Self::Origin200Ip29 => "origin200_ip29", Self::Origin2000Ip27 => "origin2000_ip27", Self::Onyx2Ip27 => "onyx2_ip27", + Self::FuelIp34 => "fuel_ip34", + Self::TezroIp53 => "tezro_ip53", } } @@ -761,16 +774,21 @@ impl MachineProfile { Self::EVERY.iter().map(|p| format!("{:<14}{}\n", p.name(), p.label())).collect() } - /// The processors fitted at most: two on the Octane, one elsewhere. + /// The processors fitted at most: four on the Tezro, two on the Octane, + /// one elsewhere. pub fn max_cpus(self) -> u8 { - if self == Self::OctaneIp30 { 2 } else { 1 } + match self { + Self::TezroIp53 => 4, + Self::OctaneIp30 => 2, + _ => 1, + } } /// Whether this profile can have `cpus` processors, and why not if it /// cannot. pub fn accepts_cpus(self, cpus: u8) -> Result<(), String> { if cpus != 1 && self.max_cpus() == 1 { - Err(format!("[machine] cpus = {cpus}: only the Octane has more than one processor")) + Err(format!("[machine] cpus = {cpus}: only the Octane and the Tezro have more than one processor")) } else if !(1..=self.max_cpus()).contains(&cpus) { Err(format!("[machine] cpus = {cpus}: IRIX runs an Octane with 1 or 2")) } else { @@ -802,6 +820,8 @@ impl MachineProfile { Self::Origin200Ip29 => "SGI Origin 200 (IP29)", Self::Origin2000Ip27 => "SGI Origin 2000 (IP27)", Self::Onyx2Ip27 => "SGI Onyx2 (IP27)", + Self::FuelIp34 => "SGI Fuel (IP34)", + Self::TezroIp53 => "SGI Tezro (IP53)", } } @@ -811,7 +831,7 @@ impl MachineProfile { /// connector), one on the Indy. pub fn scsi_controllers(self) -> u8 { match self { - Self::Indigo2Ip22 | Self::O2Ip32 | Self::OctaneIp30 | Self::Origin200Ip29 | Self::Origin2000Ip27 | Self::Onyx2Ip27 => 2, + Self::Indigo2Ip22 | Self::O2Ip32 | Self::OctaneIp30 | Self::Origin200Ip29 | Self::Origin2000Ip27 | Self::Onyx2Ip27 | Self::FuelIp34 | Self::TezroIp53 => 2, _ => 1, } } @@ -820,18 +840,23 @@ impl MachineProfile { /// buses are wide, with ids 0 to 15 and the controller on 0. The Indy's /// and Indigo2's WD33C93A chips are narrow, with ids 0 to 7. The O2's /// Adaptec buses are wide too, but stay at 7 until they are widened on - /// purpose. + /// purpose. The Fuel's and the Tezro's ISP12160 buses are wide. pub fn max_scsi_id(self) -> u8 { match self { - Self::OctaneIp30 | Self::Origin200Ip29 | Self::Origin2000Ip27 | Self::Onyx2Ip27 => 15, + Self::OctaneIp30 | Self::Origin200Ip29 | Self::Origin2000Ip27 | Self::Onyx2Ip27 | Self::FuelIp34 | Self::TezroIp53 => 15, _ => 7, } } - /// How many memory banks the machine has: four on all of them (the MC's - /// four banks; the O2's four DIMM pairs). `banks` has room for more. + /// How many memory banks the machine has: four on the MC, HEART and O2 + /// (the O2's four DIMM pairs), two on Fuel, eight on Tezro. `banks` has + /// room for more; unused entries must be zero. pub fn memory_banks(self) -> usize { - 4 + match self { + Self::FuelIp34 => 2, + Self::TezroIp53 => MAX_BANKS, + _ => 4, + } } pub fn guinness(self) -> bool { @@ -846,6 +871,8 @@ impl MachineProfile { Self::Origin200Ip29 | Self::Origin2000Ip27 | Self::Onyx2Ip27 => crate::ip27_machine::CPUS, Self::O2Ip32 => crate::ip32_machine::CPUS, Self::Indigo2Ip28 => crate::machine::IP28_CPUS, + Self::FuelIp34 => crate::ip34_machine::CPUS, + Self::TezroIp53 => crate::ip53_machine::CPUS, Self::IndyIp24 | Self::Indigo2Ip22 => crate::machine::CPUS, } } @@ -885,18 +912,18 @@ pub enum GraphicsBoard { /// VPro V8: BUZZ A, 128 MB. Octane only. #[clap(name = "v8", alias = "vpro:v8", alias = "vpro_v8")] V8, - /// VPro V10: BUZZ B, 32 MB. Octane only. It shipped only in the Octane2 - /// and needs Crossbow 1.4, so the machine takes the Octane2's Crossbow and - /// Bridge revisions. + /// VPro V10: BUZZ B, 32 MB. Octane and Fuel. On the Octane it shipped + /// only in the Octane2 and needs Crossbow 1.4, so the machine takes the + /// Octane2's Crossbow and Bridge revisions. #[clap(name = "v10", alias = "vpro:v10", alias = "vpro_v10")] V10, - /// VPro V12: BUZZ B, 128 MB. Octane only. It shipped only in the Octane2 - /// and needs Crossbow 1.4, so the machine takes the Octane2's Crossbow and - /// Bridge revisions. + /// VPro V12: BUZZ B, 128 MB. Octane, Fuel and Tezro. On the Octane it + /// shipped only in the Octane2 and needs Crossbow 1.4, so the machine + /// takes the Octane2's Crossbow and Bridge revisions. #[clap(name = "v12", alias = "vpro:v12", alias = "vpro_v12")] V12, /// No graphics board: the machine runs on its serial console. Octane - /// only. + /// and Fuel. #[clap(name = "none")] None, } @@ -959,6 +986,8 @@ impl GraphicsBoard { match profile { MachineProfile::OctaneIp30 => self.is_impact() || self.is_vpro() || self == Self::None, MachineProfile::Origin200Ip29 | MachineProfile::Origin2000Ip27 | MachineProfile::Onyx2Ip27 => self == Self::None, + MachineProfile::FuelIp34 => matches!(self, Self::V10 | Self::V12 | Self::None), + MachineProfile::TezroIp53 => matches!(self, Self::None | Self::V12), MachineProfile::Indigo2Ip22 => !self.is_vpro() && self != Self::None, _ => !self.is_vpro() && !matches!(self, Self::Extreme | Self::None), } @@ -969,6 +998,7 @@ impl GraphicsBoard { match profile { MachineProfile::OctaneIp30 => Self::SolidImpact, MachineProfile::Origin200Ip29 | MachineProfile::Origin2000Ip27 | MachineProfile::Onyx2Ip27 => Self::None, + MachineProfile::FuelIp34 | MachineProfile::TezroIp53 => Self::None, _ => Self::Newport, } } @@ -1028,7 +1058,7 @@ impl std::fmt::Display for GraphicsBoard { #[serde(deny_unknown_fields)] pub struct GraphicsSection { /// Graphics board: Newport, GR2 XZ/Extreme, IMPACT Solid/High/Maximum, - /// or none (Octane only). Unset: the machine's default + /// or none (Octane and Fuel). Unset: the machine's default /// ([`GraphicsBoard::default_for`]); read it with /// [`MachineConfig::graphics_board`]. Extreme is restricted to IP22, /// the Octane to IMPACT or none; non-Newport boards require one head @@ -1187,18 +1217,28 @@ pub struct MachineSection { pub cpu: Option, #[serde(default)] pub cache: CacheModel, - /// Processors fitted. More than one only on the Octane. - #[serde(default = "default_cpus")] - pub cpus: u8, + /// Processors fitted, or the machine's default when unset: read it with + /// [`MachineSection::cpus`]. More than one only on the Octane and the + /// Tezro. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub cpus: Option, } -fn default_cpus() -> u8 { - 1 +impl MachineSection { + /// The processors fitted: `cpus` as set, or the profile's default. A + /// machine owns its own default (the Tezro's is + /// `ip53_machine::DEFAULT_CPUS`); every other profile fits one. + pub fn cpus(&self) -> u8 { + self.cpus.unwrap_or(match self.profile { + MachineProfile::TezroIp53 => crate::ip53_machine::DEFAULT_CPUS, + _ => 1, + }) + } } impl Default for MachineSection { fn default() -> Self { - Self { profile: MachineProfile::default(), cpu: None, cache: CacheModel::default(), cpus: default_cpus() } + Self { profile: MachineProfile::default(), cpu: None, cache: CacheModel::default(), cpus: None } } } @@ -1799,6 +1839,143 @@ mod scsi_keys { } } +/// Fuel's independent serial connections. The selected console serves CI and serial_log. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum FuelConsole { + #[default] + L1, + Ioc3A, + Ioc3B, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct FuelSection { + /// Persistent 16 MiB boot flash store; absent means volatile flash. + #[serde(default)] + pub flash: Option, + #[serde(default)] + pub console: FuelConsole, + /// L1 host console TCP port; IOC3 keeps serial_port_a and serial_port_b. + #[serde(default)] + pub l1_port: Option, + /// Separate L1 log when an IOC3 port is selected as console. + #[serde(default)] + pub l1_log: Option, + /// Fit the PS/2 keyboard and mouse controller in the onboard IOC3 and + /// feed it the host's keyboard and mouse. + #[serde(default = "default_fuel_input")] + pub ps2_input: bool, + /// L1 firmware revision as "major.minor.patch", reported to request + /// 0x0005. Unset: 1.48.1. + #[serde(default)] + pub l1_revision: Option, + /// Brick serial reported to L1 request 0x0004, at most 37 bytes. Unset: + /// the marked synthetic `EMUN00`. + #[serde(default)] + pub brick_serial: Option, +} + +fn default_fuel_input() -> bool { true } + +impl Default for FuelSection { + fn default() -> Self { + Self { + flash: None, + console: FuelConsole::default(), + l1_port: None, + l1_log: None, + ps2_input: true, + l1_revision: None, + brick_serial: None, + } + } +} + +impl FuelSection { + /// Left out of a written config when nothing in it was set. + fn is_default(&self) -> bool { + self.flash.is_none() + && self.console == FuelConsole::default() + && self.l1_port.is_none() + && self.l1_log.is_none() + && self.ps2_input + && self.l1_revision.is_none() + && self.brick_serial.is_none() + } +} + +/// The Tezro's serial connections: the L1 and the IOC4's four ports. The +/// selected console serves CI and serial_log. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum TezroConsole { + #[default] + L1, + Ioc4A, + Ioc4B, + Ioc4C, + Ioc4D, +} + +impl TezroConsole { + /// The connection's index: 0 for the L1, then the IOC4's ports. + pub fn index(self) -> usize { + self as usize + } +} + +/// The processor counts a Tezro board fits. +pub const TEZRO_CPUS: [u8; 3] = [1, 2, 4]; + +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct TezroSection { + /// Persistent 16 MiB boot flash store; absent means volatile flash. + #[serde(default)] + pub flash: Option, + #[serde(default)] + pub console: TezroConsole, + /// L1 host console TCP port; IOC4 ports 0 and 1 keep serial_port_a and + /// serial_port_b. + #[serde(default)] + pub l1_port: Option, + /// Separate L1 log when an IOC4 port is selected as console. + #[serde(default)] + pub l1_log: Option, + /// Host TCP ports of IOC4 ports 2 and 3. + #[serde(default)] + pub serial_port_c: Option, + #[serde(default)] + pub serial_port_d: Option, +} + +impl Default for TezroSection { + fn default() -> Self { + Self { + flash: None, + console: TezroConsole::default(), + l1_port: None, + l1_log: None, + serial_port_c: None, + serial_port_d: None, + } + } +} + +impl TezroSection { + /// Left out of a written config when nothing in it was set. + fn is_default(&self) -> bool { + self.flash.is_none() + && self.console == TezroConsole::default() + && self.l1_port.is_none() + && self.l1_log.is_none() + && self.serial_port_c.is_none() + && self.serial_port_d.is_none() + } +} + /// Top-level machine configuration. /// /// Field order matters for TOML export: the `toml` serializer requires every @@ -1844,6 +2021,12 @@ pub struct MachineConfig { #[serde(default = "default_banks", with = "banks_serde")] pub banks: [u32; MAX_BANKS], + /// Whether `banks` was given (in the config file or by `--bank0..3`), as + /// opposed to left at the generic default, so that a value equal to the + /// default still counts as chosen. Only the Tezro reads it. + #[serde(skip)] + pub banks_set: bool, + /// Window scale factor (1 = native, 2 = 2× for HiDPI/4K). CLI --2x overrides this. #[serde(default = "default_scale")] pub scale: u32, @@ -2053,6 +2236,14 @@ pub struct MachineConfig { #[serde(default, skip_serializing_if = "Origin2000Section::is_default")] pub origin2000: Origin2000Section, + /// Serial console and flash storage for the Fuel. + #[serde(default, skip_serializing_if = "FuelSection::is_default")] + pub fuel: FuelSection, + + /// Processors, serial console and flash storage for the Tezro. + #[serde(default, skip_serializing_if = "TezroSection::is_default")] + pub tezro: TezroSection, + /// Host performance tuning (`[perf]` section). #[serde(default)] pub perf: PerfConfig, @@ -2247,6 +2438,7 @@ impl Default for MachineConfig { nvram: default_nvram(), nveeprom: default_nveeprom(), banks: default_banks(), + banks_set: false, scsi: default_scsi(), scale: default_scale(), nfs: None, @@ -2291,6 +2483,8 @@ impl Default for MachineConfig { octane: OctaneSection::default(), origin200: Origin200Section::default(), origin2000: Origin2000Section::default(), + fuel: FuelSection::default(), + tezro: TezroSection::default(), perf: PerfConfig::default(), clock: ClockConfig::default(), rtc_offset: RtcOffset::default(), @@ -2385,6 +2579,12 @@ impl MachineConfig { MachineProfile::Origin200Ip29 => Err("[virtio_serial] is not wired on the Origin 200".into()), MachineProfile::Origin2000Ip27 => Err("[virtio_serial] is not wired on the Origin 2000".into()), MachineProfile::Onyx2Ip27 => Err("[virtio_serial] is not wired on the Onyx2".into()), + MachineProfile::FuelIp34 => Err( + "[virtio_serial] is not wired on the Fuel: its XBridge has no virtio-pci attachment".into(), + ), + MachineProfile::TezroIp53 => Err( + "[virtio_serial] is not wired on the Tezro: its PIC has no virtio-pci attachment".into(), + ), MachineProfile::O2Ip32 => Err( "[virtio_serial] is not wired on the O2 yet: the virtio-pci transport exists, but the \ MACE PCI bridge has no interrupt route for a third slot (see docs/dev/virtio-serial.md)" @@ -2434,6 +2634,16 @@ impl MachineConfig { [self.serial_port_a.unwrap_or(a), self.serial_port_b.unwrap_or(b)] } + /// Parse a config file's text, remembering whether it listed `banks`, and + /// folding in the spellings older versions wrote. + fn parse_toml(text: &str) -> Result { + let mut cfg = toml::from_str::(text)?; + let table = toml::from_str::(text)?; + cfg.banks_set = table.contains_key("banks"); + cfg.fold_deprecated(); + Ok(cfg) + } + /// Load from `iris.toml` if it exists, otherwise return defaults. /// /// A *missing* file is fine (defaults are used). A file that exists but @@ -2457,11 +2667,8 @@ impl MachineConfig { let Ok(text) = std::fs::read_to_string(path) else { return Ok(Self::default()); }; - match toml::from_str::(&text) { - Ok(mut cfg) => { - cfg.fold_deprecated(); - Ok(cfg) - } + match Self::parse_toml(&text) { + Ok(cfg) => Ok(cfg), Err(e) => { let mut msg = format!("failed to parse {path}:\n{e}"); // Common footgun: backslashes in a double-quoted string (Windows @@ -2567,9 +2774,25 @@ impl MachineConfig { self.banks[..n].to_vec() } + /// The Tezro's memory: `banks` from locational bank 0, or the board's + /// 4 GiB default when `banks` was never set (the generic default names a + /// bank size no Tezro takes, so a `banks` equal to it is a mistake to + /// report, not a request for the default). + pub fn tezro_memory(&self) -> Result { + use crate::ip53_machine::MemoryPopulation; + if !self.banks_set && self.banks == default_banks() { + return Ok(MemoryPopulation::default()); + } + let n = self.banks.iter().rposition(|&b| b != 0).map_or(0, |i| i + 1); + if let Some(gap) = self.banks[..n].iter().position(|&b| b == 0) { + return Err(format!("banks: bank {gap} is empty but a later bank is populated; fill the banks from 0")); + } + MemoryPopulation::from_mib(&self.banks[..n]).map_err(|e| format!("banks: {e}")) + } + /// Validate bank sizes, returns a description of any errors. /// The graphics board: `[graphics] board`, or the machine's default - /// when it is not set (Solid IMPACT on the Octane, Newport elsewhere). + /// (Solid IMPACT on Octane, none on Fuel, Newport elsewhere). pub fn graphics_board(&self) -> GraphicsBoard { self.graphics.board.unwrap_or_else(|| GraphicsBoard::default_for(self.machine.profile)) } @@ -2601,7 +2824,7 @@ impl MachineConfig { self.xio_widgets()?; self.validate_nvme()?; self.cpu_part()?; - self.machine.profile.accepts_cpus(self.machine.cpus)?; + self.machine.profile.accepts_cpus(self.machine.cpus())?; if self.graphics.heads != 1 && self.graphics.heads != 2 { return Err(format!( "graphics.heads {} is invalid (valid: 1, 2)", @@ -2625,6 +2848,8 @@ impl MachineConfig { format!("graphics.board \"{name}\" is not modelled on the Origin 2000 (valid: none)") } else if self.machine.profile == MachineProfile::Onyx2Ip27 { format!("graphics.board \"{name}\" is not modelled on the Onyx2 (valid: none)") + } else if self.machine.profile == MachineProfile::FuelIp34 { + format!("graphics.board \"{name}\" is not a Fuel board (valid: v12, v10, none)") } else { format!("graphics.board \"{name}\" is only valid on the Octane") }); @@ -2666,9 +2891,9 @@ impl MachineConfig { "scsi.{} and scsi.{} both name {}", other, key, scsi_label(controller, id) )); } - if dev.is_daynaport() && matches!(self.machine.profile, MachineProfile::O2Ip32 | MachineProfile::Origin200Ip29 | MachineProfile::Origin2000Ip27 | MachineProfile::Onyx2Ip27) { + if dev.is_daynaport() && matches!(self.machine.profile, MachineProfile::O2Ip32 | MachineProfile::Origin200Ip29 | MachineProfile::Origin2000Ip27 | MachineProfile::Onyx2Ip27 | MachineProfile::FuelIp34 | MachineProfile::TezroIp53) { return Err(format!( - "scsi.{}: kind = \"daynaport\" is not available on the O2 (its Ethernet is built in)", key + "scsi.{}: kind = \"daynaport\" is not available on {} (its Ethernet is built in)", key, self.machine.profile.label() )); } if dev.is_daynaport() && self.network.mode == NetMode::Stream { @@ -2730,6 +2955,28 @@ impl MachineConfig { return Err(format!("bank{i} size 256 MB: the PROM disables 256 MB banks on the 180 MHz R10000 module (use 128 and 128, or the r12000 module)")); } } + } else if self.machine.profile == MachineProfile::FuelIp34 { + if self.banks[..2] != [256, 256] { + return Err("Fuel currently requires banks = [256, 256] (512 MiB)".into()); + } + if !board.supports(MachineProfile::FuelIp34) { + return Err(format!("graphics.board \"{}\" is not a Fuel board (valid: v12, v10, none)", board.as_str())); + } + if let Some(text) = &self.fuel.l1_revision { + crate::dev::l1::parse_revision(text).map_err(|e| format!("fuel.l1_revision: {e}"))?; + } + if self.fuel.brick_serial.as_ref().is_some_and(|s| s.len() > 37 || s.contains('\0')) { + return Err("fuel.brick_serial must be at most 37 bytes with no NUL".into()); + } + } else if self.machine.profile == MachineProfile::TezroIp53 { + if !TEZRO_CPUS.contains(&self.machine.cpus()) { + return Err(format!("[machine] cpus = {} is not a Tezro board's population (valid: {TEZRO_CPUS:?})", self.machine.cpus())); + } + crate::ip53_machine::chip(self.cpu()).accepts_cpus(self.machine.cpus())?; + self.tezro_memory()?.check_for_cpus(usize::from(self.machine.cpus())).map_err(|e| format!("banks: {e}"))?; + if !board.supports(MachineProfile::TezroIp53) { + return Err(format!("graphics.board \"{}\" is not a Tezro board (valid: v12, none)", board.as_str())); + } } else if self.machine.profile == MachineProfile::OctaneIp30 { // A HEART bank is a matched DIMM pair: 64 MB to 2 GB, filled from // bank 0 without a gap, 8 GB at most (SGI's memory upgrade guide). @@ -2775,8 +3022,8 @@ impl MachineConfig { } } match (self.network.mode, self.machine.profile) { - (NetMode::Off, p) if p != MachineProfile::O2Ip32 && p != MachineProfile::OctaneIp30 && p != MachineProfile::Origin200Ip29 && p != MachineProfile::Origin2000Ip27 && p != MachineProfile::Onyx2Ip27 => { - return Err("network.mode = \"off\" is only available on the O2, the Octane and the Origin 200 and 2000 for now; the Indy and Indigo2 always have their Ethernet".into()); + (NetMode::Off, p) if !matches!(p, MachineProfile::O2Ip32 | MachineProfile::OctaneIp30 | MachineProfile::Origin200Ip29 | MachineProfile::Origin2000Ip27 | MachineProfile::Onyx2Ip27 | MachineProfile::FuelIp34 | MachineProfile::TezroIp53) => { + return Err("network.mode = \"off\" is only available on the O2, the Octane, the Fuel, the Tezro and the Origin 200 and 2000 for now; the Indy and Indigo2 always have their Ethernet".into()); } (NetMode::Stream, _) => { let ep = self.network.stream.as_deref() @@ -2857,7 +3104,7 @@ impl MachineConfig { if cfg!(feature = "lightning") { return Err("jitv2.compile = \"sync\" is not available in a `lightning` build, which always compiles on the pool".into()); } - if self.machine.cpus > 1 { + if self.machine.cpus() > 1 { return Err("jitv2.compile = \"sync\" needs one CPU: several share the compile pool's threads".into()); } } @@ -3249,10 +3496,10 @@ impl Cli { if let Some(p) = &self.prom { cfg.prom = p.clone(); } if let Some(p) = &self.nvram { cfg.nvram = p.clone(); } if let Some(p) = &self.nveeprom { cfg.nveeprom = p.clone(); } - if let Some(v) = self.bank0 { cfg.banks[0] = v; } - if let Some(v) = self.bank1 { cfg.banks[1] = v; } - if let Some(v) = self.bank2 { cfg.banks[2] = v; } - if let Some(v) = self.bank3 { cfg.banks[3] = v; } + if let Some(v) = self.bank0 { cfg.banks[0] = v; cfg.banks_set = true; } + if let Some(v) = self.bank1 { cfg.banks[1] = v; cfg.banks_set = true; } + if let Some(v) = self.bank2 { cfg.banks[2] = v; cfg.banks_set = true; } + if let Some(v) = self.bank3 { cfg.banks[3] = v; cfg.banks_set = true; } if let Some(n) = self.jitv2_threads { cfg.jitv2.threads = n; } if let Some(m) = self.jitv2_compile { cfg.jitv2.compile = m; } @@ -3732,7 +3979,7 @@ mod export_tests { cfg.validate().expect("sync compile on the default machine"); cfg.machine.profile = MachineProfile::OctaneIp30; cfg.machine.cpu = Some("r10000".into()); - cfg.machine.cpus = 2; + cfg.machine.cpus = Some(2); assert!(cfg.validate().unwrap_err().contains("one CPU")); } @@ -3904,7 +4151,7 @@ mod export_tests { assert_eq!(apply(&["--ip22"]).machine.profile, MachineProfile::Indigo2Ip22); assert!(Cli::try_parse_from(["iris", "--ip22", "--machine", "o2_ip32"]).is_err()); let e = MachineProfile::from_name("vax").unwrap_err(); - assert_eq!(e, "unknown machine \"vax\" (valid: indy_ip24, indigo2_ip22, indigo2_ip28, o2_ip32, octane_ip30, origin200_ip29, origin2000_ip27, onyx2_ip27)"); + assert_eq!(e, "unknown machine \"vax\" (valid: indy_ip24, indigo2_ip22, indigo2_ip28, o2_ip32, octane_ip30, origin200_ip29, origin2000_ip27, onyx2_ip27, fuel_ip34, tezro_ip53)"); let list = MachineProfile::list(); assert_eq!(list.lines().count(), MachineProfile::EVERY.len()); assert!(list.starts_with("indy_ip24 SGI Indy (IP24)\n"), "{list}"); @@ -3913,7 +4160,8 @@ mod export_tests { for p in MachineProfile::EVERY { match p { MachineProfile::IndyIp24 | MachineProfile::Indigo2Ip22 | MachineProfile::Indigo2Ip28 - | MachineProfile::O2Ip32 | MachineProfile::OctaneIp30 | MachineProfile::Origin200Ip29 | MachineProfile::Origin2000Ip27 | MachineProfile::Onyx2Ip27 => {} + | MachineProfile::O2Ip32 | MachineProfile::OctaneIp30 | MachineProfile::Origin200Ip29 | MachineProfile::Origin2000Ip27 | MachineProfile::Onyx2Ip27 | MachineProfile::FuelIp34 + | MachineProfile::TezroIp53 => {} } let cfg: MachineConfig = toml::from_str(&format!("[machine]\nprofile = \"{}\"", p.name())).unwrap(); assert_eq!(cfg.machine.profile, p); @@ -3928,19 +4176,25 @@ mod export_tests { let accepts = |p: MachineProfile, cpu: &str| p.part(Some(cpu)).is_ok(); assert!(accepts(MachineProfile::OctaneIp30, "r14000") && !accepts(MachineProfile::OctaneIp30, "r5000")); assert!(accepts(MachineProfile::IndyIp24, "r5000") && !accepts(MachineProfile::IndyIp24, "r10000")); + assert!(accepts(MachineProfile::FuelIp34, "r14000") && !accepts(MachineProfile::FuelIp34, "r10000")); + assert!(accepts(MachineProfile::TezroIp53, "r16000") && !accepts(MachineProfile::TezroIp53, "r14000")); // The counts are what the checks in validate() and the Octane // constructor allowed before they moved here: one on every machine // but the Octane, one or two on the Octane. - for p in [MachineProfile::IndyIp24, MachineProfile::Indigo2Ip22, MachineProfile::Indigo2Ip28, MachineProfile::O2Ip32, MachineProfile::OctaneIp30, MachineProfile::Origin200Ip29] { + for p in [MachineProfile::IndyIp24, MachineProfile::Indigo2Ip22, MachineProfile::Indigo2Ip28, MachineProfile::O2Ip32, MachineProfile::OctaneIp30, MachineProfile::Origin200Ip29, MachineProfile::TezroIp53] { for cpus in 0..=4u8 { - let old_ok = if p == MachineProfile::OctaneIp30 { (1..=2).contains(&cpus) } else { cpus == 1 }; + let old_ok = match p { + MachineProfile::OctaneIp30 => (1..=2).contains(&cpus), + MachineProfile::TezroIp53 => (1..=4).contains(&cpus), + _ => cpus == 1, + }; assert_eq!(p.accepts_cpus(cpus).is_ok(), old_ok, "{p:?} with {cpus}"); } - if p != MachineProfile::OctaneIp30 { + if !matches!(p, MachineProfile::OctaneIp30 | MachineProfile::TezroIp53) { assert_eq!( p.accepts_cpus(2).unwrap_err(), - "[machine] cpus = 2: only the Octane has more than one processor" + "[machine] cpus = 2: only the Octane and the Tezro have more than one processor" ); } } @@ -3949,11 +4203,11 @@ mod export_tests { octane.machine.profile = MachineProfile::OctaneIp30; octane.machine.cpu = Some("r10000".into()); octane.banks = banks_from4([128, 0, 0, 0]); - octane.machine.cpus = 2; + octane.machine.cpus = Some(2); octane.validate().expect("two processors"); - octane.machine.cpus = 3; + octane.machine.cpus = Some(3); assert!(octane.validate().unwrap_err().contains("1 or 2")); - octane.machine.cpus = 1; + octane.machine.cpus = Some(1); octane.machine.cpu = Some("r4400".into()); assert!(octane.validate().unwrap_err().contains("takes [machine] cpu = r10000_195")); @@ -4360,3 +4614,348 @@ mod ci_overlay_tests { assert!(a.contains(&format!("iris-ci-{}-", std::process::id())), "{a}"); } } + +#[cfg(test)] +mod fuel_input_tests { + use super::*; + + /// A config saved for another machine must not grow a `[fuel]` table, + /// and a Fuel's own settings must survive a save and reload. + #[test] + fn only_a_configured_fuel_section_is_saved_and_it_round_trips() { + let other = MachineConfig::default(); + let saved = toml::to_string(&other).unwrap(); + assert!(!saved.contains("[fuel]"), "{saved}"); + let mut fuel = MachineConfig::default(); + fuel.fuel.console = FuelConsole::Ioc3B; + fuel.fuel.flash = Some("flash.bin".into()); + fuel.fuel.l1_port = Some(18882); + fuel.fuel.l1_log = Some("l1.log".into()); + fuel.fuel.ps2_input = false; + let saved = toml::to_string(&fuel).unwrap(); + assert!(saved.contains("[fuel]"), "{saved}"); + let loaded: MachineConfig = toml::from_str(&saved).unwrap(); + assert_eq!(loaded.fuel.console, FuelConsole::Ioc3B); + assert_eq!(loaded.fuel.flash.as_deref(), Some("flash.bin")); + assert_eq!(loaded.fuel.l1_port, Some(18882)); + assert_eq!(loaded.fuel.l1_log.as_deref(), Some("l1.log")); + assert!(!loaded.fuel.ps2_input); + assert!(!MachineProfile::ALL.contains(&MachineProfile::FuelIp34)); + } + + /// PS/2 is the Fuel's standard keyboard and mouse, and a `[fuel]` table + /// that does not name it keeps it. + #[test] + fn fuel_input_defaults_to_ps2() { + let none: MachineConfig = toml::from_str("[fuel]\nflash = \"flash.bin\"\n").unwrap(); + assert!(none.fuel.ps2_input); + assert!(MachineConfig::default().fuel.is_default()); + let off: MachineConfig = toml::from_str("[fuel]\nps2_input = false\n").unwrap(); + assert!(!off.fuel.is_default(), "a change is written back"); + } + + /// The Fuel's processor is fixed, so `machine.cpu` may only name it (or the + /// family, as the key did before the R14000 had a name); the default + /// `r4400` is refused rather than silently ignored. + /// The L1 identity keys parse, validate and round trip; unset they are + /// left out of a written config. + #[test] + fn fuel_l1_identity_keys_are_validated_and_round_trip() { + let base = "banks = [256, 256]\n[machine]\nprofile = \"fuel_ip34\"\ncpu = \"r14000\"\n"; + let ok: MachineConfig = toml::from_str(&format!( + "{base}[fuel]\nl1_revision = \"1.50.2\"\nbrick_serial = \"EMUN03\"\n" + )).unwrap(); + ok.validate().unwrap(); + let saved = toml::to_string(&ok).unwrap(); + let back: MachineConfig = toml::from_str(&saved).unwrap(); + assert_eq!(back.fuel.l1_revision.as_deref(), Some("1.50.2")); + assert_eq!(back.fuel.brick_serial.as_deref(), Some("EMUN03")); + let plain: MachineConfig = toml::from_str(base).unwrap(); + plain.validate().unwrap(); + assert!(plain.fuel.l1_revision.is_none() && plain.fuel.brick_serial.is_none()); + let bad_rev: MachineConfig = toml::from_str(&format!("{base}[fuel]\nl1_revision = \"1.48\"\n")).unwrap(); + assert!(bad_rev.validate().unwrap_err().contains("fuel.l1_revision")); + let long = "x".repeat(38); + let bad_serial: MachineConfig = toml::from_str(&format!("{base}[fuel]\nbrick_serial = \"{long}\"\n")).unwrap(); + assert!(bad_serial.validate().unwrap_err().contains("fuel.brick_serial")); + } + + #[test] + fn fuel_takes_only_its_r14000() { + let mut cfg = MachineConfig::default(); + cfg.machine.profile = MachineProfile::FuelIp34; + cfg.banks.fill(0); + cfg.banks[0] = 256; + cfg.banks[1] = 256; + cfg.scsi.clear(); + // Leaving `cpu` out takes the Fuel's own part. + cfg.machine.cpu = None; + cfg.validate().unwrap(); + assert_eq!(cfg.cpu(), &crate::cpu::parts::R14000_500); + for name in ["r14000", "r14000_500"] { + cfg.machine.cpu = Some(name.into()); + cfg.validate().unwrap(); + assert_eq!(cfg.cpu(), &crate::cpu::parts::R14000_500, "{name}"); + } + for name in ["r10000", "r12000_360", "r14000_600"] { + cfg.machine.cpu = Some(name.into()); + assert!(cfg.validate().unwrap_err().contains("r14000_500"), "{name}"); + } + } + + #[test] + fn fuel_requires_its_fixed_memory_population_and_a_vpro_or_no_graphics() { + let mut cfg = MachineConfig::default(); + cfg.machine.profile = MachineProfile::FuelIp34; + cfg.banks.fill(0); + cfg.banks[0] = 256; + cfg.banks[1] = 256; + cfg.scsi.clear(); + assert_eq!(cfg.graphics_board(), GraphicsBoard::None); + cfg.network.mode = NetMode::Off; + cfg.validate().unwrap(); + cfg.network.mode = NetMode::Pcap; + cfg.validate().unwrap(); + cfg.graphics.board = Some(GraphicsBoard::Newport); + assert!(cfg.validate().unwrap_err().contains("none")); + for board in [GraphicsBoard::V12, GraphicsBoard::V10] { + cfg.graphics.board = Some(board); + cfg.validate().unwrap(); + } + cfg.graphics.board = Some(GraphicsBoard::V6); + assert!(cfg.validate().unwrap_err().contains("not a Fuel board")); + cfg.graphics.board = None; + cfg.banks[1] = 0; + assert!(cfg.validate().unwrap_err().contains("256, 256")); + } + + #[test] + fn fuel_accepts_wide_scsi_ids_on_both_channels_without_changing_narrow_profiles() { + let text = "banks = [256, 256]\n[machine]\nprofile = \"fuel_ip34\"\ncpu = \"r14000\"\n[scsi.15]\npath = \"disk.raw\"\n[scsi.31]\npath = \"disk1.raw\"\n"; + let mut cfg: MachineConfig = toml::from_str(text).unwrap(); + cfg.validate().unwrap(); + // The Octane's buses are wide as well; the Indy's WD33C93A is narrow. + cfg.machine.profile = MachineProfile::IndyIp24; + cfg.machine.cpu = Some("r4400".into()); + assert!(cfg.validate().unwrap_err().contains("out of range")); + } +} + +#[cfg(test)] +mod tezro_config_tests { + use super::*; + + fn tezro() -> MachineConfig { + let mut cfg = MachineConfig::default(); + cfg.machine.profile = MachineProfile::TezroIp53; + cfg.set_cpu(&parts::R16000_700_EARLY); + cfg.scsi.clear(); + cfg + } + + /// Left at the generic default, `banks` gives the Tezro its 4 GiB layout; + /// a listed population is taken bank by bank. + #[test] + fn tezro_memory_defaults_to_the_4_gib_layout_and_takes_listed_banks() { + let mut cfg = tezro(); + cfg.validate().unwrap(); + assert_eq!(cfg.tezro_memory().unwrap(), crate::ip53_machine::MemoryPopulation::default()); + cfg.banks = [1024, 512, 0, 0, 0, 0, 0, 0]; + cfg.validate().unwrap(); + assert_eq!(cfg.tezro_memory().unwrap().banks_mib(), [1024, 512]); + cfg.banks = [1024, 0, 512, 0, 0, 0, 0, 0]; + assert!(cfg.validate().unwrap_err().contains("fill the banks")); + cfg.banks = [256, 256, 0, 0, 0, 0, 0, 0]; + assert!(cfg.validate().unwrap_err().contains("512 or 1024")); + } + + /// A `[machine]` with no `cpu` gets the Tezro's first part, the 700 MHz + /// R16000, and so does the bare family name and its full name (`r16000_700` + /// is the early part: it is the only 700 MHz one listed); a part name + /// selects that part. `--cpu` overrides all of it. + #[test] + fn an_unset_cpu_defaults_to_the_first_part_and_a_bare_family_name_takes_it() { + let load = |text: &str| MachineConfig::parse_toml(text).unwrap(); + let tezro = load("[machine]\nprofile = \"tezro_ip53\"\n"); + assert_eq!(tezro.cpu(), &parts::R16000_700_EARLY); + tezro.validate().unwrap(); + for name in ["r16000", "r16000_700"] { + let first = load(&format!("[machine]\nprofile = \"tezro_ip53\"\ncpu = \"{name}\"\n")); + assert_eq!(first.cpu_part(), Ok(&parts::R16000_700_EARLY), "{name} takes the first part"); + } + let fast = load("[machine]\nprofile = \"tezro_ip53\"\ncpu = \"r16000_800\"\n"); + assert_eq!(fast.cpu(), &parts::R16000_800); + fast.validate().unwrap(); + let mut two = fast.clone(); + two.machine.cpus = Some(2); + assert!(two.validate().unwrap_err().contains("4 processors only"), "the 800 MHz part with two"); + let cli = Cli::parse_from(["iris", "--cpu", "r16000_800"]); + assert_eq!(cli.apply(tezro).cpu(), &parts::R16000_800, "--cpu names a part"); + let fuel = load("banks = [256, 256]\n[machine]\nprofile = \"fuel_ip34\"\n"); + assert_eq!(fuel.cpu(), &parts::R14000_500); + } + + /// A part the Tezro lacks is one generic error that names the machine and + /// the parts it takes, and so is a name that is not a part. + #[test] + fn a_part_the_tezro_lacks_is_refused_naming_its_parts() { + let load = |cpu: &str| MachineConfig::parse_toml(&format!("[machine]\nprofile = \"tezro_ip53\"\ncpu = \"{cpu}\"\n")).unwrap(); + let parts = "SGI Tezro (IP53) takes [machine] cpu = r16000_700 or r16000_800 or r16000_900 or r16000_1000"; + for cpu in ["r4400", "r10000", "r12000_360", "r14000", "r16000_700_4mb", "r16000_800_8mb", "r16000_1000_16mb", "r16000_600"] { + assert_eq!(load(cpu).validate().unwrap_err(), parts, "{cpu}"); + } + } + + /// The file loader applies the same defaults as the text parser, and + /// refuses an unknown CPU value when the config is checked. + #[test] + fn try_load_applies_tezro_defaults_and_cpu_errors() { + let dir = std::env::temp_dir().join(format!("iris-tezro-try-load-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("tezro.toml"); + let load = |text: &str| { + std::fs::write(&path, text).unwrap(); + MachineConfig::try_load(path.to_str().unwrap()).unwrap() + }; + let cfg = load("[machine]\nprofile = \"tezro_ip53\"\n"); + assert_eq!(cfg.cpu(), &parts::R16000_700_EARLY); + assert!(!cfg.banks_set); + assert_eq!(cfg.tezro_memory().unwrap().total_mib(), 4096); + let e = load("[machine]\nprofile = \"tezro_ip53\"\ncpu = \"not_a_cpu\"\n").validate().unwrap_err(); + assert!(e.contains("SGI Tezro (IP53) takes"), "{e}"); + let _ = std::fs::remove_dir_all(&dir); + } + + /// A `banks` equal to the generic default is a choice, not "unset": the + /// Tezro takes no 128 MB bank, so `--bank0 128 --bank1 128` and a + /// `banks = [128, 128, 0, 0]` line are refused rather than read as the + /// board's 4 GiB default. + #[test] + fn tezro_honours_banks_that_equal_the_generic_default() { + let cli = Cli::parse_from(["iris", "--bank0", "128", "--bank1", "128"]); + let cfg = cli.apply(tezro()); + assert!(cfg.validate().unwrap_err().contains("512 or 1024")); + + let text = "banks = [128, 128, 0, 0]\n[machine]\nprofile = \"tezro_ip53\"\n"; + let cfg = MachineConfig::parse_toml(text).unwrap(); + assert!(cfg.validate().unwrap_err().contains("512 or 1024")); + + let cfg = MachineConfig::parse_toml("[machine]\nprofile = \"tezro_ip53\"\n").unwrap(); + assert_eq!(cfg.tezro_memory().unwrap(), crate::ip53_machine::MemoryPopulation::default()); + } + + /// The processor is one of the Tezro's R16000 chips, with no graphics + /// board or a V12. + #[test] + fn tezro_requires_an_r16000_and_takes_no_graphics_or_a_v12() { + let mut cfg = tezro(); + cfg.set_cpu(&parts::R16000_700_EARLY); + cfg.validate().unwrap(); + for cpu in [&parts::R4400_150, &parts::R5000_180, &parts::R12000_360, &parts::R14000_600] { + cfg.set_cpu(cpu); + let e = cfg.validate().unwrap_err(); + assert!(e.contains("SGI Tezro (IP53) takes") && e.contains("r16000_700"), "{}: {e}", cpu.name()); + } + assert_eq!(MachineProfile::TezroIp53.cpus(), [&parts::R16000_700_EARLY, &parts::R16000_800, &parts::R16000_900, &parts::R16000_1000]); + let mut cfg = tezro(); + assert_eq!(cfg.graphics_board(), GraphicsBoard::None); + cfg.graphics.board = Some(GraphicsBoard::Newport); + assert!(cfg.validate().is_err()); + cfg.graphics.board = Some(GraphicsBoard::V10); + assert!(cfg.validate().is_err(), "the V10 is not a Tezro board"); + for cpus in [1, 4] { + cfg.graphics.board = Some(GraphicsBoard::V12); + cfg.machine.cpus = Some(cpus); + cfg.validate().unwrap(); + } + let mut cfg = tezro(); + cfg.network.mode = NetMode::Off; + cfg.validate().unwrap(); + } + + /// `[machine] cpus` is the processors fitted: four on a Tezro unless + /// set, one elsewhere, and only the Octane and the Tezro take more. + #[test] + fn machine_cpus_defaults_to_four_on_the_tezro_and_one_elsewhere() { + let mut cfg = tezro(); + assert_eq!(cfg.machine.cpus(), 4); + cfg.validate().unwrap(); + for cpus in [1, 2] { + cfg.machine.cpus = Some(cpus); + cfg.validate().unwrap(); + assert_eq!(cfg.machine.cpus(), cpus); + } + assert_eq!(MachineConfig::default().machine.cpus(), 1); + let loaded: MachineConfig = toml::from_str("[machine]\nprofile = \"tezro_ip53\"\ncpu = \"r10000\"\ncpus = 2\n").unwrap(); + assert_eq!(loaded.machine.cpus(), 2); + let mut fuel = MachineConfig::default(); + fuel.machine.profile = MachineProfile::FuelIp34; + fuel.set_cpu(&parts::R14000_500); + fuel.machine.cpus = Some(2); + assert!(fuel.validate().unwrap_err().contains("only the Octane and the Tezro")); + } + + /// The 700 MHz chip is fitted as one, two or four processors, the 1000 + /// MHz chip as two or four, the 800 and 900 MHz chips as four only, and + /// the refusal says so. + #[test] + fn each_chip_is_fitted_with_the_counts_its_board_data_allows() { + for &chip in crate::ip53_machine::CPUS { + for cpus in 0..=5u8 { + let mut cfg = tezro(); + cfg.set_cpu(chip); + cfg.machine.cpus = Some(cpus); + let counts: &[u8] = match chip { + c if c == &parts::R16000_700_EARLY => &[1, 2, 4], + c if c == &parts::R16000_1000 => &[2, 4], + _ => &[4], + }; + let fits = counts.contains(&cpus); + assert_eq!(cfg.validate().is_ok(), fits, "{} with {cpus}", chip.name()); + } + if chip != &parts::R16000_700_EARLY { + let mut cfg = tezro(); + cfg.set_cpu(chip); + cfg.machine.cpus = Some(1); + let e = cfg.validate().unwrap_err(); + let only = if chip == &parts::R16000_1000 { "2, 4 processors only" } else { "4 processors only" }; + assert!(e.contains(only) && e.contains(&chip.label()), "{e}"); + } + } + } + + #[test] + fn tezro_refuses_a_processor_count_its_board_does_not_fit() { + for cpus in [0, 3, 5] { + let mut cfg = tezro(); + cfg.machine.cpus = Some(cpus); + assert!(cfg.validate().unwrap_err().contains("[machine] cpus"), "{cpus}"); + } + } + + /// The sample configuration in the repository root is a valid Tezro. + #[test] + fn the_sample_tezro_configuration_validates() { + let cfg: MachineConfig = toml::from_str(include_str!("../tezro.toml.example")).unwrap(); + assert_eq!(cfg.machine.profile, MachineProfile::TezroIp53); + cfg.validate().unwrap(); + assert_eq!(cfg.tezro_memory().unwrap().total_mib(), 4096); + } + + /// A `[tezro]` table is written only when something in it was set, and + /// what was set survives a save and reload. + #[test] + fn only_a_configured_tezro_section_is_saved_and_it_round_trips() { + let saved = toml::to_string(&MachineConfig::default()).unwrap(); + assert!(!saved.contains("[tezro]"), "{saved}"); + let text = "[machine]\nprofile = \"tezro_ip53\"\ncpu = \"r16000\"\n[tezro]\nconsole = \"ioc4_b\"\nflash = \"flash.bin\"\nl1_port = 18882\nserial_port_c = 18883\n"; + let loaded: MachineConfig = toml::from_str(text).unwrap(); + assert_eq!(loaded.machine.profile, MachineProfile::TezroIp53); + assert_eq!(loaded.tezro.console, TezroConsole::Ioc4B); + assert_eq!(loaded.tezro.console.index(), 2); + let again: MachineConfig = toml::from_str(&toml::to_string(&loaded).unwrap()).unwrap(); + assert_eq!(again.tezro.flash.as_deref(), Some("flash.bin")); + assert_eq!((again.tezro.l1_port, again.tezro.serial_port_c), (Some(18882), Some(18883))); + assert!(!MachineProfile::ALL.contains(&MachineProfile::TezroIp53)); + } +} diff --git a/src/cpu/build.rs b/src/cpu/build.rs index 7a8e1f0e9..b1956a73d 100644 --- a/src/cpu/build.rs +++ b/src/cpu/build.rs @@ -7,7 +7,8 @@ /// construction so the hot path carries no per-model branch. `$shadow` picks /// the shadow caches for the R4400 and R5000, which also have the functional /// ones (`CacheModel`). `$o2` selects the O2's R10000 and R12000 types, whose -/// secondary caches have 64-byte lines where the others have 128. +/// secondary caches have 64-byte lines where the others have 128. The R16000's +/// secondary cache size is a type parameter, so `l2_kb` picks the type here, once. #[macro_export] macro_rules! with_cpu { ($part:expr, $shadow:expr, $o2:expr, $C:ident => $body:expr) => {{ @@ -23,6 +24,24 @@ macro_rules! with_cpu { (F::R10000, _, false) => { type $C = sh::R10000ShadowCache; $body } (F::R12000 | F::R14000, _, true) => { type $C = sh::R12000O2ShadowCache; $body } (F::R12000 | F::R14000, _, false) => { type $C = sh::R12000ShadowCache; $body } + (F::R16000, _, _) if $part.l2_kb == 8192 => { type $C = sh::R16000Scache8ShadowCache; $body } + (F::R16000, _, _) if $part.l2_kb == 16384 => { type $C = sh::R16000Scache16ShadowCache; $body } + (F::R16000, _, _) => { type $C = sh::R16000ShadowCache; $body } } }}; } + +#[cfg(test)] +mod tests { + use crate::cpu::mips_cache_v2::{CpuModel, MipsCache}; + use crate::cpu::parts::{R16000_1000, R16000_700_EARLY, R16000_800, R16000_900}; + + /// `l2_kb` picks the R16000 type: 4, 8 or 16 MiB of secondary cache. + #[test] + fn an_r16000_part_gets_the_cache_type_of_its_secondary_size() { + for (part, mib) in [(&R16000_700_EARLY, 4), (&R16000_800, 4), (&R16000_900, 8), (&R16000_1000, 16)] { + let (name, bytes) = crate::with_cpu!(part, true, false, C => (::NAME, ::L2_SIZE)); + assert_eq!((name, bytes >> 20), ("R16000", mib), "{}", part.name()); + } + } +} diff --git a/src/cpu/jitv2/equiv_test.rs b/src/cpu/jitv2/equiv_test.rs index f8ec28ef6..1c2283d0b 100644 --- a/src/cpu/jitv2/equiv_test.rs +++ b/src/cpu/jitv2/equiv_test.rs @@ -4544,6 +4544,22 @@ mod tests { } } + /// The R16000 types take the R10000 family's LWR path, whatever their + /// secondary cache. + #[test] + fn jit_lwr_matches_the_interpreter_on_the_r16000() { + use crate::cpu::mips_cache_shadow::{R16000Scache16ShadowCache, R16000Scache8ShadowCache, R16000ShadowCache}; + use crate::cpu::mips_core::STATUS_KX; + fn check>>() { + let (interp, jit) = lwr_both::(STATUS_KX, 1); + assert_eq!(jit, interp, "{}", C::NAME); + assert_eq!(interp.gpr[2] >> 32, 0x1122_3344, "{} keeps the upper word", C::NAME); + } + check::(); + check::(); + check::(); + } + /// One instruction at 0x1000, interpreted and compiled, on CPU model `C` /// after `setup`. The JIT learns the model's choices from /// `install_jit_hooks`, as in production. diff --git a/src/cpu/mips_cache_shadow.rs b/src/cpu/mips_cache_shadow.rs index 785c027de..30b643675 100644 --- a/src/cpu/mips_cache_shadow.rs +++ b/src/cpu/mips_cache_shadow.rs @@ -109,7 +109,65 @@ const MRU_BIT: u64 = 1 << 63; const PD_STATE_SHIFT: u32 = 6; const SD_STATE_SHIFT: u32 = 10; +/// The primary instruction state, TagLo[6] (NEC U10278EJ3V0UM p. 285). +const PI_VALID: u64 = 1 << 6; + +/// A primary instruction word is the instruction predecoded, 36 bits (NEC +/// U10278EJ3V0UM p. 288). Neither manual gives the layout. This one is +/// inferred from the words PROM 6.210 stores, `p` the word and `f` the +/// instruction: p[25:20] = f[5:0], p[19:15] = f[10:6], p[14:10] = f[25:21], +/// p[28:26] = f[28:26], p[35:29] a class that also gives f[31:29], and p[4:0] +/// and p[9:5] the rt and rd fields in the order the class gives. Only the +/// sampled classes decode, and every row is inferred. `addiu $4,$0,2` +/// (0x24040002) is 0x3_4420_0004. +/// +/// Rows: class, f[31:29], whether p[4:0] is rt, the f[28:26] values as a bit +/// mask, and the function fields of the SPECIAL members. +const L1I_CLASSES: &[(u8, u32, bool, u8, &[u8])] = &[ + (0x00, 0, false, 1 << 0, &[0x00]), // nop + (0x08, 0, false, 1 << 0, &[0x38, 0x3c]), // dsll, dsll32 + (0x18, 0, false, 1 << 0, &[0x25]), // or + (0x1a, 1, true, 0b1010_0011, &[]), // addi, addiu, ori, lui + (0x1e, 3, true, 1 << 1, &[]), // daddiu + (0x35, 5, false, 1 << 7, &[]), // cache + (0x36, 6, true, 1 << 7, &[]), // ld + (0x3f, 2, false, 1 << 0, &[]), // mtc0 + (0x40, 0, false, 1 << 3, &[]), // jal + (0x41, 0, false, 0b101, &[0x08]), // jr, j + (0x71, 0, false, 1 << 1, &[]), // REGIMM +]; + +/// The model's own form, bits 35:32 set over an instruction no class holds. +/// Hardware would never produce it. It takes classes 0x78-0x7f, so a real +/// word of those classes would be read as raw. +const L1I_RAW: u64 = 0xf << 32; + +/// The instruction a predecoded word encodes, or the class that is refused. +fn l1i_decode(p: u64) -> Result { + if p >> 32 == L1I_RAW >> 32 { + return Ok(p as u32); + } + let (class, op, funct) = ((p >> 29) as u8 & 0x7f, (p >> 26) as u32 & 7, (p >> 20) as u8 & 0x3f); + let &(_, hi3, rt_low, ops, functs) = L1I_CLASSES.iter().find(|c| c.0 == class).ok_or(class)?; + if ops >> op & 1 == 0 || (op == 0 && !functs.is_empty() && !functs.contains(&funct)) || (class == 0 && p != 0) { + return Err(class); + } + let (low, high) = ((p & 0x1f) as u32, (p >> 5) as u32 & 0x1f); + let (rt, rd) = if rt_low { (low, high) } else { (high, low) }; + Ok(hi3 << 29 | op << 26 | ((p >> 10) as u32 & 0x1f) << 21 | rt << 16 | rd << 11 | ((p >> 15) as u32 & 0x1f) << 6 | funct as u32) +} +/// The predecoded word for `f`, or the raw form where no class holds it. +fn l1i_encode(f: u32) -> u64 { + let w = f as u64; + let (rt, rd) = (w >> 16 & 0x1f, w >> 11 & 0x1f); + L1I_CLASSES.iter().find_map(|&(class, _, rt_low, _, _)| { + let (low, high) = if rt_low { (rt, rd) } else { (rd, rt) }; + let p = (class as u64) << 29 | (w >> 26 & 7) << 26 | (w & 0x3f) << 20 | (w >> 6 & 0x1f) << 15 + | (w >> 21 & 0x1f) << 10 | high << 5 | low; + (l1i_decode(p) == Ok(f)).then_some(p) + }).unwrap_or(L1I_RAW | w) +} /// R10000 only: ways per set, as CACHE index operations address them. Bit 0 /// of the index selects the way; the set starts above the line offset. The @@ -182,6 +240,14 @@ pub struct ShadowCache< /// valid lines hold their pages. Null without one. view_space: UnsafeCell<*const PpMemSpace>, view: UnsafeCell, + /// The held pages whose generation counters are not ppmem's, with the + /// counters carrying their `GEN_HELD`: see [`Self::mark_foreign`]. + #[cfg(feature = "jitv2")] + foreign: UnsafeCell>, + /// For each primary instruction set, the 64-byte half of a held + /// secondary line that a refill there would load (`refilled_word`), or + /// `u64::MAX`. + refill: UnsafeCell>, } // Safety: the CPU thread is the only accessor, as for every other cache model @@ -240,6 +306,9 @@ impl< decode: UnsafeCell::new(crate::cpu::decode_cache::DecodeCache::new()), view_space: UnsafeCell::new(std::ptr::null()), view: UnsafeCell::new(0), + #[cfg(feature = "jitv2")] + foreign: UnsafeCell::new(Vec::new()), + refill: UnsafeCell::new(vec![u64::MAX; Self::PI_SETS].into_boxed_slice()), } } @@ -370,10 +439,19 @@ impl< if s.held.get(self.tag_slot(sel, virt_addr)) == Some(&true) { self.store_into_held_line(sel, virt_addr, phys_addr); } + if sel == CACH_PI { + // The set now holds what the guest wrote, not a refill. + if let Some(r) = unsafe { &mut *self.refill.get() }.get_mut((virt_addr as usize / IC_LINE.max(1)) % Self::PI_SETS.max(1)) { + *r = u64::MAX; + } + } } // ---- valid lines hold pages in the CPU's cached view ---- + /// Primary instruction sets, as Index Load Data (I) addresses them. + const PI_SETS: usize = if IC_LINE == 0 { 0 } else { IC_SIZE / WAYS / IC_LINE }; + fn view_space(&self) -> Option<&PpMemSpace> { unsafe { (*self.view_space.get()).as_ref() } } @@ -394,6 +472,9 @@ impl< CACH_PD if DC_LINE != 0 && (tag >> PD_STATE_SHIFT) & 3 != 0 => { Some((pa_hi | ((tag >> 8) & 0xff_ffff) << 12 | (va & 0xfff & !(DC_LINE as u64 - 1)), DC_LINE)) } + CACH_PI if IC_LINE != 0 && tag & PI_VALID != 0 => { + Some((pa_hi | ((tag >> 8) & 0xff_ffff) << 12 | (va & 0xfff & !(IC_LINE as u64 - 1)), IC_LINE)) + } CACH_SD if L2_LINE != 0 && (tag >> SD_STATE_SHIFT) & 3 != 0 => { Some((pa_hi | ((tag >> 14) & 0x3_ffff) << 18 | (va & 0x3_ffff & !(L2_LINE as u64 - 1)), L2_LINE)) } @@ -437,10 +518,23 @@ impl< /// into it from the data array, since Index Store Tag writes the tag /// alone (NEC VR5000/VR10000 instruction manual pp. 24-25). A line with /// no memory behind it holds no page and stays in the arrays, as every - /// line did before. + /// line did before, unless the machine allows lines there. fn line_enters(&self, sp: &PpMemSpace, sel: u32, idx: usize, va: u64, (pa, len): (u64, usize)) { + let unbacked = !sp.backs(pa); let Some(new) = sp.hold_line_page(self.view_no(), pa) else { return }; let base = unsafe { *self.tc_base.get() }; + if new { + self.mark_foreign(sp, pa, true); + } + if new && unbacked { + // No memory behind the page yet: it starts as what the cache's + // bus shows there. + let gran = crate::ppmem::granularity() as u64; + for a in (pa & !(gran - 1)..(pa & !(gran - 1)) + gran).step_by(8) { + let r = self.downstream.read64(a); + unsafe { *(base.add(a as usize) as *mut u64) = if r.is_ok() { r.data.rotate_left(32) } else { 0 } }; + } + } if l2_op_log(CACHE_LOG_OP) { crate::dlog!(LogModule::L2c, "shadow: -> line {pa:#x} holds its page (new: {new})"); } @@ -455,6 +549,7 @@ impl< let word = s.data[self.data_slot(sel, s.data.len(), first + (unit * k) as u64 + way)]; let at = pa + (unit * k) as u64; match sel { + CACH_PI => self.put_instruction(at, word), CACH_PD => unsafe { *(base.add(at as usize) as *mut u32) = word as u32 }, // TagHi is the doubleword's high word and TagLo its low word // (MIPS R10000 UM p. 201; NEC VR10000 UM p. 289, Fig. 13-33). @@ -468,6 +563,9 @@ impl< _ => unsafe { *(base.add(at as usize) as *mut u64) = word }, } } + if sel == CACH_SD { + self.set_refill(s.tags[idx], pa, len, true); + } } /// The line in slot `idx` of `sel` leaves, unwritten, as Index Store Tag @@ -476,12 +574,52 @@ impl< fn line_leaves(&self, sp: &PpMemSpace, sel: u32, idx: usize, (pa, len): (u64, usize)) { let s = self.shadow(sel); s.held[idx] = false; + if sel == CACH_SD { + self.set_refill(s.tags[idx], pa, len, false); + } let gone = sp.release_line_page(self.view_no(), pa); + if gone { + self.mark_foreign(sp, pa, false); + } if l2_op_log(CACHE_LOG_OP) { crate::dlog!(LogModule::L2c, "shadow: -> line {pa:#x} leaves (page released: {gone})"); } } + /// ppmem puts `GEN_HELD` on its own generation counters. Where the cache's + /// bus counts a held page's writes on a counter of its own, which compiles + /// check instead (the Fuel's reset PROM, mapped over memory), the mark + /// goes there too, and comes off with the page's last line. A page + /// outside any counter, or on ppmem's, is skipped. + #[cfg_attr(not(feature = "jitv2"), allow(unused_variables))] + fn mark_foreign(&self, sp: &PpMemSpace, pa: u64, held: bool) { + #[cfg(feature = "jitv2")] + { + use crate::cpu::jitv2::jitv2::GEN_HELD; + use crate::ppmem::MappedMemory; + use std::sync::atomic::{AtomicU64, Ordering}; + let page = pa & !(crate::ppmem::granularity() as u64 - 1); + let foreign = unsafe { &mut *self.foreign.get() }; + if held { + for a in (page..page + crate::ppmem::granularity() as u64).step_by(4096) { + let g = self.downstream.gen_ptr(a); + let own = unsafe { sp.gen_window_base().add((a >> 12) as usize) as *const AtomicU64 }; + if !g.is_null() && g != own { + unsafe { (*g).fetch_add(GEN_HELD, Ordering::Release) }; + foreign.push((page, g)); + } + } + } else { + foreign.retain(|&(p, g)| { + if p == page { + unsafe { (*g).fetch_add(1u64.wrapping_sub(GEN_HELD), Ordering::Release) }; + } + p != page + }); + } + } + } + /// Index Store Data into a held line writes through to its page, so the /// next cached access sees it. Same quadword rule as the data array. fn store_into_held_line(&self, sel: u32, va: u64, value: u64) { @@ -491,24 +629,79 @@ impl< let at = pa + (va & (len as u64 - 1) & !(unit - 1)); let base = unsafe { *self.tc_base.get() }; match sel { + CACH_PI => self.put_instruction(at, value & 0xf_ffff_ffff), CACH_PD => unsafe { *(base.add(at as usize) as *mut u32) = value as u32 }, _ => unsafe { *(base.add(at as usize) as *mut u64) = value; *(base.add((at ^ 8) as usize) as *mut u64) = 0; }, } + if sel == CACH_SD { + self.set_refill(self.shadow(sel).tags[idx], pa, len, true); + } + } + + /// A primary instruction word goes into its line's page as the + /// instruction it predecodes, so a fetch runs it. A word of a class the + /// model does not know leaves the page as it was, where hardware would + /// run it. + fn put_instruction(&self, at: u64, word: u64) { + match l1i_decode(word) { + Ok(f) => unsafe { *((*self.tc_base.get()).add(at as usize) as *mut u32) = f }, + Err(class) => crate::dlog!(LogModule::L2c, + "shadow: primary instruction word {word:#011x} at {at:#x}: predecode class {class:#04x} unknown, memory's word runs"), + } + } + + /// A held secondary line `[pa, pa + len)` with tag `tag` is what a primary + /// instruction refill in each set it maps to would load, or stops being. + /// The set is the line's virtual index, TagLo[8:7] (VA[13:12], NEC + /// U10278EJ3V0UM p. 287), over PA[11:6]. Two lines for one set: the + /// later wins. + fn set_refill(&self, tag: u64, pa: u64, len: usize, held: bool) { + let refill = unsafe { &mut *self.refill.get() }; + if !held { + // By address: the tag may already name another line. + refill.iter_mut().filter(|r| (pa..pa + len as u64).contains(*r)).for_each(|r| *r = u64::MAX); + return; + } + for half in (pa..pa + len as u64).step_by(IC_LINE.max(1)) { + if let Some(r) = refill.get_mut(((tag >> 7 & 3) << 12 | (half & 0xfff)) as usize / IC_LINE) { + *r = half; + } + } + } + + /// Index Load Data (I) in a set a held secondary line maps to, which + /// Index Store Data (I) has not written since and where no held primary + /// line is: the word a refill from that + /// line would have left, predecoded. The model never refills on a fetch, + /// so this assumes the line was refilled, whichever way the operand + /// names. PROM 6.210, the IP35 machines' PROM, runs a staged secondary + /// line and reads the primary words back to stage them itself. Index + /// Load Tag (I) still + /// reads the stored tag. + #[cold] + #[inline(never)] + fn refilled_word(&self, va: u64) -> Option { + let half = *unsafe { &*self.refill.get() }.get((va as usize / IC_LINE.max(1)) % Self::PI_SETS.max(1))?; + if half == u64::MAX || self.shadow(CACH_PI).held[self.tag_slot(CACH_PI, va)] { + return None; + } + let at = half + (va & (IC_LINE as u64 - 1) & !3); + Some(l1i_encode(unsafe { *((*self.tc_base.get()).add(at as usize) as *const u32) })) } /// How many lines hold pages. fn held_count(&self) -> usize { - [CACH_PD, CACH_SD].iter().map(|&sel| self.shadow(sel).held.iter().filter(|&&h| h).count()).sum() + [CACH_PI, CACH_PD, CACH_SD].iter().map(|&sel| self.shadow(sel).held.iter().filter(|&&h| h).count()).sum() } /// Every held line goes, with no writeback: power-on, or a snapshot /// restore. fn drop_held_lines(&self) { let Some(sp) = self.view_space() else { return }; - for sel in [CACH_PD, CACH_SD] { + for sel in [CACH_PI, CACH_PD, CACH_SD] { for i in 0..self.shadow(sel).held.len() { if self.shadow(sel).held[i] { let line = self.line_of(sel, self.shadow(sel).tags[i], self.slot_va(sel, i)).expect("a held line is valid"); @@ -554,10 +747,12 @@ impl< model::R4400 => "R4400", model::R5000 => "R5000", model::R12000 => "R12000", + model::R16000 => "R16000", _ => "shadow", }; const CACHE_KIND: &'static str = "shadow"; const R10K_CACHE_OPS: bool = Self::R10K; + const FETCH_ALIAS: bool = MODEL == model::R16000; } impl< @@ -808,7 +1003,7 @@ impl< s.mru[set] = phys_addr & MRU_BIT != 0; let old = s.tags[idx]; s.tags[idx] = phys_addr & mask & !MRU_BIT; - if Self::R10K && matches!(sel, CACH_PD | CACH_SD) && old != s.tags[idx] { + if Self::R10K && matches!(sel, CACH_PI | CACH_PD | CACH_SD) && old != s.tags[idx] { self.tag_changes(sel, idx, virt_addr, old, s.tags[idx]); } } @@ -845,6 +1040,11 @@ impl< crate::dlog!(LogModule::L2c, "shadow: -> ILD slot={slot} data={:#018x} ecc={:#x}", s.data[slot], s.ecc[slot]); } + if sel == CACH_PI { + if let Some(p) = self.refilled_word(virt_addr) { + return p; + } + } s.data[slot] } } @@ -998,6 +1198,27 @@ pub type R12000ShadowCache = pub type R12000Module4MShadowCache = ShadowCache<32768, 64, 32768, 32, 4194304, 128, true, 0x0000_0e35, 0x0000_0900, 64, { model::R12000 }>; +/// The R16000 with a 4 MB secondary cache, the Tezro's 700 and 800 MHz parts: +/// the R12000's geometry and programming model with the larger secondary (SS +/// code 3 in Config) and 128-byte secondary lines (SB: a model choice). The +/// PRId here is a placeholder: the part's own, `Part::prid`, is set at build +/// time, as for the R14000. FIR 0x0900, the one R14000 value on record; a +/// model choice: no R16000 FIR was found in the retained evidence. +pub type R16000ShadowCache = + ShadowCache<32768, 64, 32768, 32, 4194304, 128, true, 0x0000_0f41, 0x0000_0900, 64, { model::R16000 }>; + +/// The R16000 with an 8 MB secondary cache, the 900 MHz part's: the 4 MB type +/// with twice the secondary (SS code 4). The size is a type parameter so the +/// executor, the translator and the cache know it when they are built, and no +/// run-time field is read to find it. +pub type R16000Scache8ShadowCache = + ShadowCache<32768, 64, 32768, 32, 8388608, 128, true, 0x0000_0f50, 0x0000_0900, 64, { model::R16000 }>; + +/// The R16000 with a 16 MB secondary cache, the 1 GHz part's (SS code 5); +/// otherwise the 4 MB type. +pub type R16000Scache16ShadowCache = + ShadowCache<32768, 64, 32768, 32, 16777216, 128, true, 0x0000_0f50, 0x0000_0900, 64, { model::R16000 }>; + /// R12000 3.5 as the O2 has it: the Octane part with 64-byte secondary lines /// (SB clear). With 128-byte lines the O2 PROM panics before its banner /// (write TLB miss at 0x0); 64 boots. IRIX on the O2 never reads SB and sizes @@ -1005,7 +1226,6 @@ pub type R12000Module4MShadowCache = pub type R12000O2ShadowCache = ShadowCache<32768, 64, 32768, 32, 2097152, 64, true, 0x0000_0e35, 0x0000_0900, 64, { model::R12000 }>; - /// SGI O2 R10000 (IP32): the IP28 model with 64-byte secondary-cache lines. /// /// The O2 PROM (4.18 and earlier) handles only that line size. With Config's @@ -1238,6 +1458,20 @@ mod tests { } assert_eq!(run::(), 0); assert_eq!(run::(), 0x1_2345_6780); + assert_eq!(run::(), 0, "the R16000 follows the R12000 and R14000"); + } + + /// The three R16000 types differ in the secondary cache alone: size, the + /// SS code Config reads from it, and the placeholder PRId each starts + /// with; the PRId a part really presents is `Part::prid`'s. + #[test] + fn the_r16000_types_differ_only_in_the_secondary_cache() { + fn id() -> (u32, u32, usize, usize, &'static str, u32, u32, bool) { + (C::PRID, C::FIR, C::L2_SIZE, C::L2_LINE, C::NAME, C::VA_BITS, C::PA_BITS, C::R10K_CACHE_OPS) + } + assert_eq!(id::(), (0xf41, 0x900, 4 << 20, 128, "R16000", 44, 40, true)); + assert_eq!(id::(), (0xf50, 0x900, 8 << 20, 128, "R16000", 44, 40, true)); + assert_eq!(id::(), (0xf50, 0x900, 16 << 20, 128, "R16000", 44, 40, true)); } /// Geometry is what CP0 Config is built from, so it must be the real @@ -1578,6 +1812,41 @@ mod tests { assert_eq!(c.read::<4>(0, 0x6000).data, 0x600d); } + /// The R16000 parts keep the R10000 family's index operations: a valid + /// primary data line or secondary line holds its page in the CPU's view, + /// with 4, 8 and 16 MiB secondary caches, and Index Store Data fills it. + #[test] + fn the_r16000_types_hold_lines_in_the_view() { + fn run>>() { + let (space, mut banks) = PpMemSpace::with_bank_sizes(&[64]).unwrap(); + let space = Box::new(space); + let bank = Arc::new(banks.remove(0)); + let c = Box::new(C::from(bank.clone() as Arc)); + let v = space.add_cached_view(&[c.tcache_bitmap_ptr()]).unwrap(); + unsafe { + c.set_tcache_window(space.view_base(v)); + c.set_cached_view(&*space, v); + } + space.map_bank(0, 0, 64 << 20, 64 << 20).unwrap(); + bank.write32(0x2004, 0x5555_5555); + c.cache_op(C_R10K_ISD | CACH_PD, 0x8000_0004, 0xdead_beef); + c.cache_op(C_IST | CACH_PD, 0x8000_0000, pd_dirty(0x2000)); + assert_eq!(space.held_line_pages(v), vec![0x2000]); + assert_eq!(c.read::<4>(0, 0x2004).data, 0xdead_beef); + assert_eq!(bank.read32(0x2004).data, 0x5555_5555, "memory never saw the line"); + c.cache_op(C_R10K_ISD | CACH_SD, 0x8000_3008, 0x1111_2222_3333_4444); + c.cache_op(C_IST | CACH_SD, 0x8000_3000, sd_tag(0x3000, 2)); + assert_eq!(space.held_line_pages(v), vec![0x2000, 0x3000]); + assert_eq!(c.read::<4>(0, 0x3008).data, 0x3333_4444, "TagLo at the lower address"); + c.cache_op(C_IST | CACH_PD, 0x8000_0000, 0); + c.cache_op(C_IST | CACH_SD, 0x8000_3000, 0); + assert!(space.held_line_pages(v).is_empty()); + } + run::(); + run::(); + run::(); + } + /// Power-on drops every held page. A snapshot taken while lines hold /// pages is refused at restore. #[test] @@ -1591,4 +1860,77 @@ mod tests { assert!(c.debug_probe("l2", 0, 0).contains("0 valid lines hold 0 pages")); assert!(c.load_cache_state(&c.save_cache_state()).is_ok()); } + + // ---- primary instruction lines ---- + + /// Every class of the table, encoded and decoded back, one field moved + /// at a time for the example word, and what the table refuses. + #[test] + fn predecoded_words_round_trip() { + let p = 0x3_4420_0004; // addiu $4,$0,2 + for (q, f) in [(p, 0x2404_0002), (p | 0x1f << 5, 0x2404_f802), (p | 0x1f << 10, 0x27e4_0002), (p | 0x1f << 15, 0x2404_07c2)] { + assert_eq!(l1i_decode(q), Ok(f), "{q:#x}"); + } + for (class, f) in [(0x08, 0x0009_20f8), (0x08, 0x0009_20fc), (0x18, 0x0109_5025), (0x1a, 0x2084_0010), + (0x1a, 0x3484_0010), (0x1a, 0x3c04_8000), (0x1e, 0x6484_0010), (0x35, 0xbd04_0040), + (0x36, 0xdd04_0008), (0x3f, 0x4084_6000), (0x40, 0x0c00_1234), (0x41, 0x03e0_0008), + (0x41, 0x0800_1234), (0x71, 0x0481_0004), (0, 0)] { + assert_eq!((l1i_encode(f) >> 29, l1i_decode(l1i_encode(f))), (class, Ok(f)), "{f:#x}"); + } + for f in [0x0009_20c0, 0x2884_0010, 0x4422_3000] { // sll, slti, mtc1: no class + assert_eq!((l1i_encode(f), l1i_decode(L1I_RAW | f as u64)), (L1I_RAW | f as u64, Ok(f))); + } + assert_eq!((l1i_decode(1), l1i_decode(0x2a << 29)), (Err(0), Err(0x2a))); + } + + /// TagLo of a valid primary instruction line for `pa`'s page. + fn pi_valid(pa: u64) -> u64 { (pa >> 12) << 8 | PI_VALID } + + /// A valid primary instruction line puts the instructions its words + /// predecode into the page, leaving memory's word for one it cannot + /// decode, and Index Store Data (I) writes through. + #[test] + fn a_primary_instruction_line_puts_its_instructions_in_the_page() { + let (c, space, bank) = windowed(); + bank.write32(0x9008, 0x1111_1111); + let va = 0x8000_9001u64; // way 1 + c.cache_op(C_R10K_ISD | CACH_PI, va, 0x3_4420_0004); + c.cache_op(C_R10K_ISD | CACH_PI, va + 8, 0x2a << 29); + c.cache_op(C_IST | CACH_PI, va, pi_valid(0x9000)); + assert_eq!(space.held_line_pages(0), vec![0x9000]); + assert_eq!([0x9000, 0x9008].map(|a| c.read::<4>(0, a).data), [0x2404_0002, 0x1111_1111]); + c.cache_op(C_R10K_ISD | CACH_PI, va + 4, l1i_encode(0x03e0_0008)); + assert_eq!(c.read::<4>(0, 0x9004).data, 0x03e0_0008); + assert_eq!(c.cache_op(C_R10K_ILD | CACH_PI, va, 0), 0x3_4420_0004, "Index Load Data (I) reads the array"); + assert_eq!(bank.read32(0x9000).data, 0, "memory never saw it"); + c.cache_op(C_IST | CACH_PI, va, 0); + assert!(space.held_line_pages(0).is_empty()); + } + + /// A held secondary line answers Index Load Data (I) in the primary sets + /// it maps to, as the predecoded words a refill would leave, until Index + /// Store Data (I) writes the set or the line goes. A held primary line + /// answers for itself. + #[test] + fn index_load_data_i_reads_a_refill_until_the_guest_writes_the_set() { + let (c, _space, _bank) = windowed(); + c.cache_op(C_IST | CACH_SD, 0x8000_4f00, sd_tag(0x4f00, 2) | 1 << 7); // VA[13:12] = 1 + c.cache_op(C_R10K_ISD | CACH_SD, 0x8000_4f10, 0x0140_0008); // jr $10 at 0x4f10 + let set = |off: u64| 0x8000_1f00 | off; + for va in [set(0x10), set(0x10) | 1] { + assert_eq!(c.cache_op(C_R10K_ILD | CACH_PI, va, 0), l1i_encode(0x0140_0008), "either way"); + } + assert_eq!(c.cache_op(C_R10K_ILD | CACH_PI, 0x8000_0f10, 0), 0, "another set: the array"); + c.cache_op(C_R10K_ISD | CACH_PI, set(0x10), 0x5_5555_5555); + assert_eq!(c.cache_op(C_R10K_ILD | CACH_PI, set(0x10), 0), 0x5_5555_5555, "the guest's pattern"); + c.cache_op(C_R10K_ISD | CACH_SD, 0x8000_4f10, 0x0140_0008); + assert_eq!(c.cache_op(C_R10K_ILD | CACH_PI, set(0x10), 0), l1i_encode(0x0140_0008), "restaged: a refill again"); + c.cache_op(C_R10K_ISD | CACH_PI, set(0x10), 0x3_4420_0004); + c.cache_op(C_IST | CACH_PI, set(0x10), pi_valid(0x1_4000)); + c.cache_op(C_R10K_ISD | CACH_SD, 0x8000_4f10, 0x0140_0008); + assert_eq!(c.cache_op(C_R10K_ILD | CACH_PI, set(0x10), 0), 0x3_4420_0004, "a held primary line answers for itself"); + c.cache_op(C_IST | CACH_PI, set(0x10), 0); + c.cache_op(C_IST | CACH_SD, 0x8000_4f00, 0); + assert_eq!(c.cache_op(C_R10K_ILD | CACH_PI, set(0x10), 0), 0x3_4420_0004, "gone with the line: the array"); + } } diff --git a/src/cpu/mips_cache_v2.rs b/src/cpu/mips_cache_v2.rs index 30c5e4a48..a0e25d08c 100644 --- a/src/cpu/mips_cache_v2.rs +++ b/src/cpu/mips_cache_v2.rs @@ -669,6 +669,11 @@ pub trait CpuModel: MipsCache { /// to match. An uncached SC succeeding is a model choice. Loads and fetches /// still fault. const POSTS_UNCACHED_STORES: bool = Self::R10K_FAMILY; + /// Whether a board may present an uncached instruction fetch of a window at + /// another address (`MipsExecutor::set_uncached_fetch_alias`). Off except + /// for the parts of the boards that need it, so the translation paths of + /// every other model carry no check for it. + const FETCH_ALIAS: bool = false; /// Bits `Index_Store_Tag` keeps for the secondary cache: all of them, except /// on the R10000 family, whose reserved tag bits read as zero (R10000 UM /// p. 270, figure 11-31). Kept are MRU (bit 63), the physical tag (TagHi 3:0 @@ -1499,11 +1504,12 @@ pub mod model { pub const R5000: u8 = 1; pub const R10000: u8 = 2; pub const R12000: u8 = 3; + pub const R16000: u8 = 4; /// The R10000 family: the R10000 and the parts that follow it, which share /// one programming model. pub const fn r10k_family(model: u8) -> bool { - matches!(model, R10000 | R12000) + matches!(model, R10000 | R12000 | R16000) } } diff --git a/src/cpu/mips_exec.rs b/src/cpu/mips_exec.rs index 16cb38372..9e5a72889 100644 --- a/src/cpu/mips_exec.rs +++ b/src/cpu/mips_exec.rs @@ -1045,6 +1045,14 @@ enum BusErrorOp { pub struct MipsExecutor { pub core: MipsCore, pub sysad: Arc, + /// Where the JIT reads code and generation counters, and the bus its + /// compile pool is started on: cacheable memory at plain physical + /// addresses. The same object as `sysad` unless the board's uncached + /// connection decodes a plain address as something other than memory (the + /// Fuel's decodes it as Hub-special space, with RAM only in its first + /// 256 MiB), in which case [`Self::set_cached_bus`] gives it the view the + /// cache fills from. + cached_bus: Arc, /// cheritest convention (`--cheritest-dump-hook`): a guest write to CP0 26 /// dumps machine state via the test device. Off unless asked for — CP0 26 /// is ECC on a real R4400 and the PROM writes it during cache init. @@ -1303,6 +1311,15 @@ pub struct MipsExecutor { /// reach EntryLo and come out as physical address bits 32 and up. All /// ones (no fold) by default; each machine sets its own. pub phys_mask: u64, + /// A window of physical addresses the board decodes differently for an + /// uncached instruction fetch, `(base, size, target)`: the fetch is + /// presented to the bus at `target` plus the offset into the window. The + /// Fuel's reset PROM answers uncached fetches at 0x1fc00000 whatever the + /// cached view holds there, and gives that device an address of its own, + /// so the code page for it is keyed, read and invalidated by address like + /// any other. None on the other machines, whose uncached and cached + /// instruction fetches reach the same bytes. + uncached_fetch_alias: Option>>, /// Where the machine has memory, from [`MipsCpuConfig::memory`]. pub memory: Arc, /// Frames the JIT's page table has, the end of `memory` in pages. Kept @@ -1426,6 +1443,31 @@ fn translate_64_user(e: &mut MipsExecutor, va: u64, at e.translate_64bit_impl::(va, at) } +/// A window whose uncached instruction fetch the board decodes apart, and the +/// translation it wraps (`MipsExecutor::set_uncached_fetch_alias`). +struct FetchAlias { + base: u64, + size: u64, + target: u64, + inner: fn(&mut MipsExecutor, u64, AccessType) -> TranslateResult, +} + +/// The translation wrappers above for a board that decodes an uncached +/// instruction fetch apart. Only a fetch that comes out uncached is changed. +fn translate_fetch_aliased(e: &mut MipsExecutor, va: u64, at: AccessType) -> TranslateResult { + let inner = e.uncached_fetch_alias.as_ref().map_or(translate_32_kernel:: as fn(&mut MipsExecutor, u64, AccessType) -> TranslateResult, |a| a.inner); + let mut result = inner(e, va, at); + if at == AccessType::Fetch && !result.is_exception() && !result.is_cached() { + if let Some(alias) = e.uncached_fetch_alias.as_ref() { + let offset = result.phys.wrapping_sub(alias.base); + if offset < alias.size { + result.phys = alias.target + offset; + } + } + } + result +} + /// Free-standing trampoline for the CP0 Status callback installed by `install_status_cb`. /// Rust does not allow `Self` inside a nested fn, so the generic trampoline lives here. // Safety: the executor's raw pointers point into allocations owned by @@ -2630,6 +2672,17 @@ impl MipsExecutor { size * 8, self.core.pc, virt_addr, phys_addr, val, status, self.fault_note(phys_addr, size)); } + /// The bus the JIT reads code from and its compile pool is started on. + pub fn cached_bus(&self) -> &Arc { + &self.cached_bus + } + + /// Give the JIT a cacheable view of memory that differs from `sysad` (see + /// the field). A board that does so builds the cache on the same view. + pub fn set_cached_bus(&mut self, bus: Arc) { + self.cached_bus = bus; + } + /// Create a new executor from a config and a bus (sysad) and a TLB. /// The cache hierarchy is constructed internally as a unified R4000Cache. pub fn new(sysad: Arc, mut tlb: T, cfg: &MipsCpuConfig) -> Self @@ -2785,6 +2838,7 @@ impl MipsExecutor { let mut executor = Self { core, + cached_bus: sysad.clone(), sysad, cheritest_dump_hook: false, tlb, @@ -2906,6 +2960,7 @@ impl MipsExecutor { batch_hook: None, phys_alias: None, phys_mask: u64::MAX, + uncached_fetch_alias: None, memory: cfg.memory.clone(), #[cfg(feature = "jitv2")] jitv2_frames: crate::cpu::jitv2::jitv2::frames_for(&cfg.memory), @@ -3480,7 +3535,7 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", #[cfg(feature = "jitv2")] pub(crate) fn jitv2_flush_from_here(&mut self, only_if_exhausted: bool) { crate::pf_scope!(crate::perfetto::events::JIT_CPU_FLUSH); - let bus = self.sysad.clone(); + let bus = self.cached_bus.clone(); let jit = self.jitv2.clone(); let flush = || { let mut j = jit.lock(); @@ -3509,6 +3564,12 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", paddr, self.memory)) } + /// The code page the last fetch landed on, or null. For tests. + #[cfg(all(test, feature = "jitv2"))] + pub fn jitv2_current_page(&self) -> *mut crate::cpu::jitv2::PhysicalCodePage { + self.pcp + } + #[cfg(feature = "jitv2")] fn jitv2_share(&mut self, jit: Arc>, world: Arc) { { @@ -3556,7 +3617,9 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", #[inline(always)] fn jitv2_track_pcp(&mut self, phys_addr: u64) { // Code above the JIT's pfn map gets no page and runs in the - // interpreter (`step_jit` checks for the null page). + // interpreter (`step_jit` checks for the null page). That includes + // uncached attributes 1 to 3: their bits 58:57 sit in `phys_addr`, + // so the frame range check rejects them. let Some(pfn) = crate::cpu::jitv2::jitv2::pfn_of(phys_addr, self.jitv2_frames) else { self.clear_pcp(); return; @@ -3622,7 +3685,7 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", let (mut jit, lookup) = { crate::pf_scope!(crate::perfetto::events::JIT_CPU_TRACK); let mut jit = self.jitv2.lock(); - let lookup = jit.page_for(pfn, page_base, self.sysad.as_ref(), fr1); + let lookup = jit.page_for(pfn, page_base, self.cached_bus.as_ref(), fr1); (jit, lookup) }; match lookup { @@ -3663,7 +3726,7 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", // exact page this fetch already landed on. self.clear_pcp(); let mut jit = self.jitv2.lock(); - let slot = jit.page_for(pfn, page_base, self.sysad.as_ref(), fr1) + let slot = jit.page_for(pfn, page_base, self.cached_bus.as_ref(), fr1) .expect("flush must leave room for at least one page"); // Direct assignment for the same borrow reason as the // hit path above; `pfn` is what `set_pcp` would mirror. @@ -3699,6 +3762,37 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", // A partial LWR keeps the upper word below this offset: 3 in 64-bit mode // on the R10000 family, else 0. self.core.lwr_upper_below = if C::PARTIAL_LWR_KEEPS_UPPER { wide } else { 0 }; + // A board that decodes an uncached instruction fetch apart wraps the + // translation itself, so the fetch paths of the other machines stay + // as they are. + if C::FETCH_ALIAS && self.uncached_fetch_alias.is_some() { + self.wrap_translate_fn(translate_fn); + } + } + + /// `update_translate_fn`'s choice, wrapped for `uncached_fetch_alias`. + /// Out of line so the update itself stays as small as it was; the wrapper + /// reaches the choice through a pointer so that the translation bodies keep + /// the one caller they inline into. + #[cold] + #[inline(never)] + fn wrap_translate_fn(&mut self, inner: fn(&mut Self, u64, AccessType) -> TranslateResult) { + if let Some(alias) = self.uncached_fetch_alias.as_mut() { + alias.inner = inner; + } + self.translate_fn = translate_fetch_aliased::; + } + + /// Present an uncached instruction fetch of the window `base..base+size` + /// to the bus at `target` plus the offset (see [`Self::uncached_fetch_alias`]). + pub fn set_uncached_fetch_alias(&mut self, base: u64, size: u64, target: u64) { + assert!(C::FETCH_ALIAS, "{} has no uncached fetch alias", C::NAME); + // Never called: `update_translate_fn` replaces it below. + fn placeholder(_: &mut MipsExecutor, _: u64, _: AccessType) -> TranslateResult { + unreachable!("the translation wrapper is installed before it is used") + } + self.uncached_fetch_alias = Some(Box::new(FetchAlias { base, size, target, inner: placeholder:: })); + self.update_translate_fn(); } /// Re-derive `fpr_read_d/write_d/read_l/write_l` from STATUS_FR. @@ -9375,7 +9469,7 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", .is_some_and(|c| c.packing_stats().1 > crate::cpu::jitv2::CODEGEN_ARENA_FLUSH_THRESHOLD_BYTES); if over_threshold { let phys_page_base = page.pfn as u64 * crate::cpu::jitv2::PAGE_SIZE as u64; - unsafe { self.jitv2.lock().flush_from_cpu_thread(self.sysad.clone()); } + unsafe { self.jitv2.lock().flush_from_cpu_thread(self.cached_bus.clone()); } self.clear_pcp(); self.jitv2_track_pcp(phys_page_base); return EXEC_RETRY; @@ -9397,10 +9491,10 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", #[cfg(feature = "developer")] { let stats = self.jitv2.lock().stats.clone(); - ran_out_of_memory = crate::cpu::jitv2::comp::handle_request(&req, &self.sysad, &mut self.jitv2_inline_analyzer, codegen, &stats); + ran_out_of_memory = crate::cpu::jitv2::comp::handle_request(&req, &self.cached_bus, &mut self.jitv2_inline_analyzer, codegen, &stats); } #[cfg(not(feature = "developer"))] - { ran_out_of_memory = crate::cpu::jitv2::comp::handle_request(&req, &self.sysad, &mut self.jitv2_inline_analyzer, codegen); } + { ran_out_of_memory = crate::cpu::jitv2::comp::handle_request(&req, &self.cached_bus, &mut self.jitv2_inline_analyzer, codegen); } } *self.jitv2.lock().codegen.lock() = codegen; { @@ -9411,7 +9505,7 @@ va={:#018x} phys={:#010x} (code pfn {:#x}, page {:#010x}, word {}/{})", if ran_out_of_memory { let page = unsafe { &mut *self.pcp }; let phys_page_base = page.pfn as u64 * crate::cpu::jitv2::PAGE_SIZE as u64; - unsafe { self.jitv2.lock().flush_from_cpu_thread(self.sysad.clone()); } + unsafe { self.jitv2.lock().flush_from_cpu_thread(self.cached_bus.clone()); } self.clear_pcp(); self.jitv2_track_pcp(phys_page_base); return EXEC_RETRY; diff --git a/src/cpu/mips_exec/monitor.rs b/src/cpu/mips_exec/monitor.rs index 7a9fc086b..18d267b61 100644 --- a/src/cpu/mips_exec/monitor.rs +++ b/src/cpu/mips_exec/monitor.rs @@ -2027,7 +2027,7 @@ impl MipsCpu { #[cfg(not(feature = "jitv2_lockstep"))] { exec.jitv2_inline_compile = false; - let bus = exec.sysad.clone(); + let bus = exec.cached_bus.clone(); let old = exec.jitv2.lock().codegen.lock().take(); match old { Some(old) => { @@ -3030,7 +3030,7 @@ impl MipsCpu { // implementation. let path = actual_args.get(1).copied().unwrap_or("jitv2.html"); let jitv2_arc = exec.jitv2.clone(); - let bus = exec.sysad.clone(); + let bus = exec.cached_bus.clone(); crate::cpu::jitv2_html_j2wp::write_jitv2_html( &jitv2_arc, &bus, &exec.memory.clone(), &mut exec.jitv2_inline_analyzer, path, &mut *writer, )?; diff --git a/src/cpu/mips_exec_test.rs b/src/cpu/mips_exec_test.rs index 4f4d44583..909dc52a4 100644 --- a/src/cpu/mips_exec_test.rs +++ b/src/cpu/mips_exec_test.rs @@ -673,6 +673,9 @@ mod tests { check::(); check::(); check::(); + check::(); + check::(); + check::(); } /// JIT store callouts carry the same source register as the interpreter. @@ -7429,6 +7432,11 @@ mod tests { assert_eq!(entry::(0xFFFF_C000), (0xC000, 0x0300_4000, (3 << 62) | (lo & !(3 << 32)))); assert_eq!(entry::(0), (0, 0xC0_0300_4000, (3 << 62) | lo)); assert_eq!(entry::(0xFFFF_C000), (0, 0x0300_4000, (lo & 0x3FFF_FFFF) | 0xC0)); + // The R16000 types share the R10000 family's TLB. + let masked = (3 << 62) | (lo & !(3 << 32)); + assert_eq!(entry::(0xFFFF_C000), (0xC000, 0x0300_4000, masked)); + assert_eq!(entry::(0xFFFF_C000), (0xC000, 0x0300_4000, masked)); + assert_eq!(entry::(0xFFFF_C000), (0xC000, 0x0300_4000, masked)); }); } @@ -7589,6 +7597,10 @@ mod tests { assert_eq!(ss_sb::(), (1, 1)); assert_eq!(ss_sb::(), (2, 1)); assert_eq!(ss_sb::(), (2, 0)); + // 4, 8 and 16 MB with 128-byte lines: SS codes 3, 4 and 5. + assert_eq!(ss_sb::(), (3, 1)); + assert_eq!(ss_sb::(), (4, 1)); + assert_eq!(ss_sb::(), (5, 1)); }); } } diff --git a/src/cpu/parts.rs b/src/cpu/parts.rs index ceda3972c..2271e7173 100644 --- a/src/cpu/parts.rs +++ b/src/cpu/parts.rs @@ -14,6 +14,7 @@ pub enum Family { R10000 = 10000, R12000 = 12000, R14000 = 14000, + R16000 = 16000, } impl Family { @@ -24,7 +25,8 @@ impl Family { Self::R5000 => 0x23, Self::R10000 => 0x09, Self::R12000 => 0x0e, - Self::R14000 => 0x0f, + // The R16000 reuses the R14000's implementation byte. + Self::R14000 | Self::R16000 => 0x0f, } } } @@ -33,7 +35,8 @@ impl Family { pub struct Part { pub family: Family, pub mhz: u32, - /// Major and minor revision, as `hinv` prints them ("Revision: 2.7"). + /// Major and minor revision, as `hinv` prints them ("Revision: 2.7"). For + /// an R16000 the PRId's major is this major plus 2 (see [`Part::prid`]). pub rev: (u8, u8), /// Secondary cache in KB. The R5000 reads 0: its cache is not modelled. pub l2_kb: u32, @@ -52,18 +55,38 @@ pub const R10000_250: Part = Part { mhz: 250, rev: (3, 4), ..R10000_195 }; pub const R12000_360: Part = Part { family: Family::R12000, mhz: 360, rev: (3, 5), l2_kb: 2048 }; pub const R12000_400: Part = Part { mhz: 400, ..R12000_360 }; pub const R14000_600: Part = Part { family: Family::R14000, mhz: 600, rev: (2, 4), l2_kb: 2048 }; +/// The Fuel's R14000A: rev 2.3 at 500 MHz with a 2 MB secondary cache, as the IP35 +/// `hinv` dumps show it. +pub const R14000_500: Part = Part { mhz: 500, rev: (2, 3), ..R14000_600 }; + +/// The Tezro's and Origin 350's earliest R16000: rev 2.1, 4 MB secondary cache. +/// A 700 MHz R16000 also shipped as rev 2.2, in the Fuel, so this one takes +/// the `_EARLY` suffix and the plain `R16000_700` stays free for that part +/// when the Fuel takes it. The configuration name `r16000_700` is this part. +pub const R16000_700_EARLY: Part = Part { family: Family::R16000, mhz: 700, rev: (2, 1), l2_kb: 4096 }; +pub const R16000_800: Part = Part { mhz: 800, rev: (2, 2), ..R16000_700_EARLY }; +pub const R16000_900: Part = Part { mhz: 900, rev: (3, 0), l2_kb: 8192, ..R16000_700_EARLY }; +pub const R16000_1000: Part = Part { mhz: 1000, rev: (3, 0), l2_kb: 16384, ..R16000_700_EARLY }; /// Every part, for the bare-metal bench rig, which takes any. A name matches /// the first listed, so the rig runs the later silicon of a clock that has two. pub const ALL: &[&Part] = &[ &R4400_150, &R5000_180_EARLY, &R5000_180, &R10000_195, &R10000_195_EARLY, &R10000_250, - &R10000_180, &R12000_270, &R12000_360, &R12000_400, &R14000_600, + &R10000_180, &R12000_270, &R12000_360, &R12000_400, &R14000_600, &R16000_700_EARLY, &R16000_800, + &R16000_900, &R16000_1000, ]; impl Part { - /// Implementation, then major and minor revision. + /// Implementation, then major and minor revision, as the PRId register + /// holds them. `rev` is what `hinv` prints, and for an R16000 that is two + /// below the register's major: the Linux and OpenBSD CPU probes both + /// display an R16000's revision that way (`rev.0 + 2` here undoes it). pub const fn prid(&self) -> u32 { - self.family.implementation() << 8 | (self.rev.0 as u32) << 4 | self.rev.1 as u32 + let major = match self.family { + Family::R16000 => self.rev.0 + 2, + _ => self.rev.0, + }; + self.family.implementation() << 8 | (major as u32) << 4 | self.rev.1 as u32 } /// CP0 Count ticks at half the pipeline clock. @@ -105,4 +128,18 @@ mod tests { assert_eq!((R5000_180_EARLY.prid(), R5000_180.prid()), (0x2310, 0x2321)); assert_eq!(find(&[&R5000_180, &R10000_250, &R12000_400], "r10000"), Some(&R10000_250)); } + + /// An R16000 prints a major two below the register's: the Tezro's 2.1, + /// 2.2 and 3.0 are raw 0x41, 0x42 and 0x50, and the R14000 beside it in + /// the table keeps its own major. + #[test] + fn an_r16000_register_major_is_the_displayed_one_plus_two() { + assert_eq!( + (R16000_700_EARLY.prid(), R16000_800.prid(), R16000_900.prid(), R16000_1000.prid()), + (0x0f41, 0x0f42, 0x0f50, 0x0f50) + ); + assert_eq!(R14000_600.prid(), 0x0f24); + assert_eq!(find(&[&R16000_800, &R16000_1000], "r16000"), Some(&R16000_800)); + assert_eq!(find(&[&R16000_700_EARLY], "r16000_700"), Some(&R16000_700_EARLY)); + } } diff --git a/src/cpu/smp.rs b/src/cpu/smp.rs index dbac65a69..90f7d25eb 100644 --- a/src/cpu/smp.rs +++ b/src/cpu/smp.rs @@ -26,6 +26,9 @@ pub const HOST_PARK_BIT: u64 = 1 << 62; pub const HOLD_HEART: u32 = 1 << 0; /// Another thread needs the machine still ([`World::stop_others`]). pub const HOLD_WORLD: u32 = 1 << 1; +/// The board holds this processor at a firmware rendezvous until the ones +/// it must follow catch up (the IP35 PROM's presence publication). +pub const HOLD_RENDEZVOUS: u32 = 1 << 2; const NOT_A_CPU: u8 = u8::MAX; diff --git a/src/dev/bcm5701.rs b/src/dev/bcm5701.rs new file mode 100644 index 000000000..fa9b7932f --- /dev/null +++ b/src/dev/bcm5701.rs @@ -0,0 +1,602 @@ +//! The Broadcom BCM5701 gigabit Ethernet controller (PCI 14e4:1645), as the +//! single-port SGI board carries it: a descriptor-ring MAC with an integrated +//! copper PHY, which the public Linux tg3 and OpenBSD bge drivers and the +//! IRIX tg driver all drive. +//! +//! BAR0 is 64 KiB: a register file at 0x0000-0x7fff (its first 256 bytes also +//! alias the PCI configuration registers) and a 32 KiB window onto the 32 KiB +//! on-chip SRAM at 0x8000-0xffff, whose base the memory-window base register +//! selects. The driver reaches chip state three ways: the BAR directly, an +//! indirect register window through configuration registers 0x78 and 0x80, +//! and an indirect memory window through 0x7c and 0x84. All three land on the +//! same state. The configuration side is the function's own `config_dword` and +//! `config_write`, so it works behind any bridge. +//! +//! Modelled: the PCI identity; the chip id in MISC_CTL; the global reset with +//! the on-chip firmware-ready handshake in SRAM and the firmware-progress +//! register; the self-clearing mode-register resets; the MII path to the +//! PHY ([`phy`]); the station address the bootcode leaves in SRAM and in +//! address filter slot 0; the interrupt mailbox, the MISC_CTL global mask and +//! PCISTATE.INTR_NOT_ACTIVE; the buffer manager's delayed enable; and the +//! serial EEPROM controller over a caller-supplied backing ([`eeprom`]); the +//! send ring and its doorbell, the standard, jumbo and return receive rings, +//! the status block DMA, host coalescing, the interrupt it drives, the +//! transmit checksum offload and segmentation, and the receive checksum +//! report ([`dma`]). +//! +//! # What is approximated +//! +//! - Iris has no virtual clock, so timing is wall-clock. Deadlines (the PHY's +//! 100 ms settle, the buffer manager's 8 ms startup, the coalescing budgets +//! and the arrival latency) are evaluated when the device is touched and by +//! the receive thread, each at its own expiry and earliest first, as the +//! model's virtual clock would fire them. +//! - EEPROM transfers complete synchronously; the delay register only reads +//! back, and the 10 ms write spacing is not enforced. Direct serial-pin +//! control is refused and counted. +//! - The management interface ignores the PHY address field; one PHY answers. +//! - A write to a register with no modelled behaviour is stored and reads +//! back, and is recorded in [`Bcm5701::refused`]. +//! - The BCM5704 dual-port variants are not modelled. +//! - Frames reach the host through the shared network link; the chip's +//! virtio-net offload path has no counterpart, so a large TCP buffer is +//! always cut into segments in software. +//! - A transmit is carried out entirely inside the doorbell write, so there +//! is no re-entrancy guard and no deferred flush: the receive thread offers +//! a waiting frame again whenever the guest makes room. +//! +//! Ported from hw/net/bcm570x.c and hw/net/bcm570x_phy.c in +//! jamesbraid/qemu-sgi-origami. + +mod dma; +pub mod eeprom; +pub mod phy; +mod state; + +#[cfg(test)] +mod tests; + +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Weak}; +use std::time::{Duration, Instant}; + +use parking_lot::Mutex; + +use crate::irq::IrqLine; +use crate::pci::{Bar, PciConfig, PciConfigPolicy, PciFunction, PciFunctionState, PciRegisterPolicy, PCI_INTA}; +use crate::traits::{Device, Resettable}; +use eeprom::Eeprom; +pub use dma::{DmaWhat, Mailbox, Ring, RxOutcome, MAX_FRAME, RX_ARRIVAL_LATENCY_DEFAULT}; +pub use state::Refused; +use state::{cfg, misc_ctl, pcistate, Header, State}; + +pub const VENDOR_ID: u16 = 0x14e4; +pub const DEVICE_ID: u16 = 0x1645; +/// The SGI board's subsystem identity. +pub const SUBSYSTEM_VENDOR_ID: u16 = 0x10a9; +pub const SUBSYSTEM_ID: u16 = 0x8010; +/// PCI network controller, Ethernet. +const CLASS_ETHERNET: u32 = 0x02_0000; + +pub const BAR0_SIZE: u32 = 0x1_0000; +/// The low 256 bytes of BAR0 are the PCI configuration registers again. +const CFG_ALIAS: u32 = 0x100; + +/// What a board gives the chip: its station address and, optionally, the +/// serial EEPROM the board fits. +pub struct Bcm5701Config { + pub mac: [u8; 6], + pub eeprom: Option, + /// The minimum between a frame arriving and the driver being told, so a + /// completion never beats the arrival it reports. Zero publishes at once. + pub rx_arrival_latency: Duration, + /// Whether the slot drives REQ64# as reset ends. The chip samples it to + /// report a 64-bit bus in PCISTATE. + pub req64: bool, +} + +impl Bcm5701Config { + pub fn new(mac: [u8; 6]) -> Self { + Self { mac, eeprom: None, rx_arrival_latency: RX_ARRIVAL_LATENCY_DEFAULT, req64: false } + } +} + +pub struct Bcm5701 { + pci: PciFunctionState, + st: Mutex, + irq: Mutex, + /// Serialises delivery of level changes to the interrupt line. + irq_order: Mutex<()>, + link: Mutex>, + rx_wake: Mutex>>, + /// Between `start` and `stop`; the NAT engine watches it too. + running: Arc, + /// The flag the network backend watches, set and cleared with `running`. + net_running: Mutex>>, + /// The network backend, between runs. A NAT engine keeps its flows across + /// a stop. + backend: Arc>>>, + /// The NAT and receive threads, joined in `stop`. + threads: Mutex>>, + me: Weak, + /// Tests freeze the clock to step it exactly. + #[cfg(test)] + frozen: Mutex>, +} + +impl Bcm5701 { + pub fn new(config: Bcm5701Config) -> Result, String> { + if config.mac[0] & 1 != 0 { + return Err("bcm5701: station address must be unicast".into()); + } + let policy = PciConfigPolicy { + registers: &[ + // The chip id in the high half is read-only; the driver's + // byte-order and interrupt-mask bits in the low half stick. + PciRegisterPolicy { + reg_dword: cfg::MISC_CTL, + initial: state::CHIPID_BCM5701 << 16, + writable_mask: misc_ctl::WRITABLE, + }, + // The chip drives PCISTATE, which `config_dword` reports from + // its state; the guest cannot write it. It runs as + // conventional PCI and starts with the interrupt line down. + PciRegisterPolicy { + reg_dword: cfg::PCISTATE, + initial: State::pcistate_at_reset(config.req64), + writable_mask: 0, + }, + // The indirect windows live in the chip, reached by whole + // dwords through `config_dword` and `config_write`. The header + // bytes behind them are what a narrower write reaches, and + // nothing is stored there. + PciRegisterPolicy { reg_dword: cfg::REG_BASEADDR, initial: 0, writable_mask: 0 }, + PciRegisterPolicy { reg_dword: cfg::MEMWIN_BASEADDR, initial: 0, writable_mask: 0 }, + PciRegisterPolicy { reg_dword: cfg::REG_DATA, initial: 0, writable_mask: 0 }, + PciRegisterPolicy { reg_dword: cfg::MEMWIN_DATA, initial: 0, writable_mask: 0 }, + ], + ..Default::default() + }; + let pci = PciConfig::new_with_policy( + VENDOR_ID, + DEVICE_ID, + CLASS_ETHERNET, + 0, + [Bar::Mem(BAR0_SIZE), Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused], + PCI_INTA, + policy, + ); + pci.write(11, u32::from(SUBSYSTEM_ID) << 16 | u32::from(SUBSYSTEM_VENDOR_ID)); + let now = Instant::now(); + let dev = Arc::new_cyclic(|me| Self { + st: Mutex::new(State::new(Header::new(me.clone()), config.req64, config.mac, config.eeprom, config.rx_arrival_latency, now)), + pci: PciFunctionState::new(pci), + irq: Mutex::new(IrqLine::unconnected()), + irq_order: Mutex::new(()), + link: Mutex::new(None), + rx_wake: Mutex::new(None), + running: Arc::new(AtomicBool::new(false)), + net_running: Mutex::new(None), + backend: Arc::new(Mutex::new(None)), + threads: Mutex::new(Vec::new()), + me: me.clone(), + #[cfg(test)] + frozen: Mutex::new(None), + }); + Ok(dev) + } + + /// The header the bus keeps for the function. + fn config_header(&self) -> &PciConfig { + self.pci.config() + } + + fn now(&self) -> Instant { + #[cfg(test)] + if let Some(t) = *self.frozen.lock() { + return t; + } + Instant::now() + } + + /// Run `f` with the state at `now`, then deliver what it produced: any + /// interrupt level it changed, frames to send, and a wake-up for the + /// receive thread if the guest made room. + fn enter(&self, now: Instant, f: impl FnOnce(&mut State) -> R) -> R { + let (result, wire, wake, purge) = { + let mut st = self.st.lock(); + st.enter(now); + let result = f(&mut st); + ( + result, + std::mem::take(&mut st.wire), + std::mem::take(&mut st.wake_rx), + std::mem::take(&mut st.purge_rx), + ) + }; + if purge { + // A reset drops what the backend was holding for the guest, so a + // frame from before it cannot reach the new rings. + if let Some(link) = self.link.lock().as_mut() { + while link.from_net.pop().is_ok() {} + } + } + self.irq_flush(); + self.send(wire); + if wake { + self.wake_rx(); + } + result + } + + /// Deliver queued interrupt level changes, in order, with no state lock + /// held that the line could need. + fn irq_flush(&self) { + let _order = self.irq_order.lock(); + let events = std::mem::take(&mut self.st.lock().irq_events); + if events.is_empty() { + return; + } + let line = self.irq.lock(); + for level in events { + line.set(level); + } + } + + /// A machine reset: registers, SRAM, PHY and the PCI header as at + /// power-on, the interrupt dropped. The EEPROM keeps its contents. + pub fn reset(&self) { + self.pci.config().reset(); + PciFunction::reset(self); + } + + /// A whole-dword write to configuration dword `reg`, by either path to + /// the header: the chip's own registers act, anything else is the + /// header's. + fn config_write_dword(&self, reg: usize, val: u32) { + if self.config_write(reg, val) { + return; + } + self.pci.config().write(reg, val); + if reg == cfg::MISC_CTL { + self.enter(self.now(), State::misc_ctl_written); + } + } + + /// The backend's cable state; the PHY reports it as link. + pub fn set_backend_link(&self, up: bool) { + self.enter(self.now(), |st| st.phy.set_link_down(!up)); + } + + fn send(&self, frames: Vec>) { + if frames.is_empty() { + return; + } + if let Some(link) = self.link.lock().as_mut() { + for frame in frames { + let _ = link.to_net.push(frame); + } + link.tx_wake.notify(); + } + } + + fn wake_rx(&self) { + if let Some(wake) = self.rx_wake.lock().as_ref() { + wake.notify(); + } + } + + /// Connect to the backend `[network]` selected: frames sent go on + /// `link.to_net`, frames on `link.from_net` are received, and + /// `link.rx_wake` is notified when the guest does something that can + /// unblock a waiting frame. The backend and receive threads run between + /// `start` and `stop`; a NAT engine, which only trades frames with the + /// host and never reaches the guest, keeps its flows across a stop. + pub fn attach(&self, mut link: crate::net::Link) { + let backend = link.backend(); + let running = link.running(); + let rings = link.into_rings(); + *self.rx_wake.lock() = Some(rings.rx_wake.clone()); + *self.link.lock() = Some(rings); + *self.net_running.lock() = Some(running.clone()); + *self.backend.lock() = backend; + if self.running.load(Ordering::SeqCst) { + running.store(true, Ordering::SeqCst); + self.spawn_nat(); + } + } + + fn spawn_nat(&self) { + let slot = self.backend.clone(); + let Some(mut engine) = slot.lock().take() else { return }; + let t = std::thread::Builder::new() + .name("bcm5701-net".into()) + .spawn(move || { + engine.run(); + *slot.lock() = Some(engine); + }) + .expect("bcm5701 NAT thread"); + self.threads.lock().push(t); + } + + /// Take frames into guest memory as they arrive and as the guest makes + /// room, and run the device's timers, until it stops. The wait ends early + /// for a wake from the guest or the NAT engine. + fn rx_loop(me: Weak, running: Arc, wake: Arc) { + while running.load(Ordering::SeqCst) { + let timeout = { + let Some(this) = me.upgrade() else { return }; + let now = this.now(); + this.enter(now, |_| ()); + let next = this.st.lock().deadline(); + next.and_then(|d| d.checked_duration_since(now)).unwrap_or(RX_IDLE) + }; + wake.wait(timeout); + if !running.load(Ordering::SeqCst) { + return; + } + let Some(this) = me.upgrade() else { return }; + this.poll(); + } + } + + /// Take frames from the backend into the buffers the driver has posted. + /// A frame the engine cannot take yet stays at the head of the backend's + /// queue until the guest makes room; one the filter rejects is discarded. + /// Returns whether any frame was consumed. + /// + /// The link is held only to take one frame at a time, so a transmit on the + /// CPU thread never waits behind a burst of deliveries. + pub fn poll(&self) -> bool { + let mut moved = false; + loop { + let now = self.now(); + { + let mut link = self.link.lock(); + let Some(link) = link.as_mut() else { break }; + let outcome = { + let Ok(frame) = link.from_net.peek() else { break }; + let mut st = self.st.lock(); + st.enter(now); + st.receive(frame) + }; + if outcome == RxOutcome::Deferred { + break; + } + let _ = link.from_net.pop(); + moved = true; + } + self.irq_flush(); + } + moved + } + + /// Everything the model was asked for and could not do, with counts. + pub fn refused(&self) -> Vec<(Refused, u64)> { + self.st.lock().refused() + } + + /// The serial EEPROM backing's current contents, if one is fitted. + pub fn eeprom_bytes(&self) -> Option> { + self.st.lock().eeprom_bytes() + } + + /// Name the long-lived locks in the lock registry, which holds only a + /// weak reference to the device. + pub fn register_locks(self: &Arc) { + for (what, probe) in [ + ("st", (|d: &Bcm5701| d.st.is_locked()) as fn(&Bcm5701) -> bool), + ("link", |d| d.link.is_locked()), + ("irq_order", |d| d.irq_order.is_locked()), + ] { + let me = Arc::downgrade(self); + crate::locks::register_lock(format!("bcm5701::{what}"), move || me.upgrade().is_some_and(|d| probe(&d))); + } + } + + /// BAR0 accepts aligned 32-bit accesses everywhere, and byte reads of the + /// first station-address pair. + fn accepts(off: u32, width: usize, write: bool) -> bool { + match width { + 4 => off & 3 == 0, + 1 => !write && (state::reg::MAC_ADDR_0_HIGH..state::reg::MAC_ADDR_0_LOW + 4).contains(&off), + _ => false, + } + } +} + +impl PciFunction for Bcm5701 { + fn state(&self) -> &PciFunctionState { + &self.pci + } + + /// The registers the chip computes: the four indirect-window registers + /// and PCISTATE. Everything else is the header's. + fn config_dword(&self, reg: usize) -> u32 { + if reg == cfg::PCISTATE { + return self.st.lock().pcistate; + } + if !(cfg::REG_BASEADDR..=cfg::MEMWIN_DATA).contains(®) { + return self.pci.config().read(reg); + } + self.enter(self.now(), |st| match reg { + cfg::REG_BASEADDR => st.reg_win_base, + cfg::MEMWIN_BASEADDR => st.mem_win_base, + cfg::REG_DATA => st.reg_read(st.reg_win_base), + _ => st.sram_read(st.mem_win_base), + }) + } + + /// A whole-dword write to the indirect-window registers acts on the chip + /// every time; a write to MISC_CTL is stored by the header and then acted + /// on, whether or not its value changed. + fn config_write(&self, reg: usize, val: u32) -> bool { + if reg == cfg::MISC_CTL { + self.pci.config().write(reg, val); + self.enter(self.now(), State::misc_ctl_written); + return true; + } + if !(cfg::REG_BASEADDR..=cfg::MEMWIN_DATA).contains(®) { + return false; + } + self.enter(self.now(), |st| match reg { + cfg::REG_BASEADDR => st.reg_win_base = val, + cfg::MEMWIN_BASEADDR => st.mem_win_base = val, + cfg::REG_DATA => st.reg_write(st.reg_win_base, val), + _ => st.sram_write(st.mem_win_base, val), + }); + true + } + + /// A narrower write to MISC_CTL changed it: act on it as a whole-dword + /// write does. + fn config_written(&self, reg: usize, _old: u32, _new: u32) { + if reg == cfg::MISC_CTL { + self.enter(self.now(), State::misc_ctl_written); + } + } + + fn bar_read(&self, bar: usize, off: u32, width: usize) -> u32 { + // A rejected read returns zero, as the model it is ported from does + // behind its bridge; the refusal is counted. + if bar != 0 { + return 0; + } + let valid = Self::accepts(off, width, false) && off < BAR0_SIZE; + // The configuration alias goes through the function's own dwords, + // which take the state lock themselves. + if valid && width == 4 && off < CFG_ALIAS { + return self.config_dword((off / 4) as usize); + } + self.enter(self.now(), |st| { + if !valid { + st.refuse(Refused::BarAccess { offset: off, width: width as u8, write: false }); + 0 + } else if width == 1 { + (st.reg_read(off & !3) >> ((off & 3) * 8)) & 0xff + } else if off < state::REG_FILE_SIZE { + st.reg_read(off) + } else { + st.sram_read((st.mem_win_base & !(state::SRAM_SIZE - 1)) + (off - state::REG_FILE_SIZE)) + } + }) + } + + fn bar_write(&self, bar: usize, off: u32, val: u32, width: usize) { + if bar != 0 { + return; + } + let valid = Self::accepts(off, width, true) && off < BAR0_SIZE; + if valid && off < CFG_ALIAS { + // Not the registers the bus built its BAR decode from: they change + // only through its configuration cycles. + let reg = (off / 4) as usize; + if !matches!(reg, 1 | 4..=9) { + self.config_write_dword(reg, val); + } + return; + } + self.enter(self.now(), |st| { + if !valid { + st.refuse(Refused::BarAccess { offset: off, width: width as u8, write: true }); + } else if off < state::REG_FILE_SIZE { + st.reg_write(off, val); + } else { + st.sram_write((st.mem_win_base & !(state::SRAM_SIZE - 1)) + (off - state::REG_FILE_SIZE), val); + } + }); + } + + /// RST# has reset the header: the chip returns to power-on state and + /// drops its interrupt. It samples REQ64# again as reset ends. + fn reset(&self) { + self.enter(self.now(), State::reset); + } + + /// The chip's DMA, and its interrupt line, which carries the level the + /// chip already has. + fn attached(&self) { + if let Some(dma) = self.pci.dma() { + self.st.lock().dma = dma.clone(); + self.wake_rx(); + } + if let Some(line) = self.pci.irq() { + *self.irq.lock() = line.clone(); + // Present the current level to the new line. + self.st.lock().update_irq(true); + self.irq_flush(); + } + } +} + +/// How long the receive thread sleeps with no deadline to wait for: it is +/// woken by the guest and the NAT engine, so this is only a bound. +const RX_IDLE: Duration = Duration::from_secs(60); + +impl Device for Bcm5701 { + fn step(&self, _cycles: u64) {} + + /// Start the receive thread, and the NAT engine's if one is attached. + fn start(&self) { + if self.running.swap(true, Ordering::SeqCst) { + return; + } + let wake = self.rx_wake.lock().get_or_insert_with(crate::net::Wake::new).clone(); + let (me, running) = (self.me.clone(), self.running.clone()); + let rx = std::thread::Builder::new() + .name("bcm5701-rx".into()) + .spawn(move || Self::rx_loop(me, running, wake)) + .expect("bcm5701 rx thread"); + self.threads.lock().push(rx); + if let Some(r) = self.net_running.lock().as_ref() { + r.store(true, Ordering::SeqCst); + } + self.spawn_nat(); + // Frames queued while stopped. + self.wake_rx(); + } + + /// Join the threads: no frame reaches guest memory and no line moves + /// afterwards. Frames the NAT engine takes meanwhile queue for `start`. + fn stop(&self) { + if !self.running.swap(false, Ordering::SeqCst) { + return; + } + if let Some(r) = self.net_running.lock().as_ref() { + r.store(false, Ordering::SeqCst); + } + self.wake_rx(); + if let Some(l) = self.link.lock().as_ref() { + l.tx_wake.notify(); + } + // The receive thread can hold the last reference for a moment, and + // then this runs on it; it cannot join itself. + let me = std::thread::current().id(); + let threads: Vec<_> = self.threads.lock().drain(..).collect(); + for t in threads { + if t.thread().id() != me { + let _ = t.join(); + } + } + } + + fn is_running(&self) -> bool { + self.running.load(Ordering::SeqCst) + } + + fn get_clock(&self) -> u64 { + 0 + } +} + +impl Resettable for Bcm5701 { + fn power_on(&self) { + self.reset(); + } +} + +impl Drop for Bcm5701 { + fn drop(&mut self) { + Device::stop(&*self); + } +} diff --git a/src/dev/bcm5701/dma.rs b/src/dev/bcm5701/dma.rs new file mode 100644 index 000000000..a2c000258 --- /dev/null +++ b/src/dev/bcm5701/dma.rs @@ -0,0 +1,1014 @@ +//! The descriptor engines: the send ring, the standard and jumbo receive +//! rings and the return ring, the status block, and the host coalescing that +//! decides when the driver is told. +//! +//! Descriptors and the status block are the chip's big-endian internal image, +//! reaching the bus through the MODE_CTL swap controls; frame data uses the +//! frame-data controls. What the host sees of those bytes is the bridge's +//! business ([`PciDma`]). + +use std::time::Duration; + +use super::state::{reg, Refused, State}; +use crate::pci::{PciDma, PciDmaStatus}; + +/// The largest frame the engines carry, transmit or receive. The part does +/// jumbo frames; the figure matches what the other Origin adapters carry so a +/// guest configured for jumbo behaves the same whichever card it has. It is a +/// model limit, not the hardware's. +pub const MAX_FRAME: usize = 9600; + +const SRAM_SEND_RING_RCB: u32 = 0x0100; +const SRAM_RX_RETURN_RCB: u32 = 0x0200; +/// `nicNoMoreRXBDs`, the one statistics counter the model keeps: the number of +/// times the chip ran out of receive descriptors. +const SRAM_STATS_NO_MORE_RXBDS: u32 = 0x0898; +const RCB_HOSTADDR_HI: u32 = 0x00; +const RCB_HOSTADDR_LO: u32 = 0x04; +const RCB_MAXLEN_FLAGS: u32 = 0x08; +const RCB_FLAG_EXT_RX_ENABLE: u32 = 1; +const RCB_FLAG_DISABLE_RING: u32 = 2; + +const TXBD_SIZE: u64 = 16; +// Offsets in the descriptors' little-endian layout; a 16-bit field's +// image offset is its offset xor 2. +const TXBD_FLAGS: usize = 0x08; +const TXBD_LEN: usize = 0x0a; +const TXBD_MSS: usize = 0x0e; +const TXBDFLAG_TCP_UDP_CKSUM: u16 = 0x0001; +const TXBDFLAG_IP_CKSUM: u16 = 0x0002; +const TXBDFLAG_END: u16 = 0x0004; +const TXBDFLAG_INSERT_SRC_ADDR: u16 = 0x1000; +const TXBDFLAG_CHOOSE_SRC_ADDR: u16 = 0x6000; +const TXBDFLAG_SRC_ADDR_SHIFT: u16 = 13; + +const RXBD_SIZE: u64 = 32; +const RXBD_IDX_LEN: usize = 0x08; +const RXBD_TYPE_FLAGS: usize = 0x0c; +const RXBD_CSUM: usize = 0x10; +const RXBD_OPAQUE: usize = 0x1c; +const RXBDFLAG_END: u32 = 0x0004; +const RXBDFLAG_JUMBO_RING: u32 = 0x0020; +const RXBDFLAG_IP_CKSUM_OK: u32 = 0x1000; +const RXBDFLAG_TCP_UDP_CKSUM_OK: u32 = 0x2000; +const RXBDFLAG_TCP: u32 = 0x4000; +const MODE_RX_NO_PSEUDO: u32 = 1 << 23; +const MODE_NO_FRAME_CRACK: u32 = 1 << 9; + +/// The standard ring holds a fixed number of descriptors. Its control block's +/// length is the size of the buffer each descriptor points at, an MTU rather +/// than a count, while the return ring's really is a ring length. +const RX_STD_RING_LEN: u16 = 512; +const RX_JUMBO_RING_LEN: u16 = 256; +const FCS_LEN: u32 = 4; +const MIN_FRAME: usize = 60; + +const STATUS_BLK_SIZE: usize = 0x50; +const STATFLAG_UPDATED: u32 = 1; +const STATUS_RX_PROD_OFF: usize = 16; +const STATUS_TX_CONS_OFF: usize = 18; +const STATUS_TAG_OFF: usize = 4; +const STATUS_RX_STD_CONS_IMAGE_OFF: usize = 8; +const STATUS_RX_JUMBO_CONS_IMAGE_OFF: usize = 10; +const HCC_MODE_ENABLE: u32 = 2; + +/// Both coalescing tick budgets are programmed in microseconds. +const COAL_TICK: Duration = Duration::from_micros(1); +/// A status DMA the guest's address refused is retried no slower than this. +const COAL_RETRY_MAX: Duration = Duration::from_millis(250); +/// Completions the arrival line can hold; a full line refuses frames. +const RX_LATENCY_DEPTH: usize = 256; +/// The default minimum between a frame arriving and its completion. +pub const RX_ARRIVAL_LATENCY_DEFAULT: Duration = Duration::from_micros(200); + +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +pub enum Ring { + Standard, + Jumbo, +} + +/// The mailboxes whose index is range-checked. +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +pub enum Mailbox { + TxProducer, + RxStandardProducer, + RxJumboProducer, + RxReturnConsumer, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +pub enum DmaWhat { + TxDescriptor, + TxPayload, + RxDescriptor, + RxPayload, + RxCompletion, + StatusBlock, +} + +/// The timers the engines keep, ordered by when they fire. +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +pub enum Timer { + PhySettle, + BufferManager, + RxCoalescing, + TxCoalescing, + RxArrival, +} + +/// What the receive path did with a frame the backend offered. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum RxOutcome { + /// Delivered, filtered or dropped: the backend may discard it. + Consumed, + /// Not taken; the frame stays at the head of the backend's queue until + /// the guest makes room. + Deferred, +} + +/// A bus master with nowhere to go, until the bridge connects one. +pub(super) struct NullDma; +impl PciDma for NullDma { + fn dma_read(&self, _addr: u64, buf: &mut [u8]) -> PciDmaStatus { + buf.fill(0); + PciDmaStatus::MasterAbort + } + fn dma_write(&self, _addr: u64, _data: &[u8]) -> PciDmaStatus { + PciDmaStatus::MasterAbort + } +} + +fn be32(b: &[u8], off: usize) -> u32 { + u32::from_be_bytes(b[off..off + 4].try_into().unwrap()) +} + +/// A 16-bit field of the chip's image sits in the half of its 32-bit word +/// the big-endian structure puts it in. +fn ld16(b: &[u8], off: usize) -> u16 { + let o = off ^ 2; + u16::from_be_bytes([b[o], b[o + 1]]) +} + +fn st32(b: &mut [u8], off: usize, v: u32) { + b[off..off + 4].copy_from_slice(&v.to_be_bytes()); +} + +fn st16(b: &mut [u8], off: usize, v: u16) { + let o = off ^ 2; + b[o..o + 2].copy_from_slice(&v.to_be_bytes()); +} + +/// The one's-complement sum of a span, folded. +fn ones_sum(buf: &[u8], seed: u32) -> u16 { + let mut sum = seed; + for pair in buf.chunks(2) { + sum += u32::from(pair[0]) << 8 | u32::from(*pair.get(1).unwrap_or(&0)); + } + while sum >> 16 != 0 { + sum = (sum & 0xffff) + (sum >> 16); + } + sum as u16 +} + +/// Fill in the IP header and/or TCP and UDP checksums of an untagged IPv4 +/// frame. A fragment keeps its transport checksum. +fn checksum_calculate(frame: &mut [u8], ip: bool, transport: bool) { + if frame.len() < 14 + 20 || frame[14] >> 4 != 4 { + return; + } + let ihl = usize::from(frame[14] & 0xf) * 4; + if ihl < 20 || frame.len() < 14 + ihl { + return; + } + if ip { + frame[24] = 0; + frame[25] = 0; + let sum = !ones_sum(&frame[14..14 + ihl], 0); + frame[24..26].copy_from_slice(&sum.to_be_bytes()); + } + if u16::from_be_bytes([frame[20], frame[21]]) & 0x3fff != 0 { + return; + } + let total = usize::from(u16::from_be_bytes([frame[16], frame[17]])); + if total < ihl { + return; + } + let data_len = (total - ihl).min(frame.len() - 14 - ihl); + let proto = frame[23]; + let at = match proto { + 6 if transport && data_len >= 20 => 16, + 17 if transport && data_len >= 8 => 6, + _ => return, + }; + let start = 14 + ihl; + frame[start + at] = 0; + frame[start + at + 1] = 0; + let pseudo = ones_pseudo(&frame[26..34], proto, data_len); + let sum = !ones_sum(&frame[start..start + data_len], pseudo); + frame[start + at..start + at + 2].copy_from_slice(&sum.to_be_bytes()); +} + +fn ones_pseudo(addresses: &[u8], proto: u8, len: usize) -> u32 { + addresses.chunks(2).map(|p| u32::from(p[0]) << 8 | u32::from(p[1])).sum::() + u32::from(proto) + len as u32 +} + +/// Finish the checksums the driver left to the card. The lengths in the frame +/// were composed by the guest, so a header shorter than its own minimum, or +/// lengths reaching past what arrived, decline the offload rather than walk +/// off the buffer. +fn tx_offload_checksum(frame: &mut [u8], flags: u16) { + if frame.len() < 14 + 20 || frame[12..14] != [0x08, 0x00] || frame[14] >> 4 != 4 { + return; + } + let ihl = usize::from(frame[14] & 0xf) * 4; + let total = usize::from(u16::from_be_bytes([frame[16], frame[17]])); + if ihl < 20 || total < ihl || 14 + total > frame.len() { + return; + } + checksum_calculate(frame, flags & TXBDFLAG_IP_CKSUM != 0, flags & TXBDFLAG_TCP_UDP_CKSUM != 0); +} + +struct LsoLayout { + ihl: usize, + thl: usize, + hdr: usize, + payload: usize, +} + +/// Whether `frame` is an unfragmented IPv4 TCP frame that `mss` can cut up. +fn lso_layout(frame: &[u8], mss: usize) -> Option { + if frame.len() < 14 + 20 + 20 || frame[12..14] != [0x08, 0x00] || frame[14] >> 4 != 4 || frame[23] != 6 { + return None; + } + let ihl = usize::from(frame[14] & 0xf) * 4; + if ihl < 20 || 14 + ihl + 20 > frame.len() { + return None; + } + let ip_len = usize::from(u16::from_be_bytes([frame[16], frame[17]])); + if u16::from_be_bytes([frame[20], frame[21]]) & 0x3fff != 0 || ip_len < ihl + 20 || 14 + ip_len != frame.len() { + return None; + } + let thl = usize::from(frame[14 + ihl + 12] >> 4) * 4; + let hdr = 14 + ihl + thl; + if thl < 20 || ihl + thl > ip_len || mss == 0 || hdr + mss > MAX_FRAME { + return None; + } + let payload = ip_len - ihl - thl; + if payload <= mss { + return None; + } + Some(LsoLayout { ihl, thl, hdr, payload }) +} + +/// The checksums of an arriving frame, as the part reports them: the sums it +/// computed rather than a verdict, a field that checks out summing to 0xffff. +/// Anything not plainly IPv4 TCP or UDP is left unclaimed, because reporting +/// a sum for something never summed would have the stack skip its own pass. +fn rx_offload_checksum(frame: &[u8], mode: u32, flags: &mut u32) -> u32 { + if mode & MODE_NO_FRAME_CRACK != 0 { + return 0; + } + if frame.len() < 34 || frame[12..14] != [0x08, 0x00] || frame[14] >> 4 != 4 { + return 0; + } + let ihl = usize::from(frame[14] & 0xf) * 4; + let total = usize::from(u16::from_be_bytes([frame[16], frame[17]])); + if ihl < 20 || total < ihl || 14 + total > frame.len() { + return 0; + } + let ip_sum = ones_sum(&frame[14..14 + ihl], 0); + if ip_sum == 0 || ip_sum == 0xffff { + *flags |= RXBDFLAG_IP_CKSUM_OK; + } + let ip_only = u32::from(ip_sum) << 16; + let proto = frame[23]; + if proto != 6 && proto != 17 { + return ip_only; + } + if proto == 6 { + *flags |= RXBDFLAG_TCP; + } + if u16::from_be_bytes([frame[20], frame[21]]) & 0x3fff != 0 { + return ip_only; + } + let l4len = total - ihl; + if (proto == 6 && l4len < 20) || (proto == 17 && l4len < 8) { + return ip_only; + } + let mut transport_checksum_present = true; + if proto == 6 { + let tcp_hlen = usize::from(frame[14 + ihl + 12] >> 4) * 4; + if tcp_hlen < 20 || tcp_hlen > l4len { + return ip_only; + } + } else { + let udp_len = usize::from(u16::from_be_bytes([frame[14 + ihl + 4], frame[14 + ihl + 5]])); + if udp_len != l4len { + return ip_only; + } + transport_checksum_present = frame[14 + ihl + 6..14 + ihl + 8] != [0, 0]; + } + let pseudo = if mode & MODE_RX_NO_PSEUDO == 0 { ones_pseudo(&frame[26..34], proto, l4len) } else { 0 }; + let l4_sum = ones_sum(&frame[14 + ihl..14 + ihl + l4len], pseudo); + if l4_sum == 0xffff && transport_checksum_present && mode & MODE_RX_NO_PSEUDO == 0 { + *flags |= RXBDFLAG_TCP_UDP_CKSUM_OK; + } + ip_only | u32::from(l4_sum) +} + +impl State { + /// The engine state a reset returns to zero. + pub(super) fn reset_engines(&mut self) { + self.tx_cons = 0; + self.status_tx_cons = 0; + self.status_rx_prod = 0; + self.rx_prod_staged = 0; + self.rx_std_cons = 0; + self.rx_jumbo_cons = 0; + self.rx_shortage_drops = 0; + self.rx_std_shortage_drops = 0; + self.status_tag = 0; + self.rx_coal_pending = 0; + self.tx_coal_pending = 0; + self.rx_coal_expired = false; + self.tx_coal_expired = false; + self.rx_coal_timer = None; + self.tx_coal_timer = None; + self.rx_latency.clear(); + self.wire.clear(); + self.wake_rx = true; + self.purge_rx = true; + } + + /// Whatever made the chip ready again, a posted buffer or a consumer + /// index the driver moved, is the moment to offer the backend's waiting + /// frame again. + pub(super) fn retry_rx_flush(&mut self) { + self.wake_rx = true; + } + + // ---- host memory ---- + + /// The 64-bit host address a pair of control-block words names: the high + /// word at the lower offset, the low word at the next. With + /// ENDIAN_WORDSWAP set, a 32-bit target access at +0 of an on-chip + /// eight-byte field reaches its more significant word (Figure 93 of the + /// family programmer's guide), which is the setting the drivers use, so + /// the pair takes no further exchange. The swap-clear 32-bit case (the + /// low word at +0) is not modelled; no driver leaves the swap clear. + fn compose_hostaddr(&self, hi: u32, lo: u32) -> u64 { + u64::from(hi) << 32 | u64::from(lo) + } + + /// The chip's DMA byte-swap controls, MODE_CTL bits 5 and 4 for frame data + /// and bits 2 and 1 for everything else. The chip is big-endian + /// internally with a 64-bit memory: its internal byte at lane l of a + /// doubleword (lane 0 the most significant) goes on PCI byte 7 - l with no + /// swap, and the swaps exchange the halves and reverse bytes within them. + /// The frame-data combinations are established by the programmer's guide; + /// the non-frame single-swap cases are inferred. A transfer's chip-side + /// lane is its PCI address modulo 8, an approximation: the chip's + /// internal buffer alignment is not modelled. + fn lane_xor(&self, frame: bool) -> usize { + let mode = self.rf(reg::MODE_CTL); + let (byte, word) = if frame { (0x10, 0x20) } else { (0x2, 0x4) }; + 7 ^ if mode & byte != 0 { 3 } else { 0 } ^ if mode & word != 0 { 4 } else { 0 } + } + + /// Read `len` bytes of the chip's internal image from `addr`. + fn dma_read_image(&self, addr: u64, len: usize, frame: bool) -> Option> { + if len == 0 { + return Some(Vec::new()); + } + let t = self.lane_xor(frame); + let astart = addr & !7; + let lead = (addr - astart) as usize; + let span = (lead + len + 7) & !7; + let mut raw = vec![0; span]; + if !self.dma.dma_read(astart, &mut raw).is_ok() { + return None; + } + Some( + (0..len) + .map(|j| { + let p = lead + j; + raw[(p & !7) + ((p & 7) ^ t)] + }) + .collect(), + ) + } + + /// Write `data` of the chip's internal image to `addr`, touching only the + /// lanes it covers as the byte enables do: whole doublewords as one + /// transfer, the partial ends a byte at a time. + fn dma_write_image(&self, addr: u64, data: &[u8], frame: bool) -> bool { + let t = self.lane_xor(frame); + let mut i = 0; + while i < data.len() { + let a = addr + i as u64; + if a & 7 == 0 && data.len() - i >= 8 { + let run = (data.len() - i) & !7; + let mut raw = vec![0; run]; + for j in 0..run { + raw[(j & !7) + ((j & 7) ^ t)] = data[i + j]; + } + if !self.dma.dma_write(a, &raw).is_ok() { + return false; + } + i += run; + } else { + let at = (a & !7) + ((a & 7) as usize ^ t) as u64; + if !self.dma.dma_write(at, &data[i..i + 1]).is_ok() { + return false; + } + i += 1; + } + } + true + } + + // ---- status block and coalescing ---- + + /// DMA the status block to the host address the driver programmed, only + /// while host coalescing is enabled. The chip owns the block, so it is + /// built whole each time. `raise` marks an update the driver is meant to + /// be interrupted for. Returns whether it reached the host. + fn post_status_block(&mut self, raise: bool) -> bool { + if self.rf(reg::HCC_MODE) & HCC_MODE_ENABLE == 0 { + return false; + } + let base = self.compose_hostaddr(self.rf(reg::HCC_STATUSBLK_ADDR_HI), self.rf(reg::HCC_STATUSBLK_ADDR_LO)); + let next_tag = self.status_tag.wrapping_add(1); + let mut blk = [0u8; STATUS_BLK_SIZE]; + st32(&mut blk, 0, STATFLAG_UPDATED); + st32(&mut blk, STATUS_TAG_OFF, u32::from(next_tag)); + blk[STATUS_RX_STD_CONS_IMAGE_OFF..][..2].copy_from_slice(&self.rx_std_cons.to_be_bytes()); + blk[STATUS_RX_JUMBO_CONS_IMAGE_OFF..][..2].copy_from_slice(&self.rx_jumbo_cons.to_be_bytes()); + st16(&mut blk, STATUS_TX_CONS_OFF, self.status_tx_cons); + st16(&mut blk, STATUS_RX_PROD_OFF, self.status_rx_prod); + if !self.dma_write_image(base, &blk, false) { + self.refuse(Refused::DmaFault(DmaWhat::StatusBlock)); + return false; + } + self.status_tag = next_tag; + // The block now carries an update the driver has not taken. The line + // asserts while that latch is set and interrupts are unmasked, and + // drops when the driver acknowledges through the interrupt mailbox. + if raise { + self.irq_pending = true; + self.update_irq(false); + } + true + } + + fn coal_arm_one(&mut self, rx: bool) { + let ticks = self.rf(if rx { reg::HCC_RX_COAL_TICKS } else { reg::HCC_TX_COAL_TICKS }); + let deadline = (ticks != 0).then(|| self.now + COAL_TICK * ticks); + if rx { + self.rx_coal_expired = false; + self.rx_coal_timer = deadline; + } else { + self.tx_coal_expired = false; + self.tx_coal_timer = deadline; + } + } + + /// Both budgets restart together: one update reports every ring. + fn coal_arm(&mut self) { + self.coal_arm_one(true); + self.coal_arm_one(false); + } + + /// One host update. All the ring indices are in one status block, so + /// whichever criterion fired reports every ring and restarts every count + /// and timer. + fn status_update(&mut self) -> bool { + if !self.post_status_block(true) { + return false; + } + self.rx_coal_pending = 0; + self.tx_coal_pending = 0; + self.coal_arm(); + true + } + + /// Retry later instead of spinning on an address the guest has not + /// repaired. + fn coal_retry(&mut self, rx: bool) { + let ticks = self.rf(if rx { reg::HCC_RX_COAL_TICKS } else { reg::HCC_TX_COAL_TICKS }); + let wait = (COAL_TICK * ticks.max(1)).min(COAL_RETRY_MAX); + let deadline = Some(self.now + wait); + if rx { + self.rx_coal_timer = deadline; + } else { + self.tx_coal_timer = deadline; + } + } + + fn rx_coal_flush(&mut self) { + if self.rx_coal_pending == 0 || self.status_update() { + return; + } + self.coal_retry(true); + } + + /// The receive budget ran out. It reports whatever arrived since the last + /// update; with nothing to report it stays expired, and the next frame is + /// reported as it lands. + pub(super) fn rx_coal_timer_fired(&mut self) { + self.rx_coal_timer = None; + self.rx_coal_expired = true; + self.rx_coal_flush(); + } + + fn tx_coal_flush(&mut self) { + if self.tx_coal_pending == 0 || self.status_update() { + return; + } + self.coal_retry(false); + } + + pub(super) fn tx_coal_timer_fired(&mut self) { + self.tx_coal_timer = None; + self.tx_coal_expired = true; + self.tx_coal_flush(); + } + + /// Count the descriptors this doorbell completed, and report them once + /// the count criterion is met or the send budget has already run out. + /// + /// A doorbell that completes nothing is reported anyway, which no + /// criterion covers: bring-up leans on that update arriving. This is a + /// known deviation from the coalescing contract, not an established + /// hardware behaviour. + fn tx_coalesce(&mut self, completed: usize) { + let max_bds = self.rf(reg::HCC_TX_MAX_COAL_BDS); + if completed == 0 { + self.status_update(); + return; + } + self.tx_coal_pending = self.tx_coal_pending.wrapping_add(completed as u16); + if (max_bds != 0 && u32::from(self.tx_coal_pending) >= max_bds) || self.tx_coal_expired { + self.tx_coal_flush(); + } + } + + /// A received frame filled a return-ring descriptor. + fn rx_coalesce(&mut self) { + let max_bds = self.rf(reg::HCC_RX_MAX_COAL_BDS); + self.rx_coal_pending = self.rx_coal_pending.wrapping_add(1); + if (max_bds != 0 && u32::from(self.rx_coal_pending) >= max_bds) || self.rx_coal_expired { + self.rx_coal_flush(); + } + } + + pub(super) fn hcc_mode_written(&mut self) { + if self.rf(reg::HCC_MODE) & HCC_MODE_ENABLE != 0 { + // The budgets run from here until the first update reloads them. + self.coal_arm(); + self.retry_rx_flush(); + } else { + self.rx_coal_timer = None; + self.tx_coal_timer = None; + } + } + + /// A budget was reprogrammed. A running timer keeps the value it was + /// loaded with, since the chip reloads one only at a status-block update; + /// one that is not running starts here, because nothing else would. + pub(super) fn coal_ticks_written(&mut self, rx: bool) { + let (expired, running) = if rx { + (self.rx_coal_expired, self.rx_coal_timer.is_some()) + } else { + (self.tx_coal_expired, self.tx_coal_timer.is_some()) + }; + if self.rf(reg::HCC_MODE) & HCC_MODE_ENABLE == 0 || expired || running { + return; + } + self.coal_arm_one(rx); + } + + // ---- transmit ---- + + fn tx_rollback_to_frame(&mut self, frame_start: u16, completed: usize) { + self.tx_cons = frame_start; + if completed != 0 { + self.status_tx_cons = frame_start; + self.tx_coalesce(completed); + } + } + + /// Replace the frame's source address with a filter slot's, as the + /// descriptor asks. + fn tx_replace_source(&self, frame: &mut [u8], flags: u16) { + if flags & TXBDFLAG_INSERT_SRC_ADDR == 0 || frame.len() < 12 { + return; + } + let slot = u32::from((flags & TXBDFLAG_CHOOSE_SRC_ADDR) >> TXBDFLAG_SRC_ADDR_SHIFT); + let hi = self.rf(reg::MAC_ADDR_0_HIGH + slot * reg::MAC_ADDR_STRIDE); + let lo = self.rf(reg::MAC_ADDR_0_LOW + slot * reg::MAC_ADDR_STRIDE); + frame[6] = (hi >> 8) as u8; + frame[7] = hi as u8; + frame[8..12].copy_from_slice(&lo.to_be_bytes()); + } + + /// Cut one oversized TCP buffer into segments the peer will accept. The + /// guest hands down a single frame holding many segments' worth of + /// payload; each piece goes out with its own copy of the headers, its own + /// sequence number, and the finishing flags only on the last. Anything + /// that does not look like the IPv4 TCP frame this is for is sent + /// unchanged rather than guessed at. + fn tx_segment(&mut self, mut frame: Vec, mss: usize, flags: u16) { + let Some(LsoLayout { ihl, thl, hdr, payload }) = lso_layout(&frame, mss) else { + if flags & (TXBDFLAG_IP_CKSUM | TXBDFLAG_TCP_UDP_CKSUM) != 0 { + tx_offload_checksum(&mut frame, flags); + } + self.wire.push(frame); + return; + }; + let seq = be32(&frame, 14 + ihl + 4); + let ip_id = u16::from_be_bytes([frame[18], frame[19]]); + let mut off = 0; + while off < payload { + let piece = mss.min(payload - off); + let mut seg = frame[..hdr].to_vec(); + seg.extend_from_slice(&frame[hdr + off..hdr + off + piece]); + seg[16..18].copy_from_slice(&((ihl + thl + piece) as u16).to_be_bytes()); + seg[18..20].copy_from_slice(&ip_id.wrapping_add((off / mss) as u16).to_be_bytes()); + seg[14 + ihl + 4..14 + ihl + 8].copy_from_slice(&seq.wrapping_add(off as u32).to_be_bytes()); + if off + piece < payload { + // Only the final piece finishes the sender's message. + seg[14 + ihl + 13] &= !0x09; + } + seg[24..26].fill(0); + seg[14 + ihl + 16..14 + ihl + 18].fill(0); + checksum_calculate(&mut seg, true, true); + self.wire.push(seg); + off += piece; + } + } + + /// Ring the transmit doorbell: send every descriptor from the chip's + /// consumer up to the producer index the driver just wrote, then report + /// the new consumer through the status block. + /// + /// The producer is a descriptor index the driver reduces modulo the ring + /// length itself; the failure mode is an index past the ring, which would + /// leave the walk chasing a consumer it never reaches. An out-of-range + /// index is refused, and the walk is bounded by the ring length + /// regardless. A frame whose descriptors are not all there yet, or whose + /// memory cannot be read, is left to retry from its first descriptor. + pub(super) fn tx_kick(&mut self, prod_raw: u32) { + let prod = (prod_raw & 0xffff) as u16; + let ring_len = (self.sram_read(SRAM_SEND_RING_RCB + RCB_MAXLEN_FLAGS) >> 16) as u16; + if ring_len == 0 || prod >= ring_len { + self.refuse(Refused::ProducerOutOfRange(Mailbox::TxProducer)); + return; + } + let ring_base = self.compose_hostaddr( + self.sram_read(SRAM_SEND_RING_RCB + RCB_HOSTADDR_HI), + self.sram_read(SRAM_SEND_RING_RCB + RCB_HOSTADDR_LO), + ); + let mut frame: Vec = Vec::new(); + let mut walked = 0usize; + let mut completed = 0usize; + let mut frame_flags = 0u16; + let mut frame_start = self.tx_cons; + let mut frame_mss = 0usize; + let mut dropped = false; + while self.tx_cons != prod && walked < usize::from(ring_len) { + let Some(bd) = self.dma_read_image(ring_base + u64::from(self.tx_cons) * TXBD_SIZE, TXBD_SIZE as usize, false) + else { + self.refuse(Refused::DmaFault(DmaWhat::TxDescriptor)); + self.tx_rollback_to_frame(frame_start, completed); + return; + }; + let buf_addr = u64::from(be32(&bd, 0)) << 32 | u64::from(be32(&bd, 4)); + let len = usize::from(ld16(&bd, TXBD_LEN)); + let flags = ld16(&bd, TXBD_FLAGS); + if !dropped && frame.len() + len <= MAX_FRAME { + let Some(payload) = self.dma_read_image(buf_addr, len, true) else { + self.refuse(Refused::DmaFault(DmaWhat::TxPayload)); + self.tx_rollback_to_frame(frame_start, completed); + return; + }; + frame.extend_from_slice(&payload); + } else if !dropped { + // The frame overran the buffer. Drop the whole frame and stay + // dropping across the remaining fragments so a later one is + // not mistaken for a fresh packet; resume at the next END. + self.refuse(Refused::TxFrameTooLarge); + dropped = true; + } + frame_flags |= flags; + if frame_mss == 0 { + frame_mss = usize::from(ld16(&bd, TXBD_MSS)); + } + if flags & TXBDFLAG_END != 0 { + if !dropped && !frame.is_empty() { + let mut whole = std::mem::take(&mut frame); + if frame_mss == 0 { + self.tx_replace_source(&mut whole, frame_flags); + } + self.tx_segment(whole, frame_mss, frame_flags); + } + frame.clear(); + frame_flags = 0; + frame_mss = 0; + dropped = false; + } + self.tx_cons = (self.tx_cons + 1) % ring_len; + walked += 1; + if flags & TXBDFLAG_END != 0 { + frame_start = self.tx_cons; + completed = walked; + } + } + if self.tx_cons != frame_start { + self.tx_rollback_to_frame(frame_start, completed); + return; + } + self.status_tx_cons = prod; + self.tx_coalesce(completed); + } + + // ---- receive mailboxes ---- + + pub(super) fn rx_mailbox_written(&mut self, off: u32, val: u32) { + let index = (val & 0xffff) as u16; + match off { + reg::MBX_RX_JUMBO_PROD_LO => { + // Posting jumbo-ring buffers; guard the index like the others. + if index >= RX_JUMBO_RING_LEN { + self.refuse(Refused::ProducerOutOfRange(Mailbox::RxJumboProducer)); + return; + } + self.set_rf(off, val); + // A write that actually posts a buffer ends the shortage. + if index != self.rx_jumbo_cons { + self.rx_shortage_drops = 0; + } + } + reg::MBX_RX_STD_PROD_LO => { + let std_len = self.rf(reg::RX_STD_RCB_MAXLEN_FLAGS) >> 16; + if std_len == 0 || index >= RX_STD_RING_LEN { + self.refuse(Refused::ProducerOutOfRange(Mailbox::RxStandardProducer)); + return; + } + if index != self.rx_std_cons { + self.rx_std_shortage_drops = 0; + } + self.set_rf(off, val); + } + _ => { + // The driver's return-ring acknowledgement, same range guard. + let ret_len = (self.sram_read(SRAM_RX_RETURN_RCB + RCB_MAXLEN_FLAGS) >> 16) as u16; + if ret_len == 0 || index >= ret_len { + self.refuse(Refused::ProducerOutOfRange(Mailbox::RxReturnConsumer)); + return; + } + self.set_rf(off, val); + } + } + // Room where there was none: take anything the peer had waiting. + self.retry_rx_flush(); + } + + // ---- receive ---- + + /// Whether the jumbo ring can take a frame at all, independent of whether + /// one is posted: it needs a host base and must not be disabled. Extended + /// RX Enable asks for the 64-byte descriptor format this model does not + /// parse, so a ring asking for it is treated as disabled. Max_Len plays no + /// part: hardware ignores it for the jumbo ring. + fn jumbo_ring_enabled(&self) -> bool { + let flags = self.rf(reg::RX_JUMBO_RCB_MAXLEN_FLAGS); + let base = self.compose_hostaddr(self.rf(reg::RX_JUMBO_RCB_HADDR_HI), self.rf(reg::RX_JUMBO_RCB_HADDR_LO)); + base != 0 && flags & (RCB_FLAG_DISABLE_RING | RCB_FLAG_EXT_RX_ENABLE) == 0 + } + + /// Whether the engine can take a frame: the rings must be set up and host + /// coalescing enabled, and the host must have posted a buffer the chip has + /// not yet used. + pub(super) fn rx_ready(&self) -> bool { + let std_len = self.rf(reg::RX_STD_RCB_MAXLEN_FLAGS) >> 16; + let ret_len = (self.sram_read(SRAM_RX_RETURN_RCB + RCB_MAXLEN_FLAGS) >> 16) as u16; + let std_prod = (self.rf(reg::MBX_RX_STD_PROD_LO) & 0xffff) as u16; + // A completion the driver has not read yet must not be overwritten; + // when the next would land on the entry the driver has consumed up to, + // the ring is full and the frame is refused. + let ret_cons = (self.rf(reg::MBX_RX_CONS0_LO) & 0xffff) as u16; + // Either ring having a buffer posted is enough; which one a frame + // takes depends on its size. + let jumbo_prod = (self.rf(reg::MBX_RX_JUMBO_PROD_LO) & 0xffff) as u16; + let posted = (std_len != 0 && std_prod != self.rx_std_cons) + || (self.jumbo_ring_enabled() && jumbo_prod != self.rx_jumbo_cons); + if self.rx_latency.len() >= RX_LATENCY_DEPTH { + return false; + } + // Against the staged producer, not the published one: a return + // descriptor whose completion is still held has already been written. + ret_len != 0 + && posted + && self.rf(reg::HCC_MODE) & HCC_MODE_ENABLE != 0 + && (self.rx_prod_staged + 1) % ret_len != ret_cons + } + + /// The address the part filters on is MAC address filter slot 0, which the + /// driver programs, not the address the board gave the device: IRIX + /// installs the system Ethernet address over whatever the card came up + /// with. + fn frame_accepted(&self, frame: &[u8]) -> bool { + if frame.len() < 6 { + return false; + } + if frame[0] & 1 != 0 { + return true; + } + let hi = self.rf(reg::MAC_ADDR_0_HIGH); + let lo = self.rf(reg::MAC_ADDR_0_LOW); + frame[..6] == [(hi >> 8) as u8, hi as u8, (lo >> 24) as u8, (lo >> 16) as u8, (lo >> 8) as u8, lo as u8] + } + + fn count_no_more_rxbds(&mut self) { + let n = self.sram_read(SRAM_STATS_NO_MORE_RXBDS).wrapping_add(1); + self.sram_write(SRAM_STATS_NO_MORE_RXBDS, n); + } + + /// A frame from the backend. Not ready, or no buffer posted, defers it: + /// the only answer that leaves the frame at the peer's queue head. A frame + /// the address filter drops is consumed, never deferred, so the peer is + /// not wedged waiting on a frame that will never be taken. Otherwise the + /// frame is DMA'd into the next posted buffer, a completed descriptor is + /// written into the return ring, and the producer the status block reports + /// advances once the frame's arrival deadline passes. + pub(super) fn receive(&mut self, frame: &[u8]) -> RxOutcome { + // A halted receive MAC is not listening, so the frame is gone rather + // than held: a backend that queued it would hand the driver traffic + // from before the interface existed. + if self.rf(reg::RX_MODE) & 2 == 0 { + return RxOutcome::Consumed; + } + if !self.rx_ready() { + return RxOutcome::Deferred; + } + if !self.frame_accepted(frame) { + return RxOutcome::Consumed; + } + if frame.len() > MAX_FRAME { + // Drop it rather than deliver part of it: a frame cut short looks + // whole, and the failure surfaces wherever the truncated payload + // is finally parsed. + self.refuse(Refused::RxFrameTooLarge); + return RxOutcome::Consumed; + } + // Pad a short frame to the 60-byte Ethernet minimum: a backend can + // hand over a runt, and the driver rejects a descriptor below it. + let mut padded; + let buf: &[u8] = if frame.len() < MIN_FRAME { + padded = frame.to_vec(); + padded.resize(MIN_FRAME, 0); + &padded + } else { + frame + }; + let framelen = buf.len() as u32; + + // Pick the ring by what the frame fits in. The standard ring's Max_Len + // is the driver's statement of how much room its buffers hold; a + // larger frame goes to the jumbo ring when it is enabled, and is + // dropped rather than written past the end of someone's buffer when + // it is not. + let std_size = u32::from((self.rf(reg::RX_STD_RCB_MAXLEN_FLAGS) >> 16) as u16); + let std_prod = (self.rf(reg::MBX_RX_STD_PROD_LO) & 0xffff) as u16; + let jumbo_prod = (self.rf(reg::MBX_RX_JUMBO_PROD_LO) & 0xffff) as u16; + let jumbo_enabled = self.jumbo_ring_enabled(); + let (ring, ring_base, idx) = if framelen <= std_size { + if std_prod == self.rx_std_cons { + // The frame fits a standard buffer and none is posted; it + // cannot take a jumbo one, because a frame's ring is chosen by + // its length against the standard ring's Max_Len. Discard + // rather than refuse: refusing stops the backend reading and + // strands the frames behind it that the jumbo ring can carry. + // Only while a jumbo buffer is posted; without one the caller + // is owed the refusal. + if !jumbo_enabled || jumbo_prod == self.rx_jumbo_cons { + return RxOutcome::Deferred; + } + if self.rx_std_shortage_drops == 0 { + self.refuse(Refused::RxShortage(Ring::Standard)); + } + self.rx_std_shortage_drops += 1; + self.count_no_more_rxbds(); + return RxOutcome::Consumed; + } + let base = self + .compose_hostaddr(self.rf(reg::RX_STD_RCB_HADDR_HI), self.rf(reg::RX_STD_RCB_HADDR_LO)); + (Ring::Standard, base, self.rx_std_cons) + } else if jumbo_enabled { + if jumbo_prod == self.rx_jumbo_cons { + // A frame longer than the standard Max_Len consumes a + // descriptor from the jumbo ring, and nothing is posted. + // Discard rather than refuse, for the reason above, and only + // while a standard buffer is posted. + if std_size == 0 || std_prod == self.rx_std_cons { + return RxOutcome::Deferred; + } + if self.rx_shortage_drops == 0 { + self.refuse(Refused::RxShortage(Ring::Jumbo)); + } + self.rx_shortage_drops += 1; + self.count_no_more_rxbds(); + return RxOutcome::Consumed; + } + let base = self + .compose_hostaddr(self.rf(reg::RX_JUMBO_RCB_HADDR_HI), self.rf(reg::RX_JUMBO_RCB_HADDR_LO)); + (Ring::Jumbo, base, self.rx_jumbo_cons) + } else { + self.refuse(Refused::RxNoRoom(Ring::Standard)); + return RxOutcome::Consumed; + }; + + let ret_len = (self.sram_read(SRAM_RX_RETURN_RCB + RCB_MAXLEN_FLAGS) >> 16) as u16; + let ret_base = self.compose_hostaddr( + self.sram_read(SRAM_RX_RETURN_RCB + RCB_HOSTADDR_HI), + self.sram_read(SRAM_RX_RETURN_RCB + RCB_HOSTADDR_LO), + ); + + // The posted buffer's host address, from the chosen ring's descriptor. + let Some(std_bd) = self.dma_read_image(ring_base + u64::from(idx) * RXBD_SIZE, RXBD_SIZE as usize, false) + else { + self.refuse(Refused::DmaFault(DmaWhat::RxDescriptor)); + return RxOutcome::Deferred; + }; + let buf_addr = u64::from(be32(&std_bd, 0)) << 32 | u64::from(be32(&std_bd, 4)); + let mut rx_flags = RXBDFLAG_END; + if ring == Ring::Jumbo { + // The jumbo ring carries no ring-wide buffer size; the driver + // states each buffer's size in the descriptor it posts. + let buf_len = be32(&std_bd, RXBD_IDX_LEN) & 0xffff; + if framelen > buf_len { + self.refuse(Refused::RxNoRoom(Ring::Jumbo)); + return RxOutcome::Consumed; + } + rx_flags |= RXBDFLAG_JUMBO_RING; + } + if !self.dma_write_image(buf_addr, buf, true) { + self.refuse(Refused::DmaFault(DmaWhat::RxPayload)); + return RxOutcome::Deferred; + } + let csum = rx_offload_checksum(buf, self.rf(reg::MODE_CTL), &mut rx_flags); + + let mut ret_bd = [0u8; RXBD_SIZE as usize]; + st32(&mut ret_bd, 0, (buf_addr >> 32) as u32); + st32(&mut ret_bd, 4, buf_addr as u32); + // The length includes the four-byte FCS the driver strips. + st32(&mut ret_bd, RXBD_IDX_LEN, u32::from(idx) << 16 | ((framelen + FCS_LEN) & 0xffff)); + st32(&mut ret_bd, RXBD_TYPE_FLAGS, rx_flags); + st32(&mut ret_bd, RXBD_CSUM, csum); + st32(&mut ret_bd, RXBD_OPAQUE, be32(&std_bd, RXBD_OPAQUE)); + if !self.dma_write_image(ret_base + u64::from(self.rx_prod_staged) * RXBD_SIZE, &ret_bd, false) { + self.refuse(Refused::DmaFault(DmaWhat::RxCompletion)); + return RxOutcome::Deferred; + } + match ring { + Ring::Jumbo => self.rx_jumbo_cons = (self.rx_jumbo_cons + 1) % RX_JUMBO_RING_LEN, + Ring::Standard => self.rx_std_cons = (self.rx_std_cons + 1) % RX_STD_RING_LEN, + } + self.rx_prod_staged = (self.rx_prod_staged + 1) % ret_len; + // The frame and its return descriptor are in guest memory; the + // producer the status block reports, and the interrupt after it, wait + // for the frame's arrival deadline. + self.rx_hold(self.rx_prod_staged); + RxOutcome::Consumed + } + + // ---- the arrival line ---- + + fn rx_publish(&mut self, status_rx_prod: u16) { + self.status_rx_prod = status_rx_prod; + self.rx_coalesce(); + } + + /// Hold a completion until the frame's minimum arrival latency has passed. + fn rx_hold(&mut self, status_rx_prod: u16) { + if self.rx_floor.is_zero() && self.rx_latency.is_empty() { + self.rx_publish(status_rx_prod); + } else { + self.rx_latency.push_back((self.now + self.rx_floor, status_rx_prod)); + } + } + + /// The head completion's deadline passed. + pub(super) fn rx_arrival_due(&mut self) { + let mut published = false; + while self.rx_latency.front().is_some_and(|(d, _)| *d <= self.now) { + let (_, prod) = self.rx_latency.pop_front().unwrap(); + self.rx_publish(prod); + published = true; + } + if published { + self.retry_rx_flush(); + } + } +} diff --git a/src/dev/bcm5701/eeprom.rs b/src/dev/bcm5701/eeprom.rs new file mode 100644 index 000000000..b56868226 --- /dev/null +++ b/src/dev/bcm5701/eeprom.rs @@ -0,0 +1,113 @@ +//! The BCM5701's serial EEPROM, behind the controller at registers 0x6838 +//! (address and command), 0x683c (data) and 0x6840 (direct serial pins). +//! +//! The controller transfers one aligned 32-bit word at a time. The backing is +//! caller-supplied: nothing here synthesises a signature, a bootstrap, a CRC +//! or a station address, because no complete board image is established. A +//! write goes through to the backing file when there is one, so contents +//! persist across runs. Transfers complete synchronously and the serial bus +//! timing, including the 10 ms spacing between writes, is not modelled. +//! +//! Ported from hw/net/bcm570x.c in jamesbraid/qemu-sgi-origami. + +use std::path::PathBuf; + +pub const MIN_SIZE: usize = 4; +pub const MAX_SIZE: usize = 65536; + +/// Why a transfer did not happen. The register file keeps Complete clear and +/// the data register unchanged; the caller counts and logs the reason. +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +pub enum Fault { + /// No backing is configured, whatever the device ID. + NoBacking, + /// A different device ID than the one configured. + WrongDevice(u8), + /// The word lies beyond the backing. + OutOfRange(u32), + ReadOnly, + /// The backing file could not be written. + Io, +} + +#[derive(Debug)] +pub struct Eeprom { + device_id: u8, + data: Vec, + path: Option, + read_only: bool, +} + +impl Eeprom { + /// An in-memory backing. `device_id` is 0..=7; the image is 4 to 65536 + /// bytes in a multiple of four. + pub fn from_image(device_id: u8, data: Vec) -> Result { + if device_id > 7 { + return Err("bcm5701: eeprom requires a device id in 0..=7".into()); + } + if data.len() < MIN_SIZE || data.len() > MAX_SIZE || data.len() % 4 != 0 { + return Err("bcm5701: EEPROM backing must be 4 to 65536 bytes and a multiple of four".into()); + } + Ok(Self { device_id, data, path: None, read_only: false }) + } + + /// A backing read from `path` and written back to it. A file the + /// filesystem marks read-only makes a read-only backing. + pub fn from_file(device_id: u8, path: PathBuf) -> Result { + let data = std::fs::read(&path).map_err(|e| format!("bcm5701: EEPROM {}: {e}", path.display()))?; + let read_only = std::fs::metadata(&path).map(|m| m.permissions().readonly()).unwrap_or(true); + let mut eeprom = Self::from_image(device_id, data)?; + eeprom.read_only = read_only; + eeprom.path = Some(path); + Ok(eeprom) + } + + /// Refuse writes, as a backing opened read-only does. + pub fn set_read_only(&mut self, read_only: bool) { + self.read_only = read_only; + } + + pub fn device_id(&self) -> u8 { + self.device_id + } + + pub fn is_read_only(&self) -> bool { + self.read_only + } + + pub fn bytes(&self) -> &[u8] { + &self.data + } + + fn word_offset(&self, device: u8, word: u32) -> Result { + if device != self.device_id { + return Err(Fault::WrongDevice(device)); + } + let offset = word as usize * 4; + if offset + 4 > self.data.len() { + return Err(Fault::OutOfRange(word)); + } + Ok(offset) + } + + /// The word at `word`, least significant byte first: the image bytes + /// `66 99 55 aa` read as `0xaa559966`. + pub fn read_word(&self, device: u8, word: u32) -> Result { + let offset = self.word_offset(device, word)?; + Ok(u32::from_le_bytes(self.data[offset..offset + 4].try_into().unwrap())) + } + + pub fn write_word(&mut self, device: u8, word: u32, value: u32) -> Result<(), Fault> { + let offset = self.word_offset(device, word)?; + if self.read_only { + return Err(Fault::ReadOnly); + } + let mut image = self.data.clone(); + image[offset..offset + 4].copy_from_slice(&value.to_le_bytes()); + if let Some(path) = &self.path { + crate::nv_storage::save(&path.to_string_lossy(), &image).map_err(|_| Fault::Io)?; + } + self.data = image; + Ok(()) + } +} diff --git a/src/dev/bcm5701/phy.rs b/src/dev/bcm5701/phy.rs new file mode 100644 index 000000000..f6a37eed8 --- /dev/null +++ b/src/dev/bcm5701/phy.rs @@ -0,0 +1,263 @@ +//! The BCM570x's integrated copper PHY, reached through the MAC's MI_COMM +//! register. +//! +//! The PHY has no bus address of its own: software writes a register number +//! into MI_COMM and reads a word back. The contract is the ordinary MII/GMII +//! one the public brgphy driver drives. Link is modelled without a cable: out +//! of reset the PHY negotiates, and autonegotiation completes a fixed settle +//! later, which restarts when the guest restarts it. Reported link is also +//! gated on the backend's cable state. +//! +//! The identity decodes to Broadcom's organisational identifier and the +//! BCM5701 model, the pair brgphy matches to bind. +//! +//! Ported from hw/net/bcm570x_phy.c in jamesbraid/qemu-sgi-origami. + +use std::time::{Duration, Instant}; + +pub mod reg { + pub const BMCR: u8 = 0x00; + pub const BMSR: u8 = 0x01; + pub const IDR1: u8 = 0x02; + pub const IDR2: u8 = 0x03; + pub const ANAR: u8 = 0x04; + pub const ANLPAR: u8 = 0x05; + pub const GTCR: u8 = 0x09; + pub const GTSR: u8 = 0x0a; + pub const EXTSR: u8 = 0x0f; + /// Broadcom auxiliary status summary. + pub const AUXSTS: u8 = 0x19; +} + +pub const IDR1_VALUE: u16 = 0x0020; +pub const IDR2_BCM5701: u16 = 0x6110; + +pub mod bmcr { + pub const RESET: u16 = 0x8000; + pub const AUTOEN: u16 = 0x1000; + pub const ISOLATE: u16 = 0x0400; + pub const STARTNEG: u16 = 0x0200; + pub const FDX: u16 = 0x0100; + pub const SPEED1000: u16 = 0x0040; +} +mod bmsr { + pub const TX100_FDX: u16 = 0x4000; + pub const TX100_HDX: u16 = 0x2000; + pub const T10_FDX: u16 = 0x1000; + pub const T10_HDX: u16 = 0x0800; + pub const EXTSTAT: u16 = 0x0100; + pub const ANEGDONE: u16 = 0x0020; + pub const CANAUTONEG: u16 = 0x0008; + pub const LINK: u16 = 0x0004; + pub const EXTCAP: u16 = 0x0001; +} +mod anar { + pub const TX100_FDX: u16 = 0x0100; + pub const TX100_HDX: u16 = 0x0080; + pub const T10_FDX: u16 = 0x0040; + pub const T10_HDX: u16 = 0x0020; + pub const CSMA: u16 = 0x0001; + pub const ACK: u16 = 0x4000; +} +const GTCR_ADV_1000TFDX: u16 = 0x0200; +const GTSR_LINKED: u16 = 0x4000 | 0x2000 | 0x1000 | 0x0800; +const EXTSR_1000: u16 = 0x2000 | 0x1000; +const AUXSTS_UP: u16 = 0x8000 | 0x0700; + +const BMCR_RESET_VALUE: u16 = bmcr::AUTOEN | bmcr::FDX | bmcr::SPEED1000; +const ANAR_RESET_VALUE: u16 = + anar::CSMA | anar::T10_HDX | anar::T10_FDX | anar::TX100_HDX | anar::TX100_FDX; + +/// How long autonegotiation takes to settle. It has to finish inside the +/// driver's own patience: the IRIX driver gives up its link wait well before +/// a second. +pub const SETTLE: Duration = Duration::from_millis(100); + +/// What the PHY could not carry out, for the owner to count and log. +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +pub enum Refusal { + /// A read of a register the model does not implement; it reads 0, never + /// the all-ones an absent PHY returns. + ReadUnimplemented(u8), + WriteUnimplemented(u8), + /// A write to a read-only register. + WriteReadOnly(u8), +} + +pub struct Phy { + bmcr: u16, + anar: u16, + gtcr: u16, + autoneg_done: bool, + link_down: bool, + settle_armed: bool, + settle: Option, + pub refused: Vec, +} + +impl Phy { + pub fn new(now: Instant) -> Self { + let mut phy = Self { + bmcr: 0, + anar: 0, + gtcr: 0, + autoneg_done: false, + link_down: false, + settle_armed: false, + settle: None, + refused: Vec::new(), + }; + phy.reset(now); + phy + } + + /// The deadline at which the settle completes. + pub fn deadline(&self) -> Option { + self.settle + } + + /// Complete the settle if its time has come. + pub fn sync(&mut self, now: Instant) { + if self.settle.is_some_and(|d| d <= now) { + self.settle = None; + self.autoneg_done = true; + } + } + + pub fn autoneg_done(&self) -> bool { + self.autoneg_done + } + + pub fn link_up(&self) -> bool { + self.autoneg_done && !self.link_down + } + + /// The far end's cable state; not a register, so a PHY reset leaves it. + pub fn set_link_down(&mut self, down: bool) { + self.link_down = down; + } + + pub fn link_down(&self) -> bool { + self.link_down + } + + fn status(&self) -> u16 { + let mut status = bmsr::EXTCAP + | bmsr::CANAUTONEG + | bmsr::EXTSTAT + | bmsr::TX100_FDX + | bmsr::TX100_HDX + | bmsr::T10_FDX + | bmsr::T10_HDX; + if self.link_up() { + status |= bmsr::LINK | bmsr::ANEGDONE; + } + status + } + + /// A real PHY negotiates as soon as autonegotiation is enabled, so the + /// settle runs from reset and from the write that restarts it, not from + /// the driver's first look at link status. + fn arm_settle(&mut self, now: Instant) { + if !self.settle_armed && !self.autoneg_done && self.bmcr & bmcr::AUTOEN != 0 { + self.settle_armed = true; + self.settle = Some(now + SETTLE); + } + } + + fn refuse(&mut self, what: Refusal) { + if self.refused.len() < 64 { + self.refused.push(what); + } + } + + pub fn read(&mut self, register: u8, now: Instant) -> u16 { + self.sync(now); + let up = self.link_up(); + match register { + reg::BMCR => self.bmcr, + reg::BMSR => { + self.arm_settle(now); + self.status() + } + reg::IDR1 => IDR1_VALUE, + reg::IDR2 => IDR2_BCM5701, + reg::ANAR => self.anar, + // The far end advertises exactly what this end does, which puts + // the driver on the route it trusts. + reg::ANLPAR => { + if up { + self.anar | anar::ACK + } else { + 0 + } + } + reg::GTCR => self.gtcr, + reg::GTSR => { + if up { + GTSR_LINKED + } else { + 0 + } + } + reg::EXTSR => EXTSR_1000, + // The register the driver gates its link-up path on, so a poll + // that reads only this one still brings the link up. + reg::AUXSTS => { + self.arm_settle(now); + if self.link_up() { + AUXSTS_UP + } else { + 0 + } + } + other => { + self.refuse(Refusal::ReadUnimplemented(other)); + 0 + } + } + } + + pub fn write(&mut self, register: u8, value: u16, now: Instant) { + self.sync(now); + match register { + reg::BMCR => { + // Reset and restart are self-clearing. + if value & bmcr::RESET != 0 { + self.reset(now); + return; + } + // Only a restart (or isolating the part) restarts + // negotiation: a driver rewrites this register for other + // reasons, and dropping a working link each time would show + // as the carrier coming back again and again. + if value & (bmcr::ISOLATE | bmcr::STARTNEG) != 0 { + self.autoneg_done = false; + self.settle_armed = false; + self.settle = None; + } + self.bmcr = value & !(bmcr::RESET | bmcr::STARTNEG); + if value & bmcr::ISOLATE == 0 { + self.arm_settle(now); + } + } + reg::ANAR => self.anar = value, + reg::GTCR => self.gtcr = value, + reg::BMSR | reg::IDR1 | reg::IDR2 | reg::ANLPAR | reg::GTSR | reg::EXTSR => { + self.refuse(Refusal::WriteReadOnly(register)); + } + other => self.refuse(Refusal::WriteUnimplemented(other)), + } + } + + pub fn reset(&mut self, now: Instant) { + self.bmcr = BMCR_RESET_VALUE; + self.anar = ANAR_RESET_VALUE; + self.gtcr = GTCR_ADV_1000TFDX; + self.autoneg_done = false; + self.settle_armed = false; + self.settle = None; + // Out of reset autonegotiation is enabled, so the part starts it. + self.arm_settle(now); + } +} diff --git a/src/dev/bcm5701/state.rs b/src/dev/bcm5701/state.rs new file mode 100644 index 000000000..6797058d1 --- /dev/null +++ b/src/dev/bcm5701/state.rs @@ -0,0 +1,702 @@ +//! The chip's register file, on-chip SRAM, interrupt line and the engines +//! that run inside register writes. The PCI side and the threads are in the +//! parent module. + +use std::collections::{BTreeMap, VecDeque}; +use std::sync::{Arc, Weak}; +use std::time::{Duration, Instant}; + +use super::eeprom::{self, Eeprom}; +use super::phy::{self, Phy}; +use super::dma::{Ring, Timer}; +use crate::pci::PciDma; +use super::Bcm5701; + +pub const REG_FILE_SIZE: u32 = 0x8000; +pub const SRAM_SIZE: u32 = 0x8000; +const REG_WORDS: usize = (REG_FILE_SIZE / 4) as usize; +const SRAM_WORDS: usize = (SRAM_SIZE / 4) as usize; + +/// Dword indices of the PCI configuration registers the chip owns. +pub mod cfg { + pub const MISC_CTL: usize = 0x68 / 4; + pub const PCISTATE: usize = 0x70 / 4; + pub const REG_BASEADDR: usize = 0x78 / 4; + pub const MEMWIN_BASEADDR: usize = 0x7c / 4; + pub const REG_DATA: usize = 0x80 / 4; + pub const MEMWIN_DATA: usize = 0x84 / 4; +} + +pub mod misc_ctl { + pub const CLEAR_INTA: u32 = 0x1; + pub const MASK_PCI_INTR: u32 = 0x2; + pub const ENDIAN_BYTESWAP: u32 = 0x4; + pub const ENDIAN_WORDSWAP: u32 = 0x8; + /// The chip id, bits 31:16, is read-only; the low half is writable. + pub const WRITABLE: u32 = 0xffff; +} + +pub mod pcistate { + pub const INTR_NOT_ACTIVE: u32 = 0x2; + pub const BUSMODE_PCI: u32 = 0x4; + /// The chip did not see REQ64# at the end of reset. + pub const BUS_32BIT: u32 = 0x10; +} + +/// The chip id MISC_CTL[31:16] presents; the driver reads the part from it. +pub const CHIPID_BCM5701: u32 = 0x0105; + +/// Register-file offsets with modelled behaviour. +pub mod reg { + pub const MAC_MODE: u32 = 0x0400; + pub const MAC_STATUS: u32 = 0x0404; + pub const MAC_ADDR_0_HIGH: u32 = 0x0410; + pub const MAC_ADDR_0_LOW: u32 = 0x0414; + pub const MAC_ADDR_STRIDE: u32 = 8; + pub const MI_COMM: u32 = 0x044c; + pub const MI_STS: u32 = 0x0450; + pub const RX_MODE: u32 = 0x0468; + pub const HCC_MODE: u32 = 0x3c00; + pub const BUFMGR_MODE: u32 = 0x4400; + pub const BUFMGR_STATUS: u32 = 0x4404; + pub const FW_PROGRESS: u32 = 0x5034; + pub const MODE_CTL: u32 = 0x6800; + pub const MISC_CFG: u32 = 0x6804; + pub const EEPROM_ADDR: u32 = 0x6838; + pub const EEPROM_DATA: u32 = 0x683c; + pub const EEPROM_PIN: u32 = 0x6840; + pub const EEPROM_DELAY: u32 = 0x6848; + pub const MBX_IRQ0_LO: u32 = 0x0204; + pub const MBX_RX_STD_PROD_LO: u32 = 0x026c; + pub const MBX_RX_JUMBO_PROD_LO: u32 = 0x0274; + pub const MBX_RX_CONS0_LO: u32 = 0x0284; + pub const MBX_TX_PROD_LO: u32 = 0x0304; + pub const HCC_RX_COAL_TICKS: u32 = 0x3c08; + pub const HCC_TX_COAL_TICKS: u32 = 0x3c0c; + pub const HCC_RX_MAX_COAL_BDS: u32 = 0x3c10; + pub const HCC_TX_MAX_COAL_BDS: u32 = 0x3c14; + pub const HCC_STATUSBLK_ADDR_HI: u32 = 0x3c38; + pub const HCC_STATUSBLK_ADDR_LO: u32 = 0x3c3c; + pub const RX_JUMBO_RCB_HADDR_HI: u32 = 0x2440; + pub const RX_JUMBO_RCB_HADDR_LO: u32 = 0x2444; + pub const RX_JUMBO_RCB_MAXLEN_FLAGS: u32 = 0x2448; + pub const RX_STD_RCB_HADDR_HI: u32 = 0x2450; + pub const RX_STD_RCB_HADDR_LO: u32 = 0x2454; + pub const RX_STD_RCB_MAXLEN_FLAGS: u32 = 0x2458; +} + +/// Bit 0 of a mode register is a self-clearing reset; bit 0 of MISC_CFG +/// resets the whole chip. +const MODE_RESET: u32 = 1; +const MISCCFG_RESET_CORE_CLOCKS: u32 = 1; +const RXMODE_ENABLE: u32 = 2; +const RCB_FLAG_EXT_RX_ENABLE: u32 = 1; + +const BUFMGR_MODE_ENABLE: u32 = 2; +/// The buffer manager's enable reads back only after its startup; a +/// functional approximation inside the driver's 10 ms polling budget. +const BUFMGR_START: Duration = Duration::from_millis(8); + +mod eeprom_cmd { + pub const READ: u32 = 1 << 31; + pub const COMPLETE: u32 = 1 << 30; + pub const RESET: u32 = 1 << 29; + pub const DEVICE_ID: u32 = 7 << 26; + pub const START: u32 = 1 << 25; + pub const WORD_ADDR: u32 = 0x3fff << 2; +} + +/// The firmware-ready handshake. The driver writes the magic to SRAM 0x0b50 +/// and polls for its complement, the on-chip bootcode's ready signal. The +/// mailbox returns the complement of the last value written, so the magic +/// reads back as ready whether the driver writes it before the reset or after. +pub const SRAM_FW_MBOX: u32 = 0x0b50; +pub const FW_MAGIC: u32 = 0x4b65_7654; +/// The firmware-progress register holds the all-ones "no progress yet" +/// sentinel, so a driver polling it sees "unchanged" and keeps polling the +/// mailbox. +const FW_PROGRESS_INIT: u32 = 0xffff_ffff; + +/// The station address the on-chip bootcode leaves where a driver looks for +/// it: SRAM 0x0c14 tagged 0x484b in its high half, and MAC address filter +/// slot 0. +const SRAM_MAC_HI: u32 = 0x0c14; +const SRAM_MAC_LO: u32 = 0x0c18; +const MAC_TAG: u32 = 0x484b; + +mod mi { + pub const DATA: u32 = 0x0000_ffff; + pub const REG: u32 = 0x001f_0000; + pub const REG_SHIFT: u32 = 16; + /// Command and address fields, kept across a read. + pub const READ_FIELDS: u32 = 0x0fff_0000; + pub const WRITE: u32 = 0x0400_0000; + pub const READ: u32 = 0x0800_0000; + pub const BUSY: u32 = 0x2000_0000; + pub const LINK: u32 = 1; +} + +/// What the model could not carry out, counted and logged on first sight so +/// an unexpected access is never silent. +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +pub enum Refused { + /// A write to a register with no modelled behaviour; the value is stored + /// and reads back. + UnmodelledWrite(u32), + /// A write to the read-only buffer-manager status. + BufmgrStatusWrite, + /// Direct EEPROM serial-pin control. + EepromPins, + EepromFault(eeprom::Fault), + /// An MI_COMM opcode other than read and write. + MiOpcode(u32), + /// A BAR access of a width or alignment the chip does not accept. + BarAccess { offset: u32, width: u8, write: bool }, + Phy(phy::Refusal), + /// A transfer to or from host memory that did not complete. + DmaFault(super::dma::DmaWhat), + /// A ring index written to a mailbox that lies outside the ring. + ProducerOutOfRange(super::dma::Mailbox), + /// A transmit frame larger than the model carries; it is dropped. + TxFrameTooLarge, + /// A received frame larger than the model carries; it is dropped. + RxFrameTooLarge, + /// A frame discarded for want of a posted descriptor, once per drought. + RxShortage(Ring), + /// A frame too big for the ring it would use; it is dropped. + RxNoRoom(Ring), + /// The jumbo ring asks for 64-byte descriptors, which are not modelled; + /// the ring is treated as unavailable. + JumboExtendedDescriptors, +} + +/// The chip's view of its own configuration header, which the bus owns: the +/// device holds it in its function state, and the state holds the device +/// weakly. +pub(super) struct Header(Weak); + +impl Header { + pub(super) fn new(device: Weak) -> Self { + Self(device) + } + + /// MISC_CTL as the guest and the bus have left it. + fn misc_ctl(&self) -> u32 { + self.0.upgrade().map_or(0, |dev| dev.config_header().read(cfg::MISC_CTL)) + } + + /// Clear `bits` of MISC_CTL, which the chip does itself. + fn clear_misc_ctl(&self, bits: u32) { + if let Some(dev) = self.0.upgrade() { + dev.config_header().write_masked(cfg::MISC_CTL, 0, bits); + } + } +} + +pub(super) struct State { + pub cfg: Header, + /// PCISTATE, which the chip drives: the bus mode, whether it saw + /// REQ64# at the end of reset, and whether the interrupt is down. + pub pcistate: u32, + /// Whether the slot drives REQ64# as reset ends. + req64: bool, + pub mac: [u8; 6], + pub(super) regfile: Box<[u32; REG_WORDS]>, + sram: Box<[u32; SRAM_WORDS]>, + /// The two indirect-window base registers (config 0x78 and 0x7c). + pub reg_win_base: u32, + pub mem_win_base: u32, + eeprom: Option, + bufmgr_start: Option, + /// The mask set through the 0x0204 mailbox, and the latch a status-block + /// update sets and the driver clears. The line is the latch with + /// neither mask (this or MISC_CTL's) set. + pub(super) irq_masked: bool, + pub(super) irq_pending: bool, + irq_level: bool, + pub irq_events: Vec, + pub phy: Phy, + pub now: Instant, + refused: BTreeMap, + pub(super) dma: Arc, + /// The send-ring consumer, and what the status block reports. + pub(super) tx_cons: u16, + pub(super) status_tx_cons: u16, + pub(super) status_rx_prod: u16, + /// Where the next return descriptor goes; it runs ahead of + /// `status_rx_prod` while the arrival line holds a completion. + pub(super) rx_prod_staged: u16, + pub(super) rx_std_cons: u16, + pub(super) rx_jumbo_cons: u16, + /// Frames discarded since a ring's shortage began, so a drought is + /// reported at both ends rather than once per frame. + pub(super) rx_shortage_drops: u64, + pub(super) rx_std_shortage_drops: u64, + pub(super) status_tag: u8, + /// Descriptors consumed on each ring since the last status-block update, + /// and whether each coalescing budget has run out. + pub(super) rx_coal_pending: u16, + pub(super) tx_coal_pending: u16, + pub(super) rx_coal_expired: bool, + pub(super) tx_coal_expired: bool, + pub(super) rx_coal_timer: Option, + pub(super) tx_coal_timer: Option, + /// Completions held until their frame's arrival deadline, with the + /// producer index each publishes. + pub(super) rx_latency: VecDeque<(Instant, u16)>, + pub(super) rx_floor: Duration, + /// Frames the transmit engine produced, for the owner to send. + pub wire: Vec>, + /// Set when something the guest did can unblock a waiting frame. + pub wake_rx: bool, + /// A reset discards the frames the backend still holds for the guest. + pub purge_rx: bool, +} + +impl State { + pub fn new( + cfg: Header, + req64: bool, + mac: [u8; 6], + eeprom: Option, + rx_floor: Duration, + now: Instant, + ) -> Self { + let mut st = Self { + cfg, + pcistate: Self::pcistate_at_reset(req64), + req64, + mac, + regfile: Box::new([0; REG_WORDS]), + sram: Box::new([0; SRAM_WORDS]), + reg_win_base: 0, + mem_win_base: 0, + eeprom, + bufmgr_start: None, + irq_masked: true, + irq_pending: false, + irq_level: false, + irq_events: Vec::new(), + phy: Phy::new(now), + now, + refused: BTreeMap::new(), + dma: Arc::new(super::dma::NullDma), + tx_cons: 0, + status_tx_cons: 0, + status_rx_prod: 0, + rx_prod_staged: 0, + rx_std_cons: 0, + rx_jumbo_cons: 0, + rx_shortage_drops: 0, + rx_std_shortage_drops: 0, + status_tag: 0, + rx_coal_pending: 0, + tx_coal_pending: 0, + rx_coal_expired: false, + tx_coal_expired: false, + rx_coal_timer: None, + tx_coal_timer: None, + rx_latency: VecDeque::new(), + rx_floor, + wire: Vec::new(), + wake_rx: false, + purge_rx: false, + }; + st.reset(); + st + } + + /// Enter at `now`: run whatever came due since the last entry. + pub fn enter(&mut self, now: Instant) { + if now > self.now { + self.now = now; + } + self.run_timers(); + } + + /// Run whatever came due, earliest first, each at its own deadline: the + /// virtual clock the model is ported from reaches a timer's expiry before + /// it fires, so a timer that arms another counts from there, not from + /// whenever this entry happened. + fn run_timers(&mut self) { + let target = self.now; + while let Some((deadline, timer)) = self.timers().into_iter().filter(|(d, _)| *d <= target).min() { + self.now = deadline; + match timer { + Timer::PhySettle => self.phy.sync(deadline), + Timer::BufferManager => { + self.bufmgr_start = None; + let mode = self.rf(reg::BUFMGR_MODE); + self.set_rf(reg::BUFMGR_MODE, mode | BUFMGR_MODE_ENABLE); + } + Timer::RxCoalescing => self.rx_coal_timer_fired(), + Timer::TxCoalescing => self.tx_coal_timer_fired(), + Timer::RxArrival => self.rx_arrival_due(), + } + } + self.now = target; + } + + fn timers(&self) -> Vec<(Instant, Timer)> { + [ + self.phy.deadline().map(|d| (d, Timer::PhySettle)), + self.bufmgr_start.map(|d| (d, Timer::BufferManager)), + self.rx_coal_timer.map(|d| (d, Timer::RxCoalescing)), + self.tx_coal_timer.map(|d| (d, Timer::TxCoalescing)), + self.rx_latency.front().map(|(d, _)| (*d, Timer::RxArrival)), + ] + .into_iter() + .flatten() + .collect() + } + + /// The earliest moment something here needs attention. + pub fn deadline(&self) -> Option { + self.timers().into_iter().map(|(d, _)| d).min() + } + + pub fn refused(&self) -> Vec<(Refused, u64)> { + self.refused.iter().map(|(what, count)| (*what, *count)).collect() + } + + pub fn eeprom_bytes(&self) -> Option> { + self.eeprom.as_ref().map(|e| e.bytes().to_vec()) + } + + pub fn refuse(&mut self, what: Refused) { + let count = self.refused.entry(what).or_insert(0); + *count += 1; + if *count == 1 { + crate::dlog!(crate::devlog::LogModule::Net, "bcm5701: unsupported or unexpected: {what:?}"); + } + } + + fn drain_phy_refusals(&mut self) { + for what in std::mem::take(&mut self.phy.refused) { + self.refuse(Refused::Phy(what)); + } + } + + // ---- storage ---- + + pub(super) fn rf(&self, off: u32) -> u32 { + self.regfile[((off & (REG_FILE_SIZE - 1)) >> 2) as usize] + } + + pub(super) fn set_rf(&mut self, off: u32, val: u32) { + self.regfile[((off & (REG_FILE_SIZE - 1)) >> 2) as usize] = val; + } + + pub fn sram_read(&self, off: u32) -> u32 { + self.sram[((off & (SRAM_SIZE - 1)) >> 2) as usize] + } + + pub fn sram_write(&mut self, off: u32, val: u32) { + let off = off & (SRAM_SIZE - 1); + let val = if off == SRAM_FW_MBOX { !val } else { val }; + self.sram[(off >> 2) as usize] = val; + } + + // ---- resets ---- + + /// The station address, as the on-chip bootcode owns it: every reset + /// puts it back. + fn load_station_address(&mut self) { + let m = self.mac; + let hi = u32::from(m[0]) << 8 | u32::from(m[1]); + let lo = u32::from_be_bytes([m[2], m[3], m[4], m[5]]); + self.sram_write(SRAM_MAC_HI, MAC_TAG << 16 | hi); + self.sram_write(SRAM_MAC_LO, lo); + self.set_rf(reg::MAC_ADDR_0_HIGH, hi); + self.set_rf(reg::MAC_ADDR_0_LOW, lo); + } + + /// The driver's write of MISC_CFG.RESET_CORE_CLOCKS, by either access + /// path: the register file and SRAM return to their reset values, the + /// handshake mailbox re-arms to the firmware-ready complement and the + /// bootcode's station address comes back. PCI configuration, the chip id + /// in MISC_CTL among it, and the indirect-window bases are untouched. + fn global_reset(&mut self) { + self.bufmgr_start = None; + self.regfile.fill(0); + self.sram.fill(0); + self.reset_engines(); + self.irq_masked = true; + self.irq_pending = false; + self.sram_write(SRAM_FW_MBOX, FW_MAGIC); + self.load_station_address(); + self.update_irq(false); + } + + /// A cold reset: as a global reset, plus the window bases, the PHY, and + /// no firmware-ready complement (only the driver-triggered reset arranges + /// that, because it is the one the driver writes the magic for). + pub fn reset(&mut self) { + self.bufmgr_start = None; + self.regfile.fill(0); + self.sram.fill(0); + self.reg_win_base = 0; + self.mem_win_base = 0; + self.pcistate = Self::pcistate_at_reset(self.req64); + self.reset_engines(); + self.irq_masked = true; + self.irq_pending = false; + self.phy.reset(self.now); + self.load_station_address(); + self.update_irq(true); + } + + // ---- interrupt ---- + + pub(super) fn misc_ctl(&self) -> u32 { + self.cfg.misc_ctl() + } + + /// PCISTATE after reset: conventional PCI, the interrupt line down, and + /// a 32-bit bus unless the slot drove REQ64#. + pub fn pcistate_at_reset(req64: bool) -> u32 { + pcistate::BUSMODE_PCI | pcistate::INTR_NOT_ACTIVE | if req64 { 0 } else { pcistate::BUS_32BIT } + } + + /// The interrupt is a level: INTA asserts while a status-block update + /// awaits the driver and neither the mailbox nor MISC_CTL.MASK_PCI_INTR + /// masks it. PCISTATE.INTR_NOT_ACTIVE mirrors the inverse, so the + /// driver's shared-interrupt check reads true exactly when the line is + /// down. `force` delivers the level even if unchanged. + pub fn update_irq(&mut self, force: bool) { + let active = self.irq_pending && !self.irq_masked && self.misc_ctl() & misc_ctl::MASK_PCI_INTR == 0; + if active { + self.pcistate &= !pcistate::INTR_NOT_ACTIVE; + } else { + self.pcistate |= pcistate::INTR_NOT_ACTIVE; + } + if force || active != self.irq_level { + self.irq_level = active; + self.irq_events.push(active); + } + } + + pub fn irq_level(&self) -> bool { + self.irq_level + } + + /// A write to MISC_CTL acknowledges if CLEAR_INTA was set, then re-derives + /// the line. This is how the driver's "set CLEAR_INTA, clear + /// MASK_PCI_INTR" enables the interrupt. + pub fn misc_ctl_written(&mut self) { + if self.misc_ctl() & misc_ctl::CLEAR_INTA != 0 { + self.irq_pending = false; + self.cfg.clear_misc_ctl(misc_ctl::CLEAR_INTA); + } + self.update_irq(false); + } + + // ---- EEPROM ---- + + fn eeprom_access(&mut self, command: u32) -> bool { + let device = ((command & eeprom_cmd::DEVICE_ID) >> 26) as u8; + let word = (command & eeprom_cmd::WORD_ADDR) >> 2; + let result = match self.eeprom.as_mut() { + None => { + self.refuse(Refused::EepromFault(eeprom::Fault::NoBacking)); + return false; + } + Some(e) if command & eeprom_cmd::READ != 0 => e.read_word(device, word).map(Some), + Some(e) => { + let value = self.regfile[(reg::EEPROM_DATA >> 2) as usize]; + e.write_word(device, word, value).map(|()| None) + } + }; + match result { + Ok(Some(value)) => { + self.set_rf(reg::EEPROM_DATA, value); + true + } + Ok(None) => true, + Err(fault) => { + self.refuse(Refused::EepromFault(fault)); + false + } + } + } + + fn eeprom_command_write(&mut self, val: u32) { + let old = self.rf(reg::EEPROM_ADDR); + let mut command = val & !(eeprom_cmd::START | eeprom_cmd::COMPLETE | eeprom_cmd::RESET | 3); + let acknowledge = val & eeprom_cmd::COMPLETE != 0; + if val & eeprom_cmd::RESET != 0 { + self.set_rf(reg::EEPROM_ADDR, eeprom_cmd::RESET); + self.set_rf(reg::EEPROM_DATA, 0); + return; + } + if !acknowledge { + command |= old & eeprom_cmd::COMPLETE; + } + self.set_rf(reg::EEPROM_ADDR, command); + // A pending Complete blocks the next transfer until it is + // acknowledged; acknowledging and starting in one write is allowed. + if val & eeprom_cmd::START == 0 || (old & eeprom_cmd::COMPLETE != 0 && !acknowledge) { + return; + } + if self.eeprom_access(command) { + self.set_rf(reg::EEPROM_ADDR, command | eeprom_cmd::COMPLETE); + } + } + + // ---- register file ---- + + /// Register-file writes, reached identically from the BAR and from the + /// indirect register window, so a side effect fires whichever path the + /// driver uses. The 5700/5701 issue the global reset through the window. + pub fn reg_write(&mut self, off: u32, val: u32) { + let off = off & (REG_FILE_SIZE - 1); + let mut val = val; + match off { + reg::MISC_CFG => { + if val & MISCCFG_RESET_CORE_CLOCKS != 0 { + self.global_reset(); + return; + } + } + reg::EEPROM_ADDR => { + self.eeprom_command_write(val); + return; + } + reg::EEPROM_DATA | reg::EEPROM_DELAY => {} + reg::EEPROM_PIN => { + self.refuse(Refused::EepromPins); + self.set_rf(off, val); + return; + } + // The reset bit self-clears; the rest of the write persists. + reg::MAC_MODE => val &= !MODE_RESET, + reg::RX_MODE => { + // A halt takes effect at once. The chip finishes the frame in + // flight first and reads back ENABLE until it has, which a + // model that takes one frame at a time never needs. + val &= !MODE_RESET; + self.set_rf(off, val); + if val & RXMODE_ENABLE != 0 { + self.retry_rx_flush(); + } + return; + } + // Write one to clear: the bits written are the ones acknowledged. + reg::MAC_STATUS => { + let old = self.rf(off); + self.set_rf(off, old & !val); + return; + } + reg::HCC_MODE => { + val &= !MODE_RESET; + self.set_rf(off, val); + self.hcc_mode_written(); + return; + } + reg::HCC_RX_COAL_TICKS | reg::HCC_TX_COAL_TICKS => { + self.set_rf(off, val); + self.coal_ticks_written(off == reg::HCC_RX_COAL_TICKS); + return; + } + reg::MBX_TX_PROD_LO => { + // Store the producer for read-back, then send up to it. + self.set_rf(off, val); + self.tx_kick(val); + return; + } + reg::RX_JUMBO_RCB_MAXLEN_FLAGS => { + // Max_Len is unused for the jumbo ring; only Disable Ring and + // Extended RX Enable do anything, and both are read from the + // stored word. Flag the unmodelled extended format once, here, + // rather than on every frame that finds the ring unavailable. + if val & RCB_FLAG_EXT_RX_ENABLE != 0 { + self.refuse(Refused::JumboExtendedDescriptors); + } + } + reg::MBX_RX_JUMBO_PROD_LO | reg::MBX_RX_STD_PROD_LO | reg::MBX_RX_CONS0_LO => { + self.rx_mailbox_written(off, val); + return; + } + reg::BUFMGR_MODE => { + if val & MODE_RESET != 0 { + self.bufmgr_start = None; + self.set_rf(off, 0); + } + val &= !MODE_RESET; + if val & BUFMGR_MODE_ENABLE == 0 { + self.bufmgr_start = None; + } else if self.rf(off) & BUFMGR_MODE_ENABLE == 0 { + if self.bufmgr_start.is_none() { + self.bufmgr_start = Some(self.now + BUFMGR_START); + } + val &= !BUFMGR_MODE_ENABLE; + } + } + reg::BUFMGR_STATUS => { + self.refuse(Refused::BufmgrStatusWrite); + return; + } + reg::MBX_IRQ0_LO => { + // Every write clears the pending update; a nonzero value + // masks the line and zero re-enables it. + self.irq_pending = false; + self.irq_masked = val != 0; + self.set_rf(off, val); + if val == 0 { + self.retry_rx_flush(); + } + self.update_irq(false); + return; + } + reg::MI_COMM => { + self.mi_comm_write(val); + return; + } + _ => { + self.refuse(Refused::UnmodelledWrite(off)); + self.set_rf(off, val); + return; + } + } + self.set_rf(off, val); + } + + fn mi_comm_write(&mut self, command: u32) { + let register = ((command & mi::REG) >> mi::REG_SHIFT) as u8; + if command & mi::BUSY == 0 { + self.set_rf(reg::MI_COMM, command); + return; + } + let now = self.now; + let opcode = command & (mi::READ | mi::WRITE); + let val = if opcode == mi::READ { + (command & mi::READ_FIELDS) | u32::from(self.phy.read(register, now)) + } else if opcode == mi::WRITE { + self.phy.write(register, (command & mi::DATA) as u16, now); + command & !mi::BUSY + } else { + self.refuse(Refused::MiOpcode(opcode >> 26)); + command & !mi::BUSY + }; + self.set_rf(reg::MI_COMM, val); + self.drain_phy_refusals(); + } + + /// Register-file reads, reached identically from the BAR and the window. + /// Most read back their stored value; MI_STS is the PHY's live link so + /// the driver's auto-poll path sees the carrier. + pub fn reg_read(&mut self, off: u32) -> u32 { + match off & (REG_FILE_SIZE - 1) { + reg::MI_STS => { + self.phy.sync(self.now); + if self.phy.link_up() { + mi::LINK + } else { + 0 + } + } + reg::FW_PROGRESS => FW_PROGRESS_INIT, + _ => self.rf(off), + } + } +} diff --git a/src/dev/bcm5701/tests.rs b/src/dev/bcm5701/tests.rs new file mode 100644 index 000000000..47857c483 --- /dev/null +++ b/src/dev/bcm5701/tests.rs @@ -0,0 +1,2444 @@ +//! Register-contract tests for the BCM5701, ported from the QEMU model's +//! qtests. The device is built alone, with a controllable clock and an +//! interrupt probe standing in for the rest of the machine. +//! +//! Ported from tests/qtest/bcm570x-test.c in jamesbraid/qemu-sgi-origami. + +use super::eeprom::{Eeprom, Fault}; +use super::phy::{reg as phy_reg, IDR1_VALUE, IDR2_BCM5701}; +use super::state::{reg, Refused, FW_MAGIC, SRAM_FW_MBOX}; +use super::*; +use crate::irq::LevelProbe; +use crate::pci::{PciDma, PciDmaStatus}; +use std::time::Duration; + +const SRAM_WINDOW: u32 = 0x8000; +const MISC_CFG_RESET: u32 = 1; +const MODE_RESET: u32 = 1; +const RXMODE_ENABLE: u32 = 2; +const HCC_ENABLE: u32 = 2; +const BUFMGR_ENABLE: u32 = 2; +const BUFMGR_ATTN: u32 = 4; +const BUFMGR_START: Duration = Duration::from_millis(8); +const PHY_SETTLE: Duration = Duration::from_millis(100); +const MICOMM_BUSY: u32 = 0x2000_0000; +const MICMD_READ: u32 = 0x0800_0000; +const MICMD_WRITE: u32 = 0x0400_0000; +const MICOMM_READ_FIELDS: u32 = 0x0fff_0000; +const PHY_ADDR: u32 = 1; +const MAC: [u8; 6] = [0x52, 0x54, 0x00, 0xab, 0xcd, 0xef]; + +const EEPROM_READ: u32 = 1 << 31; +const EEPROM_COMPLETE: u32 = 1 << 30; +const EEPROM_RESET: u32 = 1 << 29; +const EEPROM_START: u32 = 1 << 25; + +const MEM_SIZE: usize = 0x90_0000; +const TX_RING: u64 = 0x20_0000; +const TX_BUF: u64 = 0x21_0000; +const STATUS: u64 = 0x22_0000; +const STD_RING: u64 = 0x23_0000; +const RET_RING: u64 = 0x24_0000; +const RX_BUF: u64 = 0x25_0000; +const JUMBO_RING: u64 = 0x26_0000; +const JUMBO_BUF: u64 = 0x27_0000; +const STD_STALE_BUF: u64 = 0x28_0000; +/// Past the memory: nothing backs it. +const UNMAPPED: u64 = 0x400_0000; +const TX_RING_LEN: u32 = 4; +const RX_RING_LEN: u32 = 4; +const STD_BUF_SIZE: u32 = 1536; +const JUMBO_BUF_SIZE: u32 = 9216; +const STATUS_BLK_SIZE: usize = 0x50; +const ARRIVAL: Duration = Duration::from_micros(200); +/// Every DMA swap control set: the chip's image goes on the bus as it is. +const MODE_ALL_SWAPS: u32 = 0x36; + +/// Bus-master memory with no bridge in front of it. +struct Mem(std::sync::Mutex>); + +impl PciDma for Mem { + fn dma_read(&self, addr: u64, buf: &mut [u8]) -> PciDmaStatus { + let m = self.0.lock().unwrap(); + let a = addr as usize; + if a.saturating_add(buf.len()) > m.len() { + buf.fill(0); + return PciDmaStatus::MasterAbort; + } + buf.copy_from_slice(&m[a..a + buf.len()]); + PciDmaStatus::Ok + } + fn dma_write(&self, addr: u64, data: &[u8]) -> PciDmaStatus { + let mut m = self.0.lock().unwrap(); + let a = addr as usize; + if a.saturating_add(data.len()) > m.len() { + return PciDmaStatus::MasterAbort; + } + m[a..a + data.len()].copy_from_slice(data); + PciDmaStatus::Ok + } +} + +struct Rig { + dev: Arc, + irq: Arc, + mem: Arc, + /// Frames the guest sent, and a way to hand frames to the guest. + sent: rtrb::Consumer>, + inject: rtrb::Producer>, + ring_slots: usize, +} + +impl Rig { + fn new() -> Self { + Self::with(Bcm5701Config::new(MAC)) + } + + fn with(config: Bcm5701Config) -> Self { + let dev = Bcm5701::new(config).unwrap(); + *dev.frozen.lock() = Some(Instant::now()); + let (irq, line) = LevelProbe::line(); + let mem = Arc::new(Mem(std::sync::Mutex::new(vec![0; MEM_SIZE]))); + dev.state().connect_irq(line); + dev.state().connect_dma(mem.clone()); + dev.attached(); + let (link, sent, inject) = crate::net::Link::loopback(); + dev.attach(link); + dev.reset(); + let ring_slots = inject.slots(); + Self { dev, irq, mem, sent, inject, ring_slots } + } + + /// Advance the clock and run whatever came due. + fn step(&self, by: Duration) { + { + let mut t = self.dev.frozen.lock(); + *t = Some(t.unwrap() + by); + } + self.dev.enter(self.dev.now(), |_| ()); + } + + fn mem_write(&self, addr: u64, data: &[u8]) { + let mut m = self.mem.0.lock().unwrap(); + m[addr as usize..addr as usize + data.len()].copy_from_slice(data); + } + fn mem_read(&self, addr: u64, len: usize) -> Vec { + self.mem.0.lock().unwrap()[addr as usize..addr as usize + len].to_vec() + } + fn mem_fill(&self, addr: u64, len: usize, byte: u8) { + self.mem.0.lock().unwrap()[addr as usize..addr as usize + len].fill(byte); + } + fn status_block(&self) -> Vec { + self.mem_read(STATUS, STATUS_BLK_SIZE) + } + fn status_tag(&self) -> u32 { + be32(&self.status_block(), 4) + } + fn status_tx_cons(&self) -> u16 { + get16dma(&self.status_block(), 18) + } + fn status_rx_prod(&self) -> u16 { + get16dma(&self.status_block(), 16) + } + fn line_up(&self) -> bool { + self.cfg_read(0x70 / 4) & 2 == 0 + } + fn nic_no_more_rxbds(&self) -> u32 { + self.readl(SRAM_WINDOW + 0x898) + } + /// The frames the guest has sent so far. + fn sent_frames(&mut self) -> Vec> { + std::iter::from_fn(|| self.sent.pop().ok()).collect() + } + /// Frames handed to the guest that it has not yet taken. + fn waiting(&self) -> usize { + self.ring_slots - self.inject.slots() + } + /// Offer a frame to the receive path, as the backend would. + fn deliver(&mut self, frame: &[u8]) { + self.inject.push(frame.to_vec()).unwrap(); + self.dev.poll(); + } + + fn readl(&self, off: u32) -> u32 { + self.dev.bar_read(0, off, 4) + } + fn readb(&self, off: u32) -> u32 { + self.dev.bar_read(0, off, 1) + } + fn writel(&self, off: u32, val: u32) { + self.dev.bar_write(0, off, val, 4); + } + fn cfg_read(&self, reg: usize) -> u32 { + self.dev.config_dword(reg) + } + /// A configuration write as the bus makes it: `width` bytes at byte + /// `offset`. + fn cfg_write_sized(&self, offset: usize, width: usize, val: u32) { + let (reg, shift) = (offset / 4, (offset & 3) * 8); + if width == 4 && self.dev.config_write(reg, val) { + return; + } + let config = self.dev.state().config(); + let old = config.read(reg); + config.write_masked(reg, val << shift, (u32::MAX >> (32 - 8 * width)) << shift); + let new = config.read(reg); + if new != old { + self.dev.config_written(reg, old, new); + } + } + fn cfg_write(&self, reg: usize, val: u32) { + self.cfg_write_sized(reg * 4, 4, val); + } + /// A configuration read of `width` bytes at byte `offset`. + fn cfg_read_sized(&self, offset: usize, width: usize) -> u32 { + (self.cfg_read(offset / 4) >> ((offset & 3) * 8)) & (u32::MAX >> (32 - 8 * width)) + } + + /// The indirect register window, config 0x78 and 0x80. + fn ind_reg_write(&self, off: u32, val: u32) { + self.cfg_write(0x78 / 4, off); + self.cfg_write(0x80 / 4, val); + } + fn ind_reg_read(&self, off: u32) -> u32 { + self.cfg_write(0x78 / 4, off); + self.cfg_read(0x80 / 4) + } + /// The indirect memory window, config 0x7c and 0x84. + fn ind_mem_write(&self, off: u32, val: u32) { + self.cfg_write(0x7c / 4, off); + self.cfg_write(0x84 / 4, val); + } + fn ind_mem_read(&self, off: u32) -> u32 { + self.cfg_write(0x7c / 4, off); + self.cfg_read(0x84 / 4) + } + + fn mii_read(&self, phy: u32, register: u8) -> u16 { + self.writel(reg::MI_COMM, MICOMM_BUSY | MICMD_READ | phy << 21 | u32::from(register) << 16); + self.readl(reg::MI_COMM) as u16 + } + fn mii_write(&self, phy: u32, register: u8, val: u16) { + self.writel( + reg::MI_COMM, + MICOMM_BUSY | MICMD_WRITE | phy << 21 | u32::from(register) << 16 | u32::from(val), + ); + } + fn sram_word(&self, off: u32) -> u32 { + self.dev.st.lock().sram_read(off) + } + fn split_addr(&self, hi_reg: u32, lo_reg: u32, addr: u64) { + self.writel(hi_reg, (addr >> 32) as u32); + self.writel(lo_reg, addr as u32); + } + fn refusals(&self, what: Refused) -> u64 { + self.dev.refused().iter().find(|(r, _)| *r == what).map_or(0, |(_, n)| *n) + } +} + +fn temp_file(name: &str, contents: &[u8]) -> std::path::PathBuf { + let path = std::env::temp_dir().join(format!("iris-bcm5701-{}-{name}", std::process::id())); + std::fs::write(&path, contents).unwrap(); + path +} + +/// Dword and sub-dword reads of the identity, so a hidden swap in one half +/// cannot hide behind the other. Ported from test_pci_four_tuple. +#[test] +fn the_function_presents_the_sgi_board_identity() { + let rig = Rig::new(); + assert_eq!(rig.cfg_read(0), 0x1645_14e4); + assert_eq!(rig.cfg_read(0x2c / 4), 0x8010_10a9); + assert_eq!(rig.cfg_read(0x08 / 4) >> 8, 0x02_0000); + assert_eq!(rig.cfg_read(0x08 / 4) & 0xff, 0); + assert_eq!(rig.dev.state().config().interrupt_pin(), PCI_INTA); +} + +/// The chip id in MISC_CTL decodes to the BCM5701, ASIC revision 0, and is +/// read-only while the low half sticks. Ported from test_misc_ctl_asic_revision. +#[test] +fn misc_ctl_presents_the_chip_id() { + let rig = Rig::new(); + let misc = rig.cfg_read(0x68 / 4); + assert_eq!(misc >> 16, 0x0105); + assert_eq!((misc >> 16) >> 12, 0); + rig.cfg_write(0x68 / 4, 0xffff_0004); + assert_eq!(rig.cfg_read(0x68 / 4), 0x0105_0004); +} + +/// Ported from test_indirect_register_window: the window aliases the file. +#[test] +fn the_indirect_register_window_aliases_the_register_file() { + let rig = Rig::new(); + rig.ind_reg_write(0x0410, 0x1234_5678); + assert_eq!(rig.readl(0x0410), 0x1234_5678); + assert_eq!(rig.ind_reg_read(0x0410), 0x1234_5678); + assert_eq!(rig.cfg_read(0x78 / 4), 0x0410); +} + +/// Only whole dwords reach the windows. A narrower configuration read sees +/// the header's own bytes, which stay zero because nothing writes them, as +/// in the model this is ported from, where the window state sits outside +/// the header and the header bytes are read-only. +#[test] +fn narrow_configuration_accesses_bypass_the_indirect_windows() { + let rig = Rig::new(); + rig.ind_reg_write(0x0410, 0x1234_5678); + rig.ind_mem_write(0x0c14, 0xdead_beef); + let config = rig.dev.state().config(); + for offset in 0x78..0x88 { + assert_eq!(config.read(offset / 4) >> ((offset & 3) * 8) & 0xff, 0, "byte at {offset:#x}"); + } + for offset in (0x78..0x88).step_by(2) { + assert_eq!(config.read(offset / 4) >> ((offset & 3) * 8) & 0xffff, 0, "halfword at {offset:#x}"); + } + rig.cfg_write_sized(0x80, 1, 0xff); + rig.cfg_write_sized(0x7e, 2, 0xffff); + assert_eq!(config.read(0x80 / 4) & 0xff, 0); + assert_eq!(config.read(0x7c / 4) >> 16, 0); + assert_eq!(rig.cfg_read_sized(0x78, 4), 0x0410); + assert_eq!(rig.cfg_read_sized(0x80, 4), 0x1234_5678); + assert_eq!(rig.cfg_read_sized(0x7c, 4), 0x0c14); + assert_eq!(rig.cfg_read_sized(0x84, 4), 0xdead_beef); +} + +/// Ported from test_indirect_memory_window: SRAM through config and through +/// the BAR window at 0x8000. +#[test] +fn the_indirect_memory_window_aliases_sram() { + let rig = Rig::new(); + rig.ind_mem_write(0x0c14, 0xdead_beef); + assert_eq!(rig.ind_mem_read(0x0c14), 0xdead_beef); + assert_eq!(rig.readl(SRAM_WINDOW + (0x0c14 & 0x7fff)), 0xdead_beef); + // The base register is masked to a 32 KiB boundary for the BAR window, + // and the 32 KiB SRAM has one position. + rig.cfg_write(0x7c / 4, 0x0c14); + assert_eq!(rig.readl(SRAM_WINDOW), rig.sram_word(0)); +} + +/// The magic written before the reset (the bge order) reads back as the +/// complement, and the reset self-clears its trigger and re-arms the mailbox. +/// Ported from test_reset_handshake_via_bar. +#[test] +fn the_reset_handshake_completes_when_the_magic_precedes_the_reset() { + let rig = Rig::new(); + rig.ind_mem_write(SRAM_FW_MBOX, FW_MAGIC); + assert_eq!(rig.ind_mem_read(SRAM_FW_MBOX), !FW_MAGIC); + rig.writel(reg::MISC_CFG, MISC_CFG_RESET); + assert_eq!(rig.readl(reg::MISC_CFG) & MISC_CFG_RESET, 0); + assert_eq!(rig.ind_mem_read(SRAM_FW_MBOX), !FW_MAGIC); +} + +/// The IRIX order: reset first, then the magic. Ported from +/// test_reset_handshake_magic_after_reset. +#[test] +fn the_reset_handshake_completes_when_the_magic_follows_the_reset() { + let rig = Rig::new(); + rig.writel(reg::MISC_CFG, MISC_CFG_RESET); + rig.ind_mem_write(SRAM_FW_MBOX, FW_MAGIC); + assert_eq!(rig.ind_mem_read(SRAM_FW_MBOX), 0xb49a_89ab); +} + +/// Ported from test_reset_handshake_via_indirect: the 5700/5701 reset through +/// the configuration window alone. +#[test] +fn the_reset_handshake_completes_through_the_indirect_windows() { + let rig = Rig::new(); + rig.ind_mem_write(SRAM_FW_MBOX, FW_MAGIC); + rig.ind_reg_write(reg::MISC_CFG, MISC_CFG_RESET); + assert_eq!(rig.ind_mem_read(SRAM_FW_MBOX), 0xb49a_89ab); +} + +/// Ported from test_firmware_progress_register: the all-ones "no progress +/// yet" sentinel survives a reset. +#[test] +fn the_firmware_progress_register_holds_its_sentinel() { + let rig = Rig::new(); + assert_eq!(rig.readl(reg::FW_PROGRESS), 0xffff_ffff); + assert_eq!(rig.ind_reg_read(reg::FW_PROGRESS), 0xffff_ffff); + rig.writel(reg::MISC_CFG, MISC_CFG_RESET); + assert_eq!(rig.readl(reg::FW_PROGRESS), 0xffff_ffff); +} + +/// A global reset keeps the PCI header, the chip id among it, and the +/// window bases, and restores the bootcode's station address. Ported from +/// test_mac_address_registers. +#[test] +fn a_global_reset_restores_the_bootcode_state_only() { + let rig = Rig::new(); + rig.cfg_write(0x68 / 4, 0x0000_0004); + rig.cfg_write(0x7c / 4, 0x1234); + rig.writel(reg::MAC_ADDR_0_HIGH, 0xa1a2); + rig.writel(reg::MISC_CFG, MISC_CFG_RESET); + assert_eq!(rig.cfg_read(0x68 / 4), 0x0105_0004); + assert_eq!(rig.cfg_read(0x7c / 4), 0x1234); + assert_eq!(rig.readl(reg::MAC_ADDR_0_HIGH), 0x5254); +} + +/// Ported from test_self_clearing_reset_bits. +#[test] +fn mode_register_reset_bits_self_clear_and_the_rest_persists() { + let rig = Rig::new(); + rig.writel(reg::MAC_MODE, MODE_RESET | 0x4); + assert_eq!(rig.readl(reg::MAC_MODE) & MODE_RESET, 0); + assert_eq!(rig.readl(reg::MAC_MODE) & 4, 4); + + rig.writel(reg::HCC_MODE, MODE_RESET); + assert_eq!(rig.readl(reg::HCC_MODE) & MODE_RESET, 0); + + rig.writel(reg::RX_MODE, MODE_RESET | RXMODE_ENABLE); + assert_eq!(rig.readl(reg::RX_MODE) & MODE_RESET, 0); + assert_eq!(rig.readl(reg::RX_MODE) & RXMODE_ENABLE, RXMODE_ENABLE); + // The IRIX sequence: reset on its own leaves the receiver halted. + rig.writel(reg::RX_MODE, MODE_RESET); + assert_eq!(rig.readl(reg::RX_MODE), 0); +} + +/// The driver's idle sequence writes ENABLE, then 0, and polls for the whole +/// register to read zero. Ported from test_hcc_mode_idle_poll and +/// test_hcc_mode_clears_to_zero. +#[test] +fn host_coalescing_mode_stores_and_clears() { + let rig = Rig::new(); + rig.writel(reg::HCC_MODE, HCC_ENABLE); + rig.writel(reg::RX_MODE, RXMODE_ENABLE); + assert_eq!(rig.readl(reg::HCC_MODE) & HCC_ENABLE, HCC_ENABLE); + rig.writel(reg::HCC_MODE, 0); + assert_eq!(rig.readl(reg::HCC_MODE) & HCC_ENABLE, 0); + + rig.writel(reg::HCC_MODE, 0x13e); + rig.writel(reg::HCC_MODE, 0); + assert_eq!(rig.readl(reg::HCC_MODE), 0); +} + +/// ENABLE reads back only after the buffer manager's startup, by either +/// access path. Ported from test_bufmgr_registers. +#[test] +fn the_buffer_manager_enable_reads_back_after_its_startup() { + for indirect in [false, true] { + let rig = Rig::new(); + let read = |off| if indirect { rig.ind_reg_read(off) } else { rig.readl(off) }; + let write = |off, val| if indirect { rig.ind_reg_write(off, val) } else { rig.writel(off, val) }; + let on = BUFMGR_ENABLE | BUFMGR_ATTN; + + write(reg::BUFMGR_MODE, BUFMGR_ATTN); + write(reg::BUFMGR_MODE, on); + assert_eq!(read(reg::BUFMGR_MODE), BUFMGR_ATTN); + rig.step(Duration::from_millis(7)); + assert_eq!(read(reg::BUFMGR_MODE), BUFMGR_ATTN); + // A repeated request does not restart the interval. + write(reg::BUFMGR_MODE, on); + rig.step(BUFMGR_START - Duration::from_millis(7) - Duration::from_nanos(1)); + assert_eq!(read(reg::BUFMGR_MODE), BUFMGR_ATTN); + rig.step(Duration::from_nanos(1)); + assert_eq!(read(reg::BUFMGR_MODE), on); + + // Reset cancels startup; the new request starts a fresh interval. + write(reg::BUFMGR_MODE, MODE_RESET | on); + assert_eq!(read(reg::BUFMGR_MODE), BUFMGR_ATTN); + rig.step(BUFMGR_START); + assert_eq!(read(reg::BUFMGR_MODE), on); + + // Halting is immediate and cancels a pending start. + write(reg::BUFMGR_MODE, BUFMGR_ATTN); + assert_eq!(read(reg::BUFMGR_MODE), BUFMGR_ATTN); + write(reg::BUFMGR_MODE, on); + write(reg::BUFMGR_MODE, BUFMGR_ATTN); + rig.step(BUFMGR_START); + assert_eq!(read(reg::BUFMGR_MODE), BUFMGR_ATTN); + + // The status register is read-only, and the refusal is counted. + write(reg::BUFMGR_STATUS, on); + assert_eq!(read(reg::BUFMGR_STATUS), 0); + assert!(rig.refusals(Refused::BufmgrStatusWrite) >= 1); + + // A global reset cancels a pending startup. + rig.writel(reg::BUFMGR_MODE, on); + rig.step(BUFMGR_START / 2); + rig.writel(reg::MISC_CFG, MISC_CFG_RESET); + rig.step(BUFMGR_START); + assert_eq!(rig.readl(reg::BUFMGR_MODE), 0); + } +} + +/// MAC_STATUS bits clear on a write of one, and the write is never readable +/// back as state. Ported from test_mac_status_write_one_clears. +#[test] +fn mac_status_clears_on_write_one() { + let rig = Rig::new(); + assert_eq!(rig.readl(reg::MAC_STATUS), 0); + rig.writel(reg::MAC_STATUS, 0x50); + assert_eq!(rig.readl(reg::MAC_STATUS), 0); + rig.writel(reg::MAC_STATUS, 0xffff_ffff); + assert_eq!(rig.readl(reg::MAC_STATUS), 0); +} + +/// MI_COMM completes at once; a successful read keeps the command and address +/// fields, replaces the data and clears Start and ReadFailed, whatever +/// reserved bits the request carried; a write without Start only stores; an +/// undefined opcode touches no PHY register and is counted. Ported from +/// test_mi_busy_clears. +#[test] +fn mi_comm_completes_reads_and_writes_by_either_path() { + for indirect in [false, true] { + let rig = Rig::new(); + let read = |off| if indirect { rig.ind_reg_read(off) } else { rig.readl(off) }; + let write = |off, val| if indirect { rig.ind_reg_write(off, val) } else { rig.writel(off, val) }; + let anar = |rig: &Rig| { + let command = MICOMM_BUSY | MICMD_READ | PHY_ADDR << 21 | 4 << 16; + if indirect { + rig.ind_reg_write(reg::MI_COMM, command); + rig.ind_reg_read(reg::MI_COMM) + } else { + rig.writel(reg::MI_COMM, command); + rig.readl(reg::MI_COMM) + } + }; + + // The discriminating full-word case: a PHY ID read keeps its fields. + for (command, result) in [(0x2822_ffff, 0x0822_0020), (0xf822_ffff, 0x0822_0020)] { + write(reg::MI_COMM, command); + assert_eq!(read(reg::MI_COMM), result); + } + + let comm = anar(&rig); + assert_eq!(comm & MICOMM_BUSY, 0); + let baseline = comm & 0xffff; + assert_eq!(comm, (MICOMM_BUSY | MICMD_READ | PHY_ADDR << 21 | 4 << 16) & MICOMM_READ_FIELDS | baseline); + + // The 10baseT half-duplex advertisement toggles. + let value = baseline ^ 0x20; + let write_command = MICMD_WRITE | PHY_ADDR << 21 | 4 << 16 | value; + write(reg::MI_COMM, write_command); + assert_eq!(read(reg::MI_COMM), write_command); + write(reg::MI_COMM, write_command); + assert_eq!(read(reg::MI_COMM), write_command); + + // Without Start nothing reaches the PHY. + assert_eq!(anar(&rig) & 0xffff, baseline); + + let id_command = MICMD_READ | PHY_ADDR << 21 | 2 << 16; + write(reg::MI_COMM, id_command); + assert_eq!(read(reg::MI_COMM), id_command); + write(reg::MI_COMM, id_command | MICOMM_BUSY); + assert_eq!(read(reg::MI_COMM), (id_command & MICOMM_READ_FIELDS) | u32::from(IDR1_VALUE)); + + write(reg::MI_COMM, write_command | MICOMM_BUSY); + assert_eq!(read(reg::MI_COMM) & MICOMM_BUSY, 0); + assert_eq!(anar(&rig) & 0xffff, value); + + // Undefined opcodes complete without a PHY access and are counted. + for opcode in [0, MICMD_READ | MICMD_WRITE] { + let command = MICOMM_BUSY | opcode | PHY_ADDR << 21 | 4 << 16 | 0x5a5a; + write(reg::MI_COMM, command); + assert_eq!(read(reg::MI_COMM), command & !MICOMM_BUSY); + assert_eq!(anar(&rig) & 0xffff, value); + } + assert_eq!(rig.refusals(Refused::MiOpcode(0)), 1); + assert_eq!(rig.refusals(Refused::MiOpcode(3)), 1); + } +} + +/// Ported from test_phy_readback: identity, a link that is down while +/// negotiating, a settle that must elapse, and MI_STS following the carrier. +#[test] +fn the_phy_negotiates_and_the_link_follows_the_settle() { + let rig = Rig::new(); + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::IDR1), IDR1_VALUE); + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::IDR2), IDR2_BCM5701); + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::BMSR) & 4, 0); + assert_eq!(rig.readl(reg::MI_STS) & 1, 0); + + // Restarting autonegotiation negotiates again. + rig.mii_write(PHY_ADDR, phy_reg::BMCR, 0x1000 | 0x0200); + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::BMSR) & 4, 0); + + rig.step(PHY_SETTLE + Duration::from_micros(1)); + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::BMSR) & 4, 4); + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::BMSR) & 0x20, 0x20); + assert_eq!(rig.readl(reg::MI_STS) & 1, 1); + + // A cable pulled drops the carrier without touching negotiation. + rig.dev.set_backend_link(false); + assert_eq!(rig.readl(reg::MI_STS) & 1, 0); + rig.dev.set_backend_link(true); + assert_eq!(rig.readl(reg::MI_STS) & 1, 1); +} + +/// Ported from test_phy_aux_status_link_up: never all-ones, ACOMP and the +/// 1000baseT full-duplex resolution once settled. +#[test] +fn the_phy_auxiliary_status_gates_the_link_up_path() { + let rig = Rig::new(); + let aux = rig.mii_read(PHY_ADDR, phy_reg::AUXSTS); + assert_eq!(aux & 0x8000, 0); + assert_ne!(aux, 0xffff); + rig.mii_write(PHY_ADDR, phy_reg::BMCR, 0x1000 | 0x0200); + rig.step(PHY_SETTLE + Duration::from_micros(1)); + let aux = rig.mii_read(PHY_ADDR, phy_reg::AUXSTS); + assert_eq!(aux & 0x8000, 0x8000); + assert_eq!(aux & 0x0700, 0x0700); + assert_ne!(aux, 0xffff); + // The partner advertises what this end does, and gigabit full duplex. + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::GTSR) & 0x0800, 0x0800); + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::ANLPAR), rig.mii_read(PHY_ADDR, phy_reg::ANAR) | 0x4000); +} + +/// A PHY register the model lacks reads zero, never the all-ones an absent +/// PHY returns, and the access is counted. +#[test] +fn unimplemented_phy_registers_read_zero_and_are_counted() { + let rig = Rig::new(); + assert_eq!(rig.mii_read(PHY_ADDR, 0x1c), 0); + rig.mii_write(PHY_ADDR, 0x1c, 0x1234); + rig.mii_write(PHY_ADDR, phy_reg::IDR1, 0); + let refused = rig.dev.refused(); + assert!(refused.contains(&(Refused::Phy(phy::Refusal::ReadUnimplemented(0x1c)), 1))); + assert!(refused.contains(&(Refused::Phy(phy::Refusal::WriteUnimplemented(0x1c)), 1))); + assert!(refused.contains(&(Refused::Phy(phy::Refusal::WriteReadOnly(phy_reg::IDR1)), 1))); +} + +/// The station address lands in the SRAM mailbox the driver reads first. +/// Ported from test_mac_address_mailbox. +#[test] +fn the_station_address_is_in_the_sram_mailbox() { + let rig = Rig::new(); + assert_eq!(rig.ind_mem_read(0x0c14), 0x484b_5254); + assert_eq!(rig.ind_mem_read(0x0c18), 0x00ab_cdef); +} + +/// The station address is also in filter slot 0, readable a byte at a time +/// from that pair alone. Ported from test_mac_address_registers. +#[test] +fn the_station_address_is_in_filter_slot_zero_with_narrow_reads_there_only() { + let rig = Rig::new(); + assert_eq!(rig.readl(reg::MAC_ADDR_0_HIGH), 0x0000_5254); + assert_eq!(rig.readl(reg::MAC_ADDR_0_LOW), 0x00ab_cdef); + let bytes: Vec = (0..8).map(|i| rig.readb(reg::MAC_ADDR_0_HIGH + i)).collect(); + assert_eq!(bytes, [0x54, 0x52, 0, 0, 0xef, 0xcd, 0xab, 0]); + + rig.writel(reg::MAC_ADDR_0_HIGH - 4, 0x5566_7788); + rig.writel(reg::MAC_ADDR_0_LOW + 4, 0x1122_3344); + // Adjacent byte reads and halfword accesses are rejected and counted. + assert_eq!(rig.readb(reg::MAC_ADDR_0_HIGH - 1), 0); + assert_eq!(rig.readb(reg::MAC_ADDR_0_LOW + 4), 0); + assert_eq!(rig.dev.bar_read(0, reg::MAC_ADDR_0_HIGH, 2), 0); + // Narrow writes are not part of the byte-read contract. + rig.dev.bar_write(0, reg::MAC_ADDR_0_HIGH, 0, 1); + rig.dev.bar_write(0, reg::MAC_ADDR_0_LOW, 0, 2); + assert_eq!(rig.readl(reg::MAC_ADDR_0_HIGH), 0x5254); + assert_eq!(rig.readl(reg::MAC_ADDR_0_LOW), 0x00ab_cdef); + assert_eq!(rig.readl(reg::MAC_ADDR_0_HIGH - 4), 0x5566_7788); + assert_eq!(rig.readl(reg::MAC_ADDR_0_LOW + 4), 0x1122_3344); + assert!(rig.refusals(Refused::BarAccess { offset: reg::MAC_ADDR_0_HIGH, width: 1, write: true }) >= 1); + assert!(rig.refusals(Refused::BarAccess { offset: reg::MAC_ADDR_0_HIGH, width: 2, write: false }) >= 1); +} + +/// A write to a register with no modelled behaviour reads back and is counted. +#[test] +fn an_unmodelled_register_stores_and_is_counted() { + let rig = Rig::new(); + rig.writel(0x1000, 0xcafe_f00d); + assert_eq!(rig.readl(0x1000), 0xcafe_f00d); + assert_eq!(rig.refusals(Refused::UnmodelledWrite(0x1000)), 1); +} + +/// The first 256 bytes of BAR0 are the configuration registers again; a +/// MISC_CTL acknowledge there lands on the same state. +#[test] +fn bar0_aliases_the_configuration_registers() { + let rig = Rig::new(); + assert_eq!(rig.readl(0x68), 0x0105_0000); + rig.writel(0x68, 0x0000_0008); + assert_eq!(rig.cfg_read(0x68 / 4), 0x0105_0008); + rig.writel(0x7c, 0x0c14); + assert_eq!(rig.readl(0x84), rig.ind_mem_read(0x0c14)); +} + +/// PCISTATE reports conventional PCI and a quiet interrupt line, is chip +/// driven, and CLEAR_INTA self-clears. Ported from the interrupt cases of +/// test_interrupt_mailbox that need no status block. +#[test] +fn pcistate_and_the_interrupt_mailbox_start_quiet() { + let rig = Rig::new(); + assert_eq!(rig.cfg_read(0x70 / 4), 0x16); + rig.cfg_write(0x70 / 4, 0); + assert_eq!(rig.cfg_read(0x70 / 4), 0x16); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.cfg_write(0x68 / 4, 1); + assert_eq!(rig.cfg_read(0x68 / 4) & 1, 0); + assert!(!rig.irq.level()); +} + +/// A slot that drives REQ64# makes the chip report a 64-bit bus, at attach +/// and after each reset. +#[test] +fn pcistate_reports_a_64_bit_bus_in_a_req64_slot() { + let rig = Rig::with(Bcm5701Config { req64: true, ..Bcm5701Config::new(MAC) }); + assert_eq!(rig.cfg_read(0x70 / 4), 0x06); + rig.dev.state().config().reset(); + PciFunction::reset(&*rig.dev); + assert_eq!(rig.cfg_read(0x70 / 4), 0x06); + let narrow = Rig::new(); + assert_eq!(narrow.cfg_read(0x70 / 4), 0x16); +} + +fn eeprom_rig(image: &[u8], device_id: u8) -> Rig { + Rig::with(Bcm5701Config { eeprom: Some(Eeprom::from_image(device_id, image.to_vec()).unwrap()), ..Bcm5701Config::new(MAC) }) +} + +/// Ported from test_eeprom_bar_read_write: Start cannot replace an +/// unacknowledged completion, acknowledge-and-start works in one write, +/// transfers stay word aligned, writes reach only their word, and Reset wins +/// over Start. +#[test] +fn the_eeprom_controller_reads_and_writes_aligned_words() { + let path = temp_file("bar-rw", &[0x66, 0x99, 0x55, 0xaa, 0x12, 0x34, 0x56, 0x78]); + let rig = Rig::with(Bcm5701Config { eeprom: Some(Eeprom::from_file(3, path.clone()).unwrap()), ..Bcm5701Config::new(MAC) }); + let done = |rig: &Rig| rig.readl(reg::EEPROM_ADDR) & (EEPROM_COMPLETE | EEPROM_START); + + rig.writel(reg::EEPROM_ADDR, EEPROM_READ | EEPROM_START | 3 << 26); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0xaa55_9966); + assert_eq!(done(&rig), EEPROM_COMPLETE); + + // Start cannot replace an unacknowledged completion. + rig.ind_reg_write(reg::EEPROM_ADDR, EEPROM_READ | EEPROM_START | 3 << 26 | 4); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0xaa55_9966); + assert_eq!(rig.readl(reg::EEPROM_ADDR) & EEPROM_COMPLETE, EEPROM_COMPLETE); + + // OpenBSD acknowledges the old completion with the next Start. + rig.ind_reg_write(reg::EEPROM_ADDR, EEPROM_READ | EEPROM_COMPLETE | EEPROM_START | 3 << 26 | 4); + assert_eq!(rig.ind_reg_read(reg::EEPROM_DATA), 0x7856_3412); + assert_eq!(done(&rig), EEPROM_COMPLETE); + + // Address bits 1:0 are ignored: this acknowledges and starts nothing. + rig.writel(reg::EEPROM_DATA, 0x1357_9bdf); + rig.writel(reg::EEPROM_ADDR, EEPROM_COMPLETE); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0x1357_9bdf); + assert_eq!(done(&rig), 0); + assert_eq!(std::fs::read(&path).unwrap()[4..], [0x12, 0x34, 0x56, 0x78]); + + rig.writel(reg::EEPROM_DATA, 0x89ab_cdef); + rig.ind_reg_write(reg::EEPROM_ADDR, EEPROM_START | 3 << 26 | 7); + assert_eq!(rig.readl(reg::EEPROM_ADDR) & EEPROM_COMPLETE, EEPROM_COMPLETE); + assert_eq!(rig.readl(reg::EEPROM_ADDR) & 3, 0); + assert_eq!(rig.readl(reg::EEPROM_DELAY), 0); + assert_eq!(std::fs::read(&path).unwrap(), [0x66, 0x99, 0x55, 0xaa, 0xef, 0xcd, 0xab, 0x89]); + + // Block reset clears the volatile state; the delay register only reads back. + rig.writel(reg::EEPROM_DELAY, 0x000a_2c2a); + rig.writel(reg::EEPROM_ADDR, EEPROM_RESET); + assert_eq!(rig.readl(reg::EEPROM_ADDR), EEPROM_RESET); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0); + assert_eq!(rig.readl(reg::EEPROM_DELAY), 0x000a_2c2a); + + // Reset wins over Start, including while the level remains set. + for data in [0x1122_3344, 0x5566_7788] { + rig.writel(reg::EEPROM_DATA, data); + rig.writel(reg::EEPROM_ADDR, EEPROM_RESET | EEPROM_START | 3 << 26 | 4); + assert_eq!(rig.readl(reg::EEPROM_ADDR), EEPROM_RESET); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0); + } + + // Chip and machine resets do not replace the external bytes. + rig.writel(reg::EEPROM_ADDR, 0); + rig.writel(reg::MISC_CFG, MISC_CFG_RESET); + assert_eq!(rig.readl(reg::EEPROM_DELAY), 0); + rig.writel(reg::EEPROM_ADDR, EEPROM_READ | EEPROM_START | 3 << 26 | 4); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0x89ab_cdef); + rig.dev.reset(); + rig.writel(reg::EEPROM_ADDR, EEPROM_READ | EEPROM_START | 3 << 26 | 4); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0x89ab_cdef); + + // A fresh process sees the guest write in the same file. + let again = Eeprom::from_file(3, path.clone()).unwrap(); + assert_eq!(again.read_word(3, 1), Ok(0x89ab_cdef)); + std::fs::remove_file(path).unwrap(); +} + +/// Ported from test_eeprom_missing_backing: no backing means no transfer and +/// the data register is untouched. +#[test] +fn an_eeprom_command_with_no_backing_does_not_complete() { + let rig = Rig::new(); + rig.writel(reg::EEPROM_DATA, 0x7654_3210); + rig.writel(reg::EEPROM_ADDR, EEPROM_READ | EEPROM_START); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0x7654_3210); + assert_eq!(rig.readl(reg::EEPROM_ADDR) & (EEPROM_COMPLETE | EEPROM_START), 0); + assert_eq!(rig.refusals(Refused::EepromFault(Fault::NoBacking)), 1); +} + +/// Ported from test_eeprom_read_only_backing. +#[test] +fn a_read_only_eeprom_refuses_writes() { + let mut eeprom = Eeprom::from_image(3, vec![0x66, 0x99, 0x55, 0xaa]).unwrap(); + eeprom.set_read_only(true); + assert!(eeprom.is_read_only()); + let rig = Rig::with(Bcm5701Config { eeprom: Some(eeprom), ..Bcm5701Config::new(MAC) }); + rig.writel(reg::EEPROM_DATA, 0x1234_5678); + rig.writel(reg::EEPROM_ADDR, EEPROM_START | 3 << 26); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0x1234_5678); + assert_eq!(rig.readl(reg::EEPROM_ADDR) & (EEPROM_COMPLETE | EEPROM_START), 0); + assert_eq!(rig.dev.eeprom_bytes().unwrap(), [0x66, 0x99, 0x55, 0xaa]); + assert_eq!(rig.refusals(Refused::EepromFault(Fault::ReadOnly)), 1); +} + +/// Ported from test_eeprom_bounds_and_device_id. +#[test] +fn eeprom_words_past_the_backing_and_other_device_ids_fail() { + let rig = eeprom_rig(&[0x66, 0x99, 0x55, 0xaa], 3); + rig.writel(reg::EEPROM_DATA, 0x1234_5678); + for command in [EEPROM_READ | EEPROM_START | 3 << 26 | 4, EEPROM_READ | EEPROM_START | 4 << 26] { + rig.writel(reg::EEPROM_ADDR, command); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0x1234_5678); + assert_eq!(rig.readl(reg::EEPROM_ADDR) & (EEPROM_COMPLETE | EEPROM_START), 0); + } + assert_eq!(rig.refusals(Refused::EepromFault(Fault::OutOfRange(1))), 1); + assert_eq!(rig.refusals(Refused::EepromFault(Fault::WrongDevice(4))), 1); +} + +/// Ported from test_eeprom_last_interface_word: the 64 KiB reach of the +/// address field. +#[test] +fn the_last_eeprom_interface_word_is_reachable() { + let mut image = vec![0u8; 65536]; + image[65532..].copy_from_slice(&[0x66, 0x99, 0x55, 0xaa]); + let rig = eeprom_rig(&image, 3); + rig.writel(reg::EEPROM_ADDR, EEPROM_READ | EEPROM_START | 3 << 26 | 0x3fff << 2); + assert_eq!(rig.readl(reg::EEPROM_DATA), 0xaa55_9966); + rig.writel(reg::EEPROM_DATA, 0x89ab_cdef); + rig.writel(reg::EEPROM_ADDR, EEPROM_COMPLETE | EEPROM_START | 3 << 26 | 0x3fff << 2); + assert_eq!(rig.dev.eeprom_bytes().unwrap()[65532..], [0xef, 0xcd, 0xab, 0x89]); +} + +/// Ported from test_eeprom_rejects_non_word_backing and +/// test_eeprom_id_requires_backing: the configuration is validated up front. +#[test] +fn eeprom_backings_are_validated() { + assert!(Eeprom::from_image(3, vec![1, 2, 3, 4, 5]).unwrap_err().contains("multiple of four")); + assert!(Eeprom::from_image(3, vec![]).is_err()); + assert!(Eeprom::from_image(3, vec![0; 65540]).is_err()); + assert!(Eeprom::from_image(8, vec![0; 4]).unwrap_err().contains("0..=7")); + assert!(Bcm5701::new(Bcm5701Config::new([1, 0, 0, 0, 0, 0])).is_err()); +} + +/// Direct serial-pin control is unsupported and observable, not faked. +#[test] +fn direct_eeprom_pin_control_is_refused() { + let rig = Rig::new(); + rig.writel(reg::EEPROM_PIN, 0x15); + assert_eq!(rig.readl(reg::EEPROM_PIN), 0x15); + assert_eq!(rig.refusals(Refused::EepromPins), 1); +} + +// ---- the descriptor engines ---- + +fn be32(b: &[u8], off: usize) -> u32 { + u32::from_be_bytes(b[off..off + 4].try_into().unwrap()) +} +fn put32(b: &mut [u8], off: usize, v: u32) { + b[off..off + 4].copy_from_slice(&v.to_be_bytes()); +} +/// A 16-bit descriptor field sits in the other half of its word, big-endian. +fn put16dma(b: &mut [u8], off: usize, v: u16) { + b[off ^ 2..(off ^ 2) + 2].copy_from_slice(&v.to_be_bytes()); +} +fn get16dma(b: &[u8], off: usize) -> u16 { + u16::from_be_bytes([b[off ^ 2], b[(off ^ 2) + 1]]) +} + +fn tx_bd(buf: u64, len: u16, flags: u16, mss: u16) -> [u8; 16] { + let mut bd = [0; 16]; + put32(&mut bd, 0, (buf >> 32) as u32); + put32(&mut bd, 4, buf as u32); + put16dma(&mut bd, 8, flags); + put16dma(&mut bd, 10, len); + put16dma(&mut bd, 14, mss); + bd +} + +const TXF_TCP_UDP_CKSUM: u16 = 1; +const TXF_IP_CKSUM: u16 = 2; +const TXF_END: u16 = 4; +const TXF_INSERT_SRC: u16 = 0x1000; +const TXF_CHOOSE_SRC: fn(u16) -> u16 = |n| n << 13; + +fn rx_bd(buf: u64, buf_len: u16) -> [u8; 32] { + let mut bd = [0; 32]; + put32(&mut bd, 0, (buf >> 32) as u32); + put32(&mut bd, 4, buf as u32); + put16dma(&mut bd, 8, buf_len); + bd +} + +impl Rig { + /// Send ring in memory, status block address, coalescing and the MAC on, + /// the way a driver brings the transmit side up. + fn tx_open(&self, ring_len: u32, coal_bds: u32, coal_ticks: u32) { + self.ind_reg_write(reg::MODE_CTL, MODE_ALL_SWAPS); + self.ind_mem_write(0x100, 0); + self.ind_mem_write(0x104, TX_RING as u32); + self.ind_mem_write(0x108, ring_len << 16); + self.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, STATUS); + self.writel(reg::HCC_TX_MAX_COAL_BDS, coal_bds); + self.writel(reg::HCC_TX_COAL_TICKS, coal_ticks); + self.writel(reg::HCC_MODE, HCC_ENABLE); + self.writel(reg::RX_MODE, RXMODE_ENABLE); + } + + fn tx_post(&self, slot: u64, bd: [u8; 16]) { + self.mem_write(TX_RING + slot * 16, &bd); + } + + /// One whole frame in one descriptor at `slot`, then its doorbell. + fn tx_send_one(&self, slot: u64, frame: &[u8], flags: u16) { + let buf = TX_BUF + slot * 0x800; + self.mem_write(buf, frame); + self.tx_post(slot, tx_bd(buf, frame.len() as u16, flags | TXF_END, 0)); + self.writel(reg::MBX_TX_PROD_LO, slot as u32 + 1); + } + + /// Receive side up: standard ring of `STD_BUF_SIZE` buffers, a return + /// ring of `ret_len`, one completion per update. + fn rx_open(&self, ret_len: u32, coal_bds: u32, coal_ticks: u32) { + self.ind_reg_write(reg::MODE_CTL, MODE_ALL_SWAPS); + self.split_addr(reg::RX_STD_RCB_HADDR_HI, reg::RX_STD_RCB_HADDR_LO, STD_RING); + self.writel(reg::RX_STD_RCB_MAXLEN_FLAGS, STD_BUF_SIZE << 16); + self.ind_mem_write(0x200, 0); + self.ind_mem_write(0x204, RET_RING as u32); + self.ind_mem_write(0x208, ret_len << 16); + self.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, STATUS); + self.writel(reg::HCC_RX_MAX_COAL_BDS, coal_bds); + self.writel(reg::HCC_RX_COAL_TICKS, coal_ticks); + self.writel(reg::HCC_MODE, HCC_ENABLE); + self.writel(reg::RX_MODE, RXMODE_ENABLE); + self.writel(reg::MBX_IRQ0_LO, 0); + } + + /// Post `count` standard buffers, all at `RX_BUF`. + fn rx_post_std(&self, count: u32) { + for i in 0..u64::from(count) { + self.mem_write(STD_RING + i * 32, &rx_bd(RX_BUF, 0)); + } + self.writel(reg::MBX_RX_STD_PROD_LO, count); + } + + fn rx_open_jumbo(&self) { + self.split_addr(reg::RX_JUMBO_RCB_HADDR_HI, reg::RX_JUMBO_RCB_HADDR_LO, JUMBO_RING); + self.writel(reg::RX_JUMBO_RCB_MAXLEN_FLAGS, JUMBO_BUF_SIZE << 16); + self.mem_write(JUMBO_RING, &rx_bd(JUMBO_BUF, JUMBO_BUF_SIZE as u16)); + self.writel(reg::MBX_RX_JUMBO_PROD_LO, 1); + } + + fn return_descriptor(&self, slot: u64) -> Vec { + self.mem_read(RET_RING + slot * 32, 32) + } +} + +fn broadcast(len: usize, seed: u8) -> Vec { + let mut f: Vec = (0..len).map(|i| (i as u8).wrapping_mul(seed).wrapping_add(1)).collect(); + f[..6].fill(0xff); + f +} + +/// A doorbell posts the status block with UPDATED set, big-endian, once the +/// coalescing block is enabled; disabled it posts nothing and the line stays +/// down. Ported from test_status_block_byte_order and +/// test_status_block_gated_by_hcc. +#[test] +fn the_status_block_is_posted_only_while_host_coalescing_is_enabled() { + let rig = Rig::new(); + rig.ind_mem_write(0x108, TX_RING_LEN << 16); + rig.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, STATUS); + rig.ind_reg_write(reg::MODE_CTL, MODE_ALL_SWAPS); + rig.writel(reg::MBX_IRQ0_LO, 0); + + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert_eq!(be32(&rig.status_block(), 0), 0); + assert!(!rig.line_up()); + + rig.writel(reg::HCC_MODE, HCC_ENABLE); + rig.writel(reg::RX_MODE, RXMODE_ENABLE); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert_eq!(rig.status_block()[..4], [0, 0, 0, 1]); + assert!(rig.line_up()); + assert!(rig.irq.level()); +} + +/// With ENDIAN_WORDSWAP set, a 32-bit access at +0 of an eight-byte field +/// reaches its more significant word, so the big-endian driver writes each +/// host address high word first and the pair composes with no exchange. An +/// exchange on top sends the status block to the low word shifted into the +/// high half, and IRIX's tg0 then never sees a transmit completion or a +/// received frame. Ported from test_status_block_hostaddr_wordswap. +#[test] +fn host_addresses_compose_high_word_first_under_the_word_swap() { + let intended = 0x80_0000u64; + let rig = Rig::new(); + rig.ind_mem_write(0x108, TX_RING_LEN << 16); + rig.ind_reg_write(reg::MODE_CTL, MODE_ALL_SWAPS); + rig.cfg_write(0x68 / 4, rig.cfg_read(0x68 / 4) | misc_ctl::ENDIAN_WORDSWAP); + rig.writel(reg::HCC_STATUSBLK_ADDR_HI, (intended >> 32) as u32); + rig.writel(reg::HCC_STATUSBLK_ADDR_LO, intended as u32); + rig.writel(reg::HCC_MODE, HCC_ENABLE); + rig.writel(reg::RX_MODE, RXMODE_ENABLE); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert_eq!(be32(&rig.mem_read(intended, STATUS_BLK_SIZE), 0), 1); +} + +/// Ported from test_transmit_round_trip: the descriptor and its buffer are +/// read, the frame reaches the backend byte for byte, and the status block +/// reports the consumer advanced, in the chip's image order. +#[test] +fn a_transmitted_frame_reaches_the_backend_and_the_status_block_follows() { + let mut rig = Rig::new(); + let frame: Vec = (0..60u8).map(|i| i.wrapping_mul(7).wrapping_add(1)).collect(); + rig.tx_open(TX_RING_LEN, 1, 0); + rig.tx_send_one(0, &frame, 0); + assert_eq!(rig.sent_frames(), [frame]); + let sb = rig.status_block(); + assert_eq!(be32(&sb, 0), 1); + assert_eq!(get16dma(&sb, 18), 1); + // A consumer of 1 at the field's offset xor 2, big-endian. + assert_eq!(sb[16..18], [0, 1]); +} + +/// With no swap controls the chip's image reaches the bus permuted within each +/// doubleword; the frame still arrives intact. Derived from the lane rule the +/// model documents (PG105 Tables 102-109), not a port. +#[test] +fn the_swap_controls_permute_the_bytes_within_each_doubleword() { + for (mode, xor) in [(0u32, 7usize), (0x10, 4), (0x20, 3), (0x30, 0)] { + let mut rig = Rig::new(); + rig.tx_open(TX_RING_LEN, 1, 0); + rig.ind_reg_write(reg::MODE_CTL, mode); + let frame: Vec = (0..64u8).map(|i| i.wrapping_mul(5) ^ 0x21).collect(); + let lanes = |bytes: &[u8], xor: usize| { + let mut out = vec![0; bytes.len()]; + for (i, b) in bytes.iter().enumerate() { + out[(i & !7) + ((i & 7) ^ xor)] = *b; + } + out + }; + // Descriptors take the non-frame controls, which are off here. + rig.mem_write(TX_RING, &lanes(&tx_bd(TX_BUF, 64, TXF_END, 0), 7)); + rig.mem_write(TX_BUF, &lanes(&frame, xor)); + rig.writel(reg::MBX_TX_PROD_LO, 1); + assert_eq!(rig.sent_frames(), [frame], "frame-data control {mode:#x}"); + // The status block is written through the same non-frame lanes. + let sb = lanes(&rig.status_block(), 7); + assert_eq!(be32(&sb, 0), 1); + } +} + +/// A transmit descriptor that cannot be read leaves the ring where it was and +/// the doorbell is retried later. Ported from test_transmit_descriptor_dma_error. +#[test] +fn an_unreadable_send_ring_leaves_the_frame_for_a_retry() { + let mut rig = Rig::new(); + let frame: Vec = (0..60u8).map(|i| i.wrapping_mul(7).wrapping_add(1)).collect(); + rig.tx_open(TX_RING_LEN, 0, 0); + rig.ind_mem_write(0x100, (UNMAPPED >> 32) as u32); + rig.ind_mem_write(0x104, UNMAPPED as u32); + rig.writel(reg::MBX_TX_PROD_LO, 1); + assert!(rig.sent_frames().is_empty()); + assert_eq!(rig.status_tag(), 0); + assert_eq!(rig.refusals(Refused::DmaFault(DmaWhat::TxDescriptor)), 1); + + rig.ind_mem_write(0x104, TX_RING as u32); + rig.mem_write(TX_BUF, &frame); + rig.tx_post(0, tx_bd(TX_BUF, 60, TXF_END, 0)); + rig.writel(reg::MBX_TX_PROD_LO, 1); + assert_eq!(rig.sent_frames(), [frame]); +} + +/// A later fragment's unreadable payload rolls the whole frame back, and an +/// earlier completed frame stays completed. Ported from +/// test_transmit_payload_dma_error and test_transmit_complete_then_fragment_dma_error_retry. +#[test] +fn an_unreadable_fragment_rolls_back_to_the_start_of_its_frame() { + let mut rig = Rig::new(); + let first: Vec = (0..60u8).map(|i| i + 1).collect(); + let second: Vec = (0..60u8).map(|i| i + 100).collect(); + rig.tx_open(8, 1, 0); + rig.mem_write(TX_BUF, &first); + rig.tx_post(0, tx_bd(TX_BUF, 60, TXF_END, 0)); + rig.mem_write(TX_BUF + 0x100, &second[..28]); + rig.tx_post(1, tx_bd(TX_BUF + 0x100, 28, 0, 0)); + rig.tx_post(2, tx_bd(UNMAPPED, 32, TXF_END, 0)); + rig.writel(reg::MBX_TX_PROD_LO, 3); + // The first frame went and is reported; the second waits. + assert_eq!(rig.sent_frames(), [first]); + assert_eq!(rig.status_tx_cons(), 1); + assert_eq!(rig.refusals(Refused::DmaFault(DmaWhat::TxPayload)), 1); + + rig.mem_write(TX_BUF + 0x200, &second[28..]); + rig.tx_post(2, tx_bd(TX_BUF + 0x200, 32, TXF_END, 0)); + rig.writel(reg::MBX_TX_PROD_LO, 3); + assert_eq!(rig.sent_frames(), [second]); + assert_eq!(rig.status_tx_cons(), 3); +} + +/// Ported from test_tx_producer_out_of_range: an index past the ring is +/// refused, observably, and nothing is walked. +#[test] +fn a_transmit_producer_past_the_ring_is_refused() { + let rig = Rig::new(); + rig.ind_mem_write(0x108, TX_RING_LEN << 16); + rig.writel(reg::MBX_TX_PROD_LO, TX_RING_LEN); + rig.writel(reg::MBX_TX_PROD_LO, 0xffff); + assert_eq!(rig.refusals(Refused::ProducerOutOfRange(Mailbox::TxProducer)), 2); + assert_eq!(rig.mii_read(PHY_ADDR, phy_reg::IDR1), IDR1_VALUE); +} + +/// Ported from test_transmit_zero_length_descriptor: nothing goes out, the +/// descriptor still completes. +#[test] +fn a_zero_length_descriptor_completes_without_a_frame() { + let mut rig = Rig::new(); + rig.tx_open(TX_RING_LEN, 1, 0); + rig.tx_post(0, tx_bd(TX_BUF, 0, TXF_END, 0)); + rig.writel(reg::MBX_TX_PROD_LO, 1); + assert_eq!(be32(&rig.status_block(), 0), 1); + assert_eq!(rig.status_tx_cons(), 1); + assert!(rig.sent_frames().is_empty()); +} + +/// Ported from test_transmit_source_address: the descriptor picks one of four +/// filter slots to replace the frame's source, only when it asks to. +#[test] +fn a_descriptor_can_replace_the_source_address_from_a_filter_slot() { + let mut rig = Rig::new(); + rig.tx_open(8, 0, 0); + let slots = [(0x210u32, 0x2030_4050u32), (0x211, 0x2131_4151), (0x212, 0x2232_4252), (0x213, 0x2333_4353)]; + for (i, (hi, lo)) in slots.iter().enumerate() { + rig.writel(reg::MAC_ADDR_0_HIGH + i as u32 * reg::MAC_ADDR_STRIDE, *hi); + rig.writel(reg::MAC_ADDR_0_LOW + i as u32 * reg::MAC_ADDR_STRIDE, *lo); + } + let frame: Vec = (0..60u8).map(|i| 0x80 + i).collect(); + let with_source = |slot: usize| { + let mut f = frame.clone(); + f[6..8].copy_from_slice(&(slots[slot].0 as u16).to_be_bytes()); + f[8..12].copy_from_slice(&slots[slot].1.to_be_bytes()); + f + }; + for (i, selector) in [2u16, 0, 3, 1].into_iter().enumerate() { + rig.tx_send_one(i as u64, &frame, TXF_INSERT_SRC | TXF_CHOOSE_SRC(selector)); + assert_eq!(rig.sent_frames(), [with_source(usize::from(selector))]); + } + // A selector without INSERT leaves the frame untouched. + rig.tx_send_one(4, &frame, TXF_CHOOSE_SRC(3)); + assert_eq!(rig.sent_frames(), std::slice::from_ref(&frame)); + // Split after the first source bytes: both fragments carry the flags. + rig.mem_write(TX_BUF + 5 * 0x800, &frame[..8]); + rig.mem_write(TX_BUF + 6 * 0x800, &frame[8..]); + let flags = TXF_INSERT_SRC | TXF_CHOOSE_SRC(2); + rig.tx_post(5, tx_bd(TX_BUF + 5 * 0x800, 8, flags, 0)); + rig.tx_post(6, tx_bd(TX_BUF + 6 * 0x800, 52, flags | TXF_END, 0)); + rig.writel(reg::MBX_TX_PROD_LO, 7); + assert_eq!(rig.sent_frames(), [with_source(2)]); +} + +fn lso_frame(payload: usize) -> Vec { + let mut f = vec![0u8; 54 + payload]; + f[..14].copy_from_slice(&[0x52, 0x54, 0, 0x11, 0x22, 0x33, 0x52, 0x54, 0, 0xbc, 0x57, 1, 8, 0]); + f[14] = 0x45; + f[16..18].copy_from_slice(&((40 + payload) as u16).to_be_bytes()); + f[18..20].copy_from_slice(&0x1234u16.to_be_bytes()); + f[22] = 64; + f[23] = 6; + f[26..30].copy_from_slice(&[10, 0, 2, 15]); + f[30..34].copy_from_slice(&[10, 0, 2, 2]); + f[34..36].copy_from_slice(&0x3039u16.to_be_bytes()); + f[36..38].copy_from_slice(&0x0017u16.to_be_bytes()); + f[38..42].copy_from_slice(&0x1000u32.to_be_bytes()); + f[46] = 0x50; + f[47] = 0x19; + f[48..50].copy_from_slice(&0x2000u16.to_be_bytes()); + for (i, b) in f[54..].iter_mut().enumerate() { + *b = i as u8; + } + f +} + +fn inet_sum(buf: &[u8], seed: u32) -> u16 { + let mut sum = seed; + for pair in buf.chunks(2) { + sum += u32::from(pair[0]) << 8 | u32::from(*pair.get(1).unwrap_or(&0)); + } + while sum >> 16 != 0 { + sum = (sum & 0xffff) + (sum >> 16); + } + !(sum as u16) +} + +fn tcp_pseudo(frame: &[u8], tcp_len: usize) -> u32 { + frame[26..34].chunks(2).map(|p| u32::from(p[0]) << 8 | u32::from(p[1])).sum::() + 6 + tcp_len as u32 +} + +/// Ported from test_transmit_segmentation: one buffer of three segments goes +/// out as three frames with their own headers, sequence numbers, identifiers +/// and checksums, and the finishing flags only on the last. +#[test] +fn an_oversized_tcp_buffer_is_cut_into_segments() { + const MSS: usize = 512; + let mut rig = Rig::new(); + let frame = lso_frame(MSS * 3); + rig.tx_open(TX_RING_LEN, 0, 0); + rig.mem_write(TX_BUF, &frame); + rig.tx_post(0, tx_bd(TX_BUF, frame.len() as u16, 0x7000 | TXF_END, MSS as u16)); + rig.writel(reg::MBX_TX_PROD_LO, 1); + let sent = rig.sent_frames(); + assert_eq!(sent.len(), 3); + for (i, got) in sent.iter().enumerate() { + let mut expected = frame[..54].to_vec(); + expected.extend_from_slice(&frame[54 + i * MSS..54 + (i + 1) * MSS]); + expected[16..18].copy_from_slice(&((40 + MSS) as u16).to_be_bytes()); + expected[18..20].copy_from_slice(&(0x1234 + i as u16).to_be_bytes()); + expected[38..42].copy_from_slice(&(0x1000 + (i * MSS) as u32).to_be_bytes()); + if i < 2 { + expected[47] &= !0x09; + } + expected[24..26].fill(0); + expected[50..52].fill(0); + let ip = inet_sum(&expected[14..34], 0); + expected[24..26].copy_from_slice(&ip.to_be_bytes()); + let tcp = inet_sum(&expected[34..], tcp_pseudo(&expected, 20 + MSS)); + expected[50..52].copy_from_slice(&tcp.to_be_bytes()); + assert_eq!(got, &expected, "segment {i}"); + } +} + +/// A frame no larger than one segment, or one that is not plain IPv4 TCP, is +/// sent whole. Ported from test_transmit_no_segmentation. +#[test] +fn a_frame_that_needs_no_segmentation_is_sent_whole() { + let mut rig = Rig::new(); + rig.tx_open(TX_RING_LEN, 0, 0); + let small = lso_frame(100); + rig.mem_write(TX_BUF, &small); + rig.tx_post(0, tx_bd(TX_BUF, small.len() as u16, TXF_END, 512)); + rig.writel(reg::MBX_TX_PROD_LO, 1); + assert_eq!(rig.sent_frames(), [small]); +} + +fn cksum_frame(ihl_byte: u8) -> Vec { + let mut f = vec![0u8; 62]; + f[..14].copy_from_slice(&[0x52, 0x54, 0, 0x11, 0x22, 0x33, 0x52, 0x54, 0, 0xbc, 0x57, 1, 8, 0]); + f[14] = ihl_byte; + f[17] = 48; + f[22] = 64; + f[23] = 6; + f[26..30].copy_from_slice(&[10, 0, 2, 15]); + f[30..34].copy_from_slice(&[10, 0, 2, 2]); + f[34..36].copy_from_slice(&[0x30, 0x39]); + f[37] = 0x17; + f[46] = 0x50; + f[47] = 0x02; + f[48] = 0x20; + for (i, b) in f.iter_mut().enumerate().skip(54) { + *b = i as u8; + } + f +} + +/// Ported from test_transmit_checksum_offload: a driver that leaves the +/// checksum fields for the card gets them filled. +#[test] +fn the_card_fills_in_the_checksums_the_driver_asked_for() { + let mut rig = Rig::new(); + let frame = cksum_frame(0x45); + rig.tx_open(TX_RING_LEN, 0, 0); + rig.tx_send_one(0, &frame, TXF_IP_CKSUM | TXF_TCP_UDP_CKSUM); + let got = rig.sent_frames().remove(0); + let ip = inet_sum(&frame[14..34], 0); + let tcp = inet_sum(&frame[34..], tcp_pseudo(&frame, 28)); + assert_eq!(u16::from_be_bytes([got[24], got[25]]), ip); + assert_eq!(u16::from_be_bytes([got[50], got[51]]), tcp); + assert_ne!(ip, 0); + assert_ne!(tcp, 0); + assert_eq!(got[..24], frame[..24]); +} + +/// A header the offload cannot trust is declined, not walked. Ported from +/// test_transmit_checksum_bad_header. +#[test] +fn a_checksum_offload_on_a_malformed_header_is_declined() { + let mut rig = Rig::new(); + let frame = cksum_frame(0x44); + rig.tx_open(TX_RING_LEN, 0, 0); + rig.tx_send_one(0, &frame, TXF_IP_CKSUM | TXF_TCP_UDP_CKSUM); + assert_eq!(rig.sent_frames(), [frame]); +} + +/// One update per group of descriptors, the tick budget reporting a short +/// group, a zero budget never doing so. Ported from test_transmit_coalescing, +/// test_transmit_coalescing_timer and test_transmit_coalescing_zero_ticks. +#[test] +fn transmit_updates_follow_the_count_and_the_tick_budget() { + let mut rig = Rig::new(); + let frame = broadcast(64, 3); + rig.tx_open(64, 4, 0x1000_0000); + for i in 0..8 { + rig.tx_send_one(i, &frame, 0); + } + assert_eq!(rig.sent_frames().len(), 8); + // Eight frames in groups of four are two updates, not eight. + assert_eq!(rig.status_tag(), 2); + + // A short group waits for the budget. + let rig = Rig::new(); + rig.tx_open(64, 8, 50); + for i in 0..3 { + rig.tx_send_one(i, &frame, 0); + } + assert_eq!(rig.status_tag(), 0); + rig.step(Duration::from_micros(49)); + assert_eq!(rig.status_tag(), 0); + rig.step(Duration::from_micros(1)); + assert_eq!(rig.status_tag(), 1); + assert_eq!(rig.status_tx_cons(), 3); + + // Zero disables the timer: only the count reports. + let mut rig = Rig::new(); + rig.tx_open(64, 2, 0); + rig.tx_send_one(0, &frame, 0); + rig.step(Duration::from_secs(1)); + assert_eq!(rig.status_tag(), 0); + rig.tx_send_one(1, &frame, 0); + assert_eq!(rig.status_tag(), 1); + assert_eq!(rig.status_tx_cons(), 2); + assert_eq!(rig.sent_frames().len(), 2); +} + +/// Empty and dropped frames still complete every descriptor they consume. +/// Ported from test_transmit_coalescing_empty_and_dropped. +#[test] +fn empty_and_oversize_frames_still_complete_their_descriptors() { + let mut rig = Rig::new(); + let frame = broadcast(64, 5); + rig.tx_open(64, 4, 0x1000_0000); + rig.mem_write(TX_BUF, &frame); + rig.mem_fill(TX_BUF + 0x1000, 9600, 0xa5); + rig.tx_post(0, tx_bd(TX_BUF, 64, TXF_END, 0)); + rig.tx_post(1, tx_bd(0, 0, TXF_END, 0)); + rig.tx_post(2, tx_bd(TX_BUF + 0x1000, 9600, 0, 0)); + rig.tx_post(3, tx_bd(TX_BUF + 0x4000, 1, TXF_END, 0)); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.writel(reg::MBX_TX_PROD_LO, 4); + // Only the first is a frame: the oversize pair is dropped whole. + assert_eq!(rig.sent_frames(), [frame]); + assert_eq!(rig.status_tx_cons(), 4); + assert_eq!(rig.status_tag(), 1); + assert!(rig.line_up()); + assert_eq!(rig.refusals(Refused::TxFrameTooLarge), 1); +} + +/// A doorbell that ends inside a frame leaves every fragment to retry; a +/// completed frame earlier is not rolled back. Ported from +/// test_transmit_split_doorbell and test_transmit_complete_then_split_doorbell. +#[test] +fn a_doorbell_that_ends_inside_a_frame_waits_for_the_rest() { + let mut rig = Rig::new(); + let first: Vec = (0..60u8).map(|i| i.wrapping_mul(7).wrapping_add(1)).collect(); + let second: Vec = (0..60u8).map(|i| i.wrapping_mul(3).wrapping_add(9)).collect(); + rig.tx_open(8, 1, 0x1000_0000); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.mem_write(TX_BUF, &first); + rig.tx_post(0, tx_bd(TX_BUF, 60, TXF_END, 0)); + rig.mem_write(TX_BUF + 0x100, &second[..28]); + rig.mem_write(TX_BUF + 0x200, &second[28..]); + rig.tx_post(1, tx_bd(TX_BUF + 0x100, 28, 0, 0)); + rig.tx_post(2, tx_bd(TX_BUF + 0x200, 32, TXF_END, 0)); + // The doorbell stops after the first fragment of the second frame. + rig.writel(reg::MBX_TX_PROD_LO, 2); + assert_eq!(rig.sent_frames(), [first]); + assert_eq!(rig.status_tx_cons(), 1); + let tag = rig.status_tag(); + rig.writel(reg::MBX_TX_PROD_LO, 3); + assert_eq!(rig.sent_frames(), [second]); + assert_eq!(rig.status_tx_cons(), 3); + assert_eq!(rig.status_tag(), tag + 1); +} + +/// A status DMA the guest's address refused is retried when the address is +/// repaired. Ported from test_transmit_coalescing_status_dma_retry. +#[test] +fn a_refused_status_update_is_retried_by_the_budget_timer() { + let mut rig = Rig::new(); + let frame = broadcast(64, 9); + rig.tx_open(64, 1, 50); + rig.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, UNMAPPED); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.tx_send_one(0, &frame, 0); + assert_eq!(rig.sent_frames(), [frame]); + assert_eq!(rig.status_tx_cons(), 0); + assert!(!rig.line_up()); + assert_eq!(rig.refusals(Refused::DmaFault(DmaWhat::StatusBlock)), 1); + + rig.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, STATUS); + rig.step(Duration::from_micros(51)); + assert_eq!(rig.status_tx_cons(), 1); + assert_eq!(rig.status_tag(), 1); + assert!(rig.line_up()); +} + +/// Ported from test_interrupt_mailbox and test_mailbox_nonzero_masks: the +/// mailbox masks and acknowledges, MISC_CTL's global mask holds the line down +/// without losing the update, and any nonzero value masks. +#[test] +fn the_interrupt_line_follows_the_mailbox_and_the_global_mask() { + let rig = Rig::new(); + rig.ind_mem_write(0x108, TX_RING_LEN << 16); + rig.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, STATUS); + rig.ind_reg_write(reg::MODE_CTL, MODE_ALL_SWAPS); + rig.writel(reg::HCC_MODE, HCC_ENABLE); + rig.writel(reg::RX_MODE, RXMODE_ENABLE); + + // Masked out of reset: an update posts, but the line stays down. + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert!(!rig.line_up()); + // Enable interrupts; the next update asserts the line. + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert!(rig.line_up() && rig.irq.level()); + // Write 1: masks and acknowledges. + rig.writel(reg::MBX_IRQ0_LO, 1); + assert!(!rig.line_up() && !rig.irq.level()); + // An update posted while masked is discarded by the zero write. + rig.writel(reg::MBX_TX_PROD_LO, 0); + rig.writel(reg::MBX_IRQ0_LO, 0); + assert!(!rig.line_up()); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert!(rig.line_up()); + rig.writel(reg::MBX_IRQ0_LO, 0); + assert!(!rig.line_up()); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert!(rig.line_up()); + + // The global mask suppresses the line; the update was never taken. + let misc = rig.cfg_read(0x68 / 4); + rig.cfg_write(0x68 / 4, misc | 2); + assert!(!rig.line_up() && !rig.irq.level()); + rig.cfg_write(0x68 / 4, misc); + assert!(rig.line_up() && rig.irq.level()); + // CLEAR_INTA acknowledges and self-clears. + rig.cfg_write(0x68 / 4, misc | 1); + assert!(!rig.line_up()); + assert_eq!(rig.cfg_read(0x68 / 4), misc); + + // Any nonzero In_ISR value masks while the handler runs. + rig.writel(reg::MBX_IRQ0_LO, 2); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert!(!rig.line_up()); + rig.writel(reg::MBX_IRQ0_LO, 0); + assert!(!rig.line_up()); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert!(rig.line_up()); +} + +/// A failed status write advances neither its tag nor the interrupt state. +/// Ported from test_status_dma_error. +#[test] +fn a_failed_status_write_advances_nothing() { + let rig = Rig::new(); + rig.ind_mem_write(0x108, TX_RING_LEN << 16); + rig.ind_reg_write(reg::MODE_CTL, MODE_ALL_SWAPS); + rig.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, UNMAPPED); + rig.writel(reg::HCC_MODE, HCC_ENABLE); + rig.writel(reg::RX_MODE, RXMODE_ENABLE); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert!(!rig.line_up()); + rig.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, STATUS); + rig.writel(reg::MBX_TX_PROD_LO, 0); + assert_eq!(rig.status_tag(), 1); + assert!(rig.line_up()); +} + +/// Ported from test_receive_round_trip: the frame lands in the posted buffer, +/// the return descriptor and status block describe it in the chip's image +/// order, and a second buffer exposes the next consumer. +#[test] +fn a_received_frame_lands_in_the_posted_buffer_with_its_completion() { + let mut rig = Rig::new(); + let frame = broadcast(64, 5); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.deliver(&frame); + rig.step(ARRIVAL); + assert_eq!(rig.mem_read(RX_BUF, 64), frame); + let ret = rig.return_descriptor(0); + assert_eq!(get16dma(&ret, 0x0a), 0); + assert_eq!(get16dma(&ret, 0x08), 64 + 4); + assert_eq!(get16dma(&ret, 0x0c) & 4, 4); + assert_eq!(ret[0x0a..0x0c], [0, 0x44]); + let sb = rig.status_block(); + assert_eq!(get16dma(&sb, 16), 1); + assert_eq!(sb[8..12], [0, 1, 0, 0]); + assert_eq!(sb[18..20], [0, 1]); + + // The return-ring acknowledgement is accepted; reposting exposes the + // next consumer. + rig.writel(reg::MBX_RX_CONS0_LO, 1); + rig.mem_write(STD_RING + 32, &rx_bd(RX_BUF + 0x1000, 0)); + rig.writel(reg::MBX_RX_STD_PROD_LO, 2); + rig.deliver(&frame); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 2); + assert_eq!(rig.status_block()[8..10], [0, 2]); +} + +/// The return descriptor is written when the frame lands; the producer the +/// status block reports, and the interrupt, wait for the arrival deadline. +/// Ported from test_receive_waits_for_its_arrival_deadline. +#[test] +fn a_completion_waits_for_its_arrival_deadline() { + let mut rig = Rig::new(); + let frame = broadcast(64, 7); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.deliver(&frame); + assert_eq!(rig.mem_read(RX_BUF, 64), frame); + assert_eq!(get16dma(&rig.return_descriptor(0), 0x0c) & 4, 4); + assert_eq!(rig.status_rx_prod(), 0); + assert!(!rig.line_up()); + rig.step(ARRIVAL - Duration::from_nanos(1)); + assert_eq!(rig.status_rx_prod(), 0); + assert!(!rig.line_up()); + rig.step(Duration::from_nanos(1)); + assert_eq!(rig.status_rx_prod(), 1); + assert!(rig.line_up()); +} + +/// Ported from test_receive_jumbo_frame: a frame beyond the standard buffer +/// goes to the jumbo ring and is marked so. +#[test] +fn a_frame_too_big_for_the_standard_buffer_takes_the_jumbo_ring() { + let mut rig = Rig::new(); + let frame = broadcast(9000, 7); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.rx_open_jumbo(); + rig.deliver(&frame); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + let ret = rig.return_descriptor(0); + assert_eq!(get16dma(&ret, 0x08), 9004); + assert_eq!(get16dma(&ret, 0x0c) & 0x20, 0x20); + assert_eq!(rig.mem_read(JUMBO_BUF, 9000), frame); + let sb = rig.status_block(); + assert_eq!(sb[8..12], [0, 0, 0, 1]); +} + +/// With the standard ring empty, a standard-size frame is discarded and +/// counted rather than refused, while a jumbo buffer is posted; the next jumbo +/// frame still gets through. Ported from +/// test_receive_drops_standard_with_empty_standard_ring. +#[test] +fn a_short_frame_with_no_standard_buffer_is_discarded_and_counted() { + let mut rig = Rig::new(); + let first = broadcast(64, 3); + let second = broadcast(64, 11); + let big = broadcast(9000, 13); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.rx_open_jumbo(); + rig.mem_write(STD_RING + 32, &rx_bd(STD_STALE_BUF, 0)); + assert_eq!(rig.nic_no_more_rxbds(), 0); + rig.deliver(&first); + rig.step(ARRIVAL); + rig.writel(reg::MBX_RX_CONS0_LO, 1); + rig.mem_fill(STD_STALE_BUF, 64, 0xa5); + rig.mem_fill(JUMBO_BUF, 64, 0xa5); + rig.deliver(&second); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.mem_read(STD_STALE_BUF, 64), vec![0xa5; 64]); + assert_eq!(rig.mem_read(JUMBO_BUF, 64), vec![0xa5; 64]); + assert_eq!(rig.nic_no_more_rxbds(), 1); + assert_eq!(rig.refusals(Refused::RxShortage(Ring::Standard)), 1); + // The jumbo ring still takes a jumbo frame. + rig.deliver(&big); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 2); + assert_eq!(rig.mem_read(JUMBO_BUF, 9000), big); + // Posting the standard buffer afterwards does not resurrect the frame. + rig.writel(reg::MBX_RX_STD_PROD_LO, 2); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 2); + assert_eq!(rig.nic_no_more_rxbds(), 1); +} + +/// The mirror: a jumbo frame with the jumbo ring empty. Ported from +/// test_receive_drops_oversize_with_empty_jumbo_ring. +#[test] +fn a_jumbo_frame_with_no_jumbo_buffer_is_discarded_and_counted() { + let mut rig = Rig::new(); + let first = broadcast(9000, 3); + let second = broadcast(9000, 11); + let small = broadcast(64, 15); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(2); + rig.rx_open_jumbo(); + rig.mem_write(JUMBO_RING + 32, &rx_bd(0x28_0000, JUMBO_BUF_SIZE as u16)); + rig.deliver(&first); + rig.step(ARRIVAL); + rig.writel(reg::MBX_RX_CONS0_LO, 1); + rig.mem_fill(0x28_0000, 9000, 0xa5); + rig.deliver(&second); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.mem_read(0x28_0000, 9000), vec![0xa5; 9000]); + assert_eq!(rig.nic_no_more_rxbds(), 1); + assert_eq!(rig.refusals(Refused::RxShortage(Ring::Jumbo)), 1); + rig.deliver(&small); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 2); + assert_eq!(rig.mem_read(RX_BUF, 64), small); + assert_eq!(get16dma(&rig.return_descriptor(1), 0x0c) & 0x20, 0); +} + +/// A frame too big for a standard buffer with the jumbo ring disabled is +/// dropped, observably. Ported from test_receive_oversize_dropped. +#[test] +fn an_oversize_frame_with_no_jumbo_ring_is_dropped_and_counted() { + let mut rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.mem_fill(RX_BUF, 2048, 0xa5); + rig.deliver(&broadcast(2000, 3)); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 0); + assert_eq!(rig.mem_read(RX_BUF, 2048), vec![0xa5; 2048]); + assert_eq!(rig.refusals(Refused::RxNoRoom(Ring::Standard)), 1); + // The frame was consumed, not left to wedge the queue. + assert_eq!(rig.waiting(), 0); + // And one over the model's limit is dropped too. + rig.deliver(&broadcast(9700, 3)); + assert_eq!(rig.refusals(Refused::RxFrameTooLarge), 1); +} + +/// Ported from test_receive_mac_enable: a halted receiver drops what it is +/// offered; with no buffer the frame waits for the guest to make room. +#[test] +fn a_halted_receiver_drops_and_a_full_one_waits() { + let mut rig = Rig::new(); + let frame = broadcast(64, 3); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.writel(reg::RX_MODE, 0); + rig.deliver(&frame); + assert_eq!(rig.waiting(), 0); + assert_eq!(rig.status_rx_prod(), 0); + rig.writel(reg::RX_MODE, RXMODE_ENABLE); + rig.deliver(&frame); + // The one posted buffer is used; the next frame waits at the head. + rig.deliver(&frame); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.waiting(), 1); + rig.mem_write(STD_RING + 32, &rx_bd(RX_BUF + 0x1000, 0)); + rig.writel(reg::MBX_RX_STD_PROD_LO, 2); + rig.writel(reg::MBX_RX_CONS0_LO, 1); + rig.dev.poll(); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 2); + assert_eq!(rig.mem_read(RX_BUF + 0x1000, 64), frame); +} + +/// The address filter is slot 0, which the driver programs: other unicast +/// frames are consumed without a completion. Ported from the filter cases of +/// test_receive_checksum_case's setup and test_receive_round_trip. +#[test] +fn the_receive_filter_follows_the_programmed_station_address() { + let mut rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + let mut other = broadcast(64, 3); + other[..6].copy_from_slice(&[2, 0, 0, 0, 0, 9]); + rig.deliver(&other); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 0); + assert_eq!(rig.waiting(), 0); + let mut mine = other.clone(); + mine[..6].copy_from_slice(&MAC); + rig.deliver(&mine); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + // IRIX installs its own address over the card's. + rig.writel(reg::MAC_ADDR_0_HIGH, 0x0200); + rig.writel(reg::MAC_ADDR_0_LOW, 9); + rig.mem_write(STD_RING + 32, &rx_bd(RX_BUF + 0x1000, 0)); + rig.writel(reg::MBX_RX_STD_PROD_LO, 2); + rig.writel(reg::MBX_RX_CONS0_LO, 1); + rig.deliver(&other); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 2); +} + +/// A runt is padded to the Ethernet minimum before it is delivered. +#[test] +fn a_short_frame_is_padded_to_the_ethernet_minimum() { + let mut rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + let mut runt = vec![0xff; 6]; + runt.extend_from_slice(&[1, 2, 3, 4]); + rig.deliver(&runt); + rig.step(ARRIVAL); + let mut padded = runt.clone(); + padded.resize(60, 0); + assert_eq!(rig.mem_read(RX_BUF, 60), padded); + assert_eq!(get16dma(&rig.return_descriptor(0), 0x08), 64); +} + +fn rx_frame(proto: u8, payload: &[u8], fix: impl Fn(&mut Vec)) -> Vec { + let mut f = vec![0xff; 6]; + f.extend_from_slice(&[0x52, 0x54, 0, 0xbc, 0x57, 1, 8, 0]); + let total = 20 + payload.len(); + f.extend_from_slice(&[0x45, 0]); + f.extend_from_slice(&(total as u16).to_be_bytes()); + f.extend_from_slice(&[0, 1, 0, 0, 64, proto, 0, 0, 10, 0, 2, 2, 10, 0, 2, 15]); + f.extend_from_slice(payload); + f.resize(f.len().max(60), 0); + fix(&mut f); + let ip = inet_sum(&f[14..34], 0); + f[24..26].copy_from_slice(&ip.to_be_bytes()); + f +} + +/// The receive checksum report: an intact TCP segment claims both sums +/// (0xffff) and the OK flags; a corrupt one claims the IP sum only; a frame +/// that is not IPv4 TCP or UDP claims nothing. Ported from +/// test_receive_checksum_offload, test_receive_checksum_bad_transport and +/// test_receive_checksum_not_ip. +#[test] +fn the_receive_report_claims_only_the_sums_it_computed() { + let mut tcp = vec![0u8; 20]; + tcp[0..4].copy_from_slice(&[0x30, 0x39, 0, 0x17]); + tcp[12] = 0x50; + let seal = |f: &mut Vec| { + f[14 + 20 + 16..14 + 20 + 18].fill(0); + let sum = inet_sum(&f[34..54], tcp_pseudo(f, 20)); + f[50..52].copy_from_slice(&sum.to_be_bytes()); + }; + for (label, frame, ip_ok, l4_ok, ip_report, l4_report) in [ + ("intact", rx_frame(6, &tcp, seal), true, true, 0xffffu32, 0xffffu32), + ("corrupt", rx_frame(6, &tcp, |_| ()), true, false, 0xffff, 0), + ] { + let mut rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.deliver(&frame); + rig.step(ARRIVAL); + let ret = rig.return_descriptor(0); + let flags = get16dma(&ret, 0x0c); + assert_eq!(flags & 0x1000 != 0, ip_ok, "{label} ip flag"); + assert_eq!(flags & 0x2000 != 0, l4_ok, "{label} transport flag"); + assert_eq!(flags & 0x4000, 0x4000, "{label} tcp flag"); + assert_eq!(u32::from(get16dma(&ret, 0x12)), ip_report, "{label} ip sum"); + let l4 = u32::from(get16dma(&ret, 0x10)); + if l4_ok { + assert_eq!(l4, l4_report); + } else { + assert_ne!(l4, 0xffff); + } + } + // Not IPv4: nothing is claimed. + let mut rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + let mut arp = broadcast(60, 3); + arp[12..14].copy_from_slice(&[0x08, 0x06]); + rig.deliver(&arp); + rig.step(ARRIVAL); + let ret = rig.return_descriptor(0); + assert_eq!(get16dma(&ret, 0x0c) & 0x7000, 0); + assert_eq!(be32(&ret, 0x10), 0); +} + +/// Receive coalescing: with a count of two, two frames make one update; the +/// tick budget reports a short group. Ported from +/// test_receive_interrupt_coalescing and test_receive_coalescing_timer. +#[test] +fn receive_updates_follow_the_count_and_the_tick_budget() { + let mut rig = Rig::new(); + let frame = broadcast(64, 3); + rig.rx_open(16, 2, 0x1000_0000); + rig.rx_post_std(4); + for _ in 0..4 { + rig.deliver(&frame); + } + rig.step(ARRIVAL); + assert_eq!(rig.status_tag(), 2); + assert_eq!(rig.status_rx_prod(), 4); + + let mut rig = Rig::new(); + rig.rx_open(16, 8, 1000); + rig.rx_post_std(4); + rig.deliver(&frame); + rig.deliver(&frame); + rig.step(ARRIVAL); + assert_eq!(rig.status_tag(), 0); + rig.step(Duration::from_micros(800)); + assert_eq!(rig.status_tag(), 1); + assert_eq!(rig.status_rx_prod(), 2); + + // Neither criterion armed: the guest has asked to be told nothing. + let mut rig = Rig::new(); + rig.rx_open(16, 0, 0); + rig.rx_post_std(2); + rig.deliver(&frame); + rig.step(Duration::from_secs(1)); + assert_eq!(rig.status_tag(), 0); +} + +/// Ported from test_rx_producer_out_of_range and the ring-length guards: +/// out-of-range indexes are refused, observably, and not stored. +#[test] +fn receive_mailbox_indexes_past_their_rings_are_refused() { + let rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.writel(reg::MBX_RX_STD_PROD_LO, 512); + rig.writel(reg::MBX_RX_JUMBO_PROD_LO, 256); + rig.writel(reg::MBX_RX_CONS0_LO, RX_RING_LEN); + assert_eq!(rig.readl(reg::MBX_RX_STD_PROD_LO), 0); + assert_eq!(rig.refusals(Refused::ProducerOutOfRange(Mailbox::RxStandardProducer)), 1); + assert_eq!(rig.refusals(Refused::ProducerOutOfRange(Mailbox::RxJumboProducer)), 1); + assert_eq!(rig.refusals(Refused::ProducerOutOfRange(Mailbox::RxReturnConsumer)), 1); +} + +/// A return-ring descriptor that cannot be read or written leaves the frame +/// queued; repairing the address delivers it. Ported from +/// test_receive_descriptor_dma_error and test_receive_completion_dma_error. +#[test] +fn an_unreachable_ring_defers_the_frame_until_it_is_repaired() { + let mut rig = Rig::new(); + let frame = broadcast(64, 3); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.split_addr(reg::RX_STD_RCB_HADDR_HI, reg::RX_STD_RCB_HADDR_LO, UNMAPPED); + rig.mem_write(STD_RING, &rx_bd(RX_BUF, 0)); + rig.writel(reg::MBX_RX_STD_PROD_LO, 1); + rig.deliver(&frame); + assert_eq!(rig.status_rx_prod(), 0); + assert_eq!(rig.waiting(), 1); + assert_eq!(rig.refusals(Refused::DmaFault(DmaWhat::RxDescriptor)), 1); + rig.split_addr(reg::RX_STD_RCB_HADDR_HI, reg::RX_STD_RCB_HADDR_LO, STD_RING); + rig.writel(reg::MBX_RX_STD_PROD_LO, 1); + rig.dev.poll(); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.mem_read(RX_BUF, 64), frame); + + // The same for the return ring. + let mut rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.ind_mem_write(0x204, UNMAPPED as u32); + rig.ind_mem_write(0x200, (UNMAPPED >> 32) as u32); + rig.rx_post_std(1); + rig.deliver(&frame); + assert_eq!(rig.refusals(Refused::DmaFault(DmaWhat::RxCompletion)), 1); + rig.ind_mem_write(0x200, 0); + rig.ind_mem_write(0x204, RET_RING as u32); + rig.writel(reg::MBX_RX_STD_PROD_LO, 1); + rig.dev.poll(); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); +} + +/// A global reset cancels pending coalescing work and the held completions. +/// Ported from test_global_reset_cancels_receive_coalescing and +/// test_system_reset_cancels_transmit_coalescing. +#[test] +fn a_reset_cancels_coalescing_timers_and_held_completions() { + let rig = Rig::new(); + let frame = broadcast(64, 3); + rig.tx_open(64, 8, 50); + rig.tx_send_one(0, &frame, 0); + rig.writel(reg::MISC_CFG, MISC_CFG_RESET); + rig.step(Duration::from_millis(1)); + assert_eq!(rig.status_tag(), 0); + + let mut rig = Rig::new(); + rig.rx_open(16, 1, 0); + rig.rx_post_std(1); + rig.deliver(&frame); + rig.dev.reset(); + rig.step(Duration::from_millis(1)); + assert_eq!(rig.status_rx_prod(), 0); + assert!(!rig.irq.level()); +} + +/// The receive thread takes a frame when the guest posts a buffer and +/// publishes its completion on the clock, with nothing polling by hand; stop +/// joins it. Exercises `Device::start` and `stop`, which the cases above +/// replace with explicit steps. +#[test] +fn the_receive_thread_delivers_and_publishes_on_its_own() { + let mut rig = Rig::new(); + *rig.dev.frozen.lock() = None; + let frame = broadcast(64, 3); + rig.rx_open(RX_RING_LEN, 1, 0); + Device::start(&*rig.dev); + assert!(Device::is_running(&*rig.dev)); + // A frame offered before any buffer is posted waits at the queue head. + rig.inject.push(frame.clone()).unwrap(); + rig.dev.wake_rx(); + rig.rx_post_std(1); + let deadline = Instant::now() + Duration::from_secs(5); + while rig.status_rx_prod() != 1 && Instant::now() < deadline { + std::thread::sleep(Duration::from_millis(2)); + } + Device::stop(&*rig.dev); + assert!(!Device::is_running(&*rig.dev)); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.mem_read(RX_BUF, 64), frame); + assert!(rig.line_up()); +} + +// ---- the rest of the plain-memory qtests ---- + +impl Rig { + /// What the receive thread does: take what it can, let the arrival + /// deadline pass. + fn pump(&mut self) { + self.dev.poll(); + self.step(ARRIVAL); + } + fn ret_flags_end(&self, slot: u64) -> u16 { + get16dma(&self.return_descriptor(slot), 0x0c) & 4 + } +} + +/// A completed frame ahead of a descriptor read failure stays completed; the +/// failed frame retries from its first descriptor once the address is +/// repaired. Ported from test_transmit_complete_then_fragment_dma_error_retry +/// (both its empty-first and 60-byte-first variants). +#[test] +fn a_frame_with_an_unreadable_second_fragment_retries_after_the_first_completes() { + for empty_first in [false, true] { + let mut rig = Rig::new(); + let first: Vec = (0..60u8).map(|i| i.wrapping_mul(7).wrapping_add(1)).collect(); + let second: Vec = (0..60u8).map(|i| i.wrapping_mul(5).wrapping_add(3)).collect(); + rig.tx_open(TX_RING_LEN, 1, 0); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.mem_write(TX_BUF, &first); + rig.mem_write(TX_BUF + 0x100, &second[..28]); + rig.tx_post(0, tx_bd(TX_BUF, if empty_first { 0 } else { 60 }, TXF_END, 0)); + rig.tx_post(1, tx_bd(TX_BUF + 0x100, 28, 0, 0)); + rig.tx_post(2, tx_bd(UNMAPPED, 32, TXF_END, 0)); + let tag = rig.status_tag(); + rig.writel(reg::MBX_TX_PROD_LO, 3); + let expect_first = if empty_first { vec![] } else { vec![first.clone()] }; + assert_eq!(rig.sent_frames(), expect_first); + assert_eq!(rig.status_tx_cons(), 1); + assert_eq!(rig.status_tag(), tag + 1); + assert!(rig.line_up()); + assert_eq!(rig.refusals(Refused::DmaFault(DmaWhat::TxPayload)), 1); + + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.mem_write(TX_BUF + 0x100 + 28, &second[28..]); + rig.tx_post(2, tx_bd(TX_BUF + 0x100 + 28, 32, TXF_END, 0)); + rig.writel(reg::MBX_TX_PROD_LO, 3); + assert_eq!(rig.sent_frames(), [second]); + assert_eq!(rig.status_tx_cons(), 3); + assert_eq!(rig.status_tag(), tag + 2); + assert!(rig.line_up()); + } +} + +/// The same when it is the next descriptor that cannot be read: the ring's +/// last 16 bytes sit at the end of memory. Ported from +/// test_transmit_complete_then_descriptor_dma_error_retry. +#[test] +fn a_frame_with_an_unreadable_next_descriptor_retries_after_the_first_completes() { + let mut rig = Rig::new(); + let first: Vec = (0..60u8).map(|i| i.wrapping_mul(7).wrapping_add(1)).collect(); + let second: Vec = (0..60u8).map(|i| i.wrapping_mul(5).wrapping_add(3)).collect(); + let initial_ring = MEM_SIZE as u64 - 16; + let repaired_ring = TX_RING - 16; + rig.tx_open(TX_RING_LEN, 1, 0); + rig.ind_mem_write(0x104, initial_ring as u32); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.mem_write(initial_ring, &tx_bd(TX_BUF, 60, TXF_END, 0)); + rig.mem_write(TX_BUF, &first); + let tag = rig.status_tag(); + rig.writel(reg::MBX_TX_PROD_LO, 2); + assert_eq!(rig.sent_frames(), [first]); + assert_eq!(rig.status_tx_cons(), 1); + assert_eq!(rig.status_tag(), tag + 1); + assert!(rig.line_up()); + assert_eq!(rig.refusals(Refused::DmaFault(DmaWhat::TxDescriptor)), 1); + + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.mem_write(repaired_ring, &tx_bd(TX_BUF, 60, TXF_END, 0)); + rig.mem_write(TX_RING, &tx_bd(TX_BUF + 0x100, 60, TXF_END, 0)); + rig.mem_write(TX_BUF + 0x100, &second); + rig.ind_mem_write(0x104, repaired_ring as u32); + rig.writel(reg::MBX_TX_PROD_LO, 2); + assert_eq!(rig.sent_frames(), [second]); + assert_eq!(rig.status_tx_cons(), 2); + assert_eq!(rig.status_tag(), tag + 2); +} + +/// A count of two with a completed frame, a failed one and a repaired retry: +/// the retried frame's two descriptors report at once, and the budget timer +/// has nothing left to say. Ported from +/// test_transmit_partial_error_descriptor_coalescing. +#[test] +fn a_partly_failed_doorbell_coalesces_by_the_descriptors_that_completed() { + let mut rig = Rig::new(); + let first: Vec = (0..60u8).map(|i| i.wrapping_mul(7).wrapping_add(1)).collect(); + let second: Vec = (0..60u8).map(|i| i.wrapping_mul(5).wrapping_add(3)).collect(); + rig.tx_open(TX_RING_LEN, 2, 50); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.mem_write(TX_BUF, &first); + rig.tx_post(0, tx_bd(TX_BUF, 60, TXF_END, 0)); + rig.tx_post(1, tx_bd(0, 0, TXF_END, 0)); + rig.tx_post(2, tx_bd(UNMAPPED, 60, TXF_END, 0)); + let tag = rig.status_tag(); + rig.writel(reg::MBX_TX_PROD_LO, 3); + assert_eq!(rig.sent_frames(), [first]); + assert_eq!(rig.status_tx_cons(), 2); + assert_eq!(rig.status_tag(), tag + 1); + assert!(rig.line_up()); + // The qtest's wait for the log line steps the clock a millisecond at a + // time, so the budget has run out by the retry. + rig.step(Duration::from_millis(1)); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.mem_write(TX_BUF + 0x100, &second); + rig.tx_post(2, tx_bd(TX_BUF + 0x100, 60, TXF_END, 0)); + rig.writel(reg::MBX_TX_PROD_LO, 3); + assert_eq!(rig.sent_frames(), [second]); + assert_eq!(rig.status_tx_cons(), 3); + assert_eq!(rig.status_tag(), tag + 2); + assert!(rig.line_up()); + // The budget gives up on a group that already reported. + rig.step(Duration::from_micros(1050)); + assert_eq!(rig.status_tx_cons(), 3); + assert_eq!(rig.status_tag(), tag + 2); +} + +/// A reply handed back straight after the transmit sees the transmit +/// completion already published. The qtest runs an ARP exchange through +/// QEMU's user-mode network; here the backend queue plays the gateway. Adapted +/// from test_user_backend_reply_waits_for_transmit. +#[test] +fn a_reply_to_a_transmit_finds_the_transmit_already_reported() { + let mut rig = Rig::new(); + let mut arp_request = vec![0xff; 6]; + arp_request.extend_from_slice(&[0x52, 0x54, 0, 0x12, 0x34, 0x56, 8, 6, 0, 1, 8, 0, 6, 4, 0, 1]); + arp_request.extend_from_slice(&[0x52, 0x54, 0, 0x12, 0x34, 0x56, 10, 0, 2, 15, 0, 0, 0, 0, 0, 0, 10, 0, 2, 2]); + assert_eq!(arp_request.len(), 42); + rig.writel(reg::MAC_ADDR_0_HIGH, 0x5254); + rig.writel(reg::MAC_ADDR_0_LOW, 0x0012_3456); + rig.tx_open(TX_RING_LEN, 2, 1_000_000); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.tx_send_one(0, &arp_request, 0); + assert_eq!(rig.sent_frames(), [arp_request.clone()]); + // The gateway's reply, to the address just programmed. + let mut reply = arp_request[6..12].to_vec(); + reply.extend_from_slice(&[0x52, 0x55, 10, 0, 2, 2, 8, 6, 0, 1, 8, 0, 6, 4, 0, 2]); + reply.extend_from_slice(&[0x52, 0x55, 10, 0, 2, 2, 10, 0, 2, 2]); + reply.extend_from_slice(&arp_request[22..32]); + reply.resize(60, 0); + rig.deliver(&reply); + rig.step(ARRIVAL); + assert_eq!(rig.status_tx_cons(), 1); + assert_eq!(rig.status_rx_prod(), 1); + assert!(rig.line_up()); + assert_eq!(rig.mem_read(RX_BUF, 60), reply); +} + +/// A frame bigger than the standard buffer, with no jumbo ring, must not +/// write past the buffer. Ported from test_receive_overruns_nothing. +#[test] +fn an_oversize_frame_never_writes_past_the_posted_buffer() { + let mut rig = Rig::new(); + let frame = broadcast(4096, 7); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.mem_fill(RX_BUF, 4096, 0xa5); + rig.deliver(&frame); + rig.step(ARRIVAL); + assert_eq!(rig.mem_read(RX_BUF + u64::from(STD_BUF_SIZE), 4096 - STD_BUF_SIZE as usize), vec![0xa5; 4096 - 1536]); +} + +fn lap_open(rig: &Rig, coal_bds: u32) { + rig.rx_open(8, coal_bds, 0); + rig.writel(reg::RX_STD_RCB_MAXLEN_FLAGS, 1518 << 16); + rig.rx_post_std(64); +} + +/// Five trips round an eight-entry return ring, each completion visible in +/// its slot before the driver acknowledges it. Ported from +/// test_receive_ring_laps. +#[test] +fn the_return_ring_laps_cleanly() { + let mut rig = Rig::new(); + let frame = broadcast(64, 2); + lap_open(&rig, 0); + for i in 0..40u32 { + let idx = u64::from(i % 8); + rig.mem_fill(RET_RING + idx * 32, 32, 0); + rig.deliver(&frame); + rig.step(ARRIVAL); + assert_eq!(rig.ret_flags_end(idx), 4, "frame {i}"); + assert_eq!(get16dma(&rig.return_descriptor(idx), 0x08), 68); + rig.writel(reg::MBX_RX_CONS0_LO, (i + 1) % 8); + rig.writel(reg::MBX_RX_STD_PROD_LO, (64 + i + 1) % 512); + } +} + +/// A full return ring holds frames back rather than overwrite completions the +/// driver owes; freeing space with a consumer write releases them, without +/// another interrupt acknowledgement. Ported from +/// test_receive_ring_full_recovers. +#[test] +fn a_full_return_ring_holds_frames_until_the_driver_frees_it() { + let mut rig = Rig::new(); + let frame = broadcast(64, 3); + lap_open(&rig, 1); + for _ in 0..20 { + rig.deliver(&frame); + } + rig.step(ARRIVAL * 4); + // Seven fit; the last slot stays clear and the rest wait in the backend. + assert_eq!(rig.status_rx_prod(), 7); + assert_eq!(rig.ret_flags_end(7), 0); + assert_eq!(rig.waiting(), 13); + rig.writel(reg::MBX_IRQ0_LO, 0); + assert!(!rig.line_up()); + rig.step(ARRIVAL); + assert_eq!(rig.ret_flags_end(7), 0); + for i in 0..8 { + rig.mem_fill(RET_RING + i * 32, 32, 0); + } + rig.writel(reg::MBX_RX_CONS0_LO, 7); + rig.pump(); + // The retry fills the ring a second time, wrapping to slot 6. + assert_eq!(rig.status_rx_prod(), 6); + assert_eq!(rig.ret_flags_end(6), 0); + for i in 0..8 { + rig.mem_fill(RET_RING + i * 32, 32, 0); + } + rig.writel(reg::MBX_RX_CONS0_LO, 6); + rig.pump(); + assert_eq!(rig.status_rx_prod(), 4); +} + +/// A halted receiver takes nothing and keeps nothing: frames from before it +/// is enabled are gone. Ported from test_receive_mac_enable. +#[test] +fn frames_offered_to_a_halted_receiver_are_not_kept() { + let mut rig = Rig::new(); + let frame = broadcast(64, 11); + rig.rx_open(8, 1, 0); + rig.writel(reg::RX_MODE, 0); + rig.rx_post_std(8); + assert_eq!(rig.readl(reg::RX_MODE) & RXMODE_ENABLE, 0); + for _ in 0..4 { + rig.deliver(&frame); + } + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 0); + assert_eq!(rig.waiting(), 0); + rig.writel(reg::RX_MODE, RXMODE_ENABLE); + rig.pump(); + assert_eq!(rig.status_rx_prod(), 0); + assert_eq!(rig.ret_flags_end(0), 0); + let fresh = broadcast(64, 17); + rig.deliver(&fresh); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + assert!(rig.line_up()); + assert_eq!(get16dma(&rig.return_descriptor(0), 0x08), 68); + // Halting again is not a bring-up special case. + rig.writel(reg::RX_MODE, 0); + rig.deliver(&broadcast(64, 19)); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + rig.writel(reg::RX_MODE, RXMODE_ENABLE); + rig.pump(); + assert_eq!(rig.status_rx_prod(), 1); + rig.deliver(&broadcast(64, 23)); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 2); + assert_eq!(rig.ret_flags_end(1), 4); +} + +/// A frame offered before any buffer is posted waits; posting one releases +/// it. Ported from test_queued_receive_released_by_producer. +#[test] +fn a_waiting_frame_is_released_by_the_producer_mailbox() { + let mut rig = Rig::new(); + let frame = broadcast(64, 29); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.mem_write(STD_RING, &rx_bd(RX_BUF, 0)); + rig.deliver(&frame); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 0); + assert_eq!(rig.ret_flags_end(0), 0); + assert_eq!(rig.waiting(), 1); + rig.writel(reg::MBX_RX_STD_PROD_LO, 1); + rig.pump(); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.mem_read(RX_BUF, 64), frame); + assert_eq!(get16dma(&rig.return_descriptor(0), 0x0a), 0); +} + +/// A frame queued before a reset must not reach the rings brought up after +/// it, by chip reset or machine reset. Ported from test_receive_queue_reset. +#[test] +fn a_reset_discards_frames_the_backend_was_holding() { + for machine_reset in [false, true] { + let mut rig = Rig::new(); + let stale = broadcast(64, 0x51); + let fresh = broadcast(64, 0xa7); + // Nothing is posted yet, so the stale frame waits at the head. + rig.rx_open(RX_RING_LEN, 1, 0); + rig.deliver(&stale); + assert_eq!(rig.waiting(), 1); + if machine_reset { + rig.dev.reset(); + } else { + rig.writel(reg::MISC_CFG, MISC_CFG_RESET); + } + assert_eq!(rig.waiting(), 0); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.deliver(&fresh); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.mem_read(RX_BUF, 64), fresh); + } +} + +/// An unreadable posted buffer defers the frame, leaves the descriptor +/// unconsumed, and the repaired buffer delivers it. Derived from the receive +/// payload read in the model; the qtest of the same subject is +/// test_receive_payload_dma_error, whose log-marker checks are not ported. +#[test] +fn an_unreachable_receive_buffer_defers_the_frame() { + let mut rig = Rig::new(); + let frame = broadcast(64, 3); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.mem_write(STD_RING, &rx_bd(UNMAPPED, 0)); + rig.writel(reg::MBX_RX_STD_PROD_LO, 1); + rig.deliver(&frame); + assert_eq!(rig.waiting(), 1); + assert_eq!(rig.status_rx_prod(), 0); + assert_eq!(rig.refusals(Refused::DmaFault(DmaWhat::RxPayload)), 1); + rig.mem_write(STD_RING, &rx_bd(RX_BUF, 0)); + rig.writel(reg::MBX_RX_STD_PROD_LO, 1); + rig.pump(); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.mem_read(RX_BUF, 64), frame); + assert_eq!(get16dma(&rig.return_descriptor(0), 0x0a), 0); +} + +/// With neither criterion armed the receive side stays silent however long +/// the ring is idle, and so does the transmit side. Ported from +/// test_receive_without_a_criterion_is_silent and +/// test_transmit_without_a_criterion_is_silent. +#[test] +fn without_a_criterion_no_update_is_ever_posted() { + let mut rig = Rig::new(); + rig.rx_open(64, 0, 0); + rig.rx_post_std(1); + rig.deliver(&broadcast(64, 3)); + rig.step(Duration::from_secs(1)); + assert_eq!(rig.ret_flags_end(0), 4); + assert_eq!(be32(&rig.status_block(), 0), 0); + assert_eq!(rig.status_rx_prod(), 0); + assert!(!rig.line_up()); + + let mut rig = Rig::new(); + let frame = broadcast(64, 5); + rig.tx_open(64, 0, 0); + rig.writel(reg::MBX_IRQ0_LO, 0); + rig.tx_send_one(0, &frame, 0); + assert_eq!(rig.sent_frames().len(), 1); + rig.step(Duration::from_secs(1)); + assert_eq!(be32(&rig.status_block(), 0), 0); + assert_eq!(rig.status_tx_cons(), 0); + assert!(!rig.line_up()); +} + +/// A budget that has run out reports the next event as it happens, in either +/// direction. Ported from test_receive_expired_timer_reports_at_once and +/// test_transmit_expired_timer_reports_at_once. +#[test] +fn an_expired_budget_reports_the_next_event_at_once() { + let mut rig = Rig::new(); + rig.rx_open(64, 8, 50); + rig.rx_post_std(2); + rig.step(Duration::from_micros(200)); + assert_eq!(rig.status_tag(), 0); + rig.deliver(&broadcast(64, 3)); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.status_tag(), 1); + + let mut rig = Rig::new(); + rig.tx_open(64, 8, 50); + rig.step(Duration::from_micros(200)); + assert_eq!(rig.status_tag(), 0); + rig.tx_send_one(0, &broadcast(64, 3), 0); + assert_eq!(rig.sent_frames().len(), 1); + assert_eq!(rig.status_tag(), 1); +} + +/// The producer the status block names follows its completion: the +/// descriptor is written, the next slot is still empty. Ported from +/// test_receive_producer_follows_its_completion. +#[test] +fn the_receive_producer_names_only_completed_descriptors() { + let mut rig = Rig::new(); + rig.rx_open(64, 1, 0); + rig.rx_post_std(3); + for i in 0..3u64 { + rig.deliver(&broadcast(64, 0x55)); + rig.step(ARRIVAL); + assert_eq!(u64::from(rig.status_rx_prod()), i + 1); + let bd = rig.return_descriptor(i); + assert_eq!(get16dma(&bd, 0x0c) & 4, 4); + assert_eq!(get16dma(&bd, 0x08), 68); + let next = rig.return_descriptor(i + 1); + assert_eq!(get16dma(&next, 0x08), 0); + assert_eq!(get16dma(&next, 0x0c), 0); + } +} + +/// A receive update the bus refused is retried by the budget timer once the +/// address is repaired. Ported from test_receive_coalescing_status_dma_retry. +#[test] +fn a_refused_receive_update_is_retried_by_the_budget_timer() { + let mut rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 50); + rig.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, UNMAPPED); + rig.rx_post_std(1); + rig.deliver(&broadcast(64, 0x6d)); + rig.step(ARRIVAL); + assert_eq!(rig.ret_flags_end(0), 4); + assert_eq!(rig.status_rx_prod(), 0); + assert!(!rig.line_up()); + rig.split_addr(reg::HCC_STATUSBLK_ADDR_HI, reg::HCC_STATUSBLK_ADDR_LO, STATUS); + rig.step(Duration::from_micros(1051)); + assert_eq!(rig.status_rx_prod(), 1); + assert_eq!(rig.status_tag(), 1); + assert!(rig.line_up()); +} + +fn cksum_rx_frame() -> Vec { + let mut f = cksum_frame(0x45); + f[..6].fill(0xff); + seal_tcp(&mut f); + f +} +fn seal_tcp(f: &mut [u8]) { + f[24..26].fill(0); + let ip = inet_sum(&f[14..34], 0); + f[24..26].copy_from_slice(&ip.to_be_bytes()); + f[50..52].fill(0); + let tcp = inet_sum(&f[34..62], tcp_pseudo(f, 28)); + f[50..52].copy_from_slice(&tcp.to_be_bytes()); +} +fn seal_udp(f: &mut [u8]) { + f[23] = 17; + f[16] = 0; + f[17] = 48; + f[38] = 0; + f[39] = 28; + f[40..42].fill(0); + f[24..26].fill(0); + let ip = inet_sum(&f[14..34], 0); + f[24..26].copy_from_slice(&ip.to_be_bytes()); + let pseudo: u32 = f[26..34].chunks(2).map(|p| u32::from(p[0]) << 8 | u32::from(p[1])).sum::() + 17 + 28; + let udp = inet_sum(&f[34..62], pseudo); + f[40..42].copy_from_slice(&udp.to_be_bytes()); +} +fn inet_fix_ip(f: &mut [u8]) { + f[24..26].fill(0); + let ip = inet_sum(&f[14..34], 0); + f[24..26].copy_from_slice(&ip.to_be_bytes()); +} + +/// One row of test_receive_checksum_case: the flags, the IP sum and the +/// transport sum a frame earns under a MODE_CTL setting. +fn rx_checksum_case(frame: &[u8], mode: u32, flags: u16, ip: u16, l4: u16, label: &str) { + let mut rig = Rig::new(); + rig.rx_open(RX_RING_LEN, 1, 0); + rig.rx_post_std(1); + rig.ind_reg_write(reg::MODE_CTL, MODE_ALL_SWAPS | mode); + rig.deliver(frame); + rig.step(ARRIVAL); + assert_eq!(rig.status_rx_prod(), 1, "{label}"); + let ret = rig.return_descriptor(0); + let got = get16dma(&ret, 0x0c); + assert_eq!(got, flags | 4, "{label} flags"); + assert_eq!(got & 3, 0, "{label}"); + assert_eq!(get16dma(&ret, 0x12), ip, "{label} ip sum"); + assert_eq!(get16dma(&ret, 0x10), l4, "{label} transport sum"); +} + +/// Ported from test_receive_checksum_offload, _udp, _udp_zero, +/// _malformed_udp_length, _bad_ip, _bad_transport, _no_pseudo, _padding, +/// _fragment, _later_fragment, _malformed_tcp_header, _malformed_length and +/// _raw_mode. +#[test] +fn the_receive_checksum_report_covers_every_malformed_case() { + let tcp = cksum_rx_frame(); + rx_checksum_case(&tcp, 0, 0x7000, 0xffff, 0xffff, "tcp"); + + let mut udp = cksum_frame(0x45); + udp[..6].fill(0xff); + seal_udp(&mut udp); + rx_checksum_case(&udp, 0, 0x3000, 0xffff, 0xffff, "udp"); + + // UDP with the checksum omitted: IP only, and the sum is 0xffff. + let mut udp_zero = udp.clone(); + udp_zero[42..44].copy_from_slice(&[0x62, 0x6a]); + udp_zero[40..42].fill(0); + rx_checksum_case(&udp_zero, 0, 0x1000, 0xffff, 0xffff, "udp, no checksum"); + + let mut bad_udp_len = udp.clone(); + bad_udp_len[39] = 27; + rx_checksum_case(&bad_udp_len, 0, 0x1000, 0xffff, 0, "udp length"); + + let mut bad_ip = tcp.clone(); + bad_ip[22] -= 1; + rx_checksum_case(&bad_ip, 0, 0x6000, 0xfeff, 0xffff, "bad ip"); + + let mut bad_tcp = tcp.clone(); + bad_tcp[51] += 1; + rx_checksum_case(&bad_tcp, 0, 0x5000, 0xffff, 0x0001, "bad transport"); + + rx_checksum_case(&tcp, 1 << 23, 0x5000, 0xffff, !inet_sum(&tcp[34..62], 0), "no pseudo header"); + + let mut padded = tcp.clone(); + padded.extend_from_slice(&[0xa5; 8]); + rx_checksum_case(&padded, 0, 0x7000, 0xffff, 0xffff, "padding"); + + let mut fragment = tcp.clone(); + fragment[20] = 0x20; + inet_fix_ip(&mut fragment); + rx_checksum_case(&fragment, 0, 0x5000, 0xffff, 0, "more fragments"); + + let mut later = tcp.clone(); + later[21] = 1; + inet_fix_ip(&mut later); + rx_checksum_case(&later, 0, 0x5000, 0xffff, 0, "later fragment"); + + let mut short_header = tcp.clone(); + short_header[46] = 0x40; + rx_checksum_case(&short_header, 0, 0x5000, 0xffff, 0, "tcp data offset"); + + let mut bad_length = tcp.clone(); + bad_length[16..18].copy_from_slice(&[1, 0]); + rx_checksum_case(&bad_length, 0, 0, 0, 0, "ip length"); + + rx_checksum_case(&tcp, 1 << 9, 0, 0, 0, "raw mode"); +} diff --git a/src/dev/bedrock/io.rs b/src/dev/bedrock/io.rs new file mode 100644 index 000000000..e42710bad --- /dev/null +++ b/src/dev/bedrock/io.rs @@ -0,0 +1,638 @@ +//! Bedrock's I/O block (the II) routes programmed PIO windows to crossbar +//! widgets, applies inbound permission checks to requests from widgets and +//! runs two block-transfer engines. Guests configure the widget masks before +//! touching devices; a BTE moves 128-byte lines and reports completion by a +//! memory write or an interrupt packet. +//! +//! The permission words and the translation entries every PIO and inbound +//! access checks are atomics, stored under the block's lock and read without +//! it. +//! +//! # What is approximated here +//! +//! BTE transfers and their completion run synchronously when a start is +//! written. Timing and unpublished register behavior are not modeled. +//! +//! The other register words sit in a sparse map keyed by offset, since the +//! 0x30010-byte block holds only a few dozen of them. +//! +//! Ported from hw/misc/sgi-bedrock-io.c in jamesbraid/qemu-sgi-origami. + +use super::reg::IIO_PROTECT_OVRRD_KEY; +use super::Effect; +use crate::traits::{BusDevice, BusRead64, BUS_ERR, BUS_OK}; +use parking_lot::Mutex; +use std::collections::BTreeMap; +use std::sync::atomic::{AtomicU64, Ordering::Relaxed}; +use std::sync::{Arc, OnceLock}; + +/// Length of the block-relative register aperture. +pub(super) const SIZE: u64 = 0x30010; + +pub mod reg { + pub const DEVICE_ID: u64 = 0x0; + pub const PROTECTION_KEY: u64 = 0x108; + pub const PIO_PERMISSION: u64 = 0x100; + pub const INBOUND_PERMISSION: u64 = 0x110; + pub const INBOUND_DEVICE_PERMISSION: u64 = 0x120; + pub const XIO_STATUS: u64 = 0x128; + pub const LINK_LOG: u64 = 0x130; + pub const LINK_STATUS: u64 = 0x138; + pub const BTE1_BASE: u64 = 0x10000; + pub const BTE_REGISTER_SIZE: u64 = 0x30; + pub const READ_ONLY: u64 = 0x8; + pub const BTE2_BASE: u64 = 0x20000; + pub const WIDGET_STATUS: u64 = 0x20; + pub const WIDGET_PERMISSION: u64 = 0x118; + pub const REGISTER_0150: u64 = 0x150; + pub const REGISTER_0158: u64 = 0x158; + pub const ITTE_BASE: u64 = 0x160; + pub const ITTE_LAST: u64 = 0x190; + pub const REGISTER_0198_BASE: u64 = 0x198; + pub const REGISTER_01D8_LAST: u64 = 0x1d8; + pub const READ_ONLY_ALIAS: u64 = 0x1f8; + pub const REGISTER_01E0: u64 = 0x1e0; + pub const REGISTER_01E8: u64 = 0x1e8; + pub const REGISTER_01F0: u64 = 0x1f0; + pub const REGISTER_0200: u64 = 0x200; + pub const REGISTER_03A8: u64 = 0x3a8; + pub const REGISTER_03B0: u64 = 0x3b0; + pub const REGISTER_03B8: u64 = 0x3b8; + pub const REGISTER_03C0: u64 = 0x3c0; + pub const RETAINED_WINDOW_BASE: u64 = 0x308; + pub const RETAINED_WINDOW_LAST: u64 = 0x348; + pub const RETAINED_CONFIG_BASE: u64 = 0x400; + pub const RETAINED_CONFIG_LAST: u64 = 0x5d8; + pub const GFX_MAP_BASE: u64 = 0x140; +} +pub mod bte_reg { + pub const STATUS: u64 = 0x0; + pub const SOURCE: u64 = 0x8; + pub const DESTINATION: u64 = 0x10; + pub const CONTROL: u64 = 0x18; + pub const NOTIFICATION: u64 = 0x20; + pub const INTERRUPT: u64 = 0x28; +} + +const ID: u64 = (2 << 28) | (0xc110 << 12) | (0x36 << 1); +const ADDR_MASK: u64 = 0xffffffff80; +pub(super) const BUSY: u64 = 1 << 20; +pub(super) const ERROR: u64 = 1 << 16; +/// XIO status bit 13: a crossbar peer answers the link. +const PEER: u64 = 1 << 13; +const ITTES: usize = 7; + +#[derive(Clone, Copy, Default)] +pub(super) struct Bte { + status: u64, + source: u64, + destination: u64, + control: u64, + notification: u64, + interrupt: u64, +} +struct Regs { + words: BTreeMap, + bte: [Bte; 2], +} +impl Default for Regs { + fn default() -> Self { + let mut words = BTreeMap::new(); + for (a, v) in [(0x20, 0xa), (0x128, 0x3fc0fff6403 & !PEER), (0x1e8, 0x111)] { + words.insert(a, v); + } + Self { words, bte: [Bte::default(); 2] } + } +} + +/// Where a finished BTE's interrupt packet goes: the PI it names, and the +/// pending-modify value. +pub(super) struct Completion { + pub pi: usize, + pub value: u64, +} + +pub(super) struct Io { + regs: Mutex, + /// PIO permission (0x100): bit 0 lets PIO through. + pio_permission: AtomicU64, + /// The widgets PIO may reach (0x110). + pio_widgets: AtomicU64, + /// The widgets whose requests are admitted (0x118). + widget_permission: AtomicU64, + /// Per-device admission for widgets 8..15, eight bits each (0x120). + device_permission: AtomicU64, + /// The large-window translation entries (0x160..0x190). + itte: [AtomicU64; ITTES], + /// Scratch register 0 (0x150), which the IP35 PROM uses to publish each + /// interface's processors; an atomic so a board can read it without a + /// lock. + scratch0: AtomicU64, + widgets: [OnceLock>; 16], +} + +impl Io { + pub(super) fn new() -> Self { + let io = Self { + regs: Mutex::new(Regs::default()), + pio_permission: AtomicU64::new(0), + pio_widgets: AtomicU64::new(0), + widget_permission: AtomicU64::new(0), + device_permission: AtomicU64::new(0), + itte: Default::default(), + scratch0: AtomicU64::new(0), + widgets: Default::default(), + }; + io.reset_permissions(); + io + } + fn reset_permissions(&self) { + for word in [&self.pio_permission, &self.pio_widgets, &self.widget_permission, &self.device_permission] { + word.store(!0, Relaxed); + } + for entry in &self.itte { + entry.store(0, Relaxed); + } + } + pub(super) fn locked(&self) -> bool { + self.regs.is_locked() + } + /// Connect crossbar widget `widget`. Once per widget, before the machine runs. + pub(super) fn connect_widget(&self, widget: u8, dev: Arc) { + assert!(self.widgets[usize::from(widget)].set(dev).is_ok(), "Bedrock widget {widget} connected twice"); + *self.regs.lock().words.entry(reg::XIO_STATUS).or_default() |= PEER; + } + fn peer(&self) -> bool { + self.widgets.iter().any(|w| w.get().is_some()) + } + fn permitted(&self, widget: u8, device: u8) -> bool { + widget < 16 + && device < 8 + && self.widget_permission.load(Relaxed) & (1 << widget) != 0 + && (widget < 8 || self.device_permission.load(Relaxed) & (1 << ((widget - 8) * 8 + device)) != 0) + } + /// Whether a request from `widget`'s device `device` is admitted; + /// `None` asks for every master on the widget. Widget 0, the crossbar + /// itself, is admitted by its widget bit alone. + pub(super) fn admits(&self, widget: u8, device: Option) -> bool { + let allowed = match device { + Some(device) => self.permitted(widget, device), + None => (0..8).all(|d| self.permitted(widget, d)), + }; + if !allowed { + crate::dlog!(crate::devlog::LogModule::Bedrock, + "Bedrock IO inbound access refused from widget {widget} device {device:?}"); + } + allowed + } + fn retained(a: u64) -> bool { + matches!( + a, + 0x208 | 0x210 | 0x218 | 0x220 | 0x228 | 0x300 | reg::GFX_MAP_BASE | 0x148 + | 0x388 | 0x390 | 0x398 | 0x3a0 | 0x600 | 0x608 | 0x610 | 0x30000 | 0x30008 + ) || (reg::RETAINED_WINDOW_BASE..reg::RETAINED_WINDOW_LAST).contains(&a) + || (reg::RETAINED_CONFIG_BASE..=reg::RETAINED_CONFIG_LAST).contains(&a) + } + fn mask(a: u64) -> Option { + Some(match a { + reg::WIDGET_STATUS => 0x7fe01f, + reg::PIO_PERMISSION | reg::INBOUND_PERMISSION | reg::WIDGET_PERMISSION | reg::INBOUND_DEVICE_PERMISSION | reg::REGISTER_0150 | reg::REGISTER_0158 => !0, + reg::XIO_STATUS => 0x3ff0fff7d3f, + reg::LINK_STATUS => 0x101ff7f, + reg::ITTE_BASE..=reg::ITTE_LAST => 0x1f1f, + reg::REGISTER_0198_BASE..=reg::REGISTER_01D8_LAST => !0, + reg::REGISTER_01E0 => 0x3ffffff, + reg::REGISTER_01E8 => 0x1111, + reg::REGISTER_01F0 => 0x0fffffff03ffffff, + reg::REGISTER_0200 => 0x10f, + reg::REGISTER_03A8 => 0x13ffffffff30f3, + reg::REGISTER_03B0 => 0x1ff0f, + reg::REGISTER_03B8 => 0xff, + reg::REGISTER_03C0 => 0xffffff, + _ if Self::retained(a) => !0, + _ => return None, + }) + } + /// The atomic that holds register `a`, for the words PIO and inbound + /// accesses check. + fn atomic(&self, a: u64) -> Option<&AtomicU64> { + match a { + reg::PIO_PERMISSION => Some(&self.pio_permission), + reg::INBOUND_PERMISSION => Some(&self.pio_widgets), + reg::WIDGET_PERMISSION => Some(&self.widget_permission), + reg::INBOUND_DEVICE_PERMISSION => Some(&self.device_permission), + reg::ITTE_BASE..=reg::ITTE_LAST if a % 8 == 0 => Some(&self.itte[((a - reg::ITTE_BASE) / 8) as usize]), + reg::REGISTER_0150 => Some(&self.scratch0), + _ => None, + } + } + fn bte_at(a: u64) -> Option<(usize, u64)> { + if (reg::BTE1_BASE..reg::BTE1_BASE + reg::BTE_REGISTER_SIZE).contains(&a) { + Some((0, a - reg::BTE1_BASE)) + } else if (reg::BTE2_BASE..reg::BTE2_BASE + reg::BTE_REGISTER_SIZE).contains(&a) { + Some((1, a - reg::BTE2_BASE)) + } else { + None + } + } + /// The widget and widget address a PIO offset below 4 GiB reaches: the + /// small windows directly, the large ones through their translation entry. + fn target(&self, offset: u64) -> Option<(u8, u64)> { + if offset < 0x1000_0000 { + Some(((offset >> 24) as u8, offset & 0xff_ffff)) + } else if offset >= 0x2000_0000 { + let entry = ((offset >> 29) - 1) as usize; + let itte = self.itte.get(entry)?.load(Relaxed); + Some(( + ((itte >> 8) & 15) as u8, + ((itte & 31) << 29) | (offset & 0x1fff_ffff) | if itte & (1 << 12) != 0 { 1 << 47 } else { 0 }, + )) + } else { + None + } + } + /// A processor's uncached access to the I/O view below 4 GiB. + pub(super) fn pio(&self, offset: u64, size: u8, value: Option) -> BusRead64 { + let Some((widget, target)) = self.target(offset) else { + return BusRead64::err(); + }; + if self.pio_permission.load(Relaxed) & 1 == 0 || self.pio_widgets.load(Relaxed) & (1 << widget) == 0 { + return BusRead64::err(); + } + if !matches!(size, 1 | 2 | 4 | 8) { + return BusRead64::err(); + } + if widget == 1 && target == 0 { + return BusRead64::ok(ID); + } + match self.widgets[usize::from(widget)].get() { + Some(dev) => access(dev.as_ref(), target, size, value), + None => BusRead64::err(), + } + } + pub(super) fn read(&self, a: u64) -> u64 { + if let Some(word) = self.atomic(a) { + return word.load(Relaxed); + } + let mut r = self.regs.lock(); + if let Some((n, a)) = Self::bte_at(a) { + let b = &mut r.bte[n]; + return match a { + bte_reg::STATUS => b.status, + bte_reg::SOURCE => b.source, + bte_reg::DESTINATION => b.destination, + bte_reg::CONTROL => { + b.status &= !BUSY; + b.control + } + bte_reg::NOTIFICATION => b.notification, + bte_reg::INTERRUPT => b.interrupt, + _ => 0, + }; + } + match a { + reg::DEVICE_ID => ID, + reg::READ_ONLY | reg::READ_ONLY_ALIAS => 0, + reg::LINK_STATUS => r.words.get(&a).copied().unwrap_or(0) & !(1 << 28), + reg::LINK_LOG => r.words.get(&a).copied().unwrap_or(0), + _ if Self::mask(a).is_some() => r.words.get(&a).copied().unwrap_or(0), + _ => { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO unsupported read {a:#x}"); + 0 + } + } + } + /// A doubleword store at `a`: [`Effect::Bte`] when it starts an engine. + pub(super) fn write(&self, a: u64, v: u64) -> Effect { + let mut r = self.regs.lock(); + if let Some((n, offset)) = Self::bte_at(a) { + let b = &mut r.bte[n]; + match offset { + bte_reg::STATUS => b.status = v & (0xffff | BUSY), + bte_reg::SOURCE => b.source = v & ADDR_MASK, + bte_reg::DESTINATION => b.destination = v & ADDR_MASK, + bte_reg::CONTROL => { + b.control = v & 0x111; + if b.control & 0x100 != 0 { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO BTE poison unsupported"); + } else if b.status & BUSY != 0 { + if b.status & 0xffff == 0 { + b.status = 0; + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO zero-count BTE start"); + } else { + return Effect::Bte(n); + } + } + } + bte_reg::NOTIFICATION => b.notification = v & ADDR_MASK, + bte_reg::INTERRUPT => b.interrupt = v & 0x7f01ff, + _ => {} + } + return Effect::None; + } + match a { + reg::PROTECTION_KEY => { + if v == IIO_PROTECT_OVRRD_KEY { + self.pio_permission.fetch_or(1, Relaxed); + } + } + reg::INBOUND_DEVICE_PERMISSION => { + self.device_permission.fetch_or(v, Relaxed); + } + _ if self.atomic(a).is_some() => self.atomic(a).unwrap().store(v & Self::mask(a).unwrap(), Relaxed), + reg::REGISTER_01E8 => { + *r.words.entry(a).or_default() |= v & Self::mask(a).unwrap(); + } + reg::XIO_STATUS => { + let v = v & Self::mask(a).unwrap() & if self.peer() { !0 } else { !PEER }; + r.words.insert(a, v); + } + reg::LINK_LOG => { + if v == 0 { + r.words.insert(a, 0); + } else { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO nonzero link-log write ignored"); + } + } + reg::READ_ONLY_ALIAS | reg::DEVICE_ID | reg::READ_ONLY => { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO read-only or unsupported clear write {a:#x}"); + } + _ => { + if let Some(mask) = Self::mask(a) { + r.words.insert(a, v & mask); + } else { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO unsupported write {a:#x} = {v:#x}"); + } + } + } + Effect::None + } + /// Run engine `n` against `memory`, the node's DMA view, on a node + /// numbered `node`. The completion interrupt, if the engine sends one to + /// this node, is returned for the chip to deliver. + pub(super) fn run_bte(&self, n: usize, memory: &dyn BusDevice, node: u8) -> Option { + let mut b = self.regs.lock().bte[n]; + let count = b.status & 0xffff; + let bytes = count * 128; + let overflow = (b.control & 1 == 0 && b.source > (1 << 40) - bytes) || b.destination > (1 << 40) - bytes; + let mut remaining = count; + if overflow { + b.status = remaining | ERROR; + } else { + while remaining != 0 { + let mut line = [0u64; 16]; + let mut status = BUS_OK; + if b.control & 1 == 0 { + for (i, v) in line.iter_mut().enumerate() { + let r = memory.read64(b.source + i as u64 * 8); + status = r.status; + if status != BUS_OK { + break; + } + *v = r.data; + } + } + if status == BUS_OK { + for (i, v) in line.into_iter().enumerate() { + status = memory.write64(b.destination + i as u64 * 8, v); + if status != BUS_OK { + break; + } + } + } + if status != BUS_OK { + break; + } + remaining -= 1; + b.source = (b.source + 128) & ADDR_MASK; + b.destination = (b.destination + 128) & ADDR_MASK; + } + b.status = if remaining == 0 { 0 } else { remaining | ERROR }; + } + self.regs.lock().bte[n] = b; + if b.control & 0x10 != 0 { + if memory.write64(b.notification, b.status) != BUS_OK { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO BTE notification write failed"); + } + return None; + } + if ((b.interrupt >> 1) & 255) as u8 != node { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO BTE interrupt reaches no node"); + return None; + } + Some(Completion { pi: (b.interrupt & 1) as usize, value: 0x100 | ((b.interrupt >> 16) & 127) }) + } + pub(super) fn power_on(&self) { + let mut r = self.regs.lock(); + *r = Regs::default(); + if self.peer() { + *r.words.entry(reg::XIO_STATUS).or_default() |= PEER; + } + self.reset_permissions(); + self.scratch0.store(0, Relaxed); + } + /// Scratch register 0. One atomic load. + pub(super) fn scratch0(&self) -> u64 { + self.scratch0.load(Relaxed) + } +} + +/// An access of `size` bytes to `dev`: a write when `value` is given. +pub(super) fn access(dev: &dyn BusDevice, a: u64, size: u8, value: Option) -> BusRead64 { + if let Some(v) = value { + let status = match size { + 1 => dev.write8(a, v as u8), + 2 => dev.write16(a, v as u16), + 4 => dev.write32(a, v as u32), + 8 => dev.write64(a, v), + _ => BUS_ERR, + }; + return BusRead64 { status, data: 0 }; + } + match size { + 1 => { + let r = dev.read8(a); + BusRead64 { status: r.status, data: r.data as u64 } + } + 2 => { + let r = dev.read16(a); + BusRead64 { status: r.status, data: r.data as u64 } + } + 4 => { + let r = dev.read32(a); + BusRead64 { status: r.status, data: r.data as u64 } + } + 8 => dev.read64(a), + _ => BusRead64::err(), + } +} + +#[cfg(test)] +mod tests { + // Cases follow tests/qtest/sn1-bedrock-io-test.c in + // jamesbraid/qemu-sgi-origami: inbound-protection, bte and + // bte-error-termination. + use super::super::reg::{md, pi, IO_BASE, MD_BASE, PI1_BASE}; + use super::super::Rig; + use super::*; + use crate::traits::{BusRead64, BUS_BUSY}; + + /// 64 MiB at bank 0, enabled. + const BANK: u64 = 64 << 20; + fn rig() -> Rig { + let rig = Rig::new(); + rig.w(MD_BASE + md::CONFIG, 0x11); + rig + } + #[derive(Default)] + struct Memory(Mutex>); + impl BusDevice for Memory { + fn read64(&self, a: u64) -> BusRead64 { + BusRead64::ok(self.0.lock().get(&a).copied().unwrap_or(0)) + } + fn write64(&self, a: u64, v: u64) -> u32 { + self.0.lock().insert(a, v); + BUS_OK + } + } + struct Busy; + impl BusDevice for Busy { + fn read64(&self, _: u64) -> BusRead64 { + BusRead64::busy() + } + fn write64(&self, _: u64, _: u64) -> u32 { + BUS_BUSY + } + } + fn launch(rig: &Rig, base: u64, count: u64, src: u64, dst: u64, notify: u64, control: u64) { + rig.w(IO_BASE + base, count | BUSY); + rig.w(IO_BASE + base + 8, src); + rig.w(IO_BASE + base + 0x10, dst); + rig.w(IO_BASE + base + 0x20, notify); + rig.w(IO_BASE + base + 0x18, control); + } + + /// The guest must enable widget and requester masks before PIO or inbound DMA succeeds. + #[test] + fn the_io_block_checks_pio_and_inbound_requester_permissions() { + { + let rig = rig(); + let pio = rig.chip.pio(); + let m = Arc::new(Memory::default()); + rig.chip.connect_widget(8, m.clone()); + rig.w(IO_BASE + reg::ITTE_BASE, 0x802); + assert_eq!(pio.write64(0x20000080, 0xabcd), BUS_OK); + assert_eq!(m.read64(0x40000080).data, 0xabcd); + assert_eq!(pio.read64(0x20000080).data, 0xabcd); + rig.w(IO_BASE + reg::INBOUND_PERMISSION, 0); + assert_eq!(pio.read64(0x20000080).status, BUS_ERR); + rig.w(IO_BASE + reg::INBOUND_PERMISSION, !0); + rig.w(IO_BASE + reg::PIO_PERMISSION, 0); + assert_eq!(pio.read64(0x20000080).status, BUS_ERR); + rig.w(IO_BASE + reg::PROTECTION_KEY, IIO_PROTECT_OVRRD_KEY); + assert_eq!(pio.read64(0x20000080).status, BUS_OK); + assert_eq!(pio.read64(0x10000000).status, BUS_ERR); + assert_ne!(rig.r(IO_BASE + reg::XIO_STATUS) & PEER, 0, "a widget answers the link"); + rig.chip.io.power_on(); + assert_eq!(rig.r(IO_BASE + reg::PIO_PERMISSION), !0); + assert_eq!(rig.r(IO_BASE + reg::REGISTER_01E8), 0x111); + assert_ne!(rig.r(IO_BASE + reg::XIO_STATUS) & PEER, 0, "the peer survives reset"); + } + { + let rig = rig(); + let chip = &rig.chip; + assert_eq!(chip.inbound_write(14, Some(2), 0x80, 8, 42), BUS_OK); + assert_eq!(chip.dma_view().read64(0x80).data, 42); + chip.io.disable_device(14, 2); + assert_eq!(chip.inbound_read(14, Some(2), 0x80, 8).status, BUS_ERR); + assert_eq!(chip.inbound_read(14, None, 0x80, 8).status, BUS_ERR); + assert_eq!(chip.inbound_read(14, Some(1), 0x80, 8).status, BUS_OK); + // An interrupt packet to PI0's pending-modify register sets bank 0 bit 35. + let a = (1 << 47) | 0x1800090; + assert_eq!(chip.inbound_write(15, Some(0), a, 8, 0x123), BUS_OK); + assert_eq!(rig.r(pi::PENDING_STATUS_A), 1 << 35); + assert_eq!(chip.inbound_write(0, None, a + PI1_BASE, 8, 0x101), BUS_OK, "widget 0 is admitted"); + assert_eq!(rig.r(PI1_BASE + pi::PENDING_STATUS_A), 1 << 1); + assert_eq!(chip.inbound_read(15, Some(0), a, 8).status, BUS_ERR, "only stores reach a PI"); + assert_eq!(chip.inbound_write(15, Some(0), a, 4, 0x123), BUS_ERR, "only doublewords"); + assert_eq!(chip.inbound_read(15, Some(0), a | (1 << 39), 8).status, BUS_ERR, "another node"); + assert_eq!(chip.inbound_read(15, Some(0), 1 << 48, 8).status, BUS_ERR); + assert_eq!(chip.inbound_read(15, Some(0), a + 0x800000, 8).status, BUS_ERR); + assert_eq!(chip.inbound_read(15, Some(0), BANK, 8).status, BUS_ERR, "beyond the bank"); + } + } + /// The guest observes BTE line copies and zero-fill through its completion registers. + #[test] + fn the_bte_copies_and_zeroes_lines_before_completion() { + let rig = rig(); + let m = rig.chip.dma_view(); + for i in 0..32 { + m.write64(0x1000 + i * 8, 0xfeed0000 | i); + } + m.write64(0x3000, 0xdead); + launch(&rig, reg::BTE1_BASE, 2, 0x1000, 0x2000, 0x3000, 0x10); + for i in 0..32 { + assert_eq!(m.read64(0x2000 + i * 8).data, 0xfeed0000 | i); + } + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::STATUS), 0); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::SOURCE), 0x1100); + assert_eq!(m.read64(0x3000).data, 0, "the notification holds the final status"); + rig.w(IO_BASE + reg::BTE2_BASE + bte_reg::INTERRUPT, 53 << 16); + launch(&rig, reg::BTE2_BASE, 1, 0, 0x2000, 0, 1); + for i in 0..16 { + assert_eq!(m.read64(0x2000 + i * 8).data, 0); + } + assert_eq!(rig.r(pi::PENDING_STATUS_A), 1 << 53, "the completion interrupt"); + } + /// A failed BTE line stops the transfer and reports the failing source and destination. + #[test] + fn the_bte_reports_transfer_errors_and_stops_after_a_failed_line() { + // The second line runs off the end of the bank, on the read or the write side. + for (src, dst) in [(BANK - 0x80, 0x2000), (0x1000, BANK - 0x80)] { + let rig = rig(); + launch(&rig, reg::BTE1_BASE, 3, src, dst, 0x3000, 0x10); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::STATUS), 2 | ERROR); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::SOURCE), src + 0x80); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::DESTINATION), dst + 0x80); + assert_eq!(rig.chip.dma_view().read64(0x3000).data, 2 | ERROR); + } + { + let rig = rig(); + launch(&rig, reg::BTE1_BASE, 1, 0x1000, 0x2000, BANK, 0x10); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::STATUS), 0, "a failed notification still completes"); + launch(&rig, reg::BTE1_BASE, 2, 0xffffffff80, 0x2000, 0x3000, 0x10); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::STATUS), 2 | ERROR); + } + { + let rig = rig(); + launch(&rig, reg::BTE1_BASE, 1, 0, 0, 0, 0x100); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::STATUS), 1 | BUSY); + rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::CONTROL); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::STATUS), 1); + launch(&rig, reg::BTE1_BASE, 0, 0, 0, 0, 0x10); + assert_eq!(rig.r(IO_BASE + reg::BTE1_BASE + bte_reg::STATUS), 0); + } + } + /// A widget that is busy answers busy through PIO, so the CPU retries. + #[test] + fn a_busy_widget_answers_busy_through_pio() { + let rig = rig(); + rig.chip.connect_widget(8, Arc::new(Busy)); + let pio = rig.chip.pio(); + assert_eq!(pio.read64(0x8000000).status, BUS_BUSY); + assert_eq!(pio.write64(0x8000000, 42), BUS_BUSY); + } + + impl Io { + /// Hold the block's lock, for tests of the paths that must not take it. + pub(in crate::dev::bedrock) fn hold(&self) -> impl Sized + '_ { + self.regs.lock() + } + fn disable_device(&self, widget: u8, device: u8) { + if (8..16).contains(&widget) && device < 8 { + let _r = self.regs.lock(); + self.device_permission.fetch_and(!(1 << ((widget - 8) * 8 + device)), Relaxed); + } + } + } +} diff --git a/src/dev/bedrock/lb.rs b/src/dev/bedrock/lb.rs new file mode 100644 index 000000000..0b6f07904 --- /dev/null +++ b/src/dev/bedrock/lb.rs @@ -0,0 +1,659 @@ +//! Bedrock's local block (LB) provides the node and region registers, atomic +//! scratch aliases, error and vector status, and the MicroLAN master used by +//! board identity parts. The guest polls the MicroLAN completion bit after +//! issuing a slot. +//! +//! # What is approximated here +//! +//! The slot is decoded immediately and completes after one microsecond of +//! machine time; no 1-wire part answers, so a reset pulse and a short write +//! slot read back as presence-free and as a one. Permission registers retain +//! guest writes, but the register path carries no requester identity except +//! the calling CPU thread's, which only the scratch-1 rule uses, so it does +//! not enforce those permissions. +//! +//! The plain register words sit in a sparse map keyed by offset, since the +//! 0xc040-byte block holds only a few dozen of them. +//! +//! Scratch 1 orders the PROM's reset barrier ([`Barrier`]): a processor's +//! first store after the block's reset that arrives once another +//! processor's later store has landed does not land. That is a model rule +//! with no hardware counterpart, standing in for processors that leave +//! reset on one clock edge (inferred: the processors share the node's +//! reset; no timing specification is retained). +//! +//! Ported from hw/misc/sgi-bedrock-lb.c in jamesbraid/qemu-sgi-origami. + +use super::pi::Levels; +use super::reg::IIO_PROTECT_OVRRD_KEY; +use super::{Effect, NodeReset}; +use crate::dev::w1; +use parking_lot::Mutex; +use std::collections::BTreeMap; +use std::sync::atomic::{AtomicU64, AtomicU8, Ordering}; + +/// Length of the block-relative register aperture. +pub(super) const SIZE: u64 = 0xc040; + +pub mod reg { + pub const NODE_REGION: u64 = 0x0; + pub const PIO_PERMISSION: u64 = 0x4000; + pub const PIO_PERMISSION_KEY: u64 = 0x4008; + pub const INBOUND_PERMISSION: u64 = 0x4010; + pub const SOFT_RESET: u64 = 0x4018; + pub const MICRO_LAN: u64 = 0x4030; + pub const ERROR_STATUS: u64 = 0x4040; + pub const ERROR_CLEAR: u64 = 0x4048; + pub const PIO_REGION_MASK: u64 = 0x4028; + pub const NODE_REGION_CONFIG: u64 = 0x4050; + pub const NODE_REGION_MASK: u64 = 0x4058; + pub const PIO_CONFIG: u64 = 0x4100; + pub const INBOUND_CONFIG: u64 = 0x4200; + pub const INBOUND_MASK: u64 = 0x4208; + pub const VECTOR_REQUEST: u64 = 0xc000; + pub const VECTOR_STATUS: u64 = 0xc020; + pub const VECTOR_STATUS_ALIAS: u64 = 0xc038; + pub const PIO_WINDOW: u64 = 0x8000; + pub const PIO_WINDOW_END: u64 = 0x8020; + pub const ATOMIC_SCRATCH: u64 = 0x8040; + pub const ATOMIC_SCRATCH_END: u64 = 0x8050; + pub const ATOMIC_READ0: u64 = 0x8058; + pub const ATOMIC_READ1: u64 = 0x8060; + pub const REGISTER_4020: u64 = 0x4020; + pub const REGISTER_4060: u64 = 0x4060; + pub const REGISTER_C008: u64 = 0xc008; + pub const REGISTER_C010: u64 = 0xc010; + pub const REGISTER_C028: u64 = 0xc028; + pub const REGISTER_C030: u64 = 0xc030; +} + +const ID: u64 = (0x36 << 1) | (0x3012 << 12) | (2 << 28); +/// Scratch 1 in the plain scratch window. +const SCRATCH1: u64 = reg::PIO_WINDOW + 8; +const VECTOR_MASK: u64 = 3 | (0x1fffff << 3) | (0xff << 32) | (0x7ff << 40); +struct Regs { + words: BTreeMap, + scratch: [u64; 5], + node: u8, + region: u8, + deadline: Option, + sample: bool, + barrier: Barrier, +} +impl Default for Regs { + fn default() -> Self { + let mut words = BTreeMap::new(); + words.insert(reg::PIO_PERMISSION, !0); + words.insert(reg::INBOUND_PERMISSION, !0); + Self { words, scratch: [0; 5], node: 0, region: 0, deadline: None, sample: true, barrier: Barrier::default() } + } +} + +/// The order of the processors' stores to scratch 1 since the block's last +/// reset, for the PROM's reset barrier. +/// +/// Every processor of a node clears scratch 1 on its reset path, then the +/// boot master stores its release there and the others poll for it. On +/// hardware every processor leaves reset on the same clock edge, so every +/// clear lands before the release (inferred: the processors share the +/// node's reset; no timing specification is retained). Here each processor is a host thread +/// that can start or stall late, and a late clear would erase the release: +/// the PROM then counts that processor as absent. So a processor's first +/// store of the epoch that arrives after another processor's second or +/// later store (the release) has landed does not land: it is ordered below +/// the release, as it would have been on hardware. Until that processor +/// stores again it reads the release, whatever has landed since: on +/// hardware it would have polled the release long before the others' +/// next stores (the four-processor PROM's come millions of instructions +/// later). +/// +/// Model rule, approximated: no hardware rule drops a store or answers one +/// processor differently from another. It is a function of the store +/// sequence alone (who stored, in what order), never of values or host +/// time. An access no processor makes (a monitor, a test, a peer node) +/// is plain storage and changes nothing here. +/// +/// Limitations: an L1 soft reset restarts the processors without an LB +/// reset, so a second PROM run in the same epoch gets no ordering. A +/// processor that cleared on time but stalls until after the others' joins +/// reads those, not the release. A peer node's store is plain storage, and +/// a late local clear after it is dropped, keeping the peer's value. The +/// requester is the calling thread's CPU number (`smp::on_cpu_thread`), +/// not a field of the access. +#[derive(Default)] +struct Barrier { + /// Processors, by CPU number, that have stored this epoch. + stored: u64, + /// The first processor's second or later store this epoch: the release. + release: Option, + /// Processors whose only store this epoch was ordered below the release. + behind: u64, +} + +/// What [`Barrier::store`] makes of one store. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Scratch1Store { + /// It lands. + Lands, + /// It lands, and is its processor's first after the release: the PROM's + /// presence join. Holds the processor's bit. + Joins(u64), + /// A first store after the release: it does not land, and its processor + /// reads the release until it stores again. + BelowRelease, +} + +fn cpu_bit(cpu: Option) -> Option { + cpu.and_then(|c| 1u64.checked_shl(c as u32)) +} + +impl Barrier { + /// Order a store of `value` from `cpu`, or from no processor. + fn store(&mut self, cpu: Option, value: u64) -> Scratch1Store { + let Some(bit) = cpu_bit(cpu) else { return Scratch1Store::Lands }; + if self.stored & bit == 0 { + self.stored |= bit; + if self.release.is_none() { + return Scratch1Store::Lands; + } + self.behind |= bit; + return Scratch1Store::BelowRelease; + } + self.behind &= !bit; + if self.release.is_some() { + return Scratch1Store::Joins(bit); + } + self.release = Some(value); + Scratch1Store::Lands + } + + /// What a read from `cpu` sees, if not the register: the release, for a + /// processor still behind it. + fn read(&self, cpu: Option) -> Option { + cpu_bit(cpu).filter(|bit| self.behind & bit != 0).and(self.release) + } +} +impl Regs { + fn error_level(&self) -> bool { + self.words.get(®::ERROR_STATUS).copied().unwrap_or(0) != 0 + } +} + +pub(super) struct Lb { + regs: Mutex, + /// Processors, by CPU number, whose presence join on scratch 1 has + /// landed since the block's last reset ([`Scratch1Store::Joins`]). + /// Written under the block's lock, read without it. + joined: AtomicU64, + /// Scratch 3's bit 0, which the IP35 PROM takes as its hub lock through + /// the test-and-set twin at 0x8058: kept with the register under the + /// block's lock, read without it. + scratch3: AtomicU64, + /// Processors, by CPU number, that have stored to their own PI + /// real-time compare register since their presence join: the IP35 + /// PROM's pair liveness post ([`Lb::note_rt_compare`]). Cleared with + /// the join mask. + posted: AtomicU64, +} + +impl Lb { + pub(super) fn new() -> Self { + Self { regs: Mutex::new(Regs::default()), joined: AtomicU64::new(0), scratch3: AtomicU64::new(0), posted: AtomicU64::new(0) } + } + /// Processor `cpu` stored to its own real-time compare register. The + /// first such store after its presence join is the PROM's liveness post + /// to its pair partner, whatever the value: the PROM's earlier stores + /// there are progress codes. A store before the join changes nothing. + pub(super) fn note_rt_compare(&self, cpu: usize) { + if let Some(bit) = cpu_bit(Some(cpu)).filter(|bit| self.joined.load(Ordering::Acquire) & bit != 0) { + self.posted.fetch_or(bit, Ordering::Release); + } + } + /// The processors, one bit per CPU number, that have posted since their + /// presence join. One atomic load. + pub(super) fn rt_posted(&self) -> u64 { + self.posted.load(Ordering::Acquire) + } + /// Whether scratch 3's bit 0, the PROM's hub lock, is clear. One atomic + /// load. + pub(super) fn hub_lock_free(&self) -> bool { + self.scratch3.load(Ordering::Acquire) & 1 == 0 + } + /// Whether processor `cpu`'s presence join on scratch 1 has landed since + /// the block's last reset. One atomic load. + pub(super) fn scratch1_joined(&self, cpu: usize) -> bool { + cpu_bit(Some(cpu)).is_some_and(|bit| self.joined.load(Ordering::Acquire) & bit != 0) + } + pub(super) fn locked(&self) -> bool { + self.regs.is_locked() + } + fn complete(r: &mut Regs, now: u64) { + if r.deadline.is_some_and(|d| now >= d) { + *r.words.entry(reg::MICRO_LAN).or_default() |= 2 | u64::from(r.sample); + r.deadline = None; + } + } + fn mask(a: u64) -> Option { + Some(match a { + reg::PIO_PERMISSION | reg::INBOUND_PERMISSION | reg::REGISTER_4020 | reg::REGISTER_4060 | reg::REGISTER_C008 | reg::REGISTER_C010 | reg::REGISTER_C028 | reg::REGISTER_C030 => !0, + reg::PIO_REGION_MASK => 0x80ff80ff80ff80ff, + reg::ERROR_STATUS => 0x7ff, + reg::NODE_REGION_CONFIG => 0x7f | (0x7ff << 8) | (0x7ff << 20) | (0xf << 32) | (1 << 40) | (1 << 44), + reg::NODE_REGION_MASK => (1 << 38) - 1, + reg::PIO_CONFIG => 0x3f, + reg::INBOUND_CONFIG => 1 | (0x3ff << 4) | (0x3ff << 16) | (1 << 28) | (1 << 32), + reg::INBOUND_MASK => { + 0x1f | (0xfff << 8) | (1 << 20) | (0x3ff << 24) | (0x3ff << 36) | (0x3ff << 48) + } + reg::VECTOR_REQUEST => VECTOR_MASK, + _ => return None, + }) + } + /// The block's reset, under its lock: the scratch words 1 and 2 and the + /// error status survive it. The node number returns to 0, and the reset + /// barrier starts a new epoch. + fn reset(&self, r: &mut Regs, node: &AtomicU8) { + let survivors = [r.scratch[1], r.scratch[2], r.words.get(®::ERROR_STATUS).copied().unwrap_or(0)]; + *r = Regs::default(); + self.joined.store(0, Ordering::Release); + self.posted.store(0, Ordering::Release); + self.scratch3.store(0, Ordering::Release); + r.scratch[1] = survivors[0]; + r.scratch[2] = survivors[1]; + r.words.insert(reg::ERROR_STATUS, survivors[2]); + node.store(0, Ordering::Release); + } + pub(super) fn power_on(&self, node: &AtomicU8) { + self.reset(&mut self.regs.lock(), node); + } + /// A doubleword read at `a` by processor `requester`, or by none. + pub(super) fn read(&self, a: u64, now: u64, requester: Option) -> u64 { + let mut r = self.regs.lock(); + Self::complete(&mut r, now); + match a { + reg::NODE_REGION => ID | ((r.node as u64) << 32) | ((r.region as u64) << 46), + SCRATCH1 => r.barrier.read(requester).unwrap_or(r.scratch[1]), + reg::PIO_WINDOW..=reg::PIO_WINDOW_END => r.scratch[(a - reg::PIO_WINDOW) as usize / 8], + reg::ATOMIC_READ0 | reg::ATOMIC_READ1 => { + let n = 3 + (a - reg::ATOMIC_READ0) as usize / 8; + let v = r.scratch[n]; + r.scratch[n] = 1; + self.scratch3.store(r.scratch[3], Ordering::Release); + v + } + reg::VECTOR_STATUS_ALIAS => r.words.remove(®::VECTOR_STATUS).unwrap_or(0), + reg::VECTOR_STATUS | reg::MICRO_LAN => r.words.get(&a).copied().unwrap_or(0), + _ if Self::mask(a).is_some() => r.words.get(&a).copied().unwrap_or(0), + _ => { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock LB unsupported read {a:#x}"); + 0 + } + } + } + /// A doubleword store at `a` by processor `requester`, or by none. The + /// error level reaches `levels` and the node number `node` under the + /// block's lock, so concurrent stores publish in register order. A soft + /// reset resets the block and asks for the node's. + pub(super) fn write(&self, a: u64, v: u64, now: u64, requester: Option, levels: &Levels, node: &AtomicU8) -> Effect { + let mut r = self.regs.lock(); + Self::complete(&mut r, now); + match a { + reg::NODE_REGION => { + let region = ((v >> 46) & 3) as u8; + if region == 3 { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock LB invalid region size"); + } else { + r.node = (v >> 32) as u8; + r.region = region; + node.store(r.node, Ordering::Release); + } + } + SCRATCH1 => { + let order = r.barrier.store(requester, v); + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock LB scratch 1 store {v:#x} from CPU {requester:?}: {order:?}"); + match order { + Scratch1Store::BelowRelease => {} + Scratch1Store::Lands => r.scratch[1] = v, + Scratch1Store::Joins(bit) => { + r.scratch[1] = v; + self.joined.fetch_or(bit, Ordering::Release); + } + } + } + reg::PIO_WINDOW..=reg::PIO_WINDOW_END => { + let n = (a - reg::PIO_WINDOW) as usize / 8; + r.scratch[n] = if n < 3 { v } else { v & 1 }; + } + reg::ATOMIC_SCRATCH..=reg::ATOMIC_SCRATCH_END => { + let n = (a - reg::ATOMIC_SCRATCH) as usize / 8; + if r.scratch[n] == 0 { + r.scratch[n] = v; + } + } + reg::PIO_PERMISSION_KEY => { + if v == IIO_PROTECT_OVRRD_KEY { + *r.words.entry(reg::PIO_PERMISSION).or_default() |= 1; + } + } + reg::SOFT_RESET => { + if v & 1 != 0 { + self.reset(&mut r, node); + return Effect::Reset(NodeReset::LocalBlock); + } + } + reg::MICRO_LAN => { + if r.deadline.is_some() { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock LB MicroLAN write while slot pending discarded"); + } else { + r.words.insert(a, v & ((1 << 34) - 4)); + let (pulse, _) = w1::slot::fields(v); + r.sample = pulse >= w1::slot::RESET_PULSE || pulse < w1::slot::ONE_PULSE; + r.deadline = Some(now.saturating_add(1000)); + } + } + reg::ERROR_STATUS => { + *r.words.entry(a).or_default() |= v & 0x7ff; + } + reg::ERROR_CLEAR => { + *r.words.entry(reg::ERROR_STATUS).or_default() &= !(v & 0x7ff); + } + reg::VECTOR_REQUEST => { + r.words.insert(a, v & VECTOR_MASK); + if r.words.get(®::VECTOR_STATUS).copied().unwrap_or(0) & (1 << 63) != 0 { + *r.words.entry(reg::VECTOR_STATUS).or_default() |= 1 << 62; + } else { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock LB vector launch reaches no peer; no reply latched"); + } + } + reg::VECTOR_STATUS | reg::VECTOR_STATUS_ALIAS => { + if v == 0 { + r.words.insert(reg::VECTOR_STATUS, 0); + } else { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock LB nonzero vector status write ignored"); + } + } + _ => { + if let Some(mask) = Self::mask(a) { + r.words.insert(a, v & mask); + if (a == reg::PIO_PERMISSION || a == reg::INBOUND_PERMISSION) && v != !0 { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock LB permission enforcement requires requester context"); + } + } else { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock LB unsupported write {a:#x} = {v:#x}"); + } + } + } + self.scratch3.store(r.scratch[3], Ordering::Release); + if levels.set(Levels::LB_ERROR, r.error_level()) { Effect::Reflect } else { Effect::None } + } +} + +#[cfg(test)] +mod tests { + // Cases follow tests/qtest/sn1-bedrock-lb-test.c in + // jamesbraid/qemu-sgi-origami: identity-permissions, scratch-reset and + // microlan-slot. + use super::super::reg::{pi, LB_BASE, PI1_BASE}; + use super::super::Rig; + use super::*; + use crate::traits::{BusDevice, BUS_ERR}; + + fn error_reflected(rig: &Rig) -> [bool; 2] { + [0, PI1_BASE].map(|base| rig.r(base + pi::PENDING_STATUS_B) & (1 << 55) != 0) + } + + /// The guest configures node permissions and expects selected scratch words to survive reset. + #[test] + fn permission_writes_latch_errors_and_reset_retains_scratch() { + { + let rig = Rig::new(); + let node = || rig.chip.node.load(Ordering::SeqCst); + rig.w(LB_BASE + reg::NODE_REGION, (0x5a << 32) | (2 << 46)); + assert_eq!(node(), 0x5a); + rig.w(LB_BASE + reg::NODE_REGION, 3 << 46); + assert_eq!(node(), 0x5a); + rig.w(LB_BASE + reg::PIO_PERMISSION, 0); + rig.w(LB_BASE + reg::PIO_PERMISSION_KEY, 0); + assert_eq!(rig.r(LB_BASE + reg::PIO_PERMISSION), 0); + rig.w(LB_BASE + reg::PIO_PERMISSION_KEY, IIO_PROTECT_OVRRD_KEY); + assert_eq!(rig.r(LB_BASE + reg::PIO_PERMISSION), 1); + rig.w(LB_BASE + reg::PIO_WINDOW + 8, 42); + rig.w(LB_BASE + reg::ERROR_STATUS, !0); + assert_eq!(rig.r(LB_BASE + reg::ERROR_STATUS), 0x7ff); + assert_eq!(error_reflected(&rig), [true; 2], "bank 1 bit 55 on both interfaces"); + rig.chip.lb.power_on(&rig.chip.node); + assert_eq!(rig.r(LB_BASE + reg::PIO_WINDOW + 8), 42); + assert_eq!(error_reflected(&rig), [true; 2], "the error status survives reset"); + rig.w(LB_BASE + reg::ERROR_CLEAR, 0x7ff); + assert_eq!(error_reflected(&rig), [false; 2]); + assert_eq!(node(), 0); + } + { + let rig = Rig::new(); + rig.w(LB_BASE + reg::ATOMIC_SCRATCH, 42); + rig.w(LB_BASE + reg::ATOMIC_SCRATCH, 43); + assert_eq!(rig.r(LB_BASE + reg::PIO_WINDOW), 42); + rig.w(LB_BASE + reg::REGISTER_C030, 0xdead); + rig.w(LB_BASE + reg::VECTOR_REQUEST, !0); + assert_eq!(rig.r(LB_BASE + reg::VECTOR_STATUS), 0); + assert_eq!(rig.r(LB_BASE + reg::REGISTER_C030), 0xdead); + assert_eq!(rig.regs.read32(LB_BASE + reg::PIO_WINDOW).status, BUS_ERR); + assert_eq!(rig.regs.read64(LB_BASE + 1).status, BUS_ERR); + } + } + /// The guest polls MicroLAN completion and cannot replace a slot that is still pending. + #[test] + fn microlan_slots_finish_after_one_microsecond_and_reject_overlap() { + let rig = Rig::new(); + let word = (0x66 << 27) | (60 << 10) | (0x41 << 2) | 3; + rig.w(LB_BASE + reg::MICRO_LAN, word); + assert_eq!(rig.r(LB_BASE + reg::MICRO_LAN), word & !3); + rig.w(LB_BASE + reg::MICRO_LAN, 8 << 10); + assert_eq!(rig.r(LB_BASE + reg::MICRO_LAN), word & !3); + rig.set_time(1000); + assert_eq!(rig.r(LB_BASE + reg::MICRO_LAN), (word & !3) | 2); + rig.w(LB_BASE + reg::MICRO_LAN, 8 << 10); + rig.set_time(2000); + assert_eq!(rig.r(LB_BASE + reg::MICRO_LAN), (8 << 10) | 3); + rig.w(LB_BASE + reg::MICRO_LAN, 60 << 10); + rig.chip.lb.power_on(&rig.chip.node); + rig.set_time(3000); + assert_eq!(rig.r(LB_BASE + reg::MICRO_LAN), 0); + } + const RELEASE: u64 = 0x07f8_0000; + /// The PROM's flags word that follows the joins. + const FLAGS: u64 = 0x6000_0000_0000_0000; + const SCRATCH1_AT: u64 = LB_BASE + SCRATCH1; + + /// Run `f` on a thread that is processor `cpu`. + fn as_cpu(cpu: u8, f: impl FnOnce() -> T + Send + 'static) -> T { + std::thread::spawn(move || { + crate::cpu::smp::set_current_cpu(cpu); + f() + }) + .join() + .unwrap() + } + fn store_as(rig: &Rig, cpu: u8, value: u64) { + let regs = rig.regs.clone(); + assert_eq!(as_cpu(cpu, move || regs.write64(SCRATCH1_AT, value)), crate::traits::BUS_OK); + } + fn read_as(rig: &Rig, cpu: u8) -> u64 { + let regs = rig.regs.clone(); + as_cpu(cpu, move || regs.read64(SCRATCH1_AT).data) + } + /// Scratch 1 as a reader that is no processor sees it. + fn scratch1(rig: &Rig) -> u64 { + rig.r(SCRATCH1_AT) + } + fn joined(rig: &Rig, cpu: usize) -> bool { + rig.chip.scratch1_joined(cpu) + } + + /// A processor's reset-path clear that arrives after the master's + /// release does not erase it, and that processor's next store joins. + #[test] + fn a_late_clear_lands_below_the_release() { + let rig = Rig::new(); + store_as(&rig, 0, 0); + store_as(&rig, 0, RELEASE); + store_as(&rig, 1, 0); + assert_eq!(scratch1(&rig), RELEASE, "the late clear did not land"); + assert_eq!(read_as(&rig, 1), RELEASE); + assert!(!joined(&rig, 1)); + store_as(&rig, 1, 0x101); + assert_eq!(scratch1(&rig), 0x101); + assert!(joined(&rig, 1) && !joined(&rig, 0)); + } + + /// A processor whose clear came after the release reads the release + /// until it stores again, even once another processor's next store has + /// replaced it; everyone else reads the register. + #[test] + fn a_late_processor_reads_the_release_until_it_stores_again() { + let rig = Rig::new(); + store_as(&rig, 0, 0); + store_as(&rig, 1, 0); + store_as(&rig, 0, RELEASE); + store_as(&rig, 2, 0); + store_as(&rig, 1, 0); + assert_eq!(read_as(&rig, 2), RELEASE, "CPU 2 is still behind the release"); + assert_eq!((read_as(&rig, 1), read_as(&rig, 0), scratch1(&rig)), (0, 0, 0)); + store_as(&rig, 2, 0x102); + assert_eq!(read_as(&rig, 2), 0x102); + assert!(joined(&rig, 2)); + } + + /// Clears that come before the release land, the release lands over + /// them, and the release itself is no join. + #[test] + fn a_release_after_every_clear_lands() { + let rig = Rig::new(); + for cpu in 0..4 { + store_as(&rig, cpu, 0x55); + assert_eq!(scratch1(&rig), 0x55); + } + store_as(&rig, 0, RELEASE); + assert_eq!(scratch1(&rig), RELEASE); + assert!((0..4).all(|c| !joined(&rig, c))); + for cpu in 0..4u8 { + store_as(&rig, cpu, 0x100 + u64::from(cpu)); + assert_eq!(scratch1(&rig), 0x100 + u64::from(cpu)); + assert!(joined(&rig, usize::from(cpu))); + } + } + + /// The block's reset, at power-on or from its soft-reset bit, starts a + /// new epoch: every processor's next store is a first store again, and + /// the joins are forgotten. + #[test] + fn each_block_reset_starts_a_new_barrier() { + let rig = Rig::new(); + let epoch = |late: u8| { + store_as(&rig, 1 - late, 0); + assert_eq!(scratch1(&rig), 0, "a first store"); + store_as(&rig, 1 - late, RELEASE); + store_as(&rig, late, 0); + assert_eq!(scratch1(&rig), RELEASE); + store_as(&rig, late, 0x100); + assert!(joined(&rig, usize::from(late))); + }; + epoch(1); + rig.chip.lb.power_on(&rig.chip.node); + assert!(!joined(&rig, 1)); + assert_eq!(scratch1(&rig), 0x100, "scratch 1 survives the reset"); + epoch(0); + rig.w(LB_BASE + reg::SOFT_RESET, 1); + assert!(!joined(&rig, 0)); + epoch(1); + } + + /// Stores no processor makes (a monitor, a test, a peer node's vector + /// write) always land and move nothing. + #[test] + fn a_store_from_no_processor_always_lands() { + let rig = Rig::new(); + for value in [RELEASE, 0, RELEASE, 0] { + rig.w(SCRATCH1_AT, value); + assert_eq!(scratch1(&rig), value); + } + store_as(&rig, 1, 0); + assert_eq!(scratch1(&rig), 0, "no release has landed from a processor"); + } + + /// (CPU, value stored, scratch 1 after) for the four-processor PROM's + /// stores to scratch 1 up to its flags word, in the order a Tezro boot + /// with every processor on time logged them reaching the LB: four + /// clears, CPU 0's release, each processor's join (a store of 0), then + /// each processor's flags word. + const ON_TIME_PROM_SEQUENCE: &[(u8, u64, u64)] = &[ + (0, 0, 0), (1, 0, 0), (3, 0, 0), (2, 0, 0), + (0, RELEASE, RELEASE), + (1, 0, 0), (0, 0, 0), (2, 0, 0), (3, 0, 0), + (3, FLAGS, FLAGS), (2, FLAGS, FLAGS), (1, FLAGS, FLAGS), (0, FLAGS, FLAGS), + ]; + + /// With every processor on time the rule changes nothing. + #[test] + fn the_on_time_prom_sequence_is_plain_storage() { + let rig = Rig::new(); + for &(cpu, value, after) in ON_TIME_PROM_SEQUENCE { + store_as(&rig, cpu, value); + assert_eq!(scratch1(&rig), after, "CPU {cpu} storing {value:#x}"); + } + assert!((0..4).all(|c| joined(&rig, c))); + } + + /// The same, from a four-processor Tezro boot in which CPUs 2 and 3 + /// cleared after CPU 0's release (and CPU 1 joined before either had + /// stored again). + const LATE_PROM_SEQUENCE: &[(u8, u64, u64)] = &[ + (1, 0, 0), (0, 0, 0), + (0, RELEASE, RELEASE), + (2, 0, RELEASE), (3, 0, RELEASE), + (1, 0, 0), (2, 0, 0), (3, 0, 0), (0, 0, 0), + (2, FLAGS, FLAGS), (0, FLAGS, FLAGS), (3, FLAGS, FLAGS), (1, FLAGS, FLAGS), + ]; + + /// Replayed, the late clears do not erase the release, and the late + /// processors read the release until their joins even after CPU 1's + /// join has replaced it. + #[test] + fn the_logged_late_prom_sequence_keeps_the_release() { + let rig = Rig::new(); + for (i, &(cpu, value, after)) in LATE_PROM_SEQUENCE.iter().enumerate() { + store_as(&rig, cpu, value); + assert_eq!(scratch1(&rig), after, "store {i}: CPU {cpu} storing {value:#x}"); + if i == 5 { + assert_eq!((read_as(&rig, 2), read_as(&rig, 3)), (RELEASE, RELEASE)); + } + } + assert!((0..4).all(|c| joined(&rig, c))); + } + + /// The soft-reset bit resets the block, as a power-on does, and asks the + /// board for a node reset once; the other bits do neither. + #[test] + fn the_soft_reset_bit_resets_the_block_and_asks_for_a_node_reset() { + let rig = Rig::new(); + rig.w(LB_BASE + reg::NODE_REGION, 5 << 32); + rig.w(LB_BASE + reg::PIO_WINDOW + 8, 42); + rig.w(LB_BASE + reg::REGISTER_C030, 0xdead); + rig.w(LB_BASE + reg::SOFT_RESET, 2); + assert!(rig.events.lock().try_recv().is_err()); + assert_eq!(rig.r(LB_BASE + reg::REGISTER_C030), 0xdead); + rig.w(LB_BASE + reg::SOFT_RESET, 1); + assert!(matches!(rig.events.lock().try_recv(), Ok(crate::traits::MachineEvent::HardReset))); + assert!(rig.events.lock().try_recv().is_err(), "one request"); + assert_eq!(rig.r(LB_BASE + reg::REGISTER_C030), 0); + assert_eq!(rig.r(LB_BASE + reg::PIO_WINDOW + 8), 42); + assert_eq!(rig.chip.node.load(Ordering::SeqCst), 0); + } + + impl Lb { + /// Hold the block's lock, for tests of the paths that must not take it. + pub(in crate::dev::bedrock) fn hold(&self) -> impl Sized + '_ { + self.regs.lock() + } + } +} diff --git a/src/dev/bedrock/md.rs b/src/dev/bedrock/md.rs new file mode 100644 index 000000000..d7fa279cc --- /dev/null +++ b/src/dev/bedrock/md.rs @@ -0,0 +1,1444 @@ +//! Bedrock's memory and directory controller: bank configuration, memory +//! self-test, and the directory, ECC and fetch-operation views. +//! +//! Memory is held by [`Memory`]. Cached, uncached and DMA accesses reach the +//! banks without this controller or its lock; the controller decodes the +//! banks from its configuration register, which [`Memory`] reads atomically. +//! CPU coherence requests and hardware protection enforcement are absent; +//! protection registers retain their words. Known unsupported writes are +//! counted and ignored, and unmapped accesses return a bus error. +//! +//! Plain register words, directory entries and reference counters, +//! protection entries and written ECC check bytes live in sparse maps. The +//! guest touches a few of the millions of possible entries, and an entry it +//! never wrote reads as its default, so a dense table would only cost memory. +//! +//! # What is approximated here +//! +//! Memory never has an error. The ECC view keeps the check byte software +//! writes for a doubleword and reads it back while the doubleword holds the +//! data it held then; once the data changes it reads the check byte the data +//! computes, as the controller's own write would have stored. Directory +//! entries are stored with the check bits the controller computes, unless +//! the directory configuration turns that off. Nothing compares stored +//! check bits with their data, so no access corrects a word, latches a +//! memory or directory error or interrupts a processor: the error registers +//! read zero and the ECC scrub window answers reads as the ECC view does and +//! refuses writes. PROM 6.210's memory test writes and reads back ECC check +//! bytes and directory entries this way, and in a boot of it and IRIX +//! 6.5.30 on the Fuel nothing reads memory whose stored check bits disagree +//! with its data. +//! +//! Ported from hw/misc/sgi-bedrock-md.c in jamesbraid/qemu-sgi-origami. + +use super::mem::{Memory, RamExtent, BANK_SPAN}; +use super::Bedrock; +use crate::traits::*; +use parking_lot::Mutex; +use std::collections::HashMap; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{Arc, Weak}; + +pub mod reg { + pub const CELL: u64 = 0x0; + pub const CONFIG: u64 = 0x8; + pub const ARBITRATION: u64 = 0x10; + pub const MIGRATION_CONFIG: u64 = 0x18; + /// Outgoing reply queue sizes: read-only, so a write changes nothing. + pub const REPLY_QUEUE_SIZE: u64 = 0x60; + pub const PERF_SELECT0: u64 = 0x10000; + pub const PERF_SELECT1: u64 = 0x10008; + pub const PERF_COUNTER0: u64 = 0x10010; + pub const PERF_COUNTER1: u64 = 0x10018; + pub const REFRESH: u64 = 0x20000; + /// Flash, UART and Synergy timing on the local "junk" bus. + pub const JUNK_BUS_TIMING: u64 = 0x20008; + pub const SELFTEST_COMMAND: u64 = 0x20030; + pub const SELFTEST_DATA: u64 = 0x20038; + pub const SELFTEST_ERROR: u64 = 0x20040; + pub const SELFTEST_STATUS: u64 = 0x20048; + pub const ISSUE_DEBUG: u64 = 0x20060; + pub const DIRECTORY_CONFIG: u64 = 0x40000; + pub const DIRECTORY_ERROR: u64 = 0x40010; + /// Reads the directory error record and clears it. + pub const DIRECTORY_ERROR_ALIAS: u64 = 0x40018; + pub const MIGRATION_DIFFERENCE: u64 = 0x40040; + pub const MIGRATION_VALUE: u64 = 0x40048; + /// Outgoing request queue sizes; the model keeps them fixed. + pub const REQUEST_QUEUE_SIZE: u64 = 0x40050; + pub const DIRECTORY_DEBUG: u64 = 0x40060; + pub const ECC_CONFIG: u64 = 0x60000; + pub const MEMORY_ERROR: u64 = 0x60010; + /// Reads the memory error record and clears it. + pub const MEMORY_ERROR_ALIAS: u64 = 0x60018; + pub const MEMORY_DEBUG: u64 = 0x60040; + + /// Reset sizes: 12 entries in each header field and 24 in each data field. + pub const REPLY_QUEUE_SIZES: u64 = 0x0018_1818_0c0c_0c00; + /// Reset sizes: four requests for each of the three queues. + pub const REQUEST_QUEUE_SIZES: u64 = 0x0404_0400; +} + +const CONFIG_MASK: u64 = 0xfbfbfbfb | (3 << 32) | (15 << 44) | (63 << 48) | (7 << 56) | (3 << 60); +const ECC_COLUMNS: [u8; 72] = [ + 0x13, 0x23, 0x43, 0x83, 0x2f, 0xf1, 0x0d, 0x07, 0xd0, 0x70, 0x4f, 0xf8, 0x61, 0x62, 0x64, 0x68, + 0x1c, 0x2c, 0x4c, 0x8c, 0x15, 0x25, 0x45, 0x85, 0x19, 0x29, 0x49, 0x89, 0x1a, 0x2a, 0x4a, 0x8a, + 0x51, 0x52, 0x54, 0x58, 0x91, 0x92, 0x94, 0x98, 0xa1, 0xa2, 0xa4, 0xa8, 0x31, 0x32, 0x34, 0x38, + 0x16, 0x26, 0x46, 0x86, 0x1f, 0xf2, 0x0b, 0x0e, 0xb0, 0xe0, 0x8f, 0xf4, 0xc1, 0xc2, 0xc4, 0xc8, + 1, 2, 4, 8, 16, 32, 64, 128, +]; +pub fn ecc_compute(mut value: u64) -> u8 { + let mut code = 0; + while value != 0 { + code ^= ECC_COLUMNS[value.trailing_zeros() as usize]; + value &= value - 1; + } + code +} + +struct State { + cell: u64, + config: u64, + refresh: u64, + directory_config: u64, + ecc_config: u64, + storage: HashMap, + directory: HashMap, + counters: HashMap, + protection: HashMap, + compat: HashMap, + /// Check bytes software wrote, by doubleword of the dense walk, each + /// with the data the doubleword held when it was written. + ecc: HashMap, + admitted: [u16; 2], + clock_ns: u64, + clock_anchor: Option, + selftest_deadline: Option, + selftest_result: bool, + selftest_status: u64, + selftest_error: u64, +} +impl Default for State { + fn default() -> Self { + Self { + cell: u64::MAX, + config: 0, + refresh: 0, + directory_config: 0, + ecc_config: 0, + storage: HashMap::new(), + directory: HashMap::new(), + counters: HashMap::new(), + protection: HashMap::new(), + compat: HashMap::new(), + ecc: HashMap::new(), + admitted: [0; 2], + clock_ns: 0, + clock_anchor: None, + selftest_deadline: None, + selftest_result: false, + selftest_status: 0, + selftest_error: 0, + } + } +} + +pub(super) struct Md { + memory: Arc, + clock: crate::dev::heart::Clock, + // RAM uses the banks directly. The registers, the diagnostic views and + // fetch operations take the state lock. + state: Mutex, + // A bank configuration write and a reset publish their memory maps in + // the order they changed the register. + remapping: Mutex<()>, + /// The cell register, published for the checks every register access + /// and the board's Hub-special windows make. Written under `state`. + cell: AtomicU64, + unsupported: AtomicU64, +} +#[derive(Clone, Copy)] +pub(super) enum View { + Registers, + Scrub, + Directory, + Fetchop, + Ecc, +} +/// One of MD's diagnostic views as a bus device. Weak: Physical, which the +/// chip owns, may hold it. +pub(super) struct MdBus { + chip: Weak, + view: View, +} + +impl Md { + /// A controller over `memory`, with `clock` timing its self-test. + pub(super) fn new(memory: Memory, clock: crate::dev::heart::Clock) -> Self { + Self { + memory: Arc::new(memory), + clock, + state: Mutex::new(State::default()), + remapping: Mutex::new(()), + cell: AtomicU64::new(State::default().cell), + unsupported: AtomicU64::new(0), + } + } + /// Where fitted RAM can answer, for the processor bus's cached view. + pub(super) fn ram_extent(&self) -> RamExtent { + self.memory.extent() + } + pub(super) fn state_locked(&self) -> bool { + self.state.is_locked() + } + pub(super) fn remapping_locked(&self) -> bool { + self.remapping.is_locked() + } + pub(super) fn physical(&self) -> &Arc { + self.memory.physical() + } + pub(super) fn view(&self, chip: Weak, view: View) -> Arc { + Arc::new(MdBus { chip, view }) + } + /// DMA uses the same Physical entries as the CPU, without accessing + /// ppmem's CPU-owned bitmap. Hardware protection remains unsupported. + pub(super) fn dma_view(&self) -> Arc { + Arc::new(DmaView(self.memory.clone())) + } + /// Uncached memory: the bank the configuration register places at an + /// address, with no lock and no allocation. + pub(super) fn uncached_view(&self) -> Arc { + Arc::new(UncachedView(self.memory.clone())) + } + + /// Reinstall RAM after the board sets Physical's default. Call on the + /// CPU thread or with the CPU stopped, before installing board overlays. + /// A configuration write rebuilds its own changed mapping. + pub(super) fn rebuild_memory(&self) { + let _remapping = self.remapping.lock(); + let config = self.state.lock().config; + self.memory.remap(config, true); + } + /// The stable logical-physical counters used by both compiled stores + /// and SysAd page claims. Their storage outlives the CPU via its bus. + #[cfg(feature = "jitv2")] + pub(super) fn generation_base(&self) -> *mut AtomicU64 { + self.memory.generations().base() + } + fn advance(s: &mut State, elapsed: u64) { + s.clock_ns = s.clock_ns.saturating_add(elapsed); + if s.selftest_deadline.is_some_and(|end| s.clock_ns >= end) { + s.selftest_deadline = None; + s.selftest_status = if s.selftest_result { 3 } else { 2 }; + s.selftest_error = if s.selftest_result { 0 } else { 1 << 48 }; + } + } + fn catch_up(s: &mut State, now: u64) { + if let Some(anchor) = s.clock_anchor { + s.clock_anchor = Some(anchor.max(now)); + Self::advance(s, now.saturating_sub(anchor)); + } + } + pub(super) fn start_clock(&self) { + let now = (self.clock)(); + self.state.lock().clock_anchor.get_or_insert(now); + } + pub(super) fn stop_clock(&self) { + let now = (self.clock)(); + let mut s = self.state.lock(); + Self::catch_up(&mut s, now); + s.clock_anchor = None; + } + /// Cell register bit 0, which gates the register window and the board's + /// Hub-special windows. One atomic load. + #[inline] + pub(super) fn cell_permits(&self) -> bool { + self.cell.load(Ordering::Relaxed) & 1 != 0 + } + /// Count and log an access the model does not implement; the caller + /// answers a bus error. + fn refuse(&self, what: &str, at: u64) -> Option { + self.unsupported.fetch_add(1, Ordering::Relaxed); + crate::dlog!(crate::devlog::LogModule::Bedrock, + "Bedrock MD refused {what} access at {at:#x}"); + None + } + /// An access through `view`. + pub(super) fn access(&self, view: View, addr: u64, bytes: u64, write: Option) -> Option { + if ![1, 2, 4, 8].contains(&bytes) || addr % bytes != 0 { + return self.refuse("access", addr); + } + // A bank configuration write remaps memory, on the CPU thread as + // HEART's MEMCFG does. + let reconfigure = write.is_some() && matches!(view, View::Registers) && addr == reg::CONFIG; + let remapping = reconfigure.then(|| self.remapping.lock()); + // Sample outside state: even an injected clock must not call out under + // the lock. Only register observations need the self-test timer. + let now = matches!(view, View::Registers).then(|| (self.clock)()); + let (result, config) = { + let mut s = self.state.lock(); + if let Some(now) = now { + Self::catch_up(&mut s, now); + } + let result = match view { + View::Registers => self.register(&mut s, addr, bytes, write), + View::Directory => self.directory(&mut s, addr, bytes, write), + View::Ecc => self.ecc(&mut s, addr, bytes, write), + View::Scrub if write.is_none() => self.ecc(&mut s, addr, bytes, None), + View::Scrub => self.refuse("scrub", addr), + View::Fetchop => self.fetchop(&mut s, addr, bytes, write), + }; + self.cell.store(s.cell, Ordering::Relaxed); + (result, s.config) + }; + if remapping.is_some() { + self.memory.remap(config, false); + } + result + } + fn register(&self, s: &mut State, at: u64, bytes: u64, write: Option) -> Option { + if bytes != 8 { + return self.refuse("register", at); + } + let old = match at { + reg::CELL => s.cell, + reg::CONFIG => s.config, + reg::REPLY_QUEUE_SIZE => reg::REPLY_QUEUE_SIZES, + reg::REFRESH => s.refresh, + reg::SELFTEST_ERROR => s.selftest_error, + reg::SELFTEST_STATUS => s.selftest_status, + reg::DIRECTORY_CONFIG => s.directory_config, + reg::ECC_CONFIG => s.ecc_config, + reg::REQUEST_QUEUE_SIZE => reg::REQUEST_QUEUE_SIZES, + // No error is ever latched: see the module notes. + reg::MEMORY_ERROR | reg::MEMORY_ERROR_ALIAS | reg::DIRECTORY_ERROR | reg::DIRECTORY_ERROR_ALIAS => 0, + _ if storage_mask(at).is_some() => *s.storage.get(&at).unwrap_or(&0), + _ if idle(at) => 0, + _ => return self.refuse("register", at), + }; + if let Some(value) = write { + match at { + reg::SELFTEST_COMMAND => return self.selftest(s, value), + reg::CELL => s.cell = value, + reg::CONFIG => s.config = value & CONFIG_MASK, + reg::REFRESH => s.refresh = value & ((1 << 63) | 0xfff), + reg::DIRECTORY_CONFIG => s.directory_config = value & 3, + reg::ECC_CONFIG => s.ecc_config = value & 1, + _ if storage_mask(at).is_some() => { + s.storage.insert(at, value & storage_mask(at).unwrap()); + } + _ => { + self.unsupported.fetch_add(1, Ordering::Relaxed); + crate::dlog!(crate::devlog::LogModule::Bedrock, + "Bedrock MD ignored register write at {at:#x}: {value:#x}"); + } + } + } + Some(old) + } + fn directory(&self, s: &mut State, at: u64, bytes: u64, write: Option) -> Option { + if bytes != 8 || at >= 1 << 30 { + return self.refuse("directory", at); + } + let software = at / (BANK_SPAN / 8); + let bank = software ^ ((s.config >> 32) & 3); + if at & 0x7c8 == 0x408 { + let fitted = self.memory.fitted(bank); + if fitted == 0 { + return Some(0); + } + let within = + ((((at % (BANK_SPAN / 8)) >> 11) << 14) | (((at >> 4) & 3) << 7)) % fitted; + let key = ((self.memory.dense_base(bank) + within) >> 14) * 4 + ((within >> 7) & 3); + let old = *s.protection.get(&key).unwrap_or(&0); + if let Some(v) = write { + s.protection.insert(key, v); + } + return Some(old); + } + let within = ((at % (BANK_SPAN / 8)) & !15) * 8; + let field = s.config >> (bank / 2 * 8); + let code = (field >> 4) & 15; + if field & (1 << (bank & 1)) == 0 + || !(1..=5).contains(&code) + || within >= (32 << 20) << code + { + return self.refuse("directory", at); + } + let mask = if at & 8 != 0 || s.directory_config & 1 != 0 { + u64::MAX + } else { + u32::MAX as u64 + }; + let table; + let key; + let fitted = self.memory.fitted(bank); + if within >= fitted { + if !s.compat.contains_key(&at) && s.compat.len() >= 74 { + return self.refuse("directory", at); + } + s.compat.entry(at).or_insert(0); + table = &mut s.compat; + key = at; + } else { + key = (self.memory.dense_base(bank) + within) / 128; + table = if at & 8 != 0 { + &mut s.counters + } else { + &mut s.directory + }; + } + let old = *table.get(&key).unwrap_or(&0) & mask; + if let Some(value) = write { + let value = if at & 8 == 0 && s.directory_config & 2 == 0 { + directory_encode(value, s.directory_config & 1 != 0) + } else { + value + }; + table.insert(key, value & mask); + } + Some(old) + } + /// A read or write of `bytes` at node address `address` in the bank the + /// configuration places there, for the fetch operations. + fn ram(&self, s: &State, address: u64, bytes: u64, write: Option) -> Option { + let Some((bank, offset)) = self.memory.decode(s.config, address, bytes) else { + return self.refuse("ram", address); + }; + let at = offset & !7; + let shift = (8 - (offset & 7) - bytes) * 8; + let mask = u64::MAX >> ((8 - bytes) * 8); + let old = self.memory.raw_read(bank, at); + if let Some(value) = write { + self.memory.raw_write(bank, at, (old & !(mask << shift)) | ((value & mask) << shift)); + Some(0) + } else { + Some((old >> shift) & mask) + } + } + fn fetchop(&self, s: &mut State, at: u64, bytes: u64, write: Option) -> Option { + if bytes != 4 && bytes != 8 { + return self.refuse("fetchop", at); + } + let op = at & 0x38; + if op > if write.is_some() { 32 } else { 24 } { + return self.refuse("fetchop", at); + } + let address = (at & !63) | (at & 4); + let old = self.ram(s, address, bytes, None)?; + let next = match (write, op) { + (None, 0) => return Some(old), + (Some(v), 0) => v, + (_, 8) => old.wrapping_add(1), + (_, 16) => old.wrapping_sub(1), + (None, 24) => 0, + (Some(v), 24) => old & v, + (Some(v), 32) => old | v, + _ => return self.refuse("fetchop", at), + }; + self.ram(s, address, bytes, Some(next))?; + Some(old) + } + // Approximation: report whether the selected fitting covers the + // programmed bank, after 1000 guest ns. Only the zero write phase mutates + // memory; other patterns have no published transformation here. + fn selftest(&self, s: &mut State, command: u64) -> Option { + if command == 2 || command == 4 { + s.selftest_deadline = None; + s.selftest_status = 0; + s.selftest_error = 0; + return Some(0); + } + if command & 7 != 1 || command & !0x71 != 0 { + self.unsupported.fetch_add(1, Ordering::Relaxed); + crate::dlog!(crate::devlog::LogModule::Bedrock, + "Bedrock MD ignored self-test command {command:#x}"); + return Some(0); + } + let selected = (command >> 4) & 7; + let configured = selected ^ ((s.config >> 32) & 3); + let field = s.config >> ((configured / 2) * 8); + let code = (field >> 4) & 15; + let size = if (1..=5).contains(&code) { + (32 << 20) << code + } else { + 0 + }; + let fits = field & (1 << (configured & 1)) != 0 + && size != 0 + && size <= self.memory.fitted(selected); + s.selftest_result = fits; + s.selftest_status = 0; + s.selftest_error = 0; + s.selftest_deadline = Some(s.clock_ns.saturating_add(1000)); + if fits && s.storage.get(®::SELFTEST_DATA).copied().unwrap_or(0) == 0x80 { + self.memory.zero(selected as usize, size); + // The zero fill writes every doubleword's data and check bits. + let start = self.memory.dense_base(selected) / 8; + s.ecc.retain(|&key, _| key < start || key >= start + size / 8); + } + Some(0) + } + fn ecc(&self, s: &mut State, at: u64, bytes: u64, write: Option) -> Option { + if at >= 1 << 30 || (bytes != 1 && bytes != 2) { + return self.refuse("ecc", at); + } + let software = at / (BANK_SPAN / 8); + let within = (at % (BANK_SPAN / 8)) ^ if bytes == 1 { 7 } else { 6 }; + let address = software * BANK_SPAN + within * 8; + let Some((bank, offset)) = self.memory.decode(s.config, address, bytes * 8) else { + if bytes == 2 && [0x04000006, 0x08000006].contains(&at) { + let i = usize::from(at == 0x08000006); + let old = s.admitted[i]; + if let Some(value) = write { + s.admitted[i] = value as u16; + } + return Some(old as u64); + } + return self.refuse("ecc", at); + }; + let mut old = 0; + for i in 0..bytes { + let raw = offset + i * 8; + let key = (self.memory.dense_base(bank as u64) + raw) / 8; + let data = self.memory.raw_read(bank, raw); + let computed = ecc_compute(data); + let stored = s.ecc.get(&key).filter(|&&(_, then)| then == data).map_or(computed, |&(code, _)| code); + old |= (stored as u64) << (i * 8); + if let Some(value) = write { + let code = (value >> (i * 8)) as u8; + if code == computed { + s.ecc.remove(&key); + } else { + s.ecc.insert(key, (code, data)); + } + } + } + Some(old) + } +} +impl Resettable for Md { + fn power_on(&self) { + // Reset is called on the CPU thread or with it stopped. + let _remapping = self.remapping.lock(); + let mut s = self.state.lock(); + *s = State::default(); + self.cell.store(s.cell, Ordering::Relaxed); + drop(s); + self.memory.remap(0, false); + } +} +/// Resolve each DMA access through the atomic Physical table, as HEART +/// does. Do not ask Physical for mem_ptr or gen_ptr: those read the inline +/// CPU bitmap. Default block methods split accesses at 4 KiB pages. +struct DmaView(Arc); +impl DmaView { + fn at(&self, address: u64, bytes: u64) -> Option<&dyn BusDevice> { + self.0.decode(self.0.config(), address, bytes)?; + let phys = self.0.physical(); + let entry = phys.entry_at(address); + if crate::ppmem::devmap::entry_eq(entry, phys.default_entry()) { + return None; + } + // SAFETY: the memory this view holds retains Physical, which keeps + // every entry alive across remaps. + Some(unsafe { &*entry }) + } +} +macro_rules! dma_rw { + ($read:ident, $write:ident, $result:ident, $ty:ty) => { + fn $read(&self, address: u64) -> $result { + self.at(address, std::mem::size_of::<$ty>() as u64) + .map_or($result::err(), |device| device.$read(address)) + } + fn $write(&self, address: u64, value: $ty) -> u32 { + self.at(address, std::mem::size_of::<$ty>() as u64) + .map_or(BUS_ERR, |device| { + let result = device.$write(address, value); + if result == BUS_OK { self.0.written(address); } + result + }) + } + }; +} +impl BusDevice for DmaView { + dma_rw!(read8, write8, BusRead8, u8); + dma_rw!(read16, write16, BusRead16, u16); + dma_rw!(read32, write32, BusRead32, u32); + dma_rw!(read64, write64, BusRead64, u64); + fn cas32(&self, address: u64, old: u32, new: u32) -> Option { + let swapped = self.at(address, 4)?.cas32(address, old, new)?; + if swapped { self.0.written(address); } + Some(swapped) + } + fn cas64(&self, address: u64, old: u64, new: u64) -> Option { + let swapped = self.at(address, 8)?.cas64(address, old, new)?; + if swapped { self.0.written(address); } + Some(swapped) + } + #[cfg(feature = "jitv2")] + fn gen_ptr(&self, address: u64) -> *const AtomicU64 { + self.0.gen_ptr(address) + } +} + +/// Uncached memory, as the processor's uncached memory space and the low +/// 256 MiB of its Hub-special space reach it: the bank the configuration +/// register places at the address, read and written directly. An address +/// no enabled bank holds answers a bus error. No lock and no allocation. +struct UncachedView(Arc); +macro_rules! uncached_rw { + ($read:ident, $write:ident, $result:ident, $ty:ty) => { + fn $read(&self, address: u64) -> $result { + match self.0.decode(self.0.config(), address, std::mem::size_of::<$ty>() as u64) { + Some((bank, offset)) => self.0.physical().banks()[bank].$read(offset), + None => $result::err(), + } + } + fn $write(&self, address: u64, value: $ty) -> u32 { + match self.0.decode(self.0.config(), address, std::mem::size_of::<$ty>() as u64) { + Some((bank, offset)) => { + let result = self.0.physical().banks()[bank].$write(offset, value); + if result == BUS_OK { self.0.written(address); } + result + } + None => BUS_ERR, + } + } + }; +} +impl BusDevice for UncachedView { + uncached_rw!(read8, write8, BusRead8, u8); + uncached_rw!(read16, write16, BusRead16, u16); + uncached_rw!(read32, write32, BusRead32, u32); + uncached_rw!(read64, write64, BusRead64, u64); + #[cfg(feature = "jitv2")] + fn gen_ptr(&self, address: u64) -> *const AtomicU64 { + self.0.gen_ptr(address) + } +} + +impl BusDevice for MdBus { + /// Bedrock's ECC view is the one device that sees the whole source register + /// of an uncached SH (`BusDevice::write_narrow_from`); every other view + /// takes the architectural low bits. + fn write_narrow_from(&self, addr: u64, size: usize, gpr: u64) -> u32 { + if size == 2 && matches!(self.view, View::Ecc) { + // PROM 6.210's memory test stores each check word with `sh` from a + // register whose low 16 bits are zero: the word sits elsewhere in + // the register, by the address of the memory it checks. Observed + // (memory address mod 0x40, carrier offset mod 8, register bits): + // 0x00 6 15:0 + // 0x10 4 63:48 + // 0x20 2 47:32 + // 0x30 0 31:16 + // The shift below gives exactly these four. Which lanes the R14000 + // drives for a narrow store, and how Bedrock wires them to the + // check bits, is unknown; this matches the PROM, not a hardware + // description. + let memory_address = (addr ^ 6).wrapping_mul(8); + return self.write16(addr, (gpr >> (memory_address.wrapping_neg() & 63)) as u16); + } + if size == 1 { self.write8(addr, gpr as u8) } else { self.write16(addr, gpr as u16) } + } + + fn read8(&self, a: u64) -> BusRead8 { + self.chip + .upgrade() + .and_then(|chip| chip.md.access(self.view, a, 1, None)) + .map_or(BusRead8::err(), |v| BusRead8::ok(v as u8)) + } + fn read16(&self, a: u64) -> BusRead16 { + self.chip + .upgrade() + .and_then(|chip| chip.md.access(self.view, a, 2, None)) + .map_or(BusRead16::err(), |v| BusRead16::ok(v as u16)) + } + fn read32(&self, a: u64) -> BusRead32 { + self.chip + .upgrade() + .and_then(|chip| chip.md.access(self.view, a, 4, None)) + .map_or(BusRead32::err(), |v| BusRead32::ok(v as u32)) + } + fn read64(&self, a: u64) -> BusRead64 { + self.chip + .upgrade() + .and_then(|chip| chip.md.access(self.view, a, 8, None)) + .map_or(BusRead64::err(), BusRead64::ok) + } + fn write8(&self, a: u64, v: u8) -> u32 { + self.chip + .upgrade() + .and_then(|chip| chip.md.access(self.view, a, 1, Some(v as u64))) + .map_or(BUS_ERR, |_| BUS_OK) + } + fn write16(&self, a: u64, v: u16) -> u32 { + self.chip + .upgrade() + .and_then(|chip| chip.md.access(self.view, a, 2, Some(v as u64))) + .map_or(BUS_ERR, |_| BUS_OK) + } + fn write32(&self, a: u64, v: u32) -> u32 { + self.chip + .upgrade() + .and_then(|chip| chip.md.access(self.view, a, 4, Some(v as u64))) + .map_or(BUS_ERR, |_| BUS_OK) + } + fn write64(&self, a: u64, v: u64) -> u32 { + self.chip + .upgrade() + .and_then(|chip| chip.md.access(self.view, a, 8, Some(v))) + .map_or(BUS_ERR, |_| BUS_OK) + } +} +fn storage_mask(at: u64) -> Option { + Some(match at { + reg::ARBITRATION => 0xff, + reg::MIGRATION_CONFIG => 0x3ff | (0xff << 16) | (1 << 32), + reg::PERF_SELECT0 | reg::PERF_SELECT1 => { + 3 | (15 << 4) | (127 << 8) | (15 << 16) | (0xffffffffff << 20) | (1 << 63) + } + reg::PERF_COUNTER0 | reg::PERF_COUNTER1 => (1 << 41) - 1, + reg::JUNK_BUS_TIMING => 0xffffffffffff, + reg::SELFTEST_DATA => 0xff, + reg::ISSUE_DEBUG => 3, + reg::MIGRATION_DIFFERENCE => 0x7fff | (7 << 32) | (7 << 35) | (1 << 63), + reg::MIGRATION_VALUE => 0x7fff | (7 << 32) | (7 << 35) | (3 << 62), + reg::DIRECTORY_DEBUG => 3, + reg::MEMORY_DEBUG => 1, + _ => return None, + }) +} +fn idle(at: u64) -> bool { + matches!( + at, + 0x20 | 0x28 + | 0x40 + | 0x48 + | 0x58 + | 0x20010 + | 0x20018 + | 0x20020 + | 0x20028 + | 0x20030 + | 0x20040 + | 0x20048 + | 0x40020 + | 0x40028 + | 0x40030 + | 0x40038 + | 0x40058 + | 0x60020 + | 0x60028 + | 0x60030 + | 0x60038 + ) +} + +#[cfg(test)] +mod tests { + // Cases follow tests/qtest/sn1-bedrock-md-test.c in + // jamesbraid/qemu-sgi-origami: memory-banks, reset, queue-sizes, + // writable-masks, selftest-phases, fetchop-operations, ecc-carrier and + // directory-storage. + use super::*; + use super::super::BedrockConfig; + + /// The controller inside a chip of its own, with the views MD hands out. + struct TestMd(Arc); + impl std::ops::Deref for TestMd { + type Target = Md; + fn deref(&self) -> &Md { + &self.0.md + } + } + impl TestMd { + fn new(banks: &[u32]) -> Result, String> { + Self::with_clock(banks, Arc::new(|| 0)) + } + fn with_clock(banks: &[u32], clock: crate::dev::heart::Clock) -> Result, String> { + let chip = Bedrock::new(BedrockConfig { banks_mib: banks, ..super::super::test_config(clock) })?; + Ok(Arc::new(Self(chip))) + } + fn md_bus(&self) -> Arc { + self.0.md.view(Arc::downgrade(&self.0), View::Registers) + } + fn uncached_view(&self) -> Arc { + self.0.uncached_view() + } + fn directory_bus(&self) -> Arc { + self.0.directory_bus() + } + fn ecc_bus(&self) -> Arc { + self.0.ecc_bus() + } + fn ecc_scrub_bus(&self) -> Arc { + self.0.ecc_scrub_bus() + } + fn fetchop_bus(&self) -> Arc { + self.0.fetchop_bus() + } + fn dma_view(&self) -> Arc { + self.0.dma_view() + } + /// Whether each PI has bank 1 bit 61 (a correctable memory error) + /// or error-summary bit 24 (an uncorrectable one) set. + fn reported(&self) -> [(bool, bool); 2] { + let regs = self.0.registers(); + std::array::from_fn(|n| { + let base = n as u64 * super::super::reg::PI1_BASE; + let pending = regs.read64(base + super::super::reg::pi::PENDING_STATUS_B).data; + let summary = regs.read64(base + super::super::reg::pi::ERROR_SUMMARY).data; + (pending & (1 << 61) != 0, summary & (1 << 24) != 0) + }) + } + } + impl Md { + /// Hold both locks, for tests of the paths that must not take them. + pub(in crate::dev::bedrock) fn hold(&self) -> impl Sized + '_ { + (self.remapping.lock(), self.state.lock()) + } + /// Explicit elapsed time, for tests that drive the self-test by hand. + fn advance_ns(&self, elapsed: u64) { + Self::advance(&mut self.state.lock(), elapsed); + } + fn unsupported(&self) -> u64 { + self.unsupported.load(Ordering::Relaxed) + } + } + const FUEL_BANKS: [u32; 2] = [256, 256]; + const FITTED_BANK: u64 = 256 << 20; + fn fuel() -> Arc { + TestMd::new(&FUEL_BANKS).unwrap() + } + fn configured() -> Arc { + let md = fuel(); + assert_eq!(md.md_bus().write64(8, 0x33), BUS_OK); + md + } + /// The Tezro's default 4 GiB: two 1 GiB banks and four of 512 MiB. + const TEZRO_BANKS: [u32; 6] = [1024, 1024, 512, 512, 512, 512]; + /// Banks 0 and 1 at size code 5 (1 GiB), the other four at code 4 (512 MiB). + const TEZRO_CONFIG: u64 = 0x53 | (0x43 << 8) | (0x43 << 16); + fn tezro() -> Arc { + let md = TestMd::new(&TEZRO_BANKS).unwrap(); + assert_eq!(md.md_bus().write64(reg::CONFIG, TEZRO_CONFIG), BUS_OK); + md + } + + /// Cached, uncached and DMA memory take no MD lock: each works while + /// both are held. An address no enabled bank holds answers a bus error + /// on each. + #[test] + fn cached_uncached_and_dma_memory_do_not_take_the_md_locks() { + let md = configured(); + let dma = md.dma_view(); + let uncached = md.uncached_view(); + let _held = md.hold(); + assert_eq!(md.physical().write64(0x1000, 0x1122334455667788), BUS_OK); + assert_eq!(md.physical().read64(0x1000).data, 0x1122334455667788); + assert_eq!(dma.read64(0x1000).data, 0x1122334455667788); + assert_eq!(dma.write32(0x1004, 0xabcdef01), BUS_OK); + assert_eq!(md.physical().read64(0x1000).data, 0x11223344abcdef01); + assert_eq!(uncached.read32(0x1004).data, 0xabcdef01); + assert_eq!(uncached.write8(0x1000, 0x99), BUS_OK); + assert_eq!(uncached.write16(0x1002, 0x7766), BUS_OK); + assert_eq!(md.physical().read64(0x1000).data, 0x99227766abcdef01); + assert_eq!(uncached.read64(BANK_SPAN + 0x1000).data, 0); + for view in [&dma, &uncached] { + assert_eq!(view.read64(FITTED_BANK).status, BUS_ERR); + assert_eq!(view.write64(2 * BANK_SPAN, 1), BUS_ERR); + } + } + + /// Guest code must not keep a compiled page after its backing bank changes. + /// page_for deliberately retains its generation pointer. A remap must + /// invalidate that page and subsequent writes to its new bank must keep + /// invalidating the same pointer, including after swapping back. + #[cfg(feature = "jitv2")] + #[test] + fn a_claimed_jit_page_tracks_physical_ownership_across_bank_swaps() { + use crate::cpu::jitv2::jitv2::Jitv2; + let md = configured(); + let dma = md.dma_view(); + let mut jit = Jitv2::new(4); + let slot = jit.page_for(0, 0, dma.as_ref(), false).unwrap(); + let page = jit.page_ptr(slot); + let generation = || unsafe { (*page).current_gen() }; + let pointer = dma.gen_ptr(0); + let mut entries = [0u64; crate::cpu::jitv2::jitv2::BITMAP_WORDS]; + entries[0] = 1; + // Exercise the JIT publication/invalidation protocol without + // executing a synthetic function body. + let code = std::ptr::NonNull::<()>::dangling().as_ptr() as *const (); + assert!(unsafe { (*page).publish(&entries, code, generation(), 1, 0, false) }); + assert!(unsafe { (*page).is_runnable(0) }); + let first = generation(); + md.md_bus().write64(reg::CONFIG, 0x33 | (1 << 32)); + assert_eq!(jit.page_for(0, 0, dma.as_ref(), false), Some(slot)); + assert_eq!(dma.gen_ptr(0), pointer); + assert!(generation() > first); + assert!(!unsafe { (*page).is_runnable(0) }); + let swapped = generation(); + assert!(unsafe { (*page).publish(&entries, code, swapped, 1, 0, false) }); + assert!(unsafe { (*page).is_runnable(0) }); + assert_eq!(dma.write64(0, 0x11223344), BUS_OK); + assert!(generation() > swapped); + assert!(!unsafe { (*page).is_runnable(0) }, "code published after a swap sees new-bank writes"); + assert_eq!(md.physical().banks()[1].read64(0).data, 0x11223344); + let written = generation(); + md.md_bus().write64(reg::CONFIG, 0x33); + assert!(generation() > written, "swapping back cannot lower the logical generation"); + let restored = generation(); + md.uncached_view().write64(0, 0x55667788); + assert!(generation() > restored); + let before_fetchop = generation(); + md.fetchop_bus().write64(0, 0); + assert!(generation() > before_fetchop); + md.md_bus().write64(reg::CONFIG, 0x13); + md.md_bus().write64(0x20038, 0x80); + let before_zero = generation(); + md.md_bus().write64(reg::SELFTEST_COMMAND, 1); + assert!(generation() > before_zero, "zero self-test invalidates logical code pages"); + let before_disable = generation(); + md.power_on(); + assert!(generation() > before_disable); + assert_eq!(dma.gen_ptr(0), pointer, "a disabled bank keeps its logical counter alive"); + } + + /// Directory and ECC state is keyed per bank, so banks of different sizes + /// never share an entry. Follows directory-bank-translation. + #[test] + fn directory_entries_of_different_sized_banks_stay_apart() { + let md = TestMd::new(&[256, 256, 64, 64]).unwrap(); + md.md_bus().write64(reg::CONFIG, 0x33 | (0x13 << 8)); + md.md_bus().write64(reg::DIRECTORY_CONFIG, 2); + let dir = md.directory_bus(); + // Software bank n's carriers start at n * 128 MiB. + let at = |bank: u64, offset: u64| bank * (BANK_SPAN / 8) + offset; + for bank in 0..4 { + assert_eq!(dir.write64(at(bank, 0x20), 0x1000 + bank), BUS_OK, "bank {bank}"); + } + for bank in 0..4 { + assert_eq!(dir.read64(at(bank, 0x20)).data, 0x1000 + bank, "bank {bank}"); + } + // A 64 MiB bank's directory ends at its size code's reach. + assert_eq!(dir.read64(at(2, 0x20) + (64 << 20) / 8).status, BUS_ERR); + assert_eq!(dir.read64(at(4, 0x20)).status, BUS_ERR, "no bank 4"); + } + + /// The board names what it fits; nothing else is accepted. A bank is a + /// whole number of direct-window regions and at most the largest size + /// code names, and the controller decodes eight. + #[test] + fn a_board_may_only_fit_banks_the_controller_can_decode() { + assert!(TestMd::new(&FUEL_BANKS).is_ok()); + assert!(TestMd::new(&TEZRO_BANKS).is_ok()); + assert!(TestMd::new(&[1024; 8]).is_ok()); + for banks in [&[][..], &[0, 256], &[2048], &[100], &[64; 9]] { + assert!(TestMd::new(banks).is_err(), "{banks:?}"); + } + } + + /// Where RAM can be, for a board's cached view: the swizzle permutes slots + /// within groups of four, so a slot reaches as far as the largest bank of + /// its group. + #[test] + fn the_ram_extent_reaches_as_far_as_the_largest_bank_of_a_slot_group() { + let extent = RamExtent::new(&FUEL_BANKS); + for slot in 0..4 { + assert!(extent.contains(slot * BANK_SPAN + FITTED_BANK - 8), "slot {slot}"); + assert!(!extent.contains(slot * BANK_SPAN + FITTED_BANK), "slot {slot}"); + } + assert!(!extent.contains(4 * BANK_SPAN)); + let mixed = RamExtent::new(&[64, 512]); + assert!(mixed.contains(3 * BANK_SPAN + (512 << 20) - 8), "slot 3 may hold the 512 MiB bank"); + assert!(!mixed.contains(512 << 20)); + assert_eq!(extent, fuel().ram_extent()); + let tezro = RamExtent::new(&TEZRO_BANKS); + assert!(tezro.contains(3 * BANK_SPAN + BANK_SPAN - 8), "slot 3 may hold a 1 GiB bank"); + assert!(tezro.contains(5 * BANK_SPAN + (512 << 20) - 8)); + assert!(!tezro.contains(5 * BANK_SPAN + (512 << 20))); + assert!(!tezro.contains(8 * BANK_SPAN)); + assert_eq!(tezro, TestMd::new(&TEZRO_BANKS).unwrap().ram_extent()); + } + + /// Each Tezro bank answers in its own 1 GiB slot at its own size: a + /// 1 GiB bank fills its slot, a 512 MiB bank stops halfway, and the + /// slots above bank 5 answer nothing. Banks 4 and 5 sit above 4 GiB, + /// inside the node's 16 GiB direct window. Follows the qtest memory-banks and + /// memory-size-boundaries cases in tests/qtest/sn1-bedrock-md-test.c. + #[test] + fn every_tezro_bank_answers_at_its_own_size_including_above_4_gib() { + use crate::ppmem::MappedMemory; + let md = tezro(); + let bus = md.physical(); + let sizes = [1024u64, 1024, 512, 512, 512, 512]; + for (slot, mib) in sizes.iter().enumerate() { + let base = slot as u64 * BANK_SPAN; + let size = mib << 20; + assert_eq!(bus.write64(base, 0x1111 * (slot as u64 + 1)), BUS_OK, "slot {slot}"); + assert_eq!(bus.write64(base + size - 8, 0xe0 + slot as u64), BUS_OK, "slot {slot} top"); + if size < BANK_SPAN { + assert_eq!(bus.read64(base + size).status, BUS_ERR, "slot {slot} past its bank"); + } + } + for slot in 0..6u64 { + assert_eq!(bus.read64(slot * BANK_SPAN).data, 0x1111 * (slot + 1), "slot {slot}"); + let top = (sizes[slot as usize] << 20) - 8; + assert_eq!(bus.read64(slot * BANK_SPAN + top).data, 0xe0 + slot, "slot {slot} top"); + } + for slot in [6, 7] { + assert_eq!(bus.read64(slot * BANK_SPAN).status, BUS_ERR, "slot {slot} holds no bank"); + } + assert_eq!(bus.banks()[4].read64(0).data, 0x1111 * 5, "bank 4 holds what slot 4 wrote"); + assert_eq!(bus.banks()[5].read64(0).data, 0x1111 * 6); + // The 16 GiB window maps every bank, in 256 MiB regions. + let space = bus.ppmem_space().expect("a window"); + assert_eq!(space.window_size(), crate::ppmem::LARGE_WINDOW_SIZE); + assert_eq!(space.mapped_bitmap(), 0xf | (0xf << 4) | (0x3 << 8) | (0x3 << 12) | (0x3 << 16) | (0x3 << 20)); + assert_eq!(md.dma_view().read64(5 * BANK_SPAN).data, 0x1111 * 6, "DMA reaches it too"); + assert_eq!(md.uncached_view().read64(4 * BANK_SPAN).data, 0x1111 * 5, "and uncached"); + } + + /// A node whose banks all lie below 4 GiB, as the Fuel's do, keeps the + /// 4 GiB window and its 64 MiB regions. + #[test] + fn a_node_below_4_gib_keeps_the_4_gib_window() { + use crate::ppmem::MappedMemory; + let md = configured(); + let space = md.physical().ppmem_space().expect("a window"); + assert_eq!(space.window_size(), crate::ppmem::WINDOW_SIZE); + assert_eq!(space.mapped_bitmap(), 0xf | (0xf << 16)); + } + + /// A processor's SC to a bank above 4 GiB swaps through the DMA view in one step and leaves a failed compare + /// alone; a successful one advances the page's logical counter. + #[test] + fn a_compare_and_swap_above_4_gib_is_one_step() { + let md = tezro(); + let view = md.dma_view(); + let at = 5 * BANK_SPAN + 0x40; + assert_eq!(view.write64(at, 7), BUS_OK); + #[cfg(feature = "jitv2")] + let before = unsafe { (*view.gen_ptr(at)).load(Ordering::Relaxed) }; + assert_eq!(view.cas64(at, 6, 9), Some(false)); + assert_eq!(view.read64(at).data, 7); + assert_eq!(view.cas64(at, 7, 9), Some(true)); + assert_eq!(view.read64(at).data, 9); + assert_eq!(view.cas32(at + 4, 9, 3), Some(true)); + assert_eq!(view.read64(at).data, 3); + #[cfg(feature = "jitv2")] + assert!(unsafe { (*view.gen_ptr(at)).load(Ordering::Relaxed) } > before); + assert_eq!(view.cas64(7 * BANK_SPAN, 0, 1), None, "no bank there"); + } + + /// Selector 1 swaps the two banks of each pair, so the 512 MiB banks 4 and 5 + /// trade places above 4 GiB. Follows memory-selector-high-bank. + #[test] + fn the_bank_select_swaps_banks_above_4_gib() { + let md = tezro(); + let bus = md.physical(); + assert_eq!(bus.write64(4 * BANK_SPAN + 0x28, 0xfedc_ba98_7654_3210), BUS_OK); + assert_eq!(bus.read64(5 * BANK_SPAN + 0x28).data, 0); + assert_eq!(md.md_bus().write64(reg::CONFIG, TEZRO_CONFIG | (1 << 32)), BUS_OK); + assert_eq!(bus.read64(5 * BANK_SPAN + 0x28).data, 0xfedc_ba98_7654_3210); + assert_eq!(bus.read64(4 * BANK_SPAN + 0x28).data, 0); + } + + /// A fitting that covers the bank under test zero-fills it, whichever slot + /// the bank is in; a bank that is not fitted fails. Follows the + /// selftest-bank-remap and origin300-selftest-fill cases. + #[test] + fn selftest_fills_a_tezro_bank_above_4_gib_and_fails_a_missing_bank() { + let md = TestMd::new(&TEZRO_BANKS).unwrap(); + // Every pair at size code 1 (64 MiB): small enough to fill in a test. + let regs = md.md_bus(); + regs.write64(reg::CONFIG, 0x13 | (0x13 << 8) | (0x13 << 16) | (0x13 << 24)); + let bus = md.physical(); + bus.write64(4 * BANK_SPAN + 0x40, 0x5555); + regs.write64(reg::SELFTEST_DATA, 0x80); + regs.write64(reg::SELFTEST_COMMAND, (4 << 4) | 1); + md.advance_ns(1000); + assert_eq!(regs.read64(reg::SELFTEST_STATUS).data, 3, "bank 4 is fitted"); + assert_eq!(bus.read64(4 * BANK_SPAN + 0x40).data, 0, "and zeroed"); + regs.write64(reg::SELFTEST_COMMAND, (6 << 4) | 1); + md.advance_ns(1000); + assert_eq!(regs.read64(reg::SELFTEST_STATUS).data, 2, "bank 6 is not fitted"); + } + + /// Code pages above 4 GiB have stable logical generation counters like + /// the rest, and a write bumps them. + #[cfg(feature = "jitv2")] + #[test] + fn code_pages_above_4_gib_keep_a_logical_generation() { + let md = tezro(); + let above = 4 * BANK_SPAN + 0x2000; + let pointer = md.dma_view().gen_ptr(above); + assert!(!pointer.is_null()); + assert_ne!(pointer, md.dma_view().gen_ptr(5 * BANK_SPAN + 0x2000)); + assert!(md.dma_view().gen_ptr(8 * BANK_SPAN).is_null(), "past the node's 8 GiB"); + assert!(md.dma_view().gen_ptr(4 * BANK_SPAN + (512 << 20)).is_null(), "past the largest bank there"); + let before = unsafe { (*pointer).load(Ordering::Relaxed) }; + assert_eq!(md.dma_view().write64(above, 1), BUS_OK); + assert!(unsafe { (*pointer).load(Ordering::Relaxed) } > before); + md.md_bus().write64(reg::CONFIG, TEZRO_CONFIG | (1 << 32)); + assert_eq!(md.dma_view().gen_ptr(above), pointer, "a bank swap keeps the counter"); + } + + /// Guest density settings remap fitted RAM, and reset restores the default map. + #[test] + fn bank_configuration_remaps_memory_and_reset_restores_defaults() { + let md = configured(); + let bus = md.physical(); + assert_eq!(bus.write64(0, 0x1122334455667788), BUS_OK); + assert_eq!(bus.write64(BANK_SPAN, 0xabcdef), BUS_OK); + assert_eq!(bus.read8(0).data, 0x11); + assert_eq!(bus.read32(4).data, 0x55667788); + assert_eq!(bus.read64(FITTED_BANK).status, BUS_ERR); + assert_eq!(bus.read64(2 * BANK_SPAN).status, BUS_ERR); + assert_eq!(md.md_bus().write64(8, 0x33 | 1 << 32), BUS_OK); + assert_eq!(bus.read64(0).data, 0xabcdef); + assert_eq!(bus.read64(BANK_SPAN).data, 0x1122334455667788); + assert!(bus.mem_ptr(0).is_some()); + md.power_on(); + assert_eq!(bus.read64(0).status, BUS_ERR); + bus.map_unmapped(0x1fc00000, 0x1fc01000); + for (code, size) in [(1, 64 << 20), (2, 128 << 20), (3, FITTED_BANK), + (4, FITTED_BANK), (5, FITTED_BANK)] { + md.md_bus().write64(reg::CONFIG, 1 | (code << 4)); + assert_eq!(bus.read64(size - 8).status, BUS_OK); + assert_eq!(bus.read64(size).status, BUS_ERR); + assert_eq!(bus.read64(0x1fc00000).status, BUS_OK, "a non-RAM mapping survives"); + } + md.md_bus().write64(reg::CONFIG, 0x11); + assert_eq!(bus.read64(64 << 20).status, BUS_ERR, "the old prefix is removed on shrink"); + md.md_bus().write64(reg::CONFIG, 0x61); + assert_eq!(bus.read64(0).status, BUS_ERR, "unsupported density does not map RAM"); + #[cfg(feature = "jitv2")] + { + let md = configured(); + let ram = md.physical(); + let first = ram.gen_ptr(0); + let second = ram.gen_ptr(BANK_SPAN); + assert!(!first.is_null() && !second.is_null()); + assert_ne!(first, second); + let before = unsafe { (*first).load(Ordering::Relaxed) }; + ram.write64(0, 0x55); + assert!(unsafe { (*first).load(Ordering::Relaxed) } > before); + md.md_bus().write64(8, 0x33 | (1 << 32)); + // The window address stays fixed while its backing bank changes. + assert_eq!(ram.gen_ptr(0), first); + assert_eq!(ram.gen_ptr(BANK_SPAN), second); + let bank0 = ram.banks()[0].gen_ptr(0); + let bank1 = ram.banks()[1].gen_ptr(0); + let before0 = unsafe { (*bank0).load(Ordering::Relaxed) }; + let before1 = unsafe { (*bank1).load(Ordering::Relaxed) }; + assert_eq!(ram.write64(0, 0xaa), BUS_OK); + assert_eq!(unsafe { (*bank0).load(Ordering::Relaxed) }, before0); + assert_eq!(unsafe { (*bank1).load(Ordering::Relaxed) }, before1 + 1); + assert_eq!(unsafe { (*first).load(Ordering::Relaxed) }, before1 + 1); + let view = md.uncached_view(); + let logical = view.gen_ptr(0); + assert_eq!(logical, md.memory.gen_ptr(0)); + let logical_before = unsafe { (*logical).load(Ordering::Relaxed) }; + drop(md); + assert_eq!(unsafe { (*logical).load(Ordering::Relaxed) }, logical_before, + "an external endpoint keeps its logical generation pointers alive"); + // The endpoint keeps the backing memory alive after MD disappears. + assert!(unsafe { (*bank1).load(Ordering::Relaxed) } > 0); + } + } + /// The guest reads ECC configuration and the error registers, which never + /// latch an error: an injected check byte read past with memory raises + /// none, and reaches no processor. + #[test] + fn guest_registers_expose_ecc_configuration_and_latch_no_error() { + { + let md = configured(); + let r = md.md_bus(); + // These carrier offsets are idle; they do not select storage. + for at in [0x20, 0x20010, 0x40020, 0x60020] { + let before = md.unsupported(); + assert_eq!(r.write64(at, u64::MAX), BUS_OK, "{at:#x}"); + assert_eq!(r.read64(at).data, 0, "{at:#x}"); + assert_eq!(md.unsupported(), before + 1); + } + for (at, value) in [(0x60, 0x001818180c0c0c00), (0x40050, 0x04040400)] { + let before = md.unsupported(); + assert_eq!(r.write64(at, 0), BUS_OK); + assert_eq!(r.read64(at).data, value); + assert_eq!(md.unsupported(), before + 1); + } + md.ecc_bus().write8(7, 3); + assert_eq!(md.uncached_view().read64(0).status, BUS_OK); + r.write64(0x40000, 2); + md.directory_bus().write64(0, 3 << 16); + r.write64(0x40000, 0); + assert_eq!(md.physical().read64(0).status, BUS_OK); + assert_eq!(md.dma_view().read64(0).status, BUS_OK); + for at in [0x40010, 0x40018, 0x60010, 0x60018] { + let before = md.unsupported(); + assert_eq!(r.write64(at, u64::MAX), BUS_OK); + assert_eq!(r.read64(at).data, 0, "{at:#x}"); + assert_eq!(md.unsupported(), before + 1); + } + assert_eq!(md.reported(), [(false, false); 2]); + // An unsupported self-test command leaves an in-flight command alone. + r.write64(0x20030, 1); + let before = md.unsupported(); + assert_eq!(r.write64(0x20030, 3), BUS_OK); + assert_eq!(md.unsupported(), before + 1); + md.advance_ns(1000); + assert_eq!(r.read64(0x20048).data, 3); + assert_eq!(r.write64(0x70000, 1), BUS_ERR); + assert_eq!(r.read64(0x70000).status, BUS_ERR); + assert_eq!(r.write32(0x60, 0), BUS_ERR); + assert_eq!(r.write64(0x61, 0), BUS_ERR); + } + { + let md = configured(); + let regs = md.md_bus(); + assert_eq!(regs.read64(0x60000).data, 0); + assert_eq!(regs.write64(0x60000, u64::MAX), BUS_OK); + assert_eq!(regs.read64(0x60000).data, 1); + assert_eq!(regs.write64(0x60000, !1), BUS_OK); + assert_eq!(regs.read64(0x60000).data, 0); + assert_eq!(regs.write8(0x60000, 1), BUS_ERR); + assert_eq!(regs.write64(0x60001, 1), BUS_ERR); + regs.write64(0x60000, 1); + md.power_on(); + assert_eq!(regs.read64(0x60000).data, 0); + assert_eq!(regs.read64(8).data, 0); + } + } + /// Self-test completion is visible only after its configured clock interval. + #[test] + fn selftest_waits_for_fitted_clock_cycles_and_zero_fills_ram() { + { + let now = Arc::new(AtomicU64::new(100)); + let time = now.clone(); + let md = TestMd::with_clock(&FUEL_BANKS, Arc::new(move || time.load(Ordering::Relaxed))).unwrap(); + let regs = md.md_bus(); + regs.write64(8, 0x33); + md.start_clock(); + regs.write64(0x20030, 1); + now.store(1099, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 0); + now.store(1100, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 3); + now.store(5000, Ordering::Relaxed); + regs.write64(0x20030, 1); + now.store(5500, Ordering::Relaxed); + md.stop_clock(); + now.store(15000, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 0); + md.start_clock(); + now.store(15499, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 0); + now.store(15500, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 3); + regs.write64(0x20030, 1); + now.store(15400, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 0); + now.store(16499, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 0); + now.store(16500, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 3); + regs.write64(0x20030, 1); + regs.write64(0x20030, 2); + now.store(20000, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 0); + md.power_on(); + regs.write64(8, 0x33); + md.start_clock(); + regs.write64(0x20030, 1); + now.store(20999, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 0); + now.store(21000, Ordering::Relaxed); + assert_eq!(regs.read64(0x20048).data, 3); + } + { + let md = configured(); + let r = md.md_bus(); + r.write64(0x20030, 0x21); + md.advance_ns(1000); + assert_eq!(r.read64(0x20048).data, 2); + assert_eq!(r.read64(0x20040).data, 1 << 48); + r.write64(0x20030, 1); + r.write64(0x20030, 2); + md.advance_ns(1000); + assert_eq!(r.read64(0x20048).data, 0); + assert_eq!(r.write64(0x20030, 3), BUS_OK); + } + { + // The zero fill writes the check bits too, even where the data + // was already zero. + let md = configured(); + let r = md.md_bus(); + r.write64(8, 0x13); + md.physical().write64(0, 0x1234); + md.ecc_bus().write8(7, 1); + md.ecc_bus().write8(15, 1); + r.write64(0x20038, 0x80); + r.write64(0x20030, 1); + md.advance_ns(1000); + assert_eq!(r.read64(0x20048).data, 3); + assert_eq!(md.physical().read64(0).data, 0); + assert_eq!(md.ecc_bus().read8(7).data, 0); + assert_eq!(md.ecc_bus().read8(15).data, 0); + } + } + /// Fetch-and-op registers preserve the big-endian lanes used by the processor. + #[test] + fn fetchop_preserves_big_endian_word_lanes() { + let md = configured(); + let ram = md.physical(); + let f = md.fetchop_bus(); + ram.write64(0, 0xffffffff00000009); + assert_eq!(f.read32(8).data, u32::MAX); + assert_eq!(ram.read64(0).data, 9); + assert_eq!(f.read32(12).data, 9); + assert_eq!(ram.read64(0).data, 10); + assert_eq!(f.write32(28, 3), BUS_OK); + assert_eq!(ram.read64(0).data, 2); + assert_eq!(f.write64(32, 8), BUS_OK); + assert_eq!(ram.read64(0).data, 10); + assert_eq!(f.read64(24).data, 10); + assert_eq!(ram.read64(0).data, 0); + assert_eq!(f.read64(32).status, BUS_ERR); + assert_eq!(f.write64(40, 1), BUS_ERR); + assert_eq!(f.read16(0).status, BUS_ERR); + } + /// The guest reads back the check bytes it writes, until a doubleword's + /// data changes and it reads the check byte the new data computes. The + /// scrub window reads as the ECC view and refuses writes. + #[test] + fn ecc_carriers_keep_written_check_bytes_until_the_data_changes() { + { + let md = configured(); + let ecc = md.ecc_bus(); + for carrier in 0..8 { + let source = 0x11223344556677a5u64; + assert_eq!(ecc.write8(carrier, source as u8), BUS_OK); + assert_eq!(ecc.read8(carrier).data, source as u8, "carrier {carrier}"); + } + assert_eq!(ecc.write16(4, 0x1234), BUS_OK); + assert_eq!(ecc.read16(4).data, 0x1234); + assert_eq!(ecc.write16(6, 0x2313), BUS_OK); + assert_eq!(ecc.read16(6).data, 0x2313); + assert_eq!(ecc.write16(0x04000006, 0xface), BUS_OK); + assert_eq!(ecc.read16(0x04000006).data, 0xface); + } + { + let md = configured(); + let (ecc, ram) = (md.ecc_bus(), md.uncached_view()); + assert_eq!(ecc.read8(7).data, ecc_compute(0), "nothing written: the data's own"); + ecc.write8(7, 3); + assert_eq!(ecc.read8(7).data, 3); + assert_eq!(md.ecc_scrub_bus().read8(7).data, 3); + assert_eq!(ram.read64(0).data, 0, "memory reads raise no error"); + assert_eq!(ram.write8(7, 0x5a), BUS_OK); + assert_eq!(ecc.read8(7).data, 0x82, "the new data's check byte"); + assert_eq!(ecc_compute(0x5a), 0x82); + ecc.write8(7, 0x82); + assert!(md.state.lock().ecc.is_empty(), "a computed byte is not kept"); + assert_eq!(md.ecc_scrub_bus().write64(0, 0), BUS_ERR); + assert_eq!(md.reported(), [(false, false); 2]); + } + } + /// The ECC view takes an uncached SH's check word from wherever the PROM + /// put it in the source register; other views, and bytes, take the low bits. + #[test] + fn ecc_view_reads_the_check_word_from_the_whole_source_register() { + let md = configured(); + let ecc = md.ecc_bus(); + // (carrier offset, register bits holding the check word) + for (carrier, shift) in [(6u64, 0u32), (4, 48), (2, 32), (0, 16)] { + let source = 0x1111_2222_3333_4444u64 ^ (0xa55a_u64 << shift); + let word = (source >> shift) as u16; + assert_eq!(ecc.write_narrow_from(carrier, 2, source), BUS_OK); + assert_eq!(ecc.read16(carrier).data, word, "carrier {carrier}"); + } + assert_eq!(ecc.write_narrow_from(5, 1, 0xfeed_0000_0000_0023), BUS_OK); + assert_eq!(ecc.read8(5).data, 0x23, "a byte keeps the architectural low bits"); + // The memory view is not the ECC view: its halfword is the low bits. + let view = md.uncached_view(); + assert_eq!(view.write_narrow_from(0x40, 2, 0xdead_beef_1234_5678), BUS_OK); + assert_eq!(md.dma_view().read64(0x40).data >> 48, 0x5678); + } + /// Directory entries are stored with the check bits the controller + /// computes unless the configuration turns that off, and read back as + /// stored; protection entries and counters keep their words. + #[test] + fn directory_entries_and_protection_words_are_stored() { + { + let md = configured(); + let r = md.md_bus(); + md.directory_bus().write64(0, 1); + assert_eq!(md.directory_bus().read64(0).data, 0x001f0001); + r.write64(0x40000, 2); + md.directory_bus().write64(0, 0x001e0001); + r.write64(0x40000, 0); + assert_eq!(md.directory_bus().read64(0).data, 0x001e0001); + assert_eq!(md.uncached_view().read64(0).status, BUS_OK); + assert_eq!(r.read64(0x40010).data, 0); + } + { + let md = configured(); + let d = md.directory_bus(); + assert_eq!(d.write64(0x408, 0x1234), BUS_OK); + assert_eq!(d.read64(0x2000408).data, 0x1234); + assert_eq!(d.write64(0x418, 0x5678), BUS_OK); + assert_eq!(d.read64(0x408).data, 0x1234); + assert_eq!(d.read64(0x418).data, 0x5678); + assert_eq!(d.write64(8, u64::MAX), BUS_OK); + assert_eq!(d.read64(8).data, u64::MAX); + assert_eq!(d.read32(0).status, BUS_ERR); + assert_eq!(d.read64(1 << 30).status, BUS_ERR); + } + } +} + +const DIRECTORY_PREMIUM: [u8; 64] = [ + 0x1f, 0x2f, 0x37, 0x3b, 0x3d, 0x3e, 0x07, 0x0b, 0x0d, 0x0e, 0x13, 0x15, 0x16, 0x19, 0x1a, 0x1c, + 0x7f, 0x4f, 0x57, 0x5b, 0x5d, 0x5e, 0x67, 0x6b, 0x6d, 0x6e, 0x73, 0x75, 0x76, 0x79, 0x7a, 0x7c, + 1, 2, 4, 8, 16, 32, 0x23, 0x25, 0x26, 0x29, 0x2a, 0x2c, 0x31, 0x32, 0x34, 0x38, 0x40, 0x43, + 0x45, 0x46, 0x49, 0x4a, 0x4c, 0x51, 0x52, 0x54, 0x58, 0x61, 0x62, 0x64, 0x68, 0x70, +]; +const DIRECTORY_STANDARD: [u8; 32] = [ + 0x1f, 0x2f, 0x37, 0x3b, 0x3d, 0x3e, 0x07, 0x0b, 0x0d, 0x0e, 0x13, 0x15, 0x16, 0x19, 0x1a, 0x1c, + 1, 2, 4, 8, 16, 32, 0x23, 0x25, 0x26, 0x29, 0x2a, 0x2c, 0x31, 0x32, 0x34, 0x38, +]; +fn directory_columns(premium: bool) -> &'static [u8] { + if premium { + &DIRECTORY_PREMIUM + } else { + &DIRECTORY_STANDARD + } +} +fn directory_syndrome(mut word: u64, premium: bool) -> u8 { + if !premium { + word &= u32::MAX as u64; + } + let mut check = 0; + while word != 0 { + check ^= directory_columns(premium)[word.trailing_zeros() as usize]; + word &= word - 1; + } + check +} +pub fn directory_encode(word: u64, premium: bool) -> u64 { + let word = word + & !if premium { + (63 << 32) | (1 << 48) + } else { + 63 << 16 + }; + let check = directory_syndrome(word, premium) as u64; + word | if premium { + (check & 63) << 32 | (check >> 6) << 48 + } else { + check << 16 + } +} diff --git a/src/dev/bedrock/mem.rs b/src/dev/bedrock/mem.rs new file mode 100644 index 000000000..0caebfe8f --- /dev/null +++ b/src/dev/bedrock/mem.rs @@ -0,0 +1,274 @@ +//! A Bedrock node's fitted memory: banks owned by `Physical`, with the bank +//! decode selected by Bedrock's memory configuration register. +//! +//! The board names the fitted banks, locational bank 0 upward, one size in +//! MiB each: two 256 MiB banks on the Fuel, up to eight on the Tezro. Each +//! bank answers in its own 1 GiB software slot, so banks 4 to 7 sit at 4 to +//! 8 GiB. A node with banks there asks ppmem for its 16 GiB window, as the +//! Octane's HEART does past 4 GiB, so every bank is in the CPU's direct +//! window. +//! +//! Every access reaches a bank through Physical's bank views or the bank +//! itself, without a lock. Remapping runs on the CPU thread, or with the CPU +//! stopped, as HEART's memory descriptors do: a bank configuration change +//! rebuilds the whole map. Physical publishes complete decode-table pages +//! atomically; a concurrent bus master sees an old or new entry. This does +//! not promise a transaction across a remap. +//! +//! Ported from hw/misc/sgi-bedrock-md.c in jamesbraid/qemu-sgi-origami. + +use crate::physical::{Physical, RamBank}; +use crate::traits::*; +use crate::ppmem::devmap::SLOT_SHIFT; +use crate::ppmem::{MappedMemory, BITMAP_SHIFT, LARGE_WINDOW_SIZE, WINDOW_SIZE}; +use parking_lot::Mutex; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::Arc; + +pub const BANK_SPAN: u64 = 1 << 30; +/// Software slots of [`BANK_SPAN`] in the node's 8 GiB. +pub const SLOTS: usize = 8; +/// The most memory a bank can hold: what the largest size code names. +const MAX_BANK_MIB: u32 = 1024; +/// Banks are mapped into the direct window in whole regions of this size. +const REGION_MIB: u32 = 1 << (BITMAP_SHIFT - 20); + +/// Where fitted RAM can answer: for each software slot, the most any bank the +/// bank-select swizzle can place there holds. The swizzle permutes slots +/// within groups of four, so a slot may hold any bank of its group. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct RamExtent([u64; SLOTS]); +impl RamExtent { + /// `banks` are the fitted sizes in MiB from locational bank 0. + pub fn new(banks: &[u32]) -> Self { + let fitted = |bank: usize| banks.get(bank).map_or(0, |&mib| u64::from(mib) << 20); + Self(std::array::from_fn(|slot| { + (0..4).map(|group_slot| fitted(slot / 4 * 4 + group_slot)).max().unwrap_or(0) + })) + } + /// Whether `address` can be fitted RAM under some bank-select setting. + pub fn contains(&self, address: u64) -> bool { + let slot = address / BANK_SPAN; + slot < SLOTS as u64 && address % BANK_SPAN < self.0[slot as usize] + } + /// Slots from 0 through the last that can hold RAM. + #[cfg(feature = "jitv2")] + fn slots(&self) -> usize { + self.0.iter().rposition(|&size| size != 0).map_or(0, |last| last + 1) + } + /// The most RAM software slot `slot` can hold, in bytes. + pub fn prefix(&self, slot: usize) -> u64 { self.0[slot] } +} + +#[derive(Clone, Copy, PartialEq, Eq)] +struct Run { + bank: usize, + base: u64, + size: u64, +} + +struct NoMemory; +impl BusDevice for NoMemory {} + +/// JIT pages belong to physical addresses, not to the bank currently +/// answering there. Keep these counters separate from the banks' counters: +/// a compiled page pins its pointer across later bank reconfiguration. +#[cfg(feature = "jitv2")] +pub(crate) struct Generations(Box<[AtomicU64]>); +#[cfg(feature = "jitv2")] +impl Generations { + /// One counter per 4 KiB page, up to the last slot that can hold RAM. + fn new(extent: &RamExtent) -> Self { + let pages = extent.slots() << (BANK_SPAN.trailing_zeros() - 12); + Self((0..pages).map(|_| AtomicU64::new(0)).collect::>().into_boxed_slice()) + } + pub(crate) fn base(&self) -> *mut AtomicU64 { + self.0.as_ptr() as *mut AtomicU64 + } + pub(crate) fn pages(&self) -> usize { + self.0.len() + } + fn bump(&self, address: u64) { + if let Some(counter) = self.0.get((address >> 12) as usize) { + counter.fetch_add(1, Ordering::Relaxed); + } + } + fn bump_all(&self, extent: &RamExtent) { + for slot in 0..extent.slots() { + let first = slot * (BANK_SPAN >> 12) as usize; + let pages = (extent.prefix(slot) >> 12) as usize; + for counter in &self.0[first..first + pages] { + counter.fetch_add(1, Ordering::Relaxed); + } + } + } +} + +pub struct Memory { + physical: Arc, + /// Fitted bytes in each locational bank, bank 0 first. Physical's banks + /// are these, in the same order. + fitted: Vec, + config: AtomicU64, + /// The runs the last remap mapped. + runs: Mutex>, + /// Each processor's window while its PROM cache lines stand. + extent: RamExtent, + #[cfg(feature = "jitv2")] + generations: Arc, +} +impl Memory { + /// Memory with `banks` fitted, locational bank 0 upward, each a size in + /// MiB: at most [`SLOTS`] banks, none empty or over 1 GiB (the largest + /// size code), each a whole number of direct-window regions. + pub fn new(banks: &[u32]) -> Result { + if banks.is_empty() || banks.len() > SLOTS { + return Err(format!("memory: {} banks; fit 1 to {SLOTS}", banks.len())); + } + if let Some((bank, mib)) = banks.iter().enumerate() + .find(|(_, &mib)| mib == 0 || mib > MAX_BANK_MIB || mib % REGION_MIB != 0) + { + return Err(format!( + "memory: bank {bank} is {mib} MiB; a bank is a multiple of {REGION_MIB} MiB up to {MAX_BANK_MIB} MiB")); + } + // Bank n starts at n GiB; past bank 3 the node runs over 4 GiB. + let window = if banks.len() as u64 * BANK_SPAN > WINDOW_SIZE { LARGE_WINDOW_SIZE } else { WINDOW_SIZE }; + let physical = Physical::with_window(banks.iter().map(|&mib| RamBank::new(mib as usize)).collect(), window); + physical.set_default(Arc::new(NoMemory), "Bedrock unconfigured memory"); + Ok(Self { physical, fitted: banks.iter().map(|&mib| u64::from(mib) << 20).collect(), + config: AtomicU64::new(0), runs: Mutex::new(Vec::new()), + #[cfg(feature = "jitv2")] + generations: Arc::new(Generations::new(&RamExtent::new(banks))), + extent: RamExtent::new(banks), + }) + } + /// Fitted bytes in locational bank `bank`; zero where nothing is fitted. + pub(crate) fn fitted(&self, bank: u64) -> u64 { + self.fitted.get(bank as usize).copied().unwrap_or(0) + } + /// Where the fitted banks start in a dense walk of all of them, bank 0 + /// first. Sparse tables key ECC, directory and protection state by it, so + /// two banks never share a key. + pub(crate) fn dense_base(&self, bank: u64) -> u64 { + self.fitted.iter().take(bank as usize).sum() + } + pub fn extent(&self) -> RamExtent { self.extent } + pub fn physical(&self) -> &Arc { + &self.physical + } + pub(crate) fn config(&self) -> u64 { + self.config.load(Ordering::Acquire) + } + + #[cfg(feature = "jitv2")] + pub(crate) fn generations(&self) -> Arc { self.generations.clone() } + #[cfg(feature = "jitv2")] + pub(crate) fn gen_ptr(&self, address: u64) -> *const AtomicU64 { + // Disabled RAM must still have a counter: a JIT page may already + // have claimed it before the guest enables or moves a bank. + if self.extent.contains(address) { + unsafe { self.generations.base().add((address >> 12) as usize) } + } else { std::ptr::null() } + } + pub(crate) fn written(&self, _address: u64) { + #[cfg(feature = "jitv2")] + self.generations.bump(_address); + } + + fn runs(&self, config: u64) -> Vec { + (0..SLOTS as u64).filter_map(|slot| { + let base = slot * BANK_SPAN; + let (bank, _) = self.decode(config, base, 1)?; + let code = (config >> (bank / 2 * 8 + 4)) & 15; + Some(Run { bank, base, size: self.fitted(bank as u64).min((32 << 20) << code) }) + }).collect() + } + + /// Map the banks as `config` places them. Only the CPU thread, or a + /// caller with the CPU stopped, may change ppmem's bitmap. Without + /// `force` it does nothing when the banks would answer where they do; a + /// forced rebuild follows the machine setting Physical's default. + pub(crate) fn remap(&self, config: u64, force: bool) { + let runs = self.runs(config); + let mut old = self.runs.lock(); + if !force && *old == runs { + self.config.store(config, Ordering::Release); + return; + } + let phys = &self.physical; + let space = phys.ppmem_space(); + if let Some(space) = space { + space.clear_mappings(); + } + // Restore only ranges previously owned by RAM. In particular, the + // unused part of a 1 GiB slot may contain flash or other devices. + for run in old.iter() { + let retained = runs.iter().find(|next| next.base == run.base) + .map_or(0, |next| next.size.min(run.size)); + if retained < run.size { + phys.fill_slots((run.base + retained) >> SLOT_SHIFT, + (run.size - retained) >> SLOT_SHIFT, phys.default_entry()); + } + } + for run in &runs { + phys.fill_slots(run.base >> SLOT_SHIFT, run.size >> SLOT_SHIFT, phys.bank_view(run.bank, run.base)); + if let Some(space) = space { + if let Err(error) = space.map_range(run.bank, 0, run.base, run.size) { + crate::dlog!(crate::devlog::LogModule::Bedrock, + "Bedrock RAM window at {:#x}: {error}", run.base); + } + } + } + self.config.store(config, Ordering::Release); + // Publish changed ownership before advancing the stable counters, + // so an in-flight compile cannot accept bytes from the old decode. + self.changed_decode(); + *old = runs; + } + pub(crate) fn decode(&self, config: u64, addr: u64, bytes: u64) -> Option<(usize, u64)> { + let software = addr / BANK_SPAN; + if software >= SLOTS as u64 { + return None; + } + let bank = software ^ ((config >> 32) & 3); + let field = config >> (bank / 2 * 8); + let code = (field >> 4) & 15; + if self.fitted(bank) == 0 || field & (1 << (bank & 1)) == 0 || !(1..=5).contains(&code) { + return None; + } + let limit = self.fitted(bank).min((32 << 20) << code); + let offset = addr % BANK_SPAN; + (offset < limit && bytes <= limit - offset).then_some((bank as usize, offset)) + } + pub(crate) fn raw_read(&self, bank: usize, at: u64) -> u64 { + self.physical.banks()[bank].read64(at).data + } + pub(crate) fn raw_write(&self, bank: usize, at: u64, value: u64) -> u32 { + let result = self.physical.banks()[bank].write64(at, value); + if result == BUS_OK { + let slot = bank as u64 ^ ((self.config() >> 32) & 3); + self.written(slot * BANK_SPAN + at); + } + result + } + pub(crate) fn zero(&self, bank: usize, size: u64) { + assert!(size <= self.fitted(bank as u64)); + // The bank owns this contiguous mapping; zero has the same representation + // in host word order. Self-test does not synchronize concurrent bank accesses. + unsafe { + std::ptr::write_bytes(self.physical.banks()[bank].host_ptr(), 0, size as usize); + } + #[cfg(feature = "jitv2")] + self.physical.banks()[bank].bump_gen_all(); + #[cfg(feature = "jitv2")] + self.generations.bump_all(&self.extent); + } + pub(crate) fn changed_decode(&self) { + #[cfg(feature = "jitv2")] + self.generations.bump_all(&self.extent); + #[cfg(feature = "jitv2")] + for bank in self.physical.banks() { + bank.bump_gen_all(); + } + } +} diff --git a/src/dev/bedrock/mod.rs b/src/dev/bedrock/mod.rs new file mode 100644 index 000000000..525273859 --- /dev/null +++ b/src/dev/bedrock/mod.rs @@ -0,0 +1,642 @@ +//! Bedrock, the node controller of the IP35 boards, as one device: two +//! processor interfaces (PI), the memory and directory controller (MD) with +//! its memory, the I/O block (II) that faces the crossbar widgets, the local +//! block (LB), the network interface (NI) and the crossbar block (XB). +//! +//! A board gives the chip its processors, banks and clock in a +//! [`BedrockConfig`], maps the faces it hands out into its processor bus, +//! and gives each processor's interrupt word to [`Bedrock::set_cpu_interrupts`]. +//! The units are plain fields; each locks only its own state. +//! +//! A unit holds no reference to another and no closure. What a unit's +//! access changes beyond the unit comes back as an `Effect`, which the chip +//! acts on once the unit's lock is released. The error levels LB, NI and XB +//! drive are PI level bits that each writer sets under its own lock. The +//! chip calls nothing in the board: a node reset leaves as a +//! [`MachineEvent::HardReset`] on the board's event channel. +//! +//! # Locks +//! +//! - PI: the interrupt inputs (pending banks, masks, requests, timer and +//! error status) are atomics, so a mask store, a pending-modify store and +//! a counter read take no lock. `state` holds the stored words and the +//! graphics registers; `timer` the comparators and the host timer. +//! - MD: the cell register is an atomic, and memory, cached, uncached or +//! DMA, takes no lock; `state` and `remapping` as the controller's notes +//! say. +//! - II: the permission words and translation entries are atomics, so PIO, +//! inbound memory and an inbound interrupt packet take no lock; `regs` +//! holds the rest. +//! - LB, NI and XB: `regs` each, under which a block drives its error level. +//! +//! Ported from hw/misc/sgi-bedrock*.c in jamesbraid/qemu-sgi-origami. + +mod io; +mod lb; +mod md; +mod mem; +mod ni; +mod pi; +mod xb; +#[cfg(test)] +mod tests; +#[cfg(test)] +use tests::{test_config, Rig}; + +pub use mem::{RamExtent, BANK_SPAN, SLOTS}; +pub use pi::Population; + +use crate::dev::heart::Clock; +use crate::hptimer::{TimerId, TimerManager, TimerReturn}; +use crate::irq::{InterruptController, IrqLine}; +use crate::traits::{BusDevice, BusRead16, BusRead32, BusRead64, BusRead8, Device, MachineEvent, Resettable, BUS_ERR, BUS_OK}; +use io::Io; +use md::{Md, View}; +use pi::{Levels, Pi}; +use std::sync::atomic::{AtomicU64, AtomicU8, Ordering}; +use std::sync::mpsc::Sender; +use std::sync::{Arc, OnceLock, Weak}; +use std::time::Duration; + +/// Unit bases inside the chip's 8 MiB register window, the same in both XIO +/// register forms, and the registers a board or test names. +pub mod reg { + pub const PI0_BASE: u64 = 0x00_0000; + pub const PI1_BASE: u64 = 0x20_0000; + pub const IO_BASE: u64 = 0x40_0000; + pub const LB_BASE: u64 = 0x60_0000; + pub const NI_BASE: u64 = 0x68_0000; + pub const XB_BASE: u64 = 0x70_0000; + pub const MD_BASE: u64 = 0x78_0000; + pub const WINDOW: u64 = 0x80_0000; + /// The public Linux Hub I/O protection-override key literal. + /// + /// Linux `arch/mips/include/asm/sn/sn0/hubio.h` at + /// `db2ddb87143519e20a95aa36c60b36107b736a58` defines + /// `IIO_PROTECT_OVRRD_KEY` as this value (lines 31, 47). This establishes + /// the literal's provenance only; it does not establish a register mapping. + pub const IIO_PROTECT_OVRRD_KEY: u64 = 0x53474972756c6573; + pub use super::io::{bte_reg as bte, reg as io}; + pub use super::lb::reg as lb; + pub use super::md::reg as md; + pub use super::ni::reg as ni; + pub use super::pi::reg as pi; + pub use super::xb::reg as xb; +} + +/// What a board gives its Bedrock. Fixed for the chip's life. +pub struct BedrockConfig<'a> { + /// The board's name in log lines. + pub name: &'static str, + /// Host nanoseconds: the PIs' counter and timers run on it, and the + /// other units' clocked work. + pub clock: Clock, + /// The processor slots the board fits. CPU n sits in + /// `processors.slot_of(n)`; a fitted slot starts with its enable bit set. + pub processors: Population, + /// The memory banks in MiB, locational bank 0 upward. + pub banks_mib: &'a [u32], + /// Where a node reset goes: LB's soft reset and NI's local reset each + /// send [`MachineEvent::HardReset`]. + pub events: Option>, +} + +/// What asked for a node reset. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum NodeReset { + /// LB's soft-reset bit. + LocalBlock, + /// NI's local-reset bit. + Network, +} + +/// What a unit's access changed outside the unit, returned with the unit's +/// lock released for [`Bedrock::apply`]. +#[must_use] +enum Effect { + None, + /// A bit of the chip's levels changed: both PIs publish. + Reflect, + /// Run block-transfer engine `n` now. + Bte(usize), + /// The unit has reset itself; the node resets. + Reset(NodeReset), +} + +/// A PI's host timer: the manager it runs on and the chip it fires into. +struct HostTimer<'a> { + tm: &'a TimerManager, + chip: &'a Weak, +} +impl HostTimer<'_> { + /// A one-shot `delay` nanoseconds out that latches PI `interface`'s due + /// comparators and arms the next. + fn arm(&self, interface: u8, delay: u64) -> TimerId { + self.tm.add_one_shot(Duration::from_nanos(delay), (self.chip.clone(), interface), |(chip, interface)| { + if let Some(chip) = chip.upgrade() { + chip.expire(usize::from(*interface)); + } + TimerReturn::Delete + }) + } +} + +pub struct Bedrock { + name: &'static str, + clock: Clock, + processors: Population, + pi: [Pi; 2], + md: Md, + io: Io, + lb: lb::Lb, + ni: ni::Ni, + xb: xb::Xb, + levels: Levels, + /// The node number LB's region register names; inbound packets and the + /// BTEs' interrupts check it. Set under LB's lock. + node: AtomicU8, + /// MD's DMA view: the BTEs and inbound memory requests use it. + dma: Arc, + timers: OnceLock>, + events: Option>, + weak: Weak, +} + +impl Bedrock { + pub fn new(config: BedrockConfig<'_>) -> Result, String> { + let BedrockConfig { name, clock, processors, banks_mib, events } = config; + let memory = mem::Memory::new(banks_mib)?; + let now = clock(); + let chip = Arc::new_cyclic(|weak: &Weak| { + let md = Md::new(memory, clock.clone()); + Self { + name, + pi: [Pi::new(0, processors, now), Pi::new(1, processors, now)], + dma: md.dma_view(), + md, + io: Io::new(), + lb: lb::Lb::new(), + ni: ni::Ni::new(), + xb: xb::Xb::new(), + levels: Levels::default(), + node: AtomicU8::new(0), + timers: OnceLock::new(), + events, + clock, + processors, + weak: weak.clone(), + } + }); + chip.md.rebuild_memory(); + Ok(chip) + } + + pub fn register_locks(self: &Arc) { + use crate::locks::register_mutex_fn; + register_mutex_fn("bedrock-pi0::state", self, |c| c.pi[0].state_locked()); + register_mutex_fn("bedrock-pi1::state", self, |c| c.pi[1].state_locked()); + register_mutex_fn("bedrock-pi0::timer", self, |c| c.pi[0].timer_locked()); + register_mutex_fn("bedrock-pi1::timer", self, |c| c.pi[1].timer_locked()); + register_mutex_fn("bedrock-md::state", self, |c| c.md.state_locked()); + register_mutex_fn("bedrock-md::remapping", self, |c| c.md.remapping_locked()); + register_mutex_fn("bedrock-io", self, |c| c.io.locked()); + register_mutex_fn("bedrock-lb", self, |c| c.lb.locked()); + register_mutex_fn("bedrock-ni", self, |c| c.ni.locked()); + register_mutex_fn("bedrock-xb", self, |c| c.xb.locked()); + } + + // The faces a board maps into its processor bus. Each is its own type, + // so no access tests which face it came through. + + /// The 8 MiB register window in its node XIO form. + pub fn registers(self: &Arc) -> Arc { + Arc::new(Registers::(self.clone())) + } + /// The register window in its local XIO form: PI0's range answers the + /// requester's own PI when the board fits processors on both PIs, as + /// QEMU's IP35 node overlays PI1 there for a processor on it (OpenBSD + /// expects the local form to select the requester's interface). That the + /// chip answers so is an approximation pending a hardware read. With + /// processors on PI0 alone it is the node form. + pub fn local_registers(self: &Arc) -> Arc { + if (0..2).any(|slot| self.processors.fitted(1, slot)) { + Arc::new(Registers::(self.clone())) + } else { + self.registers() + } + } + /// The II's PIO face: the processor's uncached I/O view below 4 GiB, + /// decoded to a widget. + pub fn pio(self: &Arc) -> Arc { + Arc::new(Pio(self.clone())) + } + /// Memory for the processor's cached view and DMA: Physical's entries. + pub fn dma_view(&self) -> Arc { + self.dma.clone() + } + /// Uncached memory: the banks, as MD's configuration places them. + pub fn uncached_view(&self) -> Arc { + self.md.uncached_view() + } + pub fn directory_bus(self: &Arc) -> Arc { + self.md.view(self.weak.clone(), View::Directory) + } + pub fn ecc_bus(self: &Arc) -> Arc { + self.md.view(self.weak.clone(), View::Ecc) + } + pub fn ecc_scrub_bus(self: &Arc) -> Arc { + self.md.view(self.weak.clone(), View::Scrub) + } + pub fn fetchop_bus(self: &Arc) -> Arc { + self.md.view(self.weak.clone(), View::Fetchop) + } + pub fn physical(&self) -> &Arc { + self.md.physical() + } + /// Where fitted RAM can answer under any bank-select setting. + pub fn ram_extent(&self) -> RamExtent { + self.md.ram_extent() + } + /// Reinstall RAM after the board sets Physical's default; on the CPU + /// thread or with the CPU stopped. + pub fn rebuild_memory(&self) { + self.md.rebuild_memory(); + } + /// Whether processor `cpu`'s join after the PROM's reset barrier has + /// landed on LB scratch 1 since the LB's last reset: the first store it + /// made there after the boot master's release. One atomic load. + pub fn scratch1_joined(&self, cpu: usize) -> bool { + self.lb.scratch1_joined(cpu) + } + /// II scratch 0 (0x150), where the IP35 PROM publishes each interface's + /// processors, bit 4 + n for slot n (A, B on PI0, C, D on PI1). One + /// atomic load. + pub fn io_scratch0(&self) -> u64 { + self.io.scratch0() + } + /// Whether LB scratch 3's bit 0, which the IP35 PROM takes as its hub + /// lock, is clear. One atomic load. + pub fn hub_lock_free(&self) -> bool { + self.lb.hub_lock_free() + } + /// The processors, one bit per CPU number, that have stored to their own + /// PI real-time compare register since their presence join on LB + /// scratch 1: the IP35 PROM's pair liveness posts. One atomic load. + pub fn rt_posted(&self) -> u64 { + self.lb.rt_posted() + } + #[cfg(feature = "jitv2")] + pub(crate) fn generation_base(&self) -> *mut AtomicU64 { + self.md.generation_base() + } + /// MD's cell register bit 0, which gates the register window and the + /// board's Hub-special windows. One atomic load. + #[inline] + pub fn cell_permits(&self) -> bool { + self.md.cell_permits() + } + + // Widgets. + + /// Connect crossbar widget `widget`, once, before the machine runs. + pub fn connect_widget(&self, widget: u8, dev: Arc) { + self.io.connect_widget(widget, dev); + } + /// A read arriving from widget `source`'s device `device` (`None`: every + /// master on the widget must be admitted): memory below bit 47. + pub fn inbound_read(&self, source: u8, device: Option, address: u64, size: u8) -> BusRead64 { + self.inbound(source, device, address, size, None) + } + /// A write arriving from widget `source`: memory below bit 47, or an + /// interrupt packet to a PI's pending-modify register in this node's + /// window. A packet the PI refuses answers a bus error. + pub fn inbound_write(&self, source: u8, device: Option, address: u64, size: u8, value: u64) -> u32 { + self.inbound(source, device, address, size, Some(value)).status + } + fn inbound(&self, widget: u8, device: Option, address: u64, size: u8, value: Option) -> BusRead64 { + if address >= (1 << 48) || u64::from(size) > (1 << 48) - address || !self.io.admits(widget, device) { + return BusRead64::err(); + } + if address & (1 << 47) == 0 { + return io::access(self.dma.as_ref(), address, size, value); + } + let node = ((address >> 39) & 255) as u8; + let window = address & ((1 << 39) - 1); + if node != self.node.load(Ordering::Acquire) || window & !0x7f_ffff != 0x180_0000 { + return BusRead64::err(); + } + // XIO register packets reach only the two PIs' pending-modify registers. + match (window & 0x7f_ffff, size, value) { + (offset @ (0x90 | 0x20_0090), 8, Some(value)) if self.interrupt(usize::from(offset >= reg::PI1_BASE), value) => { + BusRead64 { status: BUS_OK, data: 0 } + } + _ => BusRead64::err(), + } + } + /// An interrupt packet to PI `pi`'s pending-modify register; whether the + /// PI took it. + fn interrupt(&self, pi: usize, value: u64) -> bool { + let taken = self.pi[pi].io_modify(value, 0); + if taken { + self.pi[pi].publish(&self.levels); + } + taken + } + /// Widget `widget`'s DMA as a bus device: crosstalk addresses, with the + /// widget's inbound permissions applied. + pub fn widget_dma(self: &Arc, widget: u8) -> Arc { + Arc::new(WidgetDma { chip: self.weak.clone(), widget }) + } + + // Processors. + + /// Where CPU `cpu`'s IP2..IP6 go: Cause bits 10-14 of its interrupt word + /// (`MipsCpu::interrupts_ptr`), from the slot the board's population + /// gives it. The chip writes no other bit, and no other device may write + /// these. The word must outlive the chip. + pub fn set_cpu_interrupts(&self, cpu: usize, word: *const AtomicU64) { + if let Some((interface, slot)) = self.processors.slot_of(cpu) { + let pi = &self.pi[usize::from(interface)]; + pi.set_word(slot, word); + pi.publish(&self.levels); + } + } + /// The PIs' comparators and profile timers fire on `tm`. Held weakly. + pub fn set_timer_manager(&self, tm: &Arc) { + let _ = self.timers.set(Arc::downgrade(tm)); + let now = (self.clock)(); + let host = self.host_timer(tm); + for pi in &self.pi { + pi.reschedule(Some(&host), now); + } + } + /// Name `source` as driving PI `interface`'s pending `level` while + /// `active`, so a clear of the level while the source stays active + /// brings it back; see the PI's notes. A bridge's interrupt inputs are + /// such sources. + pub fn set_level_source(&self, interface: usize, source: usize, level: u8, active: bool) { + if let Some(pi) = self.pi.get(interface) { + pi.set_level_source(source, level, active); + } + } + /// The L1 UART's line: bank 0 bit 4 of both PIs. + pub fn uart_irq(self: &Arc) -> IrqLine { + IrqLine::new(Arc::new(Uart(self.weak.clone())), 0) + } + fn host_timer<'a>(&'a self, tm: &'a TimerManager) -> HostTimer<'a> { + HostTimer { tm, chip: &self.weak } + } + fn timer_manager(&self) -> Option> { + self.timers.get().and_then(Weak::upgrade) + } + /// PI `interface`'s host timer fired. + fn expire(&self, interface: usize) { + let Some(tm) = self.timer_manager() else { return }; + if self.pi[interface].expire(&self.host_timer(&tm), (self.clock)()) { + self.pi[interface].publish(&self.levels); + } + } + fn publish_all(&self) { + for pi in &self.pi { + pi.publish(&self.levels); + } + } + fn apply(&self, effect: Effect) { + match effect { + Effect::None => {} + Effect::Reflect => self.publish_all(), + Effect::Bte(n) => { + let node = self.node.load(Ordering::Acquire); + if let Some(done) = self.io.run_bte(n, self.dma.as_ref(), node) { + if !self.interrupt(done.pi, done.value) { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock IO BTE completion interrupt failed"); + } + } + } + Effect::Reset(cause) => { + crate::dlog!(crate::devlog::LogModule::Bedrock, "{}: {cause:?} reset from CPU {:?}; restarting the node", + self.name, crate::cpu::smp::on_cpu_thread()); + if let Some(events) = &self.events { + let _ = events.send(MachineEvent::HardReset); + } + } + } + } + + /// The PI a local-form access at offset 0 reaches: the requester's own. + /// A reader that is no processor reaches PI0. + fn own_pi(&self) -> usize { + crate::cpu::smp::on_cpu_thread() + .and_then(|cpu| self.processors.slot_of(cpu)) + .map_or(0, |(interface, _)| usize::from(interface)) + } + + /// An access of `size` bytes at `offset` in the register window, a write + /// when `value` is given; PI0's range reaches PI `pi0`. What no unit + /// answers, the padding after a unit's registers included, is a bus error. + fn access(&self, pi0: usize, offset: u64, size: u8, value: Option) -> BusRead64 { + if !self.md.cell_permits() { + return BusRead64::err(); + } + match offset { + reg::PI0_BASE..reg::PI1_BASE => self.pi_access(pi0, offset, size, value), + reg::PI1_BASE..reg::IO_BASE => self.pi_access(1, offset - reg::PI1_BASE, size, value), + reg::IO_BASE..reg::MD_BASE => self.block_access(offset, size, value), + reg::MD_BASE..reg::WINDOW => { + let result = self.md.access(View::Registers, offset - reg::MD_BASE, u64::from(size), value); + result.map_or(BusRead64::err(), BusRead64::ok) + } + _ => BusRead64::err(), + } + } + fn pi_access(&self, index: usize, a: u64, size: u8, value: Option) -> BusRead64 { + let pi = &self.pi[index]; + let now = (self.clock)(); + match (size, value) { + (8, None) if a < pi::SIZE && a & 7 == 0 => { + let (data, latched) = pi.read(a, now, &self.levels); + if latched { + pi.publish(&self.levels); + } + data.map_or(BusRead64::err(), BusRead64::ok) + } + (8, Some(v)) if a < pi::SIZE && a & 7 == 0 => { + if a == pi::reg::RT_COMPARE_A || a == pi::reg::RT_COMPARE_B { + let slot = (index as u8, usize::from(a == pi::reg::RT_COMPARE_B)); + if let Some(cpu) = crate::cpu::smp::on_cpu_thread().filter(|&cpu| self.processors.slot_of(cpu) == Some(slot)) { + self.lb.note_rt_compare(cpu); + } + } + if pi.write(a, v, now) { + let tm = self.timer_manager(); + pi.reschedule(tm.as_deref().map(|tm| self.host_timer(tm)).as_ref(), now); + } + pi.publish(&self.levels); + BusRead64 { status: BUS_OK, data: 0 } + } + // The counter's two halves, high word first. + (4, None) if a & !4 == pi::reg::COUNTER => { + BusRead64::ok(pi.count(now) >> (32 - (a & 4) * 8) & 0xffff_ffff) + } + _ => { + pi.note(a, value.is_some()); + BusRead64::err() + } + } + } + /// An access to the II, LB, NI or XB at `offset` in the window: aligned + /// doublewords only. + fn block_access(&self, offset: u64, size: u8, value: Option) -> BusRead64 { + let (base, end) = match offset { + reg::IO_BASE..reg::LB_BASE => (reg::IO_BASE, io::SIZE), + reg::LB_BASE..reg::NI_BASE => (reg::LB_BASE, lb::SIZE), + reg::NI_BASE..reg::XB_BASE => (reg::NI_BASE, ni::SIZE), + _ => (reg::XB_BASE, xb::SIZE), + }; + let a = offset - base; + if size != 8 || a & 7 != 0 || a >= end { + return BusRead64::err(); + } + let now = (self.clock)(); + let (data, effect) = match (base, value) { + (reg::IO_BASE, None) => (self.io.read(a), Effect::None), + (reg::IO_BASE, Some(v)) => (0, self.io.write(a, v)), + (reg::LB_BASE, None) => (self.lb.read(a, now, crate::cpu::smp::on_cpu_thread()), Effect::None), + (reg::LB_BASE, Some(v)) => (0, self.lb.write(a, v, now, crate::cpu::smp::on_cpu_thread(), &self.levels, &self.node)), + (reg::NI_BASE, None) => self.ni.read(a, &self.levels), + (reg::NI_BASE, Some(v)) => (0, self.ni.write(a, v, &self.levels)), + (_, None) => (self.xb.read(a), Effect::None), + (_, Some(v)) => (0, self.xb.write(a, v, &self.levels)), + }; + self.apply(effect); + BusRead64::ok(data) + } +} + +impl Device for Bedrock { + fn step(&self, _cycles: u64) {} + /// The counters run and the PIs' timers arm. + fn start(&self) { + let now = (self.clock)(); + self.md.start_clock(); + let tm = self.timer_manager(); + let host = tm.as_deref().map(|tm| self.host_timer(tm)); + for pi in &self.pi { + pi.start(host.as_ref(), now); + } + self.publish_all(); + } + /// The counters pause where they are and the PIs' timers stop. + fn stop(&self) { + let now = (self.clock)(); + let tm = self.timer_manager(); + let host = tm.as_deref().map(|tm| self.host_timer(tm)); + for pi in &self.pi { + pi.stop(host.as_ref(), now); + } + self.md.stop_clock(); + self.publish_all(); + } + fn is_running(&self) -> bool { + self.pi[0].is_running() + } + fn get_clock(&self) -> u64 { + self.pi[0].count((self.clock)()) + } +} + +impl Resettable for Bedrock { + /// Every unit's reset. Memory keeps its contents. + fn power_on(&self) { + let now = (self.clock)(); + let tm = self.timer_manager(); + let host = tm.as_deref().map(|tm| self.host_timer(tm)); + for pi in &self.pi { + pi.power_on(now); + pi.reschedule(host.as_ref(), now); + } + self.md.power_on(); + self.lb.power_on(&self.node); + self.io.power_on(); + let _ = self.ni.power_on(&self.levels); + let _ = self.xb.power_on(&self.levels); + self.publish_all(); + } +} + +/// The register window. With `LOCAL`, PI0's range answers the requester's +/// own PI. +struct Registers(Arc); +impl Registers { + #[inline] + fn access(&self, offset: u64, size: u8, value: Option) -> BusRead64 { + let pi0 = if LOCAL { self.0.own_pi() } else { 0 }; + self.0.access(pi0, offset, size, value) + } +} +macro_rules! face_access { + ($r:ident, $w:ident, $ty:ty, $result:ident, $size:expr) => { + fn $r(&self, a: u64) -> $result { + let r = self.access(a, $size, None); + $result { data: r.data as $ty, status: r.status } + } + fn $w(&self, a: u64, v: $ty) -> u32 { + self.access(a, $size, Some(v as u64)).status + } + }; +} +impl BusDevice for Registers { + face_access!(read8, write8, u8, BusRead8, 1); + face_access!(read16, write16, u16, BusRead16, 2); + face_access!(read32, write32, u32, BusRead32, 4); + face_access!(read64, write64, u64, BusRead64, 8); +} + +/// The II's PIO face. +struct Pio(Arc); +impl Pio { + #[inline] + fn access(&self, a: u64, size: u8, value: Option) -> BusRead64 { + self.0.io.pio(a, size, value) + } +} +impl BusDevice for Pio { + face_access!(read8, write8, u8, BusRead8, 1); + face_access!(read16, write16, u16, BusRead16, 2); + face_access!(read32, write32, u32, BusRead32, 4); + face_access!(read64, write64, u64, BusRead64, 8); +} + +/// A widget's DMA as inbound traffic. Weak: the widget's device may hold +/// it, and the chip holds the widget. +struct WidgetDma { + chip: Weak, + widget: u8, +} +impl WidgetDma { + #[inline] + fn access(&self, a: u64, size: u8, value: Option) -> BusRead64 { + match self.chip.upgrade() { + Some(chip) => chip.inbound(self.widget, None, a, size, value), + None => BusRead64::err(), + } + } +} +impl BusDevice for WidgetDma { + face_access!(read8, write8, u8, BusRead8, 1); + face_access!(read16, write16, u16, BusRead16, 2); + face_access!(read32, write32, u32, BusRead32, 4); + face_access!(read64, write64, u64, BusRead64, 8); +} + +/// The L1 UART's interrupt line. +struct Uart(Weak); +impl InterruptController for Uart { + fn set_line(&self, _: u32, level: bool) { + if let Some(chip) = self.0.upgrade() { + if chip.levels.set(Levels::UART, level) { + chip.publish_all(); + } + } + } +} diff --git a/src/dev/bedrock/ni.rs b/src/dev/bedrock/ni.rs new file mode 100644 index 000000000..d124f9d32 --- /dev/null +++ b/src/dev/bedrock/ni.rs @@ -0,0 +1,306 @@ +//! Bedrock's network interface (NI) exposes node routing, sideband and error +//! registers. The Fuel runs as one node without a NUMAlink peer, but the +//! guest still reads and programs these local control words during machine +//! setup. +//! +//! # What is approximated here +//! +//! Routing and sideband writes are retained; the network fabric and packet +//! delivery are not simulated. Unspecified power-on fields start at zero, and +//! the sideband word survives reset as the current compatibility behavior. +//! +//! Ported from hw/misc/sgi-bedrock-ni.c and hw/misc/sgi-bedrock-ni-regs.c +//! in jamesbraid/qemu-sgi-origami. + +use super::pi::Levels; +use super::{Effect, NodeReset}; +use parking_lot::Mutex; + +/// Length of the block-relative register aperture. +pub(super) const SIZE: u64 = 0x2008; + +/// Port status bits 1:0 of a node with no cable. Bit 1, link up, is clear: +/// real Fuels' PROM 6.210 consoles print "Local hub NUMAlink is down" +/// (comp.sys.sgi.admin, Atro Tossavainen 2005-04-13, Tristram Scott +/// 2007-07-18). Bit 0 is chosen to keep IRIX quiet and is unconfirmed: with +/// bits 1:0 at 0 and reset enable 0, the IRIX 6.5.30 bus-error handler prints +/// "Detected a link reset due to other partition reset" on every faulting +/// widget probe, and the 2005 Fuel's IRIX took a PIO read error with no such +/// line. Which nonzero value hardware reads is unknown; a PROM-monitor read +/// of this register on an uncabled node would settle it. +const PORT_STATUS_UNCABLED: u64 = 1; + +pub mod reg { + /// Port status, hub-relative `0x680000` (the first word of the block). + pub const PORT_STATUS: u64 = 0x0; + pub const LOCAL_RESET: u64 = 0x8; + pub const CONTROL_0: u64 = 0x10; + pub const CONTROL_1: u64 = 0x18; + pub const CONTROL_2: u64 = 0x20; + pub const CONTROL_3: u64 = 0x30; + pub const CONTROL_4: u64 = 0x38; + pub const ERROR_STATUS: u64 = 0x100; + pub const ERROR_STATUS_UPPER: u64 = 0x108; + pub const ERROR_STATUS_LOWER: u64 = 0x110; + pub const SIDE_BAND: u64 = 0x118; + pub const ERROR_CLEAR: u64 = 0x120; + pub const ROUTES: u64 = 0x1000; + pub const ROUTES_END: u64 = 0x1400; + pub const GLOBAL_CONTROL: u64 = 0x2000; +} + +#[derive(Clone)] +struct Regs { + control: [u64; 8], + routes: [u64; 128], + global: u64, + errors: u64, + sideband: u64, +} +impl Default for Regs { + fn default() -> Self { + Self { control: [0; 8], routes: [0; 128], global: 0, errors: 0, sideband: 0 } + } +} +impl Regs { + fn error_level(&self) -> bool { + self.errors & 0x1fffffff000000 != 0 + } +} + +pub(super) struct Ni { + regs: Mutex, +} + +impl Ni { + pub(super) fn new() -> Self { + Self { regs: Mutex::new(Regs::default()) } + } + pub(super) fn locked(&self) -> bool { + self.regs.is_locked() + } + fn mask(a: u64) -> u64 { + match a { + reg::CONTROL_0 => 1, + reg::CONTROL_1 => 7, + reg::CONTROL_2 => 0x1ff, + reg::CONTROL_3 => 0x7efffffff, + reg::CONTROL_4 => 0xffff, + _ => 0, + } + } + fn reflect(r: &Regs, levels: &Levels) -> Effect { + if levels.set(Levels::NI_ERROR, r.error_level()) { Effect::Reflect } else { Effect::None } + } + pub(super) fn power_on(&self, levels: &Levels) -> Effect { + let mut r = self.regs.lock(); + let sideband = r.sideband; + *r = Regs::default(); + r.sideband = sideband; + Self::reflect(&r, levels) + } + /// A doubleword read. Reading the error-clear word clears the errors, so + /// the error level reaches `levels` under the block's lock. + pub(super) fn read(&self, a: u64, levels: &Levels) -> (u64, Effect) { + let mut r = self.regs.lock(); + let v = if Self::mask(a) != 0 { + r.control[a as usize / 8] + } else if (reg::ROUTES..reg::ROUTES_END).contains(&a) { + r.routes[(a - reg::ROUTES) as usize / 8] + } else { + match a { + reg::PORT_STATUS => PORT_STATUS_UNCABLED, + reg::ERROR_STATUS => r.errors, + reg::ERROR_STATUS_UPPER | reg::ERROR_STATUS_LOWER => 0, + reg::SIDE_BAND => r.sideband, + reg::ERROR_CLEAR => { + let v = r.errors; + r.errors = 0; + return (v, Self::reflect(&r, levels)); + } + reg::GLOBAL_CONTROL => r.global, + _ => { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock NI unsupported read {a:#x}"); + 0 + } + } + }; + (v, Effect::None) + } + /// A doubleword store. The local-reset bit (bit 2 of the reset word) + /// asks for the node's reset; the block's own registers are left to it. + pub(super) fn write(&self, a: u64, v: u64, levels: &Levels) -> Effect { + let mut r = self.regs.lock(); + if Self::mask(a) != 0 { + r.control[a as usize / 8] = v & Self::mask(a); + } else if (reg::ROUTES..reg::ROUTES_END).contains(&a) { + r.routes[(a - reg::ROUTES) as usize / 8] = v & 31; + } else { + match a { + reg::LOCAL_RESET => { + if v & 4 != 0 { + return Effect::Reset(NodeReset::Network); + } + } + reg::ERROR_STATUS => r.errors = v & ((1 << 53) - 1), + reg::SIDE_BAND => r.sideband = v & 0xfffff, + reg::GLOBAL_CONTROL => r.global = v & 31, + _ => crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock NI unsupported or read-only write {a:#x} = {v:#x}"), + } + } + Self::reflect(&r, levels) + } +} + +#[cfg(test)] +mod tests { + // Cases ported from tests/qtest/sn1-bedrock-test.c in jamesbraid/qemu-sgi-origami. + use super::super::reg::{pi, NI_BASE, PI1_BASE}; + use super::super::Rig; + use super::*; + use crate::traits::{BusDevice, BUS_ERR}; + + fn error_level(rig: &Rig) -> bool { + let levels = [0, PI1_BASE].map(|base| rig.r(base + pi::PENDING_STATUS_B) & (1 << 62) != 0); + assert_eq!(levels[0], levels[1], "both interfaces reflect bit 62"); + levels[0] + } + fn resets(rig: &Rig) -> usize { + rig.events.lock().try_iter().count() + } + + /// The guest programs route fields and reads the error register to acknowledge faults. + #[test] + fn route_registers_store_fields_and_error_reads_clear_pending_errors() { + let rig = Rig::new(); + for (a, mask) in [ + (reg::CONTROL_0, 1), + (reg::CONTROL_1, 7), + (reg::CONTROL_2, 0x1ff), + (reg::CONTROL_3, 0x7efffffff), + (reg::CONTROL_4, 0xffff), + ] { + rig.w(NI_BASE + a, !0); + assert_eq!(rig.r(NI_BASE + a), mask); + } + for a in [reg::ROUTES, reg::ROUTES_END - 8] { + rig.w(NI_BASE + a, !0); + assert_eq!(rig.r(NI_BASE + a), 31); + } + rig.w(NI_BASE + reg::GLOBAL_CONTROL, !0); + assert_eq!(rig.r(NI_BASE + reg::GLOBAL_CONTROL), 31); + rig.w(NI_BASE + reg::CONTROL_0, 0); + rig.w(NI_BASE + reg::LOCAL_RESET, 4); + assert_eq!(resets(&rig), 1); + rig.w(NI_BASE + reg::ERROR_STATUS, 0xffffff); + assert!(!error_level(&rig)); + rig.w(NI_BASE + reg::ERROR_STATUS, 1 << 24); + assert!(error_level(&rig)); + rig.r(NI_BASE + reg::ERROR_CLEAR); + assert!(!error_level(&rig)); + rig.w(NI_BASE + reg::SIDE_BAND, !0); + let _ = rig.chip.ni.power_on(&rig.chip.levels); + assert_eq!(rig.r(NI_BASE + reg::SIDE_BAND), 0xfffff); + assert_eq!(rig.r(NI_BASE + reg::CONTROL_3), 0); + } + + /// The PROM's one power-on pass writes a sign-extended port-parameter constant and zero + /// to the error and sideband words; each must read back through its defined bits only. + #[test] + fn power_on_programming_pass_reads_back_through_the_defined_bits() { + let rig = Rig::new(); + rig.w(NI_BASE + reg::CONTROL_0, 1); + rig.w(NI_BASE + reg::CONTROL_1, 0); + rig.w(NI_BASE + reg::CONTROL_2, 0x166); + rig.w(NI_BASE + reg::CONTROL_3, 0xffff_ffff_a3ff_1990); + rig.w(NI_BASE + reg::CONTROL_4, 0xc800); + rig.w(NI_BASE + reg::ERROR_STATUS, 0); + rig.r(NI_BASE + reg::ERROR_CLEAR); + rig.w(NI_BASE + reg::SIDE_BAND, 0); + for (a, want) in [ + (reg::CONTROL_0, 1), + (reg::CONTROL_1, 0), + (reg::CONTROL_2, 0x166), + (reg::CONTROL_3, 0x7_a3ff_1990), + (reg::CONTROL_4, 0xc800), + (reg::ERROR_STATUS, 0), + (reg::SIDE_BAND, 0), + (reg::ERROR_CLEAR, 0), + ] { + assert_eq!(rig.r(NI_BASE + a), want, "register {a:#x}"); + } + } + + /// Only the local-reset bit of the reset word reaches the node; the other bits must + /// not restart it, and the interface's own registers are left to the board's reset. + #[test] + fn only_the_local_reset_bit_requests_a_node_reset() { + let rig = Rig::new(); + for enable in 0..2 { + rig.w(NI_BASE + reg::CONTROL_0, enable); + for command in 0..8u64 { + rig.w(NI_BASE + reg::LOCAL_RESET, command); + assert_eq!(resets(&rig), usize::from(command & 4 != 0), "command {command:#x}"); + } + } + // The request does not clear this block. + assert_eq!(rig.r(NI_BASE + reg::CONTROL_0), 1); + } + + /// A latched port error raises the interrupt line through its interrupting bits only; + /// drivers rely on the line dropping when the status is rewritten or cleared. + #[test] + fn stored_error_status_drives_the_error_line_through_the_interrupting_bits() { + let rig = Rig::new(); + for (value, high) in [(1u64 << 30, true), (0xff, false), (1 << 52, true), (0, false)] { + rig.w(NI_BASE + reg::ERROR_STATUS, value); + assert_eq!(error_level(&rig), high, "status {value:#x}"); + } + // Bit 53 is beyond the register and must not be retained. + rig.w(NI_BASE + reg::ERROR_STATUS, 1 << 53); + assert_eq!(rig.r(NI_BASE + reg::ERROR_STATUS), 0); + } + + /// Reset values of the programmable words are unknown on hardware, so the model clears them; + /// a guest that re-runs its setup after a reset must see zeroes, not stale routes. + #[test] + fn reset_clears_every_programmable_register_and_misaligned_accesses_fault() { + let rig = Rig::new(); + let offsets = [ + reg::CONTROL_0, reg::CONTROL_1, reg::CONTROL_2, reg::CONTROL_3, reg::CONTROL_4, + reg::ROUTES, reg::ROUTES_END - 8, reg::GLOBAL_CONTROL, + ]; + for a in offsets { + rig.w(NI_BASE + a, !0); + } + // A misaligned write must not land in the neighbouring register. + assert_eq!(rig.regs.write64(NI_BASE + reg::CONTROL_4 + 1, 0), BUS_ERR); + assert_eq!(rig.regs.read64(NI_BASE + reg::CONTROL_4 + 1).status, BUS_ERR); + assert_eq!(rig.r(NI_BASE + reg::CONTROL_4), 0xffff); + let _ = rig.chip.ni.power_on(&rig.chip.levels); + for a in offsets { + assert_eq!(rig.r(NI_BASE + a), 0, "register {a:#x}"); + } + } + + /// An uncabled one-node port reads status bits 1:0 = 1: link down, bit 0 + /// set. Reading 0 there makes the guest's bus-error handler print a + /// link-reset notice. Bit 18 stays clear (a one-node machine) and a + /// write does not change the value. + #[test] + fn uncabled_port_status_reads_one_and_ignores_writes() { + let rig = Rig::new(); + assert_eq!(rig.r(NI_BASE + reg::PORT_STATUS), 1); + rig.w(NI_BASE + reg::PORT_STATUS, !0); + assert_eq!(rig.r(NI_BASE + reg::PORT_STATUS), 1); + let _ = rig.chip.ni.power_on(&rig.chip.levels); + assert_eq!(rig.r(NI_BASE + reg::PORT_STATUS), 1); + } + + impl Ni { + /// Hold the block's lock, for tests of the paths that must not take it. + pub(in crate::dev::bedrock) fn hold(&self) -> impl Sized + '_ { + self.regs.lock() + } + } +} diff --git a/src/dev/bedrock/pi.rs b/src/dev/bedrock/pi.rs new file mode 100644 index 000000000..c31a652e6 --- /dev/null +++ b/src/dev/bedrock/pi.rs @@ -0,0 +1,957 @@ +//! A Bedrock processor interface: processor-slot control, pending +//! interrupts, the real-time counter and its comparators, the profile timer, +//! error records and the graphics flow-control registers. A node has two +//! PIs with two processor slots each, A and B; the board names which of the +//! four slots hold a processor ([`Population`]). The Fuel fits PI0 slot A. +//! +//! Each slot drives IP2..IP6 of its processor: bank 0 under the slot's bank +//! 0 mask, bank 1 under its bank 1 mask, its RT comparator, the profile +//! timer and the error summary under its error cause. The inputs of those +//! five levels are atomics. A register store or an inbound packet changes +//! one and then publishes the slot's levels into Cause bits 10-14 of its +//! processor's interrupt word, without a lock; see [`Pi::publish`]. +//! +//! # What is approximated here +//! +//! Every fitted slot runs its processor from reset, with its CPU-enable bit +//! set, whatever the register later holds: no cold-reset value for the bit +//! is known and the register is not known to stop a processor. A write that +//! clears a running slot's enable is stored and logged, not carried out. +//! +//! For a slot with no processor nothing is delivered. A store that arms +//! such a slot is logged under the `bedrock` module instead of being dropped +//! silently: a CPU enable, an NMI, a nonzero interrupt mask (an +//! interprocessor interrupt needs one), a timer enable and an error cause. +//! The register itself is stored, as hardware holds it. No processor model +//! has an NMI entry, so every NMI is logged. +//! +//! The CPU-number register answers the reading processor's slice: its slot +//! in bit 0 and the interface in bit 1. A read from a thread that runs no +//! processor (a device, the monitor, a test) answers slot A; hardware always +//! has a requester. +//! +//! The model accepts Linux's public `SGIrules` key at offset 0x8 for +//! compatibility. Linux documents the literal for Hub I/O, but the available +//! evidence does not establish that this PI offset maps to the same register. +//! +//! A board can name the sources of the pending levels a widget's interrupt +//! packets set ([`Pi::set_level_source`]). When the processor clears such a +//! level while its source stays active, the PI sets it again a millisecond +//! later, as QEMU's PI does for a level-triggered PCI input whose packet +//! would not be sent again: a host bridge sends a packet on an edge only. +//! That reassertion and its delay are QEMU's choices, not a measured +//! Bedrock behaviour. +//! +//! The graphics flow-control registers store what software writes and count +//! nothing: no uncached store into the graphics page takes a credit and no +//! board returns one. +//! +//! Ported from hw/misc/sgi-bedrock-pi.c in jamesbraid/qemu-sgi-origami. + +use super::reg::IIO_PROTECT_OVRRD_KEY; +use crate::dev::odyssey::{PiGfx, PI_GFX_REGS}; +use super::HostTimer; +use crate::hptimer::TimerId; +use parking_lot::Mutex; +use std::sync::atomic::{fence, AtomicPtr, AtomicU64, Ordering::{Acquire, Relaxed, Release, SeqCst}}; + +pub(super) const RT_MASK: u64 = (1 << 55) - 1; +pub(super) const ERROR_MASK: u64 = (1 << 35) - 1; +pub(super) const MEMORY_ERROR: u64 = 1 << 24; +/// Bank 1 bits that reflect another block's level; a pending-modify write +/// cannot change them. +pub(super) const REFLECT: u64 = (1 << 54) | (1 << 55) | (1 << 62) | (1 << 63); +/// Nanoseconds per real-time counter tick: the counter runs at 1.25 MHz. +pub(super) const TICK: u64 = 800; +/// Length of a PI's register aperture. +pub(super) const SIZE: u64 = 0x30108; +/// IP2 is Cause bit 10. +const CAUSE_IP2: u32 = 10; +/// The request region the requests and the protection key name. +const REQUEST_REGION: u64 = 0; +/// How long after the processor clears a level whose source is still active +/// the level is set again, in host nanoseconds. +const REASSERT_NS: u64 = 1_000_000; +/// Level sources a board can name: 16 inputs for each of up to 16 bridges. +const SOURCES: usize = 256; + +pub mod reg { + pub const NODE_REGION: u64 = 0x0; + pub const PROTECTION_KEY: u64 = 0x8; + pub const OFFSET_0010: u64 = 0x10; + pub const OFFSET_0018: u64 = 0x18; + pub const INTERFACE_ID: u64 = 0x20; + pub const CACHED_ALIAS: u64 = 0x28; + pub const OFFSET_0030: u64 = 0x30; + pub const OFFSET_0038: u64 = 0x38; + pub const INTERFACE_SLOT_A: u64 = 0x40; + pub const INTERFACE_SLOT_B: u64 = 0x48; + pub const CPU_A_ENABLE: u64 = 0x50; + pub const CPU_B_ENABLE: u64 = 0x58; + pub const NMI_SLOT_A: u64 = 0x70; + pub const NMI_SLOT_B: u64 = 0x78; + pub const PENDING_MODIFY: u64 = 0x90; + pub const PENDING_STATUS_A: u64 = 0x98; + pub const PENDING_STATUS_B: u64 = 0xa0; + pub const OFFSET_00A8: u64 = 0xa8; + pub const OFFSET_00B0: u64 = 0xb0; + pub const OFFSET_00B8: u64 = 0xb8; + pub const OFFSET_00C0: u64 = 0xc0; + pub const REQUEST_A: u64 = 0xc8; + pub const REQUEST_B: u64 = 0xd0; + pub const PENDING_A_CLEAR: u64 = 0xd8; + pub const PENDING_B_CLEAR: u64 = 0xe0; + pub const OFFSET_00E8: u64 = 0xe8; + pub const OFFSET_00F0: u64 = 0xf0; + pub const RT_COMPARE_A: u64 = 0x108; + pub const RT_COMPARE_B: u64 = 0x110; + pub const PROFILE_COMPARE: u64 = 0x118; + pub const RT_STATUS_A: u64 = 0x120; + pub const RT_STATUS_B: u64 = 0x128; + pub const PROFILE_STATUS_A: u64 = 0x130; + pub const PROFILE_STATUS_B: u64 = 0x138; + pub const RT_ENABLE_A: u64 = 0x140; + pub const RT_ENABLE_B: u64 = 0x148; + pub const PROFILE_ENABLE_A: u64 = 0x150; + pub const PROFILE_ENABLE_B: u64 = 0x158; + pub const PENDING_MODIFY_B: u64 = 0x180; + pub const ERROR_SUMMARY_ALIAS: u64 = 0x3f8; + pub const ERROR_SUMMARY: u64 = 0x400; + pub const ERROR_CAUSE_A: u64 = 0x408; + pub const ERROR_CAUSE_B: u64 = 0x410; + pub const ERROR_ADDRESS_A: u64 = 0x418; + pub const ERROR_ADDRESS_B: u64 = 0x420; + pub const ERROR_CONTROL: u64 = 0x428; + pub const ERROR_CAPTURE_A: u64 = 0x438; + pub const ERROR_CAPTURE_B: u64 = 0x448; + pub const ERROR_CAPTURE_C: u64 = 0x458; + pub const ERROR_CAPTURE_D: u64 = 0x468; + pub const ERROR_SOURCE_A: u64 = 0x490; + pub const ERROR_SOURCE_B: u64 = 0x4c0; + pub const ERROR_DETAIL_A: u64 = 0x4a8; + pub const ERROR_DETAIL_B: u64 = 0x4b0; + pub const ERROR_DETAIL_C: u64 = 0x4b8; + pub const OFFSET_0430: u64 = 0x430; + pub const OFFSET_0440: u64 = 0x440; + pub const OFFSET_0450: u64 = 0x450; + pub const OFFSET_0460: u64 = 0x460; + pub const OFFSET_04C8: u64 = 0x4c8; + pub const OFFSET_04D0: u64 = 0x4d0; + pub const CREDIT_MODE: u64 = 0x20068; + pub const COUNTER: u64 = 0x30100; + pub const GRAPHICS_BASE: u64 = 0x300; + pub const GRAPHICS_END: u64 = 0x350; +} + +/// The processor slots a board fits, one bit per slot: bit `interface * 2 + +/// slot`, slot 0 being A. CPU 0 sits in PI0 slot A, so every population has +/// it; a board fills the others in the order PI0/A, PI1/A, PI0/B, PI1/B, so +/// two processors sit one per interface. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Population(u8); +impl Default for Population { + fn default() -> Self { + Self::SINGLE + } +} +impl Population { + /// One processor, in PI0 slot A. + pub const SINGLE: Self = Self(0b0001); + /// Slots named by `mask`. CPU 0 runs in PI0/A, so it must be fitted, and + /// only the four slots exist. + pub fn new(mask: u8) -> Result { + if mask & !0b1111 != 0 { + return Err(format!("processor mask {mask:#x} names a slot outside the node's four")); + } + if mask & 1 == 0 { + return Err(format!("processor mask {mask:#x} leaves PI0 slot A empty, where CPU 0 runs")); + } + Ok(Self(mask)) + } + /// `count` processors in the board's order: PI0/A, PI1/A, PI0/B, PI1/B. + pub fn fitted_cpus(count: usize) -> Result { + if !(1..=4).contains(&count) { + return Err(format!("{count} processors; a node fits 1 to 4")); + } + Self::new(ORDER[..count].iter().fold(0, |mask, bit| mask | (1 << bit))) + } + /// The slot (interface, 0 = A or 1 = B) CPU `cpu` sits in: the board puts + /// CPU n in the nth slot of this set in its order. + pub fn slot_of(self, cpu: usize) -> Option<(u8, usize)> { + ORDER.iter().filter(|&&bit| self.0 & (1 << bit) != 0).nth(cpu).map(|&bit| (bit / 2, usize::from(bit % 2))) + } + pub fn mask(self) -> u8 { + self.0 + } + pub fn count(self) -> usize { + self.0.count_ones() as usize + } + /// Whether a processor is fitted in `slot` (0 = A, 1 = B) of `interface`. + pub fn fitted(self, interface: u8, slot: usize) -> bool { + self.0 & (1 << (usize::from(interface) * 2 + slot)) != 0 + } +} +/// The board's population order, as slot numbers `interface * 2 + slot`: +/// PI0/A, PI1/A, PI0/B, PI1/B. +const ORDER: [u8; 4] = [0, 2, 1, 3]; + +/// Bank 0 bit 4 and bank 1 bits 54, 55 and 62: levels other blocks of the +/// chip drive, which both PIs reflect. Each writer sets its own bit under +/// its own lock, so two writers' levels land in register order. +#[derive(Default)] +pub(super) struct Levels([AtomicU64; 2]); +impl Levels { + pub(super) const UART: (usize, u32) = (0, 4); + pub(super) const XB_ERROR: (usize, u32) = (1, 54); + pub(super) const LB_ERROR: (usize, u32) = (1, 55); + pub(super) const NI_ERROR: (usize, u32) = (1, 62); + /// Drive `(bank, bit)` to `level`. Whether the bit changed. + pub(super) fn set(&self, (bank, bit): (usize, u32), level: bool) -> bool { + let mask = 1u64 << bit; + let before = if level { + self.0[bank].fetch_or(mask, SeqCst) + } else { + self.0[bank].fetch_and(!mask, SeqCst) + }; + (before & mask != 0) != level + } + fn bank(&self, bank: usize) -> u64 { + self.0[bank].load(SeqCst) + } +} + +/// The real-time counter's origin, read without a lock. The counter is +/// `base + (time - epoch) / TICK`, `time` being the pause instant while the +/// machine is stopped. A writer holds the PI's timer lock and makes `seq` +/// odd while it changes the three words; a reader retries while `seq` is +/// odd or moved. +struct Counter { + seq: AtomicU64, + base: AtomicU64, + epoch: AtomicU64, + /// The pause instant, or `RUNNING`. + paused: AtomicU64, +} +const RUNNING: u64 = u64::MAX; +#[derive(Clone, Copy)] +struct Origin { + base: u64, + epoch: u64, + paused: u64, +} +impl Origin { + fn time(&self, now: u64) -> u64 { + if self.paused == RUNNING { now } else { self.paused } + } + fn count(&self, now: u64) -> u64 { + self.base.wrapping_add(self.time(now).saturating_sub(self.epoch) / TICK) & RT_MASK + } + /// The start of the tick `time` falls in. + fn tick_start(&self, time: u64) -> u64 { + time - time.saturating_sub(self.epoch) % TICK + } +} +impl Counter { + fn new(now: u64) -> Self { + Self { seq: AtomicU64::new(0), base: AtomicU64::new(0), epoch: AtomicU64::new(now), paused: AtomicU64::new(now) } + } + fn read(&self) -> Origin { + loop { + let seq = self.seq.load(Acquire); + let origin = Origin { base: self.base.load(Relaxed), epoch: self.epoch.load(Relaxed), paused: self.paused.load(Relaxed) }; + fence(Acquire); + if seq & 1 == 0 && self.seq.load(Relaxed) == seq { + return origin; + } + std::hint::spin_loop(); + } + } + /// Publish `origin`. The caller holds the timer lock, so writers are one at a time. + fn write(&self, origin: Origin) { + let seq = self.seq.load(Relaxed); + self.seq.store(seq + 1, Relaxed); + fence(Release); + self.base.store(origin.base, Relaxed); + self.epoch.store(origin.epoch, Relaxed); + self.paused.store(origin.paused, Relaxed); + self.seq.store(seq + 2, Release); + } +} + +/// The comparators and their deadlines, in host nanoseconds, which change +/// together and with the counter's origin. +struct Timer { + rt_compare: [u64; 2], + profile_compare: u64, + rt_due: [u128; 2], + profile_due: u128, + /// When each pending level whose source is active comes back, 0 for + /// none. + reassert: [u64; 128], + /// The host timer that fires at the earliest deadline. + id: Option, +} + +/// The registers that are not interrupt inputs: stored words and the +/// graphics registers. +struct State { + w: [u64; 0x4d8 / 8], + gfx: PiGfx, + credit_mode: u64, +} +impl State { + fn new() -> Self { + let mut w = [0; 0x4d8 / 8]; + w[reg::NODE_REGION as usize / 8] = !0; + w[reg::OFFSET_0018 as usize / 8] = !0; + Self { w, gfx: PiGfx::default(), credit_mode: 0 } + } +} + +pub(super) struct Pi { + interface: u8, + processors: Population, + pending: [AtomicU64; 2], + /// Each slot's bank 0 and bank 1 masks: 0xa8 and 0xb0 for A, 0xb8 and + /// 0xc0 for B. + mask: [[AtomicU64; 2]; 2], + /// Requests A and B and their enable, which make bank 0 bits 5 and 6. + request: [AtomicU64; 2], + request_enable: AtomicU64, + /// The regions an inbound interrupt packet may come from (0x10). + region: AtomicU64, + rt_status: [AtomicU64; 2], + rt_enable: [AtomicU64; 2], + profile_status: [AtomicU64; 2], + profile_enable: [AtomicU64; 2], + error_summary: AtomicU64, + error_cause: [AtomicU64; 2], + /// The pending levels with an active source, a bit per level in two + /// banks, kept from `sources` for the clear path. + sourced: [AtomicU64; 2], + /// Which level each named source drives, plus one; 0 while inactive. + sources: Mutex<[u8; SOURCES]>, + /// Each slot's processor's interrupt word, null for an empty slot. + words: [AtomicPtr; 2], + counter: Counter, + /// The earliest deadline in host nanoseconds; `u64::MAX` while stopped. + next_due: AtomicU64, + timer: Mutex, + state: Mutex, +} + +impl Pi { + pub(super) fn new(interface: u8, processors: Population, now: u64) -> Self { + let pi = Self { + interface, + processors, + pending: Default::default(), + mask: Default::default(), + request: Default::default(), + request_enable: AtomicU64::new(0), + region: AtomicU64::new(!0), + rt_status: Default::default(), + rt_enable: Default::default(), + profile_status: Default::default(), + profile_enable: Default::default(), + error_summary: AtomicU64::new(0), + error_cause: Default::default(), + sourced: Default::default(), + sources: Mutex::new([0; SOURCES]), + words: Default::default(), + counter: Counter::new(now), + next_due: AtomicU64::new(RUNNING), + timer: Mutex::new(Timer { + rt_compare: [0; 2], profile_compare: 0, rt_due: [0; 2], profile_due: 0, reassert: [0; 128], id: None, + }), + state: Mutex::new(State::new()), + }; + let mut s = pi.state.lock(); + for slot in 0..2 { + s.w[reg::CPU_A_ENABLE as usize / 8 + slot] = u64::from(pi.started(slot)); + } + drop(s); + pi.arm_all(&mut pi.timer.lock(), now); + pi + } + + pub(super) fn state_locked(&self) -> bool { + self.state.is_locked() + } + pub(super) fn timer_locked(&self) -> bool { + self.timer.is_locked() + } + /// Where slot `slot`'s IP2..IP6 go. + pub(super) fn set_word(&self, slot: usize, word: *const AtomicU64) { + self.words[slot].store(word as *mut AtomicU64, SeqCst); + } + + /// Whether a processor sits in `slot` of this interface. + fn started(&self, slot: usize) -> bool { + self.processors.fitted(self.interface, slot) + } + + /// Report an access to a register the model does not implement. + pub(super) fn note(&self, off: u64, write: bool) { + crate::dlog!(crate::devlog::LogModule::Bedrock, + "Bedrock PI{} unsupported {} at {off:#x}", self.interface, + if write { "write" } else { "read" }); + } + + /// Log a store to slot `slot` this interface does not carry out. + fn refuse(&self, what: &str, slot: usize) { + crate::dlog!(crate::devlog::LogModule::Bedrock, + "Bedrock PI{} slot {}: not carried out: {what}", self.interface, if slot == 0 { 'A' } else { 'B' }); + } + + fn pending0(&self, levels: &Levels) -> u64 { + let enable = self.request_enable.load(SeqCst); + self.pending[0].load(SeqCst) + | levels.bank(0) + | u64::from(self.request[0].load(SeqCst) & enable != 0) << 5 + | u64::from(self.request[1].load(SeqCst) & enable != 0) << 6 + } + fn pending1(&self, levels: &Levels) -> u64 { + (self.pending[1].load(SeqCst) & !REFLECT) | levels.bank(1) + } + + /// Slot `slot`'s IP2..IP6, bit 0 being IP2. + fn outputs(&self, slot: usize, levels: &Levels) -> u64 { + u64::from(self.pending0(levels) & self.mask[slot][0].load(SeqCst) != 0) + | u64::from(self.pending1(levels) & self.mask[slot][1].load(SeqCst) != 0) << 1 + | (self.rt_status[slot].load(SeqCst) & self.rt_enable[slot].load(SeqCst)) << 2 + | (self.profile_status[slot].load(SeqCst) & self.profile_enable[slot].load(SeqCst)) << 3 + | u64::from(self.error_summary.load(SeqCst) & self.error_cause[slot].load(SeqCst) != 0) << 4 + } + + /// Bring each processor's IP2..IP6 to what the registers say. Every + /// input change is a sequentially consistent store or read-modify-write + /// followed by a publish, so no lock is needed: take the last write to a + /// word. Its writer read the inputs again after it, and would have + /// written again had any change before that read left the word stale. A + /// change after that read is followed by its own publish, which would + /// make a later write. So once writers stop, the word matches the + /// registers. A publish that changes nothing reads the inputs and the + /// word and stores nothing. Only the PI writes Cause bits 10-14. + pub(super) fn publish(&self, levels: &Levels) { + for slot in 0..2 { + let p = self.words[slot].load(Acquire); + if p.is_null() { + continue; + } + // SAFETY: `Bedrock::set_cpu_interrupts`' contract: the word + // outlives the chip. + let word = unsafe { &*p }; + loop { + let want = self.outputs(slot, levels); + let have = (word.load(SeqCst) >> CAUSE_IP2) & 0x1f; + if want == have { + break; + } + let (set, clear) = (want & !have, have & !want); + if set != 0 { + word.fetch_or(set << CAUSE_IP2, SeqCst); + #[cfg(feature = "idle-pause")] + crate::cpu::idle_park::wake(); + } + if clear != 0 { + word.fetch_and(!(clear << CAUSE_IP2), SeqCst); + } + } + } + } + + /// Set or clear one pending bit, as a pending-modify store does: bit 8 + /// sets, bit 6 picks the bank and bits 5:0 the bit. The reflected bits + /// of bank 1 are left alone. + fn modify(&self, value: u64) { + let bank = ((value >> 6) & 1) as usize; + let mask = 1u64 << (value & 63); + if bank == 1 && mask & REFLECT != 0 { + return; + } + if value & 0x100 != 0 { + self.pending[bank].fetch_or(mask, SeqCst); + } else { + self.pending[bank].fetch_and(!mask, SeqCst); + } + } + + /// An interrupt packet from region `region` to the pending-modify + /// register. Refused, and false, unless the region may send one. + pub(super) fn io_modify(&self, value: u64, region: u8) -> bool { + if region >= 64 || self.region.load(SeqCst) & (1 << region) == 0 { + self.note(reg::PENDING_MODIFY, true); + return false; + } + self.modify(value); + true + } + + /// Name `source` as driving pending `level` (7 to 127) while `active`, + /// or as inactive. A clear of a level whose source is active brings the + /// level back after [`REASSERT_NS`]. + pub(super) fn set_level_source(&self, source: usize, level: u8, active: bool) { + if source >= SOURCES || (active && !(7..=127).contains(&level)) { + return; + } + let mut sources = self.sources.lock(); + sources[source] = if active { level + 1 } else { 0 }; + let mut sourced = [0u64; 2]; + for &v in sources.iter().filter(|&&v| v != 0) { + let level = v - 1; + sourced[usize::from(level / 64)] |= 1 << (level % 64); + } + for (bank, bits) in sourced.into_iter().enumerate() { + self.sourced[bank].store(bits, SeqCst); + } + } + + /// Whether pending `level` (bank 1 from 64) has an active source. + fn is_sourced(&self, level: usize) -> bool { + self.sourced[level / 64].load(SeqCst) & (1 << (level % 64)) != 0 + } + + /// The real-time counter at host time `now`. Lock-free. + pub(super) fn count(&self, now: u64) -> u64 { + self.counter.read().count(now) + } + + fn arm_rt(&self, t: &mut Timer, o: &Origin, slot: usize, now: u64) { + let time = o.time(now); + let ticks = (t.rt_compare[slot].wrapping_sub(o.count(now)).wrapping_sub(1) & RT_MASK) + 1; + t.rt_due[slot] = u128::from(o.tick_start(time)) + u128::from(ticks) * u128::from(TICK); + } + fn arm_profile(&self, t: &mut Timer, o: &Origin, now: u64) { + let time = o.time(now); + let ticks = (t.profile_compare as u32).wrapping_sub(o.count(now) as u32); + let ticks = if ticks == 0 { 1u64 << 32 } else { u64::from(ticks) }; + t.profile_due = u128::from(o.tick_start(time)) + u128::from(ticks) * u128::from(TICK); + } + fn arm_all(&self, t: &mut Timer, now: u64) { + let o = self.counter.read(); + self.arm_rt(t, &o, 0, now); + self.arm_rt(t, &o, 1, now); + self.arm_profile(t, &o, now); + self.set_next_due(t, &o); + } + fn set_next_due(&self, t: &Timer, o: &Origin) { + let due = if o.paused != RUNNING { + RUNNING + } else { + t.rt_due.iter().copied().chain([t.profile_due]) + .chain(t.reassert.iter().filter(|&&due| due != 0).map(|&due| u128::from(due))) + .min().unwrap().min(u128::from(RUNNING - 1)) as u64 + }; + self.next_due.store(due, SeqCst); + } + + /// Latch the comparators that are due at `now`, under the timer lock. + /// Whether one latched. + fn latch(&self, t: &mut Timer, now: u64) -> bool { + let o = self.counter.read(); + let time = o.time(now); + let mut latched = false; + for slot in 0..2 { + if u128::from(time) >= t.rt_due[slot] { + self.rt_status[slot].store(1, SeqCst); + self.arm_rt(t, &o, slot, now); + latched = true; + } + } + if u128::from(time) >= t.profile_due { + self.profile_status[0].store(1, SeqCst); + self.profile_status[1].store(1, SeqCst); + self.arm_profile(t, &o, now); + latched = true; + } + for level in 7..128 { + let due = t.reassert[level]; + if due != 0 && time >= due { + t.reassert[level] = 0; + if self.is_sourced(level) { + self.pending[level / 64].fetch_or(1 << (level % 64), SeqCst); + latched = true; + } + } + } + self.set_next_due(t, &o); + latched + } + + /// Latch what is due at `now`: one load when nothing is. Whether a + /// comparator latched, so the caller publishes. + pub(super) fn sync(&self, now: u64) -> bool { + if now < self.next_due.load(SeqCst) { + return false; + } + self.latch(&mut self.timer.lock(), now) + } + + /// The host timer fired: latch what is due and arm the next deadline. + pub(super) fn expire(&self, host: &HostTimer, now: u64) -> bool { + let mut t = self.timer.lock(); + let latched = self.latch(&mut t, now); + self.arm_host(&mut t, host, now); + latched + } + + /// (Re)arm the host timer for the earliest deadline, under the timer + /// lock. A deadline more than a second out is armed for a second. + fn arm_host(&self, t: &mut Timer, host: &HostTimer, now: u64) { + if let Some(id) = t.id.take() { + host.tm.remove(id); + } + let due = self.next_due.load(SeqCst); + if due == RUNNING { + return; + } + let delay = due.saturating_sub(now).clamp(1, 1_000_000_000); + t.id = Some(host.arm(self.interface, delay)); + } + + /// Re-arm after the comparators or the counter moved. + pub(super) fn reschedule(&self, host: Option<&HostTimer>, now: u64) { + if let Some(host) = host { + self.arm_host(&mut self.timer.lock(), host, now); + } + } + + /// The machine started at `now`: the counter resumes from where it paused. + pub(super) fn start(&self, host: Option<&HostTimer>, now: u64) { + let mut t = self.timer.lock(); + let mut o = self.counter.read(); + if o.paused == RUNNING { + return; + } + let delta = now.saturating_sub(o.paused); + o.epoch = o.epoch.saturating_add(delta); + o.paused = RUNNING; + self.counter.write(o); + for due in &mut t.rt_due { + *due += u128::from(delta); + } + t.profile_due += u128::from(delta); + for due in t.reassert.iter_mut().filter(|due| **due != 0) { + *due = due.saturating_add(delta); + } + self.set_next_due(&t, &o); + if let Some(host) = host { + self.arm_host(&mut t, host, now); + } + } + + /// The machine stopped at `now`: latch what is due, then freeze the counter. + pub(super) fn stop(&self, host: Option<&HostTimer>, now: u64) -> bool { + let mut t = self.timer.lock(); + let latched = self.latch(&mut t, now); + let mut o = self.counter.read(); + if o.paused == RUNNING { + o.paused = now; + self.counter.write(o); + } + self.next_due.store(RUNNING, SeqCst); + if let (Some(host), Some(id)) = (host, t.id.take()) { + host.tm.remove(id); + } + latched + } + + pub(super) fn is_running(&self) -> bool { + self.counter.read().paused == RUNNING + } + + /// The CPU-number register: the reading processor's slot in bit 0 and + /// this interface in bit 1; slot A for a reader that is no processor + /// here. + fn cpu_number(&self) -> u64 { + let slot = crate::cpu::smp::on_cpu_thread() + .and_then(|cpu| self.processors.slot_of(cpu)) + .filter(|&(interface, _)| interface == self.interface) + .map_or(0, |(_, slot)| slot as u64); + slot | u64::from(self.interface) << 1 + } + + /// A doubleword read at `off`, and whether a comparator latched on the + /// way, so the caller publishes. `None` for a register the model does + /// not have. + pub(super) fn read(&self, off: u64, now: u64, levels: &Levels) -> (Option, bool) { + let mut latched = false; + let value = match off { + reg::INTERFACE_ID => self.cpu_number(), + reg::INTERFACE_SLOT_A => u64::from(self.started(0)), + reg::INTERFACE_SLOT_B => u64::from(self.started(1)), + reg::OFFSET_0010 => self.region.load(SeqCst), + reg::PENDING_STATUS_A | reg::PENDING_STATUS_B => { + // A level due to come back shows in the read. + latched = self.sync(now); + if off == reg::PENDING_STATUS_A { self.pending0(levels) } else { self.pending1(levels) } + } + reg::OFFSET_00A8 | reg::OFFSET_00B0 | reg::OFFSET_00B8 | reg::OFFSET_00C0 => { + let n = (off - reg::OFFSET_00A8) as usize / 8; + self.mask[n / 2][n % 2].load(SeqCst) + } + reg::REQUEST_A | reg::PENDING_A_CLEAR => self.request[0].load(SeqCst), + reg::REQUEST_B | reg::PENDING_B_CLEAR => self.request[1].load(SeqCst), + reg::OFFSET_00E8 => self.request_enable.load(SeqCst), + reg::RT_COMPARE_A | reg::RT_COMPARE_B => self.timer.lock().rt_compare[((off - reg::RT_COMPARE_A) / 8) as usize], + reg::PROFILE_COMPARE => self.timer.lock().profile_compare, + reg::RT_STATUS_A | reg::RT_STATUS_B => { + latched = self.sync(now); + self.rt_status[((off - reg::RT_STATUS_A) / 8) as usize].load(SeqCst) + } + reg::PROFILE_STATUS_A | reg::PROFILE_STATUS_B => { + latched = self.sync(now); + self.profile_status[((off - reg::PROFILE_STATUS_A) / 8) as usize].load(SeqCst) + } + reg::RT_ENABLE_A | reg::RT_ENABLE_B => self.rt_enable[((off - reg::RT_ENABLE_A) / 8) as usize].load(SeqCst), + reg::PROFILE_ENABLE_A | reg::PROFILE_ENABLE_B => self.profile_enable[((off - reg::PROFILE_ENABLE_A) / 8) as usize].load(SeqCst), + reg::ERROR_SUMMARY | reg::ERROR_SUMMARY_ALIAS => self.error_summary.load(SeqCst), + reg::ERROR_CAUSE_A | reg::ERROR_CAUSE_B => self.error_cause[((off - reg::ERROR_CAUSE_A) / 8) as usize].load(SeqCst), + reg::COUNTER => self.count(now), + reg::GRAPHICS_BASE..reg::GRAPHICS_END if off % 8 == 0 => { + let n = ((off - reg::GRAPHICS_BASE) / 8) as u32; + debug_assert!(n < PI_GFX_REGS); + self.state.lock().gfx.read(n) + } + reg::CREDIT_MODE => self.state.lock().credit_mode, + reg::ERROR_CAPTURE_A | reg::ERROR_CAPTURE_B | reg::ERROR_CAPTURE_C | reg::ERROR_CAPTURE_D => { + self.state.lock().w[(off - reg::PROTECTION_KEY) as usize / 8] + } + _ if stored(off) => self.state.lock().w[off as usize / 8], + _ => { + self.note(off, false); + return (None, false); + } + }; + (Some(value), latched) + } + + /// A doubleword store at `off`. Whether the comparators or the counter + /// moved, so the caller re-arms the host timer. The caller publishes. + pub(super) fn write(&self, off: u64, value: u64, now: u64) -> bool { + let index = |base: u64| ((off - base) / 8) as usize; + match off { + reg::OFFSET_0010 => self.region.store(value, SeqCst), + reg::OFFSET_00A8 | reg::OFFSET_00B0 | reg::OFFSET_00B8 | reg::OFFSET_00C0 => { + let n = index(reg::OFFSET_00A8); + self.mask[n / 2][n % 2].store(value, SeqCst); + if value != 0 && !self.started(n / 2) { + self.refuse("interrupt mask for an empty slot", n / 2); + } + } + reg::PENDING_MODIFY | reg::PENDING_MODIFY_B => { + self.modify(value); + let level = ((value >> 6) & 1) as usize * 64 + (value & 63) as usize; + if value & 0x100 == 0 && self.is_sourced(level) { + let mut t = self.timer.lock(); + let o = self.counter.read(); + t.reassert[level] = o.time(now).saturating_add(REASSERT_NS); + self.set_next_due(&t, &o); + return true; + } + } + reg::REQUEST_A | reg::REQUEST_B => { + self.request[index(reg::REQUEST_A)].fetch_or(1 << REQUEST_REGION, SeqCst); + } + reg::PENDING_A_CLEAR | reg::PENDING_B_CLEAR => { + self.request[index(reg::PENDING_A_CLEAR)].fetch_and(!value, SeqCst); + } + reg::OFFSET_00E8 => self.request_enable.store(value, SeqCst), + reg::RT_STATUS_A | reg::RT_STATUS_B => { + // Latch a deadline that has passed first, so the clear takes it. + self.sync(now); + self.rt_status[index(reg::RT_STATUS_A)].store(0, SeqCst); + } + reg::PROFILE_STATUS_A | reg::PROFILE_STATUS_B => { + self.sync(now); + self.profile_status[index(reg::PROFILE_STATUS_A)].store(0, SeqCst); + } + reg::RT_ENABLE_A | reg::RT_ENABLE_B | reg::PROFILE_ENABLE_A | reg::PROFILE_ENABLE_B => { + let rt = matches!(off, reg::RT_ENABLE_A | reg::RT_ENABLE_B); + let slot = index(if rt { reg::RT_ENABLE_A } else { reg::PROFILE_ENABLE_A }); + if rt { &self.rt_enable[slot] } else { &self.profile_enable[slot] }.store(value & 1, SeqCst); + if value & 1 != 0 && !self.started(slot) { + self.refuse("timer enable for an empty slot", slot); + } + } + reg::ERROR_SUMMARY_ALIAS => { + // Software sets every bit but the memory error, which only MD sets. + let mut summary = self.error_summary.load(SeqCst); + while let Err(now) = self.error_summary.compare_exchange_weak( + summary, (summary & MEMORY_ERROR) | (value & ERROR_MASK & !MEMORY_ERROR), SeqCst, SeqCst, + ) { + summary = now; + } + } + reg::ERROR_SUMMARY => { + self.error_summary.fetch_and(!(value & ERROR_MASK), SeqCst); + } + reg::ERROR_CAUSE_A | reg::ERROR_CAUSE_B => { + let slot = index(reg::ERROR_CAUSE_A); + self.error_cause[slot].store(value & ERROR_MASK, SeqCst); + if value & ERROR_MASK != 0 && !self.started(slot) { + self.refuse("error cause for an empty slot", slot); + } + } + reg::RT_COMPARE_A | reg::RT_COMPARE_B | reg::PROFILE_COMPARE | reg::COUNTER => { + let mut t = self.timer.lock(); + self.latch(&mut t, now); + let mut o = self.counter.read(); + match off { + reg::RT_COMPARE_A | reg::RT_COMPARE_B => { + let slot = index(reg::RT_COMPARE_A); + t.rt_compare[slot] = value & RT_MASK; + self.arm_rt(&mut t, &o, slot, now); + } + reg::PROFILE_COMPARE => { + t.profile_compare = value & 0xffff_ffff; + self.arm_profile(&mut t, &o, now); + } + _ => { + o.base = value & RT_MASK; + o.epoch = o.time(now); + self.counter.write(o); + self.arm_rt(&mut t, &o, 0, now); + self.arm_rt(&mut t, &o, 1, now); + self.arm_profile(&mut t, &o, now); + } + } + self.set_next_due(&t, &o); + return true; + } + reg::GRAPHICS_BASE..reg::GRAPHICS_END if off % 8 == 0 => { + let n = ((off - reg::GRAPHICS_BASE) / 8) as u32; + debug_assert!(n < PI_GFX_REGS); + self.state.lock().gfx.write(n, value); + } + reg::CREDIT_MODE => self.state.lock().credit_mode = value & 1, + _ => self.write_stored(off, value), + } + false + } + + /// The stores that only change a stored word. + fn write_stored(&self, off: u64, value: u64) { + let mut s = self.state.lock(); + let w = |s: &mut State, at: u64, v: u64| s.w[at as usize / 8] = v; + match off { + reg::NODE_REGION | reg::OFFSET_0018 | reg::OFFSET_04C8 | reg::OFFSET_04D0 => w(&mut s, off, value), + reg::PROTECTION_KEY => { + if value == IIO_PROTECT_OVRRD_KEY { + s.w[0] |= 1 << REQUEST_REGION; + } + } + reg::CACHED_ALIAS => w(&mut s, off, value & 15), + reg::OFFSET_0030 => w(&mut s, off, value & 255), + reg::OFFSET_0038 => w(&mut s, off, value & 0xffffff), + reg::CPU_A_ENABLE | reg::CPU_B_ENABLE => { + let slot = ((off - reg::CPU_A_ENABLE) / 8) as usize; + w(&mut s, off, value & 1); + drop(s); + if value & 1 != 0 && !self.started(slot) { + self.refuse("CPU enable for an empty slot", slot); + } else if value & 1 == 0 && self.started(slot) { + self.refuse("CPU disable; the processor keeps running", slot); + } + } + reg::NMI_SLOT_A | reg::NMI_SLOT_B => { + drop(s); + if value & 1 != 0 { + self.refuse("NMI; no processor model takes one", ((off - reg::NMI_SLOT_A) / 8) as usize); + } else { + self.note(off, true); + } + } + reg::OFFSET_00F0 => { + w(&mut s, off, value & 31); + drop(s); + if value & 31 != 0 { + self.note(off, true); + } + } + reg::ERROR_ADDRESS_A | reg::ERROR_ADDRESS_B => w(&mut s, off, value & 0x1fffffff8), + reg::ERROR_CONTROL => { + w(&mut s, off, value & 15); + drop(s); + if value & 15 != 0 { + self.note(off, true); + } + } + reg::ERROR_CAPTURE_A | reg::ERROR_CAPTURE_B | reg::ERROR_CAPTURE_C | reg::ERROR_CAPTURE_D => { + let slot = usize::from(off >= 0x450); + s.w[0x430 / 8 + slot * 4] = 0; + s.w[0x440 / 8 + slot * 4] = 0; + } + reg::ERROR_SOURCE_A | reg::ERROR_SOURCE_B => w(&mut s, off, value & 63), + reg::ERROR_DETAIL_A | reg::ERROR_DETAIL_B => w(&mut s, off, value & 0x1fffff), + reg::ERROR_DETAIL_C => w(&mut s, off, value & 0xfffff), + _ => { + drop(s); + self.note(off, true); + } + } + } + + /// The PI block's local reset at `now`. The controls and captures + /// hardware keeps across it survive; the interrupt inputs, timers, + /// counter and graphics registers start over. + pub(super) fn power_on(&self, now: u64) { + let mut s = self.state.lock(); + let old = s.w; + *s = State::new(); + for off in [0, 0x18, 0x28, 0x50, 0x58, 0x418, 0x420, 0x428, 0x430, 0x440, 0x450, 0x460, 0x4c8, 0x4d0] { + s.w[off / 8] = old[off / 8]; + } + drop(s); + for bank in &self.pending { + bank.store(0, SeqCst); + } + for a in self.mask.iter().flatten().chain(&self.request).chain([&self.request_enable]) + .chain(&self.rt_status).chain(&self.rt_enable).chain(&self.profile_status) + .chain(&self.profile_enable).chain(&self.error_cause) + { + a.store(0, SeqCst); + } + let mut t = self.timer.lock(); + t.rt_compare = [0; 2]; + t.profile_compare = 0; + t.reassert = [0; 128]; + let o = self.counter.read(); + let time = o.time(now); + self.counter.write(Origin { base: 0, epoch: time, paused: o.paused }); + self.arm_all(&mut t, now); + } +} + +/// The doubleword registers that only hold what was stored. +fn stored(off: u64) -> bool { + matches!( + off, + reg::NODE_REGION + | reg::OFFSET_0018 + | reg::CACHED_ALIAS + | reg::OFFSET_0030 + | reg::OFFSET_0038 + | reg::CPU_A_ENABLE + | reg::CPU_B_ENABLE + | reg::OFFSET_00F0 + | reg::ERROR_ADDRESS_A + | reg::ERROR_ADDRESS_B + | reg::ERROR_CONTROL + | reg::OFFSET_0430 + | reg::OFFSET_0440 + | reg::OFFSET_0450 + | reg::OFFSET_0460 + | reg::ERROR_SOURCE_A + | reg::ERROR_DETAIL_A + | reg::ERROR_DETAIL_B + | reg::ERROR_DETAIL_C + | reg::ERROR_SOURCE_B + | reg::OFFSET_04C8 + | reg::OFFSET_04D0 + ) +} + +#[cfg(test)] +mod tests; diff --git a/src/dev/bedrock/pi/tests.rs b/src/dev/bedrock/pi/tests.rs new file mode 100644 index 000000000..0caf89d49 --- /dev/null +++ b/src/dev/bedrock/pi/tests.rs @@ -0,0 +1,334 @@ +// Cases follow tests/qtest/sn1-bedrock-pi-test.c in jamesbraid/qemu-sgi-origami: +// power-on-values, local-reset, pending-reflection, real-time-comparators, +// profile-counter-wrap, cpu-number-fallback, cpu-enable-lines and nmi, and +// sn1-cpu-identity-test's four-processors-know-their-slots. + +use super::super::reg::PI1_BASE; +use super::super::Rig; +use super::*; +use crate::traits::{BusDevice, Device, BUS_ERR, BUS_OK}; +use std::sync::atomic::Ordering; + +/// Drive the reflected bank 1 bit 55, as LB's error level does. +fn reflect(rig: &Rig, level: bool) { + if rig.chip.levels.set((1, 55), level) { + rig.chip.publish_all(); + } +} +use std::sync::Arc; +use std::time::Duration; + +/// The PROM reads PI slot presence and relies on the documented reset survivors. +#[test] +fn the_pi_reports_interface_state_and_retains_registers_across_reset() { + { + let rig = Rig::new(); + assert_eq!(rig.r(reg::INTERFACE_ID), 0); + assert_eq!(rig.r(reg::INTERFACE_SLOT_A), 1); + assert_eq!(rig.r(reg::INTERFACE_SLOT_B), 0); + rig.w(reg::NODE_REGION, 0); + rig.w(reg::PROTECTION_KEY, 0x53474972756c6573); + assert_eq!(rig.r(reg::NODE_REGION), 1); + assert_eq!(rig.r(PI1_BASE + reg::INTERFACE_ID), 2); + assert_eq!(rig.r(PI1_BASE + reg::INTERFACE_SLOT_A), 0); + assert_eq!(rig.regs.read32(reg::INTERFACE_ID).status, BUS_ERR); + assert_eq!(rig.regs.write32(reg::COUNTER, 0), BUS_ERR); + } + { + let rig = Rig::new(); + rig.w(reg::PENDING_MODIFY, 0x10c); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 1 << 12); + rig.w(reg::PENDING_MODIFY_B, 12); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 0); + reflect(&rig, true); + rig.w(reg::PENDING_MODIFY, 64 + 55); + assert_eq!(rig.r(reg::PENDING_STATUS_B), 1 << 55); + rig.w(reg::PENDING_MODIFY, 0x100 | 64 | 63); + assert_eq!(rig.r(reg::PENDING_STATUS_B), 1 << 55); + rig.w(reg::REQUEST_A, 0); + rig.w(reg::OFFSET_00E8, 1); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 1 << 5); + rig.w(reg::PENDING_A_CLEAR, 1); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 0); + } + { + let rig = Rig::new(); + rig.w(reg::CACHED_ALIAS, 9); + rig.w(reg::CPU_A_ENABLE, 1); + rig.w(reg::ERROR_ADDRESS_A, !0); + rig.chip.pi[0].error_summary.fetch_or(MEMORY_ERROR, Ordering::SeqCst); + rig.w(reg::ERROR_SUMMARY_ALIAS, 3); + rig.w(reg::RT_COMPARE_A, 0x24); + rig.chip.pi[0].power_on(0); + assert_eq!(rig.r(reg::ERROR_SUMMARY), MEMORY_ERROR | 3); + assert_eq!(rig.r(reg::CACHED_ALIAS), 9); + assert_eq!(rig.r(reg::CPU_A_ENABLE), 1); + assert_eq!(rig.r(reg::ERROR_ADDRESS_A), 0x1fffffff8); + assert_eq!(rig.r(reg::RT_COMPARE_A), 0); + assert_eq!(rig.r(reg::COUNTER), 0); + rig.w(reg::ERROR_SUMMARY_ALIAS, 0); + assert_eq!(rig.r(reg::ERROR_SUMMARY), MEMORY_ERROR); + rig.w(reg::ERROR_SUMMARY, MEMORY_ERROR); + assert_eq!(rig.r(reg::ERROR_SUMMARY), 0); + } +} + +/// The counter wraps, the comparators latch at their tick, and the counter +/// pauses while the machine is stopped. +#[test] +fn the_pi_timer_wraps_and_pauses() { + { + let rig = Rig::new(); + rig.w(reg::COUNTER, 0x1_0000_0000); + rig.w(reg::RT_COMPARE_A, 0x1_0000_0002); + rig.set_time(1599); + assert_eq!(rig.r(reg::RT_STATUS_A), 0); + rig.set_time(1600); + assert_eq!(rig.r(reg::RT_STATUS_A), 1); + assert_eq!(rig.regs.read32(reg::COUNTER).data, 1); + assert_eq!(rig.regs.read32(reg::COUNTER + 4).data, 2); + rig.w(reg::RT_STATUS_A, !0); + assert_eq!(rig.r(reg::RT_STATUS_A), 0); + rig.w(reg::COUNTER, RT_MASK); + rig.w(reg::PROFILE_COMPARE, 0); + rig.set_time(2400); + assert_eq!(rig.r(reg::COUNTER), 0); + assert_eq!(rig.r(reg::PROFILE_STATUS_A), 1); + assert_eq!(rig.r(reg::PROFILE_STATUS_B), 1); + rig.w(reg::PROFILE_STATUS_A, 0); + assert_eq!(rig.r(reg::PROFILE_STATUS_A), 0); + assert_eq!(rig.r(reg::PROFILE_STATUS_B), 1); + } + { + let rig = Rig::new(); + rig.set_time(1_000_000); + rig.w(reg::COUNTER, 0); + rig.w(reg::RT_COMPARE_A, 2); + rig.set_time(1_000_400); + rig.chip.stop(); + rig.set_time(2_000_000); + assert_eq!(rig.r(reg::COUNTER), 0); + assert_eq!(rig.r(reg::RT_STATUS_A), 0, "no time passes while stopped"); + rig.chip.start(); + rig.set_time(2_001_199); + assert_eq!(rig.r(reg::RT_STATUS_A), 0); + rig.set_time(2_001_200); + assert_eq!(rig.r(reg::RT_STATUS_A), 1); + } +} + +/// Each of a slot's five outputs follows its own inputs into Cause bits +/// 10-14 of the processor's word, and nothing else in the word moves. +#[test] +fn pi_interrupt_outputs_follow_masks_into_the_cpus_word() { + let rig = Rig::new(); + rig.word.store(0xffff_0000_0000_83ff & !(0x1f << 10), Ordering::SeqCst); + rig.w(reg::OFFSET_00A8, 1 << 12); + rig.w(reg::OFFSET_00B0, 1 << 55); + rig.w(reg::PENDING_MODIFY, 0x10c); + assert_eq!(rig.cause(), 1); + reflect(&rig, true); + assert_eq!(rig.cause(), 3); + rig.w(reg::RT_COMPARE_A, 2); + rig.w(reg::PROFILE_COMPARE, 2); + rig.w(reg::RT_ENABLE_A, 1); + rig.w(reg::PROFILE_ENABLE_A, 1); + rig.w(reg::ERROR_CAUSE_A, 1); + rig.w(reg::ERROR_SUMMARY_ALIAS, 1); + assert_eq!(rig.cause(), 0b10011); + rig.set_time(1600); + assert_eq!(rig.r(reg::RT_STATUS_A), 1); + assert_eq!(rig.cause(), 31); + rig.w(reg::RT_STATUS_A, 0); + assert_eq!(rig.cause(), 27); + rig.w(reg::PENDING_MODIFY, 12); + reflect(&rig, false); + rig.w(reg::PROFILE_STATUS_A, 0); + rig.w(reg::ERROR_SUMMARY, 1); + assert_eq!(rig.cause(), 0); + assert_eq!(rig.word.load(Ordering::SeqCst), 0xffff_0000_0000_83ff & !(0x1f << 10), "only bits 10-14 are the PI's"); +} + +/// A guest can wait for a timer interrupt without touching a register: the +/// host timer latches the comparator and raises IP4. +#[test] +fn the_timer_manager_delivers_expirations_without_mmio_polling() { + let config = super::super::test_config(crate::dev::heart::host_clock()); + let chip = super::super::Bedrock::new(config).unwrap(); + let word = Arc::new(std::sync::atomic::AtomicU64::new(0)); + chip.set_cpu_interrupts(0, &*word); + let tm = Arc::new(crate::hptimer::TimerManager::new()); + chip.set_timer_manager(&tm); + let regs = chip.registers(); + assert_eq!(regs.write64(reg::RT_COMPARE_A, 12500), BUS_OK); + assert_eq!(regs.write64(reg::RT_ENABLE_A, 1), BUS_OK); + chip.start(); + let deadline = std::time::Instant::now() + Duration::from_secs(5); + while word.load(Ordering::SeqCst) & (1 << 12) == 0 { + assert!(std::time::Instant::now() < deadline, "IP4 never rose"); + std::thread::sleep(Duration::from_millis(1)); + } + chip.stop(); + assert!(!chip.is_running()); +} + +/// The PROM counts processors from the present bits. A board fills PI0/A, +/// then PI1/A, then PI0/B and PI1/B, so two processors sit one per interface. +#[test] +fn every_fitted_slot_reads_present_and_starts_enabled() { + for (mask, present) in [(0b0001, [1, 0, 0, 0]), (0b0101, [1, 0, 1, 0]), (0b0111, [1, 1, 1, 0]), (0b1111, [1, 1, 1, 1])] { + let rig = Rig::with(Population::new(mask).unwrap()); + let read = |interface: u64, off| rig.r(interface * PI1_BASE + off); + assert_eq!([read(0, reg::INTERFACE_SLOT_A), read(0, reg::INTERFACE_SLOT_B), + read(1, reg::INTERFACE_SLOT_A), read(1, reg::INTERFACE_SLOT_B)], present, "{mask:#06b}"); + assert_eq!([read(0, reg::CPU_A_ENABLE), read(0, reg::CPU_B_ENABLE), + read(1, reg::CPU_A_ENABLE), read(1, reg::CPU_B_ENABLE)], present, "{mask:#06b}: enables"); + assert_eq!((read(0, reg::INTERFACE_ID), read(1, reg::INTERFACE_ID)), (0, 2), "{mask:#06b}: no requester"); + } +} + +/// A population that cannot hold CPU 0, or names a fifth slot, is refused +/// rather than built; the board's order puts CPU n in the nth fitted slot, +/// and one, two and four processors fill PI0/A, then PI1/A, then the B slots. +#[test] +fn a_population_without_cpu_0_or_beyond_four_slots_is_refused() { + for mask in [0, 0b0010, 0b1100, 0b1_0001, 0xff] { + assert!(Population::new(mask).is_err(), "{mask:#x}"); + } + let four = Population::new(0b1111).unwrap(); + assert_eq!((0..4).map(|cpu| four.slot_of(cpu).unwrap()).collect::>(), [(0, 0), (1, 0), (0, 1), (1, 1)]); + assert_eq!(four.slot_of(4), None); + assert_eq!(Population::new(0b0101).unwrap().slot_of(1), Some((1, 0))); + assert_eq!(Population::SINGLE.count(), 1); + assert_eq!([1, 2, 4].map(|n| Population::fitted_cpus(n).unwrap().mask()), [0b0001, 0b0101, 0b1111]); + assert!(Population::fitted_cpus(0).is_err() && Population::fitted_cpus(5).is_err()); +} + +/// Each processor reads its own slice from the CPU-number register of its +/// interface, and a reader that is no processor reads slot A. The local +/// register form answers the requester's own PI at offset 0. +/// (sn1-cpu-identity-test four-processors-know-their-slots) +#[test] +fn four_processors_know_their_slots_and_their_own_pi() { + let rig = Arc::new(Rig::with(Population::new(0b1111).unwrap())); + assert_eq!((rig.r(reg::INTERFACE_ID), rig.r(PI1_BASE + reg::INTERFACE_ID)), (0, 2), "no issuing processor"); + let local = rig.chip.local_registers(); + let slices: Vec<(u64, u64)> = (0..4u8) + .map(|cpu| { + let (rig, local) = (rig.clone(), local.clone()); + std::thread::spawn(move || { + crate::cpu::smp::set_current_cpu(cpu); + let (interface, _) = Population::new(0b1111).unwrap().slot_of(cpu as usize).unwrap(); + (rig.r(u64::from(interface) * PI1_BASE + reg::INTERFACE_ID), local.read64(reg::INTERFACE_ID).data) + }) + .join() + .unwrap() + }) + .collect(); + assert_eq!(slices, [(0, 0), (2, 2), (1, 1), (3, 3)], "slices A, C, B, D"); + // With processors on PI0 alone, the local form is the node form. + let fuel = Rig::new(); + std::thread::spawn(move || { + crate::cpu::smp::set_current_cpu(0); + assert_eq!(fuel.chip.local_registers().read64(reg::INTERFACE_ID).data, 0); + }) + .join() + .unwrap(); +} + +/// Each processor's IP2..IP6 come from its own slot: an interrupt pending +/// in PI1's bank 0 under slot A's mask reaches CPU 1 and no other. +#[test] +fn each_processor_takes_the_interrupts_of_its_own_slot() { + let rig = Rig::with(Population::new(0b1111).unwrap()); + let words: Vec> = (0..4).map(|_| Arc::new(Default::default())).collect(); + for (cpu, word) in words.iter().enumerate() { + rig.chip.set_cpu_interrupts(cpu, &**word); + } + rig.w(PI1_BASE + reg::OFFSET_00A8, 1 << 9); + rig.w(PI1_BASE + reg::PENDING_MODIFY, 0x109); + let ip2 = |cpu: usize| (words[cpu].load(Ordering::SeqCst) >> 10) & 1; + assert_eq!([ip2(0), ip2(1), ip2(2), ip2(3)], [0, 1, 0, 0]); +} + +/// Enabling a processor that is not fitted, or clearing the fitted one's +/// bit, is stored as the register holds it and starts or stops nothing. An +/// NMI reaches no processor. +#[test] +fn enables_and_nmis_the_model_does_not_carry_out_are_stored() { + let rig = Rig::new(); + rig.w(reg::OFFSET_00A8, 1 << 9); + rig.w(reg::CPU_B_ENABLE, 1); + rig.w(PI1_BASE + reg::CPU_A_ENABLE, 3); + assert_eq!((rig.r(reg::CPU_B_ENABLE), rig.r(PI1_BASE + reg::CPU_A_ENABLE)), (1, 1)); + rig.w(reg::CPU_A_ENABLE, 0); + assert_eq!(rig.r(reg::CPU_A_ENABLE), 0); + rig.w(reg::NMI_SLOT_A, 1); + rig.w(PI1_BASE + reg::NMI_SLOT_B, u64::MAX); + // CPU 0 still takes its interrupts. + rig.w(reg::PENDING_MODIFY, 0x109); + assert_eq!(rig.cause(), 1); +} + +/// A nonzero mask, timer enable or error cause for a slot with no processor +/// is stored and reaches no processor; an interprocessor interrupt to PI1/A +/// lands in its pending bank and goes nowhere. +#[test] +fn arming_a_slot_with_no_processor_reaches_no_processor() { + let rig = Rig::new(); + rig.w(reg::OFFSET_00C0, 1 << 3); + rig.w(PI1_BASE + reg::OFFSET_00A8, 1 << 7); + rig.w(reg::RT_ENABLE_B, 1); + rig.w(reg::PROFILE_ENABLE_B, 1); + rig.w(PI1_BASE + reg::ERROR_CAUSE_B, MEMORY_ERROR); + assert_eq!((rig.r(reg::OFFSET_00C0), rig.r(PI1_BASE + reg::OFFSET_00A8)), (1 << 3, 1 << 7)); + assert_eq!(rig.r(reg::RT_ENABLE_B), 1); + rig.w(PI1_BASE + reg::PENDING_MODIFY, 0x107); + assert_eq!(rig.r(PI1_BASE + reg::PENDING_STATUS_A) & (1 << 7), 1 << 7); + assert_eq!(rig.cause(), 0); +} + +impl Pi { + /// Hold both locks, for tests of the paths that must not take them. + pub(in crate::dev::bedrock) fn hold(&self) -> impl Sized + '_ { + (self.state.lock(), self.timer.lock()) + } +} + +/// A level whose named source stays active comes back a millisecond after +/// the processor clears it, and only then; a source that went inactive, a +/// level with no source and a clear that arrives as a packet bring nothing +/// back. A pending-status read latches a level that is due. +/// (sgi-pic-test runtime-profile-bedrock-rearm-one-level) +#[test] +fn a_cleared_level_with_an_active_source_comes_back_after_a_millisecond() { + let rig = Rig::new(); + rig.w(reg::OFFSET_00A8, 1 << 12); + rig.chip.set_level_source(0, 3, 12, true); + rig.w(reg::PENDING_MODIFY, 0x10c); + assert_eq!(rig.cause(), 1); + rig.w(reg::PENDING_MODIFY, 12); + assert_eq!(rig.cause(), 0); + rig.set_time(999_999); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 0, "not before a millisecond"); + rig.set_time(1_000_000); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 1 << 12); + assert_eq!(rig.cause(), 1, "the latch publishes"); + // The source going inactive ends it. + rig.w(reg::PENDING_MODIFY, 12); + rig.chip.set_level_source(0, 3, 12, false); + rig.set_time(3_000_000); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 0); + // A level no source names, and a packet's clear, are not reasserted. + rig.chip.set_level_source(0, 4, 13, true); + rig.w(reg::PENDING_MODIFY, 0x10d); + rig.w(reg::PENDING_MODIFY, 12); + assert_eq!(rig.chip.inbound_write(10, None, (1 << 47) | 0x180_0090, 8, 13), BUS_OK, "the packet clears 13"); + rig.set_time(5_000_000); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 0); + // Levels below 7 are not sources. + rig.chip.set_level_source(0, 5, 6, true); + rig.w(reg::PENDING_MODIFY, 6); + rig.set_time(7_000_000); + assert_eq!(rig.r(reg::PENDING_STATUS_A), 0); +} diff --git a/src/dev/bedrock/tests.rs b/src/dev/bedrock/tests.rs new file mode 100644 index 000000000..c93b33ef4 --- /dev/null +++ b/src/dev/bedrock/tests.rs @@ -0,0 +1,306 @@ +//! Tests of the chip as a whole: what one unit's access does to another, +//! the lock-free interrupt publish, and the paths that take no lock. + +use super::reg::{io, lb, md, pi, LB_BASE, MD_BASE, PI1_BASE}; +use super::*; +use crate::traits::{BUS_ERR, BUS_OK}; + +/// The L1 UART's line is bank 0 bit 4 of both PIs. +#[test] +fn the_uart_line_reaches_both_interfaces() { + let rig = Rig::new(); + rig.w(pi::OFFSET_00A8, 1 << 4); + let line = rig.chip.uart_irq(); + line.set(true); + assert_eq!(rig.r(pi::PENDING_STATUS_A), 1 << 4); + assert_eq!(rig.r(PI1_BASE + pi::PENDING_STATUS_A), 1 << 4); + assert_eq!(rig.cause(), 1); + line.set(false); + assert_eq!(rig.cause(), 0); +} + +/// Pending-modify stores, mask stores and a reflected level race on four +/// threads. Each publish is lock-free; once every writer has stopped, the +/// processor's word matches what the registers say, whatever the +/// interleaving. +#[test] +fn racing_stores_leave_the_word_matching_the_registers() { + const ROUNDS: u64 = 20_000; + let rig = Arc::new(Rig::new()); + let writers: Vec<_> = (0..4u64) + .map(|n| { + let rig = rig.clone(); + let uart = rig.chip.uart_irq(); + std::thread::spawn(move || { + for i in 0..ROUNDS { + match n { + // Two writers set and clear bank 0 bits 1 and 2. + 0 | 1 => rig.w(pi::PENDING_MODIFY, if (i + n) % 2 == 0 { 0x100 } else { 0 } | (n + 1)), + // One flips bank 0's mask between them. + 2 => rig.w(pi::OFFSET_00A8, if i % 3 == 0 { 0b010 } else { 0b110 }), + // One drives the UART's level, bank 0 bit 4. + _ => uart.set(i % 2 == 0), + } + } + }) + }) + .collect(); + for writer in writers { + writer.join().unwrap(); + } + let pending = rig.r(pi::PENDING_STATUS_A); + let mask = rig.r(pi::OFFSET_00A8); + assert_eq!(rig.cause(), u64::from(pending & mask != 0), "pending {pending:#x} mask {mask:#x}"); + rig.w(pi::OFFSET_00A8, 1 << 4); + rig.chip.uart_irq().set(true); + assert_eq!(rig.cause(), 1); +} + +/// Run `work` on another thread while this one holds every lock of the +/// chip; it must finish anyway. +fn without_locks(rig: &Arc, work: impl FnOnce(&Rig) + Send + 'static) { + let chip = &rig.chip; + let held = ( + chip.pi[0].hold(), + chip.pi[1].hold(), + chip.md.hold(), + chip.io.hold(), + chip.lb.hold(), + chip.ni.hold(), + chip.xb.hold(), + ); + let (done, finished) = std::sync::mpsc::channel(); + let other = rig.clone(); + let worker = std::thread::spawn(move || { + work(&other); + done.send(()).unwrap(); + }); + let result = finished.recv_timeout(std::time::Duration::from_secs(10)); + drop(held); + worker.join().unwrap(); + assert!(result.is_ok(), "the path waited for a lock"); +} + +/// The PI paths a running guest takes most take no lock: a mask store, a +/// pending-modify store and a counter read, each publishing as it goes. +#[test] +fn the_pi_hot_paths_take_no_lock() { + let rig = Arc::new(Rig::new()); + without_locks(&rig, |rig| { + rig.w(pi::OFFSET_00A8, 1 << 9); + rig.w(pi::PENDING_MODIFY, 0x109); + assert_eq!(rig.cause(), 1); + rig.w(pi::PENDING_MODIFY, 0x009); + assert_eq!(rig.cause(), 0); + rig.set_time(8000); + assert_eq!(rig.r(pi::COUNTER), 10); + assert_eq!(rig.regs.read32(pi::COUNTER + 4).data, 10); + }); +} + +/// With MD's cell register bit 0 clear, the register window refuses every +/// access, MD's own registers included. The padding after a unit's +/// registers, and anything past the window, answers a bus error. +#[test] +fn the_cell_bit_gates_the_window_and_padding_answers_an_error() { + let rig = Rig::new(); + assert!(rig.chip.cell_permits()); + assert_eq!(rig.regs.read64(super::pi::SIZE).status, BUS_ERR, "past PI0's registers"); + assert_eq!(rig.regs.read64(MD_BASE - 8).status, BUS_ERR); + assert_eq!(rig.regs.read64(super::reg::WINDOW).status, BUS_ERR); + rig.w(MD_BASE + md::CELL, 0); + assert!(!rig.chip.cell_permits()); + assert_eq!(rig.regs.read64(pi::INTERFACE_ID).status, BUS_ERR); + assert_eq!(rig.regs.read64(MD_BASE + md::CELL).status, BUS_ERR); +} + +/// An inbound interrupt packet's address for PI `pi` of node `node`. +fn packet_address(node: u64, pi: u64) -> u64 { + (1 << 47) | (node << 39) | 0x180_0000 | (pi * PI1_BASE) | 0x90 +} + +/// The PI region mask gates interrupt packets: a refused packet answers a +/// bus error, so the sender knows the level was never set, and sets nothing. +#[test] +fn a_refused_interrupt_packet_answers_an_error_from_inbound_write() { + let rig = Rig::new(); + let chip = &rig.chip; + rig.w(pi::OFFSET_00A8, 1 << 9); + rig.w(pi::OFFSET_0010, 0); + assert_eq!(chip.inbound_write(15, Some(0), packet_address(0, 0), 8, 0x109), BUS_ERR); + assert_eq!(rig.r(pi::PENDING_STATUS_A), 0); + assert_eq!(rig.cause(), 0); + rig.w(pi::OFFSET_0010, 1); + assert_eq!(chip.inbound_write(15, Some(0), packet_address(0, 0), 8, 0x109), BUS_OK); + assert_eq!(rig.r(pi::PENDING_STATUS_A), 1 << 9); + assert_eq!(rig.cause(), 1, "IP2"); + assert_eq!(chip.inbound_write(15, Some(0), packet_address(0, 0), 8, 0x009), BUS_OK); + assert_eq!(rig.cause(), 0); +} + +/// LB's region register names the node; inbound packets and the BTEs' +/// interrupts answer only to that node from then on. +#[test] +fn the_local_blocks_node_number_moves_the_hub_window() { + let rig = Rig::new(); + let chip = &rig.chip; + rig.w(LB_BASE + lb::NODE_REGION, 3 << 32); + assert_eq!(chip.inbound_write(15, Some(0), packet_address(0, 0), 8, 0x101), BUS_ERR); + assert_eq!(chip.inbound_write(15, Some(0), packet_address(3, 0), 8, 0x101), BUS_OK); + assert_eq!(rig.r(pi::PENDING_STATUS_A), 1 << 1); + // A BTE completion interrupt names its node in bits 8:1. + rig.w(MD_BASE + md::CONFIG, 0x11); + rig.w(super::reg::IO_BASE + io::BTE1_BASE + super::reg::bte::INTERRUPT, (5 << 16) | (3 << 1)); + rig.w(super::reg::IO_BASE + io::BTE1_BASE, 1 | super::io::BUSY); + rig.w(super::reg::IO_BASE + io::BTE1_BASE + super::reg::bte::CONTROL, 0); + assert_eq!(rig.r(pi::PENDING_STATUS_A), (1 << 1) | (1 << 5)); +} + +/// A node reset leaves as one event on the board's channel, for either +/// cause, and the chip calls nothing else in the board. +#[test] +fn a_node_reset_is_one_event_on_the_boards_channel() { + let rig = Rig::new(); + rig.w(LB_BASE + lb::SOFT_RESET, 1); + rig.w(super::reg::NI_BASE + super::reg::ni::LOCAL_RESET, 4); + let events: Vec<_> = rig.events.lock().try_iter().collect(); + assert!(matches!(events[..], [MachineEvent::HardReset, MachineEvent::HardReset]), "{events:?}"); +} + +/// Inbound set and clear packets race mask stores and LB's reflected error +/// level on other threads; once every writer has stopped, the processor's +/// word matches what the registers say. +#[test] +fn racing_packets_and_reflected_levels_leave_the_word_matching_the_registers() { + const ROUNDS: u64 = 20_000; + let rig = Arc::new(Rig::new()); + rig.w(pi::OFFSET_00B0, 1 << 55); + let writers: Vec<_> = (0..4u64) + .map(|n| { + let rig = rig.clone(); + std::thread::spawn(move || { + for i in 0..ROUNDS { + match n { + // Two packet sources set and clear bits 1 and 2. + 0 | 1 => { + let set = if (i + n) % 2 == 0 { 0x100 } else { 0 }; + let status = rig.chip.inbound_write(15, Some(0), packet_address(0, 0), 8, set | (n + 1)); + assert_eq!(status, BUS_OK); + } + // One store flips bank 0's mask between the two bits. + 2 => rig.w(pi::OFFSET_00A8, if i % 3 == 0 { 0b010 } else { 0b110 }), + // One drives LB's error level, bank 1 bit 55. + _ => rig.w(LB_BASE + if i % 2 == 0 { lb::ERROR_STATUS } else { lb::ERROR_CLEAR }, 1), + } + } + }) + }) + .collect(); + for writer in writers { + writer.join().unwrap(); + } + let pending = rig.r(pi::PENDING_STATUS_A); + let mask = rig.r(pi::OFFSET_00A8); + let reflected = rig.r(pi::PENDING_STATUS_B) & rig.r(pi::OFFSET_00B0); + let want = u64::from(pending & mask != 0) | u64::from(reflected != 0) << 1; + assert_eq!(rig.cause(), want, "pending {pending:#x} mask {mask:#x} reflected {reflected:#x}"); +} + +/// Two stores to LB's error registers race; each sets the reflected bit +/// under LB's lock, so the level both PIs see is the one the register +/// holds when the stores are done. +#[test] +fn a_reflected_level_follows_its_register_under_racing_stores() { + let rig = Arc::new(Rig::new()); + let writers: Vec<_> = [lb::ERROR_STATUS, lb::ERROR_CLEAR] + .into_iter() + .map(|at| { + let rig = rig.clone(); + std::thread::spawn(move || { + for _ in 0..20_000 { + rig.w(LB_BASE + at, 1); + } + }) + }) + .collect(); + for writer in writers { + writer.join().unwrap(); + } + let level = rig.r(LB_BASE + lb::ERROR_STATUS) != 0; + for base in [0, PI1_BASE] { + assert_eq!(rig.r(base + pi::PENDING_STATUS_B) & (1 << 55) != 0, level); + } +} + +/// The crossbar's paths take no lock either: an inbound interrupt packet, +/// an 8-byte inbound memory write and read, and PIO to a widget. +#[test] +fn the_inbound_and_pio_paths_take_no_lock() { + struct Stub; + impl BusDevice for Stub { + fn read64(&self, _: u64) -> BusRead64 { + BusRead64::ok(0x5a) + } + } + let rig = Arc::new(Rig::new()); + rig.w(MD_BASE + md::CONFIG, 0x11); + rig.chip.connect_widget(8, Arc::new(Stub)); + rig.w(pi::OFFSET_00A8, 1 << 9); + without_locks(&rig, |rig| { + assert_eq!(rig.chip.inbound_write(15, Some(0), packet_address(0, 0), 8, 0x109), BUS_OK); + assert_eq!(rig.cause(), 1); + assert_eq!(rig.chip.inbound_write(14, None, 0x1000, 8, 0x1234), BUS_OK); + assert_eq!(rig.chip.inbound_read(14, None, 0x1000, 8).data, 0x1234); + assert_eq!(rig.chip.pio().read64(0x800_0000).data, 0x5a); + }); +} + +/// A test chip's configuration: the Fuel's one processor and one 64 MiB +/// bank, on `clock`. +pub(super) fn test_config(clock: Clock) -> BedrockConfig<'static> { + BedrockConfig { name: "test", clock, processors: Population::SINGLE, banks_mib: &[64], events: None } +} + +/// A chip on a hand-driven clock, for the unit tests: its register window +/// and CPU 0's interrupt word. +pub(super) struct Rig { + pub(super) chip: Arc, + now: Arc, + pub(super) word: Arc, + pub(super) regs: Arc, + /// The node's events. + pub(super) events: parking_lot::Mutex>, +} +impl Rig { + /// The Fuel's one processor. + pub(super) fn new() -> Self { + Self::with(Population::SINGLE) + } + pub(super) fn with(processors: Population) -> Self { + let now = Arc::new(AtomicU64::new(0)); + let clock = now.clone(); + let config = test_config(Arc::new(move || clock.load(Ordering::SeqCst))); + let (events, receiver) = std::sync::mpsc::channel(); + let chip = Bedrock::new(BedrockConfig { processors, events: Some(events), ..config }).unwrap(); + let word = Arc::new(AtomicU64::new(0)); + chip.set_cpu_interrupts(0, &*word); + chip.start(); + let regs = chip.registers(); + Self { chip, now, word, regs, events: parking_lot::Mutex::new(receiver) } + } + pub(super) fn r(&self, at: u64) -> u64 { + let r = self.regs.read64(at); + assert_eq!(r.status, BUS_OK, "read at {at:#x}"); + r.data + } + pub(super) fn w(&self, at: u64, value: u64) { + assert_eq!(self.regs.write64(at, value), BUS_OK, "write at {at:#x}"); + } + pub(super) fn set_time(&self, ns: u64) { + self.now.store(ns, Ordering::SeqCst); + } + /// CPU 0's IP2..IP6, bit 0 being IP2. + pub(super) fn cause(&self) -> u64 { + (self.word.load(Ordering::SeqCst) >> 10) & 0x1f + } +} diff --git a/src/dev/bedrock/xb.rs b/src/dev/bedrock/xb.rs new file mode 100644 index 000000000..bee1d3609 --- /dev/null +++ b/src/dev/bedrock/xb.rs @@ -0,0 +1,255 @@ +//! Bedrock's crossbar block (XB) records the fabric error queues and lets +//! software service queued errors one at a time; its error level reaches both +//! processor interfaces. +//! +//! # What is approximated here +//! +//! Arbitration timing and diagnostic outputs are not modeled. Unsupported +//! queue reads return zero, while writes are logged and ignored. +//! +//! Ported from hw/misc/sgi-bedrock-xb.c in jamesbraid/qemu-sgi-origami. + +use super::pi::Levels; +use super::Effect; +use parking_lot::Mutex; + +/// Length of the block-relative register aperture. +pub(super) const SIZE: u64 = 0x198; + +pub mod reg { + pub const CONTROL_BASE: u64 = 0x0; + pub const CONTROL_END: u64 = 0x18; + pub const QUEUE_BASE: u64 = 0x20; + pub const QUEUE_END: u64 = 0x88; + pub const FIRST_ERROR: u64 = 0x90; + pub const DEBUG_BASE: u64 = 0x98; + pub const DEBUG_END: u64 = 0xa0; + pub const CLEAR_QUEUE_BASE: u64 = 0x120; + pub const CLEAR_QUEUE_END: u64 = 0x188; + pub const CLEAR_FIRST_ERROR: u64 = 0x190; +} + +const MASKS: [u64; 14] = [ + 0x3333, 0x33, 0, 0, 0x3330, 0, 0, 0xf0, 0xfff0, 0, 0, 0, 0x3303, 0x11, +]; +struct Regs { + control: [u64; 4], + queues: [u64; 14], + first: u64, + debug: [u64; 2], + unserviced: u16, +} +impl Default for Regs { + fn default() -> Self { + Self { + control: [0x0008000080000021, 0, 0, 0], + queues: [0; 14], + first: 0, + debug: [0; 2], + unserviced: 0, + } + } +} +impl Regs { + fn error_level(&self) -> bool { + self.first & 0x1000 != 0 + } +} + +pub(super) struct Xb { + regs: Mutex, +} + +impl Xb { + pub(super) fn new() -> Self { + Self { regs: Mutex::new(Regs::default()) } + } + pub(super) fn locked(&self) -> bool { + self.regs.is_locked() + } + fn reflect(r: &Regs, levels: &Levels) -> Effect { + if levels.set(Levels::XB_ERROR, r.error_level()) { Effect::Reflect } else { Effect::None } + } + pub(super) fn power_on(&self, levels: &Levels) -> Effect { + let mut r = self.regs.lock(); + *r = Regs::default(); + Self::reflect(&r, levels) + } + pub(super) fn read(&self, a: u64) -> u64 { + let r = self.regs.lock(); + match a { + reg::CONTROL_BASE..=reg::CONTROL_END => r.control[a as usize / 8], + reg::QUEUE_BASE..=reg::QUEUE_END => { + let n = (a - reg::QUEUE_BASE) as usize / 8; + if MASKS[n] == 0 { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock XB unsupported queue {n}"); + } + r.queues[n] + } + reg::FIRST_ERROR => r.first, + reg::DEBUG_BASE | reg::DEBUG_END => r.debug[(a - reg::DEBUG_BASE) as usize / 8], + _ => { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock XB unsupported read {a:#x}"); + 0 + } + } + } + /// A doubleword store. The error level reaches `levels` under the + /// block's lock. + pub(super) fn write(&self, a: u64, v: u64, levels: &Levels) -> Effect { + let mut r = self.regs.lock(); + match a { + reg::CONTROL_BASE..=reg::CONTROL_END => { + let n = a as usize / 8; + r.control[n] = v & [0x000ffffffffffff1, 0xffffffff, 0x7ff, 0xf][n]; + if a == 0x10 && r.control[n] != 0 { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock XB traffic observation unsupported"); + } + } + reg::QUEUE_BASE..=reg::QUEUE_END => { + let n = (a - reg::QUEUE_BASE) as usize / 8; + let v = v & MASKS[n]; + if MASKS[n] == 0 { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock XB unsupported queue {n}"); + } + if v != 0 { + r.queues[n] |= v; + r.unserviced |= 1 << n; + if r.first & 0x1000 == 0 { + r.first = 0x1000 | ((15 - n) as u64) << 8; + } + } + } + reg::CLEAR_QUEUE_BASE..=reg::CLEAR_QUEUE_END => { + let n = (a - reg::CLEAR_QUEUE_BASE) as usize / 8; + r.queues[n] = 0; + r.unserviced &= !(1 << n); + } + reg::CLEAR_FIRST_ERROR => { + if r.first & 0x1000 != 0 { + let n = 15 - ((r.first >> 8) & 15) as usize; + if r.unserviced & (1 << n) != 0 { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock XB first error cleared before queue"); + } else { + r.first = 0; + for n in 0..14 { + if r.queues[n] != 0 { + r.first = 0x1000 | ((15 - n) as u64) << 8; + break; + } + } + } + } + } + reg::DEBUG_BASE | reg::DEBUG_END => { + let n = (a - reg::DEBUG_BASE) as usize / 8; + r.debug[n] = v & [0xffff, 7][n]; + if r.debug[n] != 0 { + crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock XB debug output unsupported"); + } + } + _ => crate::dlog!(crate::devlog::LogModule::Bedrock, "Bedrock XB unsupported or read-only write {a:#x} = {v:#x}"), + } + Self::reflect(&r, levels) + } +} + +#[cfg(test)] +mod tests { + // Cases follow tests/qtest/sn1-bedrock-xb-test.c in + // jamesbraid/qemu-sgi-origami: control-storage, service-order and + // cold-reset. + use super::super::reg::{pi, PI1_BASE, XB_BASE}; + use super::super::Rig; + use super::*; + use crate::traits::{BusDevice, BUS_ERR}; + + fn error_level(rig: &Rig) -> bool { + let levels = [0, PI1_BASE].map(|base| rig.r(base + pi::PENDING_STATUS_B) & (1 << 54) != 0); + assert_eq!(levels[0], levels[1], "both interfaces reflect bit 54"); + levels[0] + } + + /// The guest sees masked crossbar control words and cleared errors after reset. + #[test] + fn crossbar_control_masks_apply_and_reset_clears_errors() { + let rig = Rig::new(); + for (a, mask) in [ + (reg::CONTROL_BASE, 0x000ffffffffffff1), + (reg::CONTROL_BASE + 8, 0xffffffff), + (reg::CONTROL_BASE + 0x10, 0x7ff), + (reg::CONTROL_END, 15), + (reg::DEBUG_BASE, 0xffff), + (reg::DEBUG_END, 7), + ] { + rig.w(XB_BASE + a, !0); + assert_eq!(rig.r(XB_BASE + a), mask); + } + // Keep expected register values independent of the decoder's mask table. + for (address, expected) in [ + (reg::QUEUE_BASE, 0x3333), + (reg::QUEUE_BASE + 8, 0x0033), + (reg::QUEUE_BASE + 0x10, 0), + (reg::QUEUE_BASE + 0x18, 0), + (reg::QUEUE_BASE + 0x20, 0x3330), + (reg::QUEUE_BASE + 0x28, 0), + (reg::QUEUE_BASE + 0x30, 0), + (reg::QUEUE_BASE + 0x38, 0x00f0), + (reg::QUEUE_BASE + 0x40, 0xfff0), + (reg::QUEUE_BASE + 0x48, 0), + (reg::QUEUE_BASE + 0x50, 0), + (reg::QUEUE_BASE + 0x58, 0), + (reg::QUEUE_BASE + 0x60, 0x3303), + (reg::QUEUE_END, 0x0011), + ] { + rig.w(XB_BASE + address, !0); + assert_eq!(rig.r(XB_BASE + address), expected, "register {address:#x}"); + } + assert!(error_level(&rig)); + let _ = rig.chip.xb.power_on(&rig.chip.levels); + assert_eq!(rig.r(XB_BASE + reg::CONTROL_BASE), 0x0008000080000021); + assert!(!error_level(&rig)); + assert_eq!(rig.regs.read32(XB_BASE + reg::CONTROL_BASE).status, BUS_ERR); + } + /// The guest services queued crossbar faults in order, one error at a time. + #[test] + fn servicing_one_crossbar_error_exposes_the_next_pending_error() { + { + let rig = Rig::new(); + rig.w(XB_BASE + reg::QUEUE_BASE, 1); + assert!(error_level(&rig)); + assert_eq!(rig.r(XB_BASE + reg::FIRST_ERROR), 0x1f00); + rig.w(XB_BASE + reg::CLEAR_FIRST_ERROR, !0); + assert_eq!(rig.r(XB_BASE + reg::FIRST_ERROR), 0x1f00); + rig.w(XB_BASE + reg::CLEAR_QUEUE_END, !0); + rig.w(XB_BASE + reg::CLEAR_QUEUE_BASE, 0); + rig.w(XB_BASE + reg::CLEAR_FIRST_ERROR, 0); + assert!(!error_level(&rig)); + let _ = rig.chip.xb.power_on(&rig.chip.levels); + assert!(!error_level(&rig)); + } + { + let rig = Rig::new(); + rig.w(XB_BASE + reg::QUEUE_END, 0x10); + rig.w(XB_BASE + reg::QUEUE_BASE, 1); + assert_eq!(rig.r(XB_BASE + reg::FIRST_ERROR), 0x1200); + rig.w(XB_BASE + reg::CLEAR_QUEUE_END, 0); + rig.w(XB_BASE + reg::CLEAR_FIRST_ERROR, 0); + assert_eq!(rig.r(XB_BASE + reg::FIRST_ERROR), 0x1f00); + rig.w(XB_BASE + reg::CLEAR_QUEUE_BASE, 0); + rig.w(XB_BASE + reg::QUEUE_BASE, 0x10); + rig.w(XB_BASE + reg::CLEAR_FIRST_ERROR, 0); + assert_eq!(rig.r(XB_BASE + reg::FIRST_ERROR), 0x1f00); + rig.w(XB_BASE + reg::CLEAR_QUEUE_BASE, 0); + rig.w(XB_BASE + reg::CLEAR_FIRST_ERROR, 0); + assert!(!error_level(&rig)); + } + } + + impl Xb { + /// Hold the block's lock, for tests of the paths that must not take it. + pub(in crate::dev::bedrock) fn hold(&self) -> impl Sized + '_ { + self.regs.lock() + } + } +} diff --git a/src/dev/bedrock_flash.rs b/src/dev/bedrock_flash.rs new file mode 100644 index 000000000..1256eb061 --- /dev/null +++ b/src/dev/bedrock_flash.rs @@ -0,0 +1,1030 @@ +//! Bedrock's boot flash: two independent 8 MiB arrays, selected 32 MiB +//! apart, with the AMD command set and CFI query data. +//! +//! Boot aliases and executable views share the same array bytes. Each array +//! has its own command state; instruction fetches read stored bytes even +//! while register reads return command status. +//! +//! Ported from hw/misc/sgi-bedrock-flash.c and hw/mips/sgi/initial-flash.c +//! in jamesbraid/qemu-sgi-origami. + + + +use super::sflash::amd::{AmdCore, Effect, SECTOR_SIZE}; +use super::sflash::storage::FlashBacking; +use crate::traits::{ + BusDevice, BusRead16, BusRead32, BusRead64, BusRead8, Resettable, BUS_ERR, BUS_OK, +}; +use parking_lot::Mutex; +use std::io; +use std::path::Path; +use std::sync::{ + atomic::{AtomicU64, Ordering}, + Arc, +}; + +pub const PART_SIZE: usize = 8 << 20; +pub const FLASH_SIZE: usize = 2 * PART_SIZE; +pub const SECOND_POSITION: u64 = 32 << 20; + +fn sum(bytes: &[u8]) -> u8 { + bytes.iter().fold(0u8, |s, b| s.wrapping_add(*b)) +} + +/// The fields of the PROM's configuration block that depend on the board. +/// +/// The PROM reads them from flash before it touches the processor, so they +/// must agree with the CPU the machine builds: a mismatch in the boot-mode +/// word sends it to its error routine before it initialises GP. The values +/// are not a measured flash dump: the Tezro rows' mode words and clocks are +/// rows of the PROM's speed table where it has one, and each row's doc says +/// so or says what was derived. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct FirmwareRecord { + /// The CPU boot-mode word the PROM compares with the processor's mode. + /// Its bits 24:0 are the mode inputs the processor latches into CP0 + /// Config at reset (the R10000 family's reset-mode inputs). + pub cpu_mode_word: u32, + /// The processor clock the PROM reports and times against, in Hz. + pub cpu_clock_hz: u32, + /// The record's word at +0x30, which the PROM's start-up check compares + /// with a machine class of its own. When the node board's L1 record + /// carries a CPU configuration, the PROM derives that class from the + /// board word's top nibble (0x4 gives 2, 0x8 gives 1, any other 0; read + /// from the stock PROM 6.210's code, established for that PROM). The + /// Fuel's record is 2, SN11: a Fuel's PROM flash session takes 2 from + /// "(0)SN1 (1)SN10 (2)SN11 (3)SN12" and its IRIX logs "Selecting SN11" + /// (just.graphica.com.au/tips/sgi-o350-chimera-notes). With 1, PROM + /// 6.210 prints "CPU configuration values mismatch", which a real Fuel's + /// console does not. + pub machine_type: u32, + /// How the record reaches the configuration block. + pub header: Header, +} + +/// How a board's record is put into the configuration block at 0x60. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Header { + /// Rewrite the board's fields over the payload's own block, keeping its + /// byte sum, whenever a store is formatted or opened. QEMU configures a + /// Fuel's flash this way. + Refreshed, + /// Replace the whole block with the one authored for the board's flash + /// seed, once, when a store is formatted, with the compensation byte the + /// PROM's editor would choose (the block's byte sum is the replaced + /// block's, so the image still sums to zero). An opened store keeps the + /// block it holds, so a flash update the PROM wrote survives. + Authored, +} + +impl FirmwareRecord { + /// The Fuel's 500 MHz R14000 with a 2 MiB secondary cache. + pub const FUEL: Self = Self { + cpu_mode_word: 0x549a_a985, + cpu_clock_hz: 500_000_000, + machine_type: 2, + header: Header::Refreshed, + }; + /// The Tezro's 700 MHz R16000 with a 4 MiB secondary cache. The mode word + /// is the row of the stock IP35 PROM's speed table for 700 MHz, 4 MiB and + /// cache tap 0xc (established), and its low 25 bits are the Config word the + /// Tezro board resets this chip with. The machine type is 0, the class the + /// PROM derives from the Tezro's board word 0x2001, so that the block + /// agrees with the CPU configuration the node board's L1 record carries + /// (see `ip53_machine`). The speed-table row itself carries 2, the class + /// of a board word in 0x4000, as the Fuel's is: copying it made the PROM + /// print "CPU configuration values mismatch" at every start. The board + /// word is a software input and a real Tezro's flash has not been read, + /// so 0 is approximated, chosen to agree with the modelled board word as + /// QEMU's Tezro does. A real Tezro may instead carry a board word in the + /// 0x4000 class, for which the PROM expects 2. Every IP53 tuple that has + /// a row in the PROM's fallback table matches a machine-type-2 row + /// (Origami IP53 processor-assemblies datasheet). That is consistent with + /// the 0x4000 class but does not settle it, because the printed fields + /// come from the node EEPROM first. + pub const TEZRO: Self = Self { + cpu_mode_word: 0x589b_ad85, + cpu_clock_hz: 700_000_000, + machine_type: 0, + header: Header::Authored, + }; + /// The Tezro's 800 MHz R16000 with a 4 MiB secondary cache: the PROM speed + /// table's row for 800 MHz, 4 MiB and tap 0xa (established; the table has a + /// second row with tap 0xb and the same low 25 bits, and the 0xa one is + /// what 800 MHz Tezro inventories print). + pub const TEZRO_800: Self = Self { + cpu_mode_word: 0x549b_af85, + cpu_clock_hz: 800_000_000, + machine_type: 0, + header: Header::Authored, + }; + /// The Tezro's 900 MHz R16000 with an 8 MiB secondary cache: the PROM + /// speed table's row for 900 MHz (established; it is the table's only 900 + /// MHz row, tap 0xb, which a 900 MHz inventory prints). That a Tezro was + /// ever sold or flashed with this part is not established. + pub const TEZRO_900: Self = Self { + cpu_mode_word: 0x569c_b185, + cpu_clock_hz: 900_000_000, + machine_type: 0, + header: Header::Authored, + }; + /// The Tezro's 1 GHz R16000 with a 16 MiB secondary cache. The PROM's + /// speed table, whose top clock is 900 MHz, has no row for it, and no + /// capture records the word: it is computed by formula, so the record is + /// approximated. The clock-dependent fields follow the formulas the + /// table's rows decode by, and bit 23 is set as in every row of 500 MHz + /// and up. + /// EC code 9 is 2 * 1000 / 200 - 1, SC code 5 gives the printed 333 MHz + /// cache clock (2 * 1000 / (SC + 1)), SS code 5 is 16 MiB, and the printed + /// tap 0x15 is 0x5 in bits 28:25 with the fifth bit at bit 22. + pub const TEZRO_1000: Self = Self { + cpu_mode_word: 0x4aed_b385, + cpu_clock_hz: 1_000_000_000, + machine_type: 0, + header: Header::Authored, + }; + + /// CP0 Config at reset: the fixed 32 KiB primary cache sizes (fields 3 + /// in bits 31:29 and 28:26) with the board's mode bits. + pub const fn reset_config0(&self) -> u32 { + (3 << 29) | (3 << 26) | self.reset_mode() + } + /// The mode bits the processor latches into Config at reset: the mode + /// word's bits 24:0. + pub const fn reset_mode(&self) -> u32 { + self.cpu_mode_word & 0x01ff_ffff + } + + /// The configuration block written into an IP53 seed: the format word, + /// the board's mode word, the `ip27conf` magic, the processor and + /// 200 MHz controller clocks as 64-bit words, the real-time clock's + /// 1,250,000 Hz, the machine type, a zero compensation byte (the caller sets + /// it), an edit counter of one, and the PROM's 6.210 version fields. + /// The layout is the PROM editor's. + fn authored_block(&self) -> [u8; CONFIG_LEN] { + let mut block = [0u8; CONFIG_LEN]; + let mut put = |offset: usize, bytes: &[u8]| block[offset..offset + bytes.len()].copy_from_slice(bytes); + put(0x00, &21u32.to_be_bytes()); + put(0x04, &self.cpu_mode_word.to_be_bytes()); + put(0x08, b"ip27conf"); + put(0x10, &u64::from(self.cpu_clock_hz).to_be_bytes()); + put(0x18, &200_000_000u64.to_be_bytes()); + put(0x20, &1_250_000u64.to_be_bytes()); + for (offset, value) in [ + (0x28, 1), + (0x2c, 16), + (0x30, self.machine_type), + (0x34, 0), + (0x38, 1), + (0x3c, 16), + (0x40, 6), + (0x44, 210), + (0x48, 0), + ] { + put(offset, &u32::to_be_bytes(value)); + } + block + } +} + +/// Where the PROM's configuration block sits in the payload, and its size. +const CONFIG_BASE: usize = 0x60; +const CONFIG_LEN: usize = 0x50; + +/// Put the board's record into a store being formatted. +fn format_header(data: &mut [u8], record: &FirmwareRecord) { + match record.header { + Header::Refreshed => configure(data, record), + Header::Authored => { + // The compensation byte is the PROM editor's rule: the new block + // sums, byte for byte, to what the block it replaces did, so the + // image still sums to zero (the check the PROM runs on itself). + let old = sum(&data[CONFIG_BASE..CONFIG_BASE + CONFIG_LEN]); + let mut block = record.authored_block(); + block[0x37] = old.wrapping_sub(sum(&block)); + data[CONFIG_BASE..CONFIG_BASE + CONFIG_LEN].copy_from_slice(&block); + } + } +} + +/// Put the board's record into a store that was opened rather than formatted. +fn reopen_header(data: &mut [u8], record: &FirmwareRecord) { + match record.header { + Header::Refreshed => configure(data, record), + Header::Authored => {} + } +} + +/// Apply the board's firmware record while retaining the configuration sum. +fn configure(data: &mut [u8], record: &FirmwareRecord) { + let config = &mut data[CONFIG_BASE..CONFIG_BASE + CONFIG_LEN]; + let old = sum(config); + config[0x34..0x38].fill(0); + config[4..8].copy_from_slice(&record.cpu_mode_word.to_be_bytes()); + config[0x14..0x18].copy_from_slice(&record.cpu_clock_hz.to_be_bytes()); + config[0x30..0x34].copy_from_slice(&record.machine_type.to_be_bytes()); + config[0x38..0x3c].copy_from_slice(&1u32.to_be_bytes()); + let adjustment = old.wrapping_sub(sum(config)); + config[0x34..0x38].copy_from_slice(&(adjustment as u32).to_be_bytes()); +} + +struct State { + data: Vec, + parts: [AmdCore; 2], + backing: Option, +} + +impl State { + fn new(data: Vec) -> Self { + Self { + data, + parts: std::array::from_fn(|_| AmdCore::default()), + backing: None, + } + } + // Keep the in-memory array unchanged until the backing write succeeds. + fn persist(&mut self, ranges: &[(usize, &[u8])]) -> io::Result<()> { + let Some(backing) = &mut self.backing else { + return Ok(()); + }; + for &(offset, bytes) in ranges { + backing.write_range(offset, bytes)?; + } + Ok(()) + } + fn replace(&mut self, offset: usize, bytes: &[u8]) -> io::Result { + if self.data[offset..offset + bytes.len()] == *bytes { + return Ok(false); + } + self.persist(&[(offset, bytes)])?; + self.data[offset..offset + bytes.len()].copy_from_slice(bytes); + Ok(true) + } + fn erase(&mut self, part: usize, sectors: &[usize]) -> io::Result { + let erased = vec![0xff; SECTOR_SIZE]; + let ranges: Vec<_> = sectors + .iter() + .map(|sector| (part * PART_SIZE + sector * SECTOR_SIZE, erased.as_slice())) + .collect(); + self.persist(&ranges)?; + let mut changed = false; + for (offset, bytes) in ranges { + let data = &mut self.data[offset..offset + bytes.len()]; + changed |= data.iter().any(|byte| *byte != 0xff); + data.fill(0xff); + } + Ok(changed) + } +} + +pub struct Flash { + st: Mutex, + gen: AtomicU64, +} + +impl Flash { + /// Format a fresh two-position store from a decoded payload, with the + /// board's firmware record. + pub fn new(payload: &[u8], record: FirmwareRecord) -> io::Result { + if payload.len() > PART_SIZE { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + "PROM payload exceeds 8 MiB", + )); + } + let mut data = vec![0xff; FLASH_SIZE]; + data[..payload.len()].copy_from_slice(payload); + let label = PART_SIZE + 0x1e0000; + data[label + 0x10..label + 0x14].copy_from_slice(&0x504c4f49u32.to_be_bytes()); + data[label + 0x14..label + 0x18].copy_from_slice(&1u32.to_be_bytes()); + data[label + 0x18..label + 0x1c].copy_from_slice(&1u32.to_be_bytes()); + format_header(&mut data, &record); + Ok(Self { + st: Mutex::new(State::new(data)), + gen: AtomicU64::new(0), + }) + } + + /// Load an exact 16 MiB store, or seed a missing/empty one. Wrong sizes are + /// refused without writes. Reopening reapplies a refreshed firmware record + /// and leaves an authored one as the store holds it. + /// Completed guest programming and erasure write through to this file. + pub fn with_file(payload: &[u8], path: impl AsRef, record: FirmwareRecord) -> io::Result { + let mut data = Self::new(payload, record)?.st.into_inner().data; + let mut backing = FlashBacking::open(path, &mut data).map_err(|error| { + if error.kind() == io::ErrorKind::InvalidData { + io::Error::new( + io::ErrorKind::InvalidData, + "flash store must be exactly 16 MiB (or empty)", + ) + } else { + error + } + })?; + reopen_header(&mut data, &record); + // Opening an existing store loads it before a refreshed record's + // fields are applied; an empty store was seeded from this same data. + backing.write_range(0, &data)?; + let flash = Self { + st: Mutex::new(State { + data, + parts: std::array::from_fn(|_| AmdCore::default()), + backing: Some(backing), + }), + gen: AtomicU64::new(0), + }; + Ok(flash) + } + + pub fn contents(&self) -> Vec { + self.st.lock().data.clone() + } + pub fn generation(&self) -> u64 { + self.gen.load(Ordering::Acquire) + } + pub fn reset(&self) { + let mut st = self.st.lock(); + for part in &mut st.parts { + part.reset(); + } + self.gen.fetch_add(1, Ordering::Release); + } + /// Advance the model clock independently of flash reads. + pub fn advance_ns(&self, ns: u64) { + let mut st = self.st.lock(); + for part in 0..2 { + let selecting = st.parts[part].selecting(); + let complete = st.parts[part].advance_ns(ns); + if selecting && !st.parts[part].selecting() { + self.gen.fetch_add(1, Ordering::Release); + } + if !complete { + continue; + } + let sectors: Vec<_> = st.parts[part].selected_sectors().collect(); + let success = match st.erase(part, §ors) { + Ok(_) => true, + Err(e) => { + log::error!("bedrock-flash: position {part} erase backing write failed: {e}"); + false + } + }; + st.parts[part].finish_erase(success); + self.gen.fetch_add(1, Ordering::Release); + } + } + pub fn register_locks(self: &Arc) { + let me = self.clone(); + crate::locks::register_lock_fn("bedrock_flash::data", move || me.st.is_locked()); + } + /// Use the same shared flash for local and remote boot spaces. + pub fn boot_view(self: &Arc) -> FlashView { + FlashView { + flash: self.clone(), + range: None, + } + } + /// A contiguous executable alias. Offsets refer to the 16 MiB backing. + pub fn array_view(self: &Arc, offset: usize, len: usize) -> io::Result { + if offset.checked_add(len).is_none_or(|end| end > FLASH_SIZE) { + return Err(io::Error::new( + io::ErrorKind::InvalidInput, + "Fuel flash alias exceeds backing", + )); + } + Ok(FlashView { + flash: self.clone(), + range: Some((offset, len)), + }) + } +} +impl Resettable for Flash { + fn power_on(&self) { + self.reset(); + } +} + +pub struct FlashView { + flash: Arc, + range: Option<(usize, usize)>, +} +impl FlashView { + fn offset(&self, addr: u64, width: usize) -> Option { + if let Some((base, len)) = self.range { + let off = usize::try_from(addr).ok()?; + return off + .checked_add(width) + .filter(|end| *end <= len) + .map(|_| base + off); + } + let (base, within) = if addr < PART_SIZE as u64 { + (0, addr) + } else if addr >= SECOND_POSITION { + (PART_SIZE, addr - SECOND_POSITION) + } else { + return None; + }; + let within = usize::try_from(within).ok()?; + within + .checked_add(width) + .filter(|end| *end <= PART_SIZE) + .map(|_| base + within) + } + fn read(&self, addr: u64, width: usize) -> Option { + let off = self.offset(addr, width)?; + let mut st = self.flash.st.lock(); + if self.range.is_none() { + let part = off / PART_SIZE; + if !st.parts[part].array_reads() && (width != 2 || off & 1 != 0) { + return None; + } + if width == 2 && off & 1 == 0 { + if let Some(value) = st.parts[part].read16(off % PART_SIZE) { + return Some(value as u64); + } + } + } + Some( + st.data[off..off + width] + .iter() + .fold(0, |v, b| (v << 8) | *b as u64), + ) + } + fn refuse(&self, addr: u64, width: usize, value: u64) -> u32 { + crate::dlog!( + crate::devlog::LogModule::Bedrock, + "bedrock-flash: unsupported {width}-byte store {value:#x} at {addr:#x}; invalid bus cycle or executable view" + ); + BUS_ERR + } +} +impl BusDevice for FlashView { + fn read8(&self, a: u64) -> BusRead8 { + self.read(a, 1) + .map_or_else(BusRead8::err, |v| BusRead8::ok(v as u8)) + } + fn read16(&self, a: u64) -> BusRead16 { + self.read(a, 2) + .map_or_else(BusRead16::err, |v| BusRead16::ok(v as u16)) + } + fn read32(&self, a: u64) -> BusRead32 { + self.read(a, 4) + .map_or_else(BusRead32::err, |v| BusRead32::ok(v as u32)) + } + fn read64(&self, a: u64) -> BusRead64 { + self.read(a, 8).map_or_else(BusRead64::err, BusRead64::ok) + } + fn write8(&self, a: u64, v: u8) -> u32 { + self.refuse(a, 1, v as u64) + } + fn write16(&self, a: u64, v: u16) -> u32 { + self.refuse(a, 2, v as u64) + } + fn write32(&self, a: u64, v: u32) -> u32 { + self.refuse(a, 4, v as u64) + } + fn write64(&self, a: u64, v: u64) -> u32 { + if self.range.is_some() || a & 7 != 0 || v > u16::MAX as u64 { + return self.refuse(a, 8, v); + } + let (part, within) = if a < SECOND_POSITION { + (0, a) + } else { + (1, a - SECOND_POSITION) + }; + let Some(end) = within.checked_add(8) else { + return self.refuse(a, 8, v); + }; + if end > SECOND_POSITION { + return self.refuse(a, 8, v); + } + let offset = (within / 4) as usize; + let mut st = self.flash.st.lock(); + let effect = match st.parts[part].write16(offset, v as u16) { + Ok(effect) => effect, + Err(()) => { + self.flash.gen.fetch_add(1, Ordering::Release); + return BUS_ERR; + } + }; + if let Effect::Program { offset, value } = effect { + let backing = part * PART_SIZE + offset; + let old = u16::from_be_bytes([st.data[backing], st.data[backing + 1]]); + match st.replace(backing, &(old & value).to_be_bytes()) { + Ok(_) => {} + Err(e) => { + log::error!("bedrock-flash: program backing write at {backing:#x} failed: {e}"); + self.flash.gen.fetch_add(1, Ordering::Release); + return BUS_ERR; + } + } + } + self.flash.gen.fetch_add(1, Ordering::Release); + BUS_OK + } + #[cfg(feature = "jitv2")] + fn gen_ptr(&self, _addr: u64) -> *const AtomicU64 { + &self.flash.gen + } +} + +#[cfg(test)] +mod tests { + // Cases follow tests/qtest/sn1-bedrock-flash-test.c in + // jamesbraid/qemu-sgi-origami: program, sector-erase, erase-polling, + // reset-states and backing-file. + use super::*; + fn unlock(v: &FlashView, part: u64) { + assert_eq!(v.write64(part + 8 * 0x555, 0xaa), crate::traits::BUS_OK); + assert_eq!(v.write64(part + 8 * 0x2aa, 0x55), crate::traits::BUS_OK); + } + fn program(v: &FlashView, part: u64, off: u64, value: u16) { + unlock(v, part); + assert_eq!(v.write64(part + 8 * 0x555, 0xa0), crate::traits::BUS_OK); + assert_eq!( + v.write64(part + 4 * off, value as u64), + crate::traits::BUS_OK + ); + } + fn erase_select(v: &FlashView, part: u64, offset: u64) { + unlock(v, part); + assert_eq!(v.write64(part + 8 * 0x555, 0x80), BUS_OK); + unlock(v, part); + assert_eq!(v.write64(part + 4 * offset, 0x30), BUS_OK); + } + + /// The PROM reads CPU configuration and the volume label from its boot flash. + #[test] + fn the_seed_exposes_fuel_configuration_and_volume_label() { + let seed = vec![0x21; 0xb0]; + let f = Flash::new(&seed, FirmwareRecord::FUEL).unwrap(); + let bytes = f.contents(); + assert_eq!(sum(&bytes[0x60..0xb0]), sum(&seed[0x60..0xb0])); + assert_eq!(&bytes[0x64..0x68], &0x549aa985u32.to_be_bytes()); + assert_eq!(&bytes[0x74..0x78], &500_000_000u32.to_be_bytes()); + assert_eq!(&bytes[0x90..0x94], &2u32.to_be_bytes(), "machine type: SN11"); + let label = PART_SIZE + 0x1e0000; + assert_eq!(&bytes[label..label + 16], &[0xff; 16]); + assert_eq!( + &bytes[label + 16..label + 28], + &[0x50, 0x4c, 0x4f, 0x49, 0, 0, 0, 1, 0, 0, 0, 1] + ); + assert!(bytes[label + 28..].iter().all(|b| *b == 0xff)); + assert!(Flash::new(&vec![0; PART_SIZE + 1], FirmwareRecord::FUEL).is_err()); + } + /// The stock image's factory block: the 400 MHz mode word, machine type 0 + /// and a zero check adjustment, before any board configures it. + fn factory_block() -> Vec { + let mut image = vec![0x21; 0xb0]; + image[0x60..0xb0].fill(0); + image[0x64..0x68].copy_from_slice(&0x5219a785u32.to_be_bytes()); + image[0x70..0x78].copy_from_slice(&400_000_000u64.to_be_bytes()); + image + } + + /// A refreshed record leaves the block's byte sum unchanged: the check + /// adjustment absorbs the difference. + #[test] + fn a_refreshed_record_configures_the_block_and_keeps_its_byte_sum() { + let record = FirmwareRecord::FUEL; + let image = factory_block(); + let bytes = Flash::new(&image, record).unwrap().contents(); + let block = &bytes[0x60..0xb0]; + assert_eq!(&block[4..8], &record.cpu_mode_word.to_be_bytes()); + assert_eq!(&block[0x14..0x18], &record.cpu_clock_hz.to_be_bytes()); + assert_eq!(&block[0x30..0x34], &record.machine_type.to_be_bytes()); + assert_eq!(&block[0x38..0x3c], &1u32.to_be_bytes(), "flash program count"); + assert_eq!(&block[0x34..0x38], &[0, 0, 0, 0x9b]); + assert_eq!(sum(block), sum(&image[0x60..0xb0])); + } + + /// The compensation keeps the image's byte sum whatever block the + /// payload carried, and changing any one byte of a seeded block off its + /// compensated value breaks the sum. + #[test] + fn the_compensation_byte_keeps_the_images_sum_and_any_change_breaks_it() { + // An image that sums to zero, as the PROM's own does, still sums to + // zero once the record's fields are seeded into it. + let mut zero = vec![0x21; 0x200]; + zero[0x60..0xb0].copy_from_slice(&factory_block()[0x60..0xb0]); + zero[0x1ff] = 0u8.wrapping_sub(sum(&zero[..0x1ff])); + assert_eq!(sum(&zero), 0); + assert_eq!(sum(&Flash::new(&zero, FirmwareRecord::FUEL).unwrap().contents()[..0x200]), 0); + for payload in [zero.clone(), vec![0x5a; 0x200], factory_block()] { + let end = 0x200.min(payload.len()); + let before = sum(&payload[..end]); + let bytes = Flash::new(&payload, FirmwareRecord::FUEL).unwrap().contents(); + assert_eq!(sum(&bytes[..end]), before, "the image's byte sum"); + // A record that differs in one field is compensated to the same sum. + let other = FirmwareRecord { cpu_mode_word: FirmwareRecord::FUEL.cpu_mode_word ^ 0x0100_0000, ..FirmwareRecord::FUEL }; + assert_eq!(sum(&Flash::new(&payload, other).unwrap().contents()[..end]), before, "another mode word"); + for at in 0x60..0xb0 { + let mut broken = bytes[..end].to_vec(); + broken[at] ^= 1; + assert_ne!(sum(&broken), before, "byte {at:#x}"); + } + } + } + + /// The Config the CPU resets to carries the mode word's low 25 bits under + /// the fixed primary cache sizes, so the PROM's check of one against the + /// other passes: the Fuel's 500 MHz record gives 0x6c9aa985. + #[test] + fn the_reset_config_carries_the_mode_words_low_bits() { + assert_eq!(FirmwareRecord::FUEL.reset_mode(), 0x009a_a985); + assert_eq!(FirmwareRecord::FUEL.reset_config0(), 0x6c9a_a985); + let other = FirmwareRecord { cpu_mode_word: 0xfe00_0000 | 0x0012_3456, ..FirmwareRecord::FUEL }; + assert_eq!(other.reset_mode(), 0x0012_3456); + } + + /// Reopening a store reapplies the record, so a store follows the + /// machine that opens it. + #[test] + fn reopening_a_store_reapplies_the_record() { + let dir = std::env::temp_dir().join(format!("bedrock-flash-record-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("fuel.bin"); + let other = FirmwareRecord { cpu_mode_word: 0x1234_5678, machine_type: 0, ..FirmwareRecord::FUEL }; + drop(Flash::with_file(&factory_block(), &path, other).unwrap()); + assert_eq!(&std::fs::read(&path).unwrap()[0x64..0x68], &0x1234_5678u32.to_be_bytes()); + let fuel = Flash::with_file(&[], &path, FirmwareRecord::FUEL).unwrap().contents(); + assert_eq!(&fuel[0x64..0x68], &0x549aa985u32.to_be_bytes()); + assert_eq!(&fuel[0x90..0x94], &2u32.to_be_bytes(), "machine type: SN11"); + assert_eq!(std::fs::read(&path).unwrap(), fuel, "the store holds what the machine sees"); + std::fs::remove_dir_all(&dir).unwrap(); + } + const TEZRO_RECORDS: [(&str, FirmwareRecord); 4] = [ + ("700", FirmwareRecord::TEZRO), + ("800", FirmwareRecord::TEZRO_800), + ("900", FirmwareRecord::TEZRO_900), + ("1000", FirmwareRecord::TEZRO_1000), + ]; + + /// The stock block the PROM ships: the 400 MHz R10000 and R12000 row, + /// machine type 0, no compensation, edit counter 0. Built field by field, not + /// copied; its byte sum is 0xac, as the PROM's own checksum rule sees it. + fn stock_block() -> [u8; CONFIG_LEN] { + let mut block = FirmwareRecord { cpu_mode_word: 0x5219_a785, cpu_clock_hz: 400_000_000, machine_type: 0, ..FirmwareRecord::TEZRO } + .authored_block(); + block[0x38..0x3c].fill(0); + assert_eq!(sum(&block), 0xac, "the stock block's byte sum"); + block + } + + /// An image whose configuration block is the stock one, as the PROM's own + /// image has it. + fn stock_image() -> Vec { + let mut image = vec![0x21; 0x200]; + image[0x60..0xb0].copy_from_slice(&stock_block()); + image + } + + /// The seed's block follows the PROM editor's layout and rule: the mode + /// word, the clock, the Bedrock and real-time clocks, machine type 0 (the + /// class the PROM derives from the Tezro's board word) and an edit + /// counter of one, and a compensation byte that makes the block sum to + /// what the stock block did, 0xac, so the image keeps summing to zero: + /// 0xac less the sum of the other bytes. The PROM's editor, copying a + /// speed-table row's class 2, computes 0xd8, 0xfd (the 800 MHz row) and + /// 0x1c for one edit of a stock-seeded image; class 0 makes each two more. + #[test] + fn the_tezro_seed_writes_the_blocks_the_proms_editor_would() { + let image = stock_image(); + for ((name, record), compensation) in TEZRO_RECORDS.iter().zip([0xdau8, 0xff, 0x1e, 0xfa]) { + let bytes = Flash::new(&image, *record).unwrap().contents(); + let block = &bytes[0x60..0xb0]; + assert_eq!(&block[0..4], &21u32.to_be_bytes(), "{name}"); + assert_eq!(&block[4..8], &record.cpu_mode_word.to_be_bytes(), "{name}"); + assert_eq!(&block[8..16], b"ip27conf", "{name}"); + assert_eq!(&block[0x10..0x18], &u64::from(record.cpu_clock_hz).to_be_bytes(), "{name}"); + assert_eq!(&block[0x18..0x20], &200_000_000u64.to_be_bytes(), "{name}: Bedrock clock"); + assert_eq!(&block[0x20..0x28], &1_250_000u64.to_be_bytes(), "{name}: real-time clock"); + assert_eq!(&block[0x30..0x34], &0u32.to_be_bytes(), "{name}: the board word's machine class"); + assert_eq!(&block[0x38..0x3c], &1u32.to_be_bytes(), "{name}: one edit"); + assert_eq!(sum(block), 0xac, "{name}: the stock block's sum"); + assert_eq!(block[0x37], compensation, "{name}: the compensation the PROM editor computes"); + assert_eq!(&bytes[..0x60], &image[..0x60], "{name}: the payload around the block is untouched"); + assert_eq!(&bytes[0xb0..image.len()], &image[0xb0..]); + } + } + + /// The compensation follows whatever block the payload carried (the whole + /// image keeps its sum), and changing any one byte of a seeded block off + /// its compensated value breaks the sum. + #[test] + fn a_tezro_compensation_byte_keeps_the_images_sum_and_any_change_breaks_it() { + // An image that sums to zero, as the PROM's own does, still sums to + // zero once a record's block is seeded into it. + let mut zero = stock_image(); + zero[0x1ff] = 0u8.wrapping_sub(sum(&zero[..0x1ff])); + assert_eq!(sum(&zero), 0); + for (name, record) in TEZRO_RECORDS { + assert_eq!(sum(&Flash::new(&zero, record).unwrap().contents()[..0x200]), 0, "{name}: the seeded image sums to zero"); + } + for payload in [stock_image(), vec![0x5a; 0x200], factory_block()] { + let before = sum(&payload[..0x200.min(payload.len())]); + for (name, record) in TEZRO_RECORDS { + let bytes = Flash::new(&payload, record).unwrap().contents(); + let end = 0x200.min(payload.len()); + assert_eq!(sum(&bytes[..end]), before, "{name}: the image's byte sum"); + // A record that differs in one field is compensated to the same sum. + let other = FirmwareRecord { cpu_mode_word: record.cpu_mode_word ^ 0x0100_0000, ..record }; + assert_eq!(sum(&Flash::new(&payload, other).unwrap().contents()[..end]), before, "{name}: another mode word"); + for at in 0x60..0xb0 { + let mut broken = bytes[..end].to_vec(); + broken[at] ^= 1; + assert_ne!(sum(&broken), before, "{name}: byte {at:#x}"); + } + } + } + } + + /// The Tezro records carry the PROM table's mode words and the Config the + /// CPU resets to carries the same low 25 bits, so the PROM's check of one + /// against the other passes; the mode words of the three chips the PROM's + /// speed table lists are its rows, and the 1 GHz one decodes by the + /// editor's formulas. + #[test] + fn the_tezro_records_carry_their_mode_words_and_the_configs_they_imply() { + // Config bits 24:0 are the mode inputs; the cache sizes above are fixed. + const CONFIG0_MODE_MASK: u32 = 0x01ff_ffff; + // (mode word, clock MHz, Bedrock MHz = 200): the PROM table's rows. + let rows = [(0x589bad85u32, 700u32, 0x6c9b_ad85u32), (0x549baf85, 800, 0x6c9b_af85), (0x569cb185, 900, 0x6c9c_b185)]; + for ((_, record), (word, mhz, config)) in TEZRO_RECORDS.iter().zip(rows) { + assert_eq!(record.cpu_mode_word, word); + assert_eq!(record.cpu_clock_hz, mhz * 1_000_000); + assert_eq!(record.reset_config0(), config); + } + let field = |word: u32, lo: u32, bits: u32| (word >> lo) & ((1 << bits) - 1); + let decoded = [(350, 4u32 << 20), (400, 4 << 20), (450, 8 << 20), (333, 16 << 20)]; + for ((name, r), (cache_mhz, cache_bytes)) in TEZRO_RECORDS.iter().zip(decoded) { + let (w, mhz) = (r.cpu_mode_word, r.cpu_clock_hz / 1_000_000); + assert_eq!(w & CONFIG0_MODE_MASK, r.reset_mode(), "{name}"); + assert_eq!((field(w, 0, 3), field(w, 7, 2), field(w, 13, 1), field(w, 15, 1)), (5, 3, 1, 1), "{name}: K0 PM SB BE"); + assert_eq!(field(w, 9, 4), 2 * mhz / 200 - 1, "{name}: EC = 2 * CPU / Bedrock - 1"); + assert_eq!(2 * mhz / (field(w, 19, 3) + 1), cache_mhz, "{name}: cache clock = 2 * CPU / (SC + 1)"); + assert_eq!(1u32 << (field(w, 16, 3) + 19), cache_bytes, "{name}: SS is 512 KiB << SS"); + } + // The tap the PROM prints: tap[3:0] in bits 28:25, tap[4] at bit 22. + let tap = |word: u32| field(word, 25, 4) | (field(word, 22, 1) << 4); + assert_eq!(TEZRO_RECORDS.map(|(_, r)| tap(r.cpu_mode_word)), [0xc, 0xa, 0xb, 0x15]); + assert_eq!(FirmwareRecord::TEZRO_1000.cpu_mode_word, 0x4aed_b385); + assert_eq!(FirmwareRecord::TEZRO_1000.reset_config0(), 0x6ced_b385); + } + + /// Reopening a store reapplies a refreshed record, so a Fuel follows the + /// machine that opens it, and leaves an authored block as stored, so a + /// block the PROM rewrote on a Tezro survives the next start. + #[test] + fn reopening_keeps_a_tezros_stored_block_and_refreshes_a_fuels() { + let dir = std::env::temp_dir().join(format!("bedrock-flash-tezro-record-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let fuel_path = dir.join("fuel.bin"); + drop(Flash::with_file(&factory_block(), &fuel_path, FirmwareRecord::TEZRO).unwrap()); + let fuel = Flash::with_file(&[], &fuel_path, FirmwareRecord::FUEL).unwrap().contents(); + assert_eq!(&fuel[0x64..0x68], &0x549aa985u32.to_be_bytes()); + assert_eq!(&fuel[0x90..0x94], &2u32.to_be_bytes(), "machine type: SN11"); + + let tezro_path = dir.join("tezro.bin"); + let seeded = Flash::with_file(&factory_block(), &tezro_path, FirmwareRecord::TEZRO).unwrap().contents(); + assert_eq!(std::fs::read(&tezro_path).unwrap(), seeded, "the seed is what the store holds"); + let mut stored = seeded.clone(); + stored[0x64..0x68].copy_from_slice(&0x1234_5678u32.to_be_bytes()); + std::fs::write(&tezro_path, &stored).unwrap(); + let reopened = Flash::with_file(&factory_block(), &tezro_path, FirmwareRecord::TEZRO).unwrap().contents(); + assert_eq!(reopened, stored); + std::fs::remove_dir_all(&dir).unwrap(); + } + /// The PROM programs through one view while executable aliases must + /// keep reading array bytes. + #[test] + fn programming_changes_shared_bytes_without_exposing_status_to_instruction_fetches() { + { + let f = Arc::new(Flash::new(&[], FirmwareRecord::FUEL).unwrap()); + let local = f.boot_view(); + let remote = f.boot_view(); + let execute = f.array_view(0, PART_SIZE).unwrap(); + assert_eq!(local.write64(SECOND_POSITION, 0xf0), crate::traits::BUS_OK); + for part in [0, SECOND_POSITION] { + assert_eq!(local.write64(part + 8 * 0x55, 0x98), crate::traits::BUS_OK); + assert_eq!(remote.read16(part + 0x20).data, b'Q' as u16); + assert_eq!(remote.read16((SECOND_POSITION - part) + 0x20).data, 0xffff); + assert_eq!(execute.read16(0x20).data, 0xffff); + assert_eq!(remote.write64(part, 0xf0), crate::traits::BUS_OK); + assert_eq!(local.write64(part + 8 * 0x555, 0xaa), crate::traits::BUS_OK); + assert_eq!( + remote.write64(part + 8 * 0x2aa, 0x55), + crate::traits::BUS_OK + ); + assert_eq!(local.write64(part + 8 * 0x555, 0xa0), crate::traits::BUS_OK); + assert_eq!( + remote.write64(part + 4 * 0x200, 0x1234), + crate::traits::BUS_OK + ); + assert_eq!(local.read16(part + 0x200).data, 0x1234); + } + assert_eq!(execute.read16(0x200).data, 0x1234); + assert!(!local.read8(PART_SIZE as u64).is_ok()); + } + { + let f = Arc::new(Flash::new(&[], FirmwareRecord::FUEL).unwrap()); + let v = f.boot_view(); + program(&v, 0, 0x200, 0x1234); + program(&v, 0, 0x200, 0xf0ff); + assert_eq!(v.read16(0x200).data, 0x1034); + unlock(&v, 0); + assert_eq!(v.write64(8 * 0x555, 0xa0), crate::traits::BUS_OK); + let gen = f.generation(); + for (addr, value) in [ + (0x800, 0x10000), + (0x804, 0), + (64 << 20, 0), + ] { + assert_eq!(v.write64(addr, value), BUS_ERR); + } + assert_eq!(v.write16(0x800, 0), BUS_ERR); + assert_eq!(f.generation(), gen); + assert_eq!(v.write64(0x800, 0x00ff), crate::traits::BUS_OK); + assert_eq!(v.read16(0x200).data, 0x0034); + let execute = f.array_view(0, PART_SIZE).unwrap(); + unlock(&v, 0); + assert_eq!(execute.write64(8 * 0x555, 0xa0), BUS_ERR); + assert_eq!(v.write64(8 * 0x555, 0xa0), crate::traits::BUS_OK); + assert_eq!(v.write64(0x808, 0x5555), crate::traits::BUS_OK); + assert_eq!(execute.read16(0x202).data, 0x5555); + } + { + let f = Arc::new(Flash::new(&[], FirmwareRecord::FUEL).unwrap()); + let v = f.boot_view(); + assert_eq!(v.write64(0x550, 0x98), BUS_ERR); + assert_eq!(v.read16(0x20).data, 0xffff); + assert_eq!(v.write64(8 * 0x55, 0x98), BUS_OK); + let expected = [ + 0x51, 0x52, 0x59, 2, 0, 0, 0, 0, 0, 0, 0, 0x27, 0x36, 0, 0, 3, 0, 9, 0, 4, 0, 4, 0, 23, + 1, 0, 0, 0, 1, 0x7f, 0, 0, 1, + ]; + for (index, value) in expected.into_iter().enumerate() { + assert_eq!(v.read16(((index + 0x10) * 2) as u64).data, value); + } + assert_eq!(v.read16(0x100).data, 0); + assert!(!v.read8(0x20).is_ok()); + assert!(!v.read16(0x21).is_ok()); + f.reset(); + assert_eq!(v.read16(0x20).data, 0xffff); + } + } + /// The PROM polls erase completion and can reset the chip before + /// selected sectors change. + #[test] + fn erase_completes_only_for_selected_sectors_and_reset_cancels_it() { + let f = Arc::new(Flash::new(&[], FirmwareRecord::FUEL).unwrap()); + let v = f.boot_view(); + program(&v, 0, 0x200, 0); + program(&v, 0, 0x10000, 0); + erase_select(&v, 0, 0x200); + assert!(!v.read8(0x200).is_ok()); + let a = v.read16(0x200).data; + let b = v.read16(0x200).data; + assert_eq!((a ^ b) & 0x44, 0x44); + let a = v.read16(0x10000).data; + let b = v.read16(0x10000).data; + assert_eq!((a ^ b) & 0x44, 0x40); + f.advance_ns(40_000); + assert_eq!(v.write64(4 * 0x10000, 0x30), BUS_OK); + f.advance_ns(40_000); + assert_eq!(v.read16(0x10000).data & 8, 0); + assert_eq!(v.write64(SECOND_POSITION + 8 * 0x55, 0x98), BUS_OK); + assert_eq!(v.read16(SECOND_POSITION + 0x20).data, b'Q' as u16); + f.advance_ns(10_000); + assert_eq!(v.read16(0x200).data & 8, 8); + assert_eq!(v.write64(0, 0xf0), BUS_OK); + assert_eq!(v.read16(0x200).data & 0x80, 0); + f.reset(); + f.advance_ns(u64::MAX); + assert_eq!(v.read16(0x200).data, 0); + assert_eq!(v.read16(0x10000).data, 0); + assert_eq!(v.read16(SECOND_POSITION + 0x20).data, 0xffff); + erase_select(&v, 0, 0x200); + assert_eq!(v.write64(0, 0xf0), BUS_OK); + f.advance_ns(u64::MAX); + assert_eq!(v.read16(0x200).data, 0); + erase_select(&v, 0, 0x200); + assert_eq!(v.write64(4 * 0x10000, 0x30), BUS_OK); + f.advance_ns(50_000 + 512_000_000); + assert_eq!(v.read16(0x200).data, 0xffff); + assert_eq!(v.read16(0x10000).data, 0xffff); + unlock(&v, 0); + assert_eq!(v.write64(8 * 0x555, 0xa0), BUS_OK); + f.reset(); + assert_eq!(v.write64(4 * 0x200, 0x1234), BUS_ERR); + assert_eq!(v.read16(0x200).data, 0xffff); + program(&v, 0, 0x200, 0); + program(&v, 0, 0x10000, 0x3456); + erase_select(&v, 0, 0x200); + f.advance_ns(50_000 + 512_000_000); + assert_eq!(v.read16(0x200).data, 0xffff); + assert_eq!(v.read16(0x10000).data, 0x3456); + } + /// Firmware updates must persist across emulator restarts without + /// overwriting a supplied image. + #[test] + fn completed_programming_and_erases_survive_reopening_the_backing_file() { + { + let path = std::env::temp_dir().join(format!("iris-bedrock-store-{}", std::process::id())); + let mut bytes = vec![0x5a; FLASH_SIZE]; + bytes[PART_SIZE] = 0xc7; + std::fs::write(&path, &bytes).unwrap(); + let f = Flash::with_file(&[0x11], &path, FirmwareRecord::FUEL).unwrap(); + assert_eq!(f.contents()[0], 0x5a); + assert_eq!(f.contents()[PART_SIZE], 0xc7); + assert_eq!(std::fs::read(&path).unwrap(), f.contents()); + drop(f); + std::fs::write(&path, [1, 2, 3]).unwrap(); + assert!(Flash::with_file(&[], &path, FirmwareRecord::FUEL).is_err()); + assert_eq!(std::fs::read(&path).unwrap(), [1, 2, 3]); + std::fs::write(&path, []).unwrap(); + let g = Flash::with_file(&[0x34], &path, FirmwareRecord::FUEL).unwrap(); + assert_eq!(g.contents()[0], 0x34); + assert_eq!(std::fs::metadata(&path).unwrap().len(), FLASH_SIZE as u64); + drop(g); + std::fs::remove_file(path).unwrap(); + } + { + let path = std::env::temp_dir().join(format!("iris-amd-persist-{}", std::process::id())); + std::fs::write(&path, []).unwrap(); + let f = Arc::new(Flash::with_file(&[], &path, FirmwareRecord::FUEL).unwrap()); + let v = f.boot_view(); + program(&v, 0, 0x200, 0x1234); + program(&v, SECOND_POSITION, (PART_SIZE - 2) as u64, 0x5678); + drop(v); + drop(f); + let f = Arc::new(Flash::with_file(&[], &path, FirmwareRecord::FUEL).unwrap()); + let v = f.boot_view(); + assert_eq!(v.read16(0x200).data, 0x1234); + assert_eq!( + v.read16(SECOND_POSITION + PART_SIZE as u64 - 2).data, + 0x5678 + ); + unlock(&v, SECOND_POSITION); + assert_eq!(v.write64(SECOND_POSITION + 8 * 0x555, 0x80), BUS_OK); + unlock(&v, SECOND_POSITION); + assert_eq!( + v.write64(SECOND_POSITION + 4 * (PART_SIZE as u64 - 2), 0x30), + BUS_OK + ); + f.advance_ns(50_000 + 512_000_000); + drop(v); + drop(f); + let f = Arc::new(Flash::with_file(&[], &path, FirmwareRecord::FUEL).unwrap()); + let v = f.boot_view(); + assert_eq!(v.read16(0x200).data, 0x1234); + assert_eq!( + v.read16(SECOND_POSITION + PART_SIZE as u64 - 2).data, + 0xffff + ); + std::fs::remove_file(path).unwrap(); + } + } + + /// A host write failure must preserve flash contents and remain visible + /// to the guest. + #[test] + fn failed_backing_writes_leave_array_bytes_unchanged() { + { + let f = Arc::new(Flash::new(&[], FirmwareRecord::FUEL).unwrap()); + // A read-only descriptor makes writes fail without a special backend. + let path = std::env::temp_dir().join(format!("iris-amd-read-only-{}", std::process::id())); + std::fs::write(&path, []).unwrap(); + f.st.lock().backing = Some(FlashBacking::from_file( + std::fs::File::open(&path).unwrap(), + FLASH_SIZE, + )); + let v = f.boot_view(); + unlock(&v, 0); + assert_eq!(v.write64(8 * 0x555, 0xa0), BUS_OK); + assert_eq!(v.write64(4 * 0x200, 0x1234), BUS_ERR); + assert_eq!(v.read16(0x200).data, 0xffff); + std::fs::remove_file(path).unwrap(); + } + { + let f = Arc::new(Flash::new(&[], FirmwareRecord::FUEL).unwrap()); + let v = f.boot_view(); + program(&v, 0, 0x200, 0); + let path = + std::env::temp_dir().join(format!("iris-amd-erase-failure-{}", std::process::id())); + std::fs::write(&path, []).unwrap(); + f.st.lock().backing = Some(FlashBacking::from_file( + std::fs::File::open(&path).unwrap(), + FLASH_SIZE, + )); + erase_select(&v, 0, 0x200); + f.advance_ns(50_000 + 512_000_000); + assert_eq!(v.read16(0x200).data & 0x20, 0x20); + assert_eq!(f.contents()[0x200], 0); + assert_eq!(v.write64(0, 0xf0), BUS_OK); + assert_eq!(v.read16(0x200).data, 0); + std::fs::remove_file(path).unwrap(); + } + } + +} diff --git a/src/dev/bridge.rs b/src/dev/bridge.rs index d49d63919..1ecad3b47 100644 --- a/src/dev/bridge.rs +++ b/src/dev/bridge.rs @@ -52,6 +52,22 @@ //! threads' edges on one pin reach HEART in the order they were decided. //! DMA goes to the memory every bus master shares (`Arc`). //! +//! # The XBridge +//! +//! The IP35 boards' XBridge holds two PCI heads, each this chip's register +//! file, windows, interrupt pins and DMA windows at a crossbar port of its +//! own, built as [`Part::XBridge`]. Where the head differs the part decides, +//! in the slow paths only: identification, the Control straps and writable +//! masks, a 1024-entry translation table of which 512 entries map a window +//! with a byte-swap alias at bit 29, a swap bit at PCI bit 55 for 64-bit +//! DMA, the force-interrupt and multiple-error registers, the 64-bit and +//! I/O aliases at 2 GB and 4 GB, and what an unclaimed or undefined access +//! answers. A processor reaches a head through [`Bridge::head_window`], +//! whose decode is the head's own; the Bridge's own window decode is +//! untouched. The head's behaviour follows hw/pci-host/sgi-xbridge.c in +//! jamesbraid/qemu-sgi-origami, not a published specification; it has +//! no flash, SSRAM or diagnostic engines. +//! //! Ported from hw/pci-host/sgi-bridge.c in jamesbraid/qemu-sgi-origami. use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, AtomicU8, Ordering}; @@ -118,6 +134,14 @@ pub mod reg { pub const RRB_ODD: u32 = 0x288; pub const RRB_STATUS: u32 = 0x290; pub const RRB_CLEAR: u32 = 0x298; + /// The XBridge's multiple-error register: every error it latched since + /// the groups were last cleared. + pub const INT_MULTIPLE: u32 = 0x170; + /// The XBridge's force registers: a write to the first eight sends + /// pin n's packet, to the next eight only while pin n is high. + pub const FORCE_ALWAYS: u32 = 0x180; + pub const FORCE_PIN: u32 = 0x1c0; + pub const FORCE_PIN_LAST: u32 = 0x1f8; } /// A register's name for the `bridge` log, from its slot base. @@ -184,6 +208,28 @@ const LONG_MASK: u64 = LONG_WINDOW_SIZE - 1; const ALIAS_WINDOW: usize = DEVICES; pub const PINS: usize = 8; pub const ATE_ENTRIES: usize = 128; +/// The XBridge head's table, and the entries of it its mapped window +/// reaches: 512 pages, at 0x4000_0000 and again with bytes swapped at +/// 0x6000_0000. Its lower halves read back at 0x12000, eight bytes apart. +pub const XBRIDGE_ATE_ENTRIES: usize = 1024; +const XBRIDGE_ATE_MAPPED: usize = 512; +const XBRIDGE_ATE_LOW_BASE: u32 = 0x01_2000; +const XBRIDGE_ATE_END: u32 = 0x01_4000; +const XBRIDGE_MAPPED_SPAN: u32 = 0x80_0000; +const XBRIDGE_MAPPED_SWAP: u32 = 1 << 29; +/// The address bits an XBridge table entry keeps. +const XBRIDGE_ATE_KEPT: u64 = (1 << 48) - 1; +/// An XBridge table entry's own swap bit. +const XBRIDGE_ATE_SWAP: u64 = 1 << 5; +/// The 64 KB below the mapped window the XBridge answers as its flush +/// target: a read finds zeros and a write is taken and dropped. +const XBRIDGE_FLUSH: std::ops::Range = 0x3fff_0000..0x4000_0000; +/// PCI address bit 55 asks the XBridge to swap a 64-bit direct transfer. +const XBRIDGE_DMA64_SWAP: u64 = 1 << 55; +/// The XBridge head's aliases of PCI space: 32-bit memory from 1 GB, +/// 64-bit memory from 2 GB and I/O from 4 GB. +const XBRIDGE_ALIASES: [(u64, u64, Space); 3] = + [(0x4000_0000, 0x4000_0000, Space::Mem), (0x8000_0000, 0x4000_0000, Space::Mem), (0x1_0000_0000, 0x1_0000_0000, Space::Io)]; /// Offset of `slot`'s device window in the small window. The first two are /// 2 MB, the rest 1 MB. @@ -219,14 +265,41 @@ const ID_PART_SHIFT: u32 = 12; const ID_MANUFACTURER_SHIFT: u32 = 1; /// The silicon revision a Bridge reports until the board names its own. const REVISION: u32 = 1; -const PART: u32 = 0xc002; -const MANUFACTURER: u32 = 0x036; + +/// Which member of the Bridge family a [`Bridge`] is: see the module +/// documentation for where they differ. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Part { + /// The Octane's and the IP27 boards' Bridge. + Bridge, + /// One PCI head of the IP35 boards' XBridge. + XBridge, +} + +impl Part { + /// The part number and manufacturer in the identification word. + const fn identity(self) -> (u32, u32) { + match self { + Part::Bridge => (0xc002, 0x036), + Part::XBridge => (0xd002, 0x024), + } + } + + /// Translation entries the chip holds. + pub const fn ate_entries(self) -> usize { + match self { + Part::Bridge => ATE_ENTRIES, + Part::XBridge => XBRIDGE_ATE_ENTRIES, + } + } +} const STATUS_FLASH_SELECT: u32 = 1 << 6; const STATUS_PCI_MODE: u32 = 1 << 5; pub const CONTROL_FLASH_WRITE_EN: u32 = 1 << 31; pub const CONTROL_MEM_SWAP: u32 = 1 << 22; +pub const CONTROL_IO_SWAP: u32 = 1 << 23; pub const CONTROL_PAGE_16K: u32 = 1 << 21; /// Control 27:24, RST_PIN_N(3:0): the four PCI reset pins, active low, all /// high (deasserted) at reset. Established: the bits, their reset value and @@ -245,6 +318,8 @@ pub const CONTROL_PAGE_16K: u32 = 1 << 21; /// does not say that either does. const CONTROL_RST_PINS: u32 = 0xf << 24; const CONTROL_RESET: u32 = (1 << 30) | (1 << 29) | (1 << 28) | CONTROL_RST_PINS | (3 << 12) | (1 << 9) | (0x1f << 4) | 0xf; +/// The widget ID the chip puts on the packets it sends. +const CONTROL_WIDGET_ID: u32 = 0xf; /// Two strobes that clear the link retry counters and always read zero. const CONTROL_RETRY_CLEAR: u32 = 0x3 << 10; @@ -281,9 +356,14 @@ const INT_PCI_GROUP: u32 = 0x3f << 10; pub const INT_XREAD_REQ_TIMEOUT: u32 = 1 << 9; const INT_RESP_BUF_GROUP: u32 = (1 << 28) | (1 << 26) | (1 << 9); pub const INT_MULTI_ERR: u32 = 1 << 31; +/// A transfer through a translation entry that is not valid, or past the +/// ones the chip has; the XBridge alone raises it. +pub const INT_PAGE_FAULT: u32 = 1 << 30; +const INT_SSRAM_GROUP: u32 = INT_PAGE_FAULT | (1 << 16); const INT_ENABLE_WRITABLE: u32 = 0x7fff_ffff; const INT_RESET_GROUPS: u32 = 0x7f; const INT_RESET_PCI_GRP: u32 = 1 << 0; +const INT_RESET_SSRAM_GRP: u32 = 1 << 1; const INT_RESET_REQ_DSP_GRP: u32 = 1 << 3; const INT_RESET_RESP_BUF_GRP: u32 = 1 << 4; const INT_RESET_MULTI_ERR: u32 = 1 << 6; @@ -352,10 +432,36 @@ const fn info(access: Access, reset: u32, writable: u32) -> Option { Some(RegInfo { access, reset, writable }) } +/// The XBridge head's registers where they are not the Bridge's: Control's +/// PCI speed strap (bits 7:4) and the bits it lacks read only, narrower +/// fields elsewhere, and the multiple-error and force-interrupt registers. +/// `None` is the Bridge's register. +fn xbridge_reg_info(slot_base: u32) -> Option> { + use reg::*; + use Access::*; + Some(match slot_base { + CONTROL => info(Rw, 0x0f00_320f, 0x0fe1_f20f), + DIRECT_MAP => info(Rw, DIRECT_ADD_512MB, 0xf3_ffff), + BUS_TIMEOUT => info(Rw, BUS_TIMEOUT_RESET, 0x1f_03ff), + INT_DEST_UPPER => info(Rw, 0, 0xf_ffff), + TEST_PIN_CONTROL => info(Rw, 0, 0xff_ffff), + INT_MULTIPLE => info(Live, 0, 0), + 0x178 => info(Ro, 0, 0), + FORCE_ALWAYS..=FORCE_PIN_LAST if slot_base % 8 == 0 => info(Wo, 0, 0), + 0x200..=0x238 if slot_base % 8 == 0 => info(Rw, device_reset((slot_base - 0x200) / 8), 0x1eef_ffff), + _ => return None, + }) +} + /// Decode, reset value and writable mask from one table, by slot base. -fn reg_info(slot_base: u32) -> Option { +fn reg_info(part: Part, slot_base: u32) -> Option { use reg::*; use Access::*; + if part == Part::XBridge { + if let Some(info) = xbridge_reg_info(slot_base) { + return info; + } + } match slot_base { IDENTIFICATION | ERR_UPPER | ERR_LOWER | ERR_CMD_WORD | TARGET_FLUSH | AUX_ERR_CMD_WORD | RESP_ERR_UPPER | RESP_ERR_LOWER | SSRAM_PARITY_ERR | PCI_ERR_UPPER | PCI_ERR_LOWER | RRB_STATUS => info(Ro, 0, 0), @@ -393,9 +499,16 @@ fn int_dest(upper: u32, lower: u32) -> IntDest { IntDest { widget: ((upper >> 16) & 0xf) as u8, address: (((upper & 0xffff) as u64) << 32) | lower as u64 } } +/// The widget a packet says it came from: the ID the guest gave the chip in +/// Control bits 3:0. +fn source(st: &State) -> u8 { + (st.r(reg::CONTROL) & CONTROL_WIDGET_ID) as u8 +} + struct State { reg: [u32; REG_SLOTS], - ate: [u64; ATE_ENTRIES], + /// Sized for the largest part; a part uses its own count. + ate: [u64; XBRIDGE_ATE_ENTRIES], /// Latched error bits of the interrupt status register (31:8). int_error: u32, /// The eight pins' driven levels, which the status register reports. @@ -406,6 +519,8 @@ struct State { /// The transfer the last DMA translation served, for error capture. dma_pci_address: u64, dma_device: u32, + /// The XBridge's multiple-error register ([`reg::INT_MULTIPLE`]). + multiple: u32, } impl State { @@ -419,22 +534,26 @@ impl State { /// The chip's own core: the register file, the table and the latched /// errors. The interrupt pins' levels belong to whatever drives them. - fn reset_core(&mut self, flash_strap: bool, revision: u32) { + fn reset_core(&mut self, part: Part, flash_strap: bool, revision: u32, control_strap: u32) { for (i, slot) in self.reg.iter_mut().enumerate() { - *slot = reg_info(i as u32 * 8).map_or(0, |r| r.reset); + *slot = reg_info(part, i as u32 * 8).map_or(0, |r| r.reset); } + let (number, manufacturer) = part.identity(); self.set_r( reg::IDENTIFICATION, - ((revision & 0xf) << ID_REVISION_SHIFT) | (PART << ID_PART_SHIFT) | (MANUFACTURER << ID_MANUFACTURER_SHIFT) | 1, + ((revision & 0xf) << ID_REVISION_SHIFT) | (number << ID_PART_SHIFT) | (manufacturer << ID_MANUFACTURER_SHIFT) | 1, ); if flash_strap { let s = self.r(reg::STATUS); self.set_r(reg::STATUS, s | STATUS_FLASH_SELECT); } + let c = self.r(reg::CONTROL); + self.set_r(reg::CONTROL, c | control_strap); self.int_error = 0; + self.multiple = 0; // Whole entries, not just valid bits: a stale page address behind a // clear valid bit is a working translation waiting for one write. - self.ate = [0; ATE_ENTRIES]; + self.ate.fill(0); } } @@ -445,8 +564,17 @@ struct FlashPart { size: u32, } -/// The packets a locked section decided to send: (host widget, vector, set). -type Packets = Vec<(u8, u8, bool)>; +/// An interrupt packet a locked section decided to send: from the widget +/// Control names, to `widget` at `address`, setting or clearing `vector`. +struct Packet { + source: u8, + widget: u8, + address: u64, + vector: u8, + set: bool, +} + +type Packets = Vec; /// The onboard Bridge. Hand it out as the `BusDevice` of the widget's small /// window: `Arc` coerces to `Arc`. As an @@ -454,7 +582,7 @@ type Packets = Vec<(u8, u8, bool)>; pub struct Bridge { st: Mutex, /// Held across "decide a packet, send it", never with `st`. - /// Nothing but the host's `set_line` is called under it. + /// Nothing but the host's interrupt input is called under it. irq_order: Mutex<()>, /// The host chips this Bridge's transfers and interrupt packets can go /// to, by the widget number the guest programs. @@ -478,6 +606,10 @@ pub struct Bridge { link_alive: RwLock bool + Send + Sync>>, /// The Crossbow port this Bridge is on: 0xf for the onboard one. widget: AtomicU8, + part: Part, + /// Control bits the board straps, kept through a reset: the XBridge + /// head's PCI bus speed in 7:4. + control_strap: AtomicU32, } /// A function's way to memory, tagged with its slot: the swap bits that @@ -508,6 +640,15 @@ enum DmaBuf<'a> { Write(&'a [u8]), } +/// Where a DMA translation sends a transfer. +#[derive(Clone, Copy)] +enum Route { + /// To the widget's XIO address, with the bytes placed so. + Host(u64, Placement, u8), + /// To the XBridge's flush target. + Flush, +} + /// How a transfer's bytes sit in memory relative to PCI address order. #[derive(Clone, Copy, PartialEq, Eq)] enum Placement { @@ -535,16 +676,22 @@ impl Bridge { /// programs, its PCI bus wired as `wiring`. Registers start at their /// power-up values. pub fn new(up: Upstream, wiring: &'static IntxRouting) -> Arc { + Self::with_part(Part::Bridge, up, wiring) + } + + /// As [`Bridge::new`], for `part` of the family. + pub fn with_part(part: Part, up: Upstream, wiring: &'static IntxRouting) -> Arc { let mut st = State { reg: [0; REG_SLOTS], - ate: [0; ATE_ENTRIES], + ate: [0; XBRIDGE_ATE_ENTRIES], int_error: 0, pin_level: 0, dma_64bit: 0, dma_pci_address: 0, dma_device: 0, + multiple: 0, }; - st.reset_core(false, REVISION); + st.reset_core(part, false, REVISION, 0); let hosts = (0..16u8).filter(|&w| up.get(w).is_some()).fold(0u16, |m, w| m | 1 << w); let this = Arc::new_cyclic(|me: &Weak| Self { st: Mutex::new(st), @@ -559,6 +706,8 @@ impl Bridge { bar_devices: Default::default(), link_alive: RwLock::new(Box::new(move |w| hosts.checked_shr(w as u32).is_some_and(|m| m & 1 != 0))), widget: AtomicU8::new(WIDGET_BRIDGE), + part, + control_strap: AtomicU32::new(0), }); this.bus.set_host_windows(&|| this.windows()); this @@ -583,9 +732,13 @@ impl Bridge { let _order = self.irq_order.lock(); let mut packets = Packets::new(); let ret = f(&mut self.st.lock(), &mut packets); - for (widget, vector, set) in packets { + for Packet { source, widget, address, vector, set } in packets { match self.up.get(widget) { - Some(h) if h.link_up() => h.irq.set_line(vector as u32, set), + Some(h) if h.link_up() => { + if !h.irq.packet(source, address, vector as u32, set) { + log::debug!("bridge: widget {widget} refused interrupt packet {address:#x} vector {vector}"); + } + } _ => log::debug!("bridge: interrupt packet to widget {widget} dropped; no live link to a host"), } } @@ -597,7 +750,10 @@ impl Bridge { /// After them, the 32-bit alias of PCI memory in the 2 GB window. fn windows(&self) -> Vec { let st = self.st.lock(); - let alias = CpuWindow { space: Space::Mem, cpu: PIO32_ALIAS as u64, pci: 0, len: PIO32_ALIAS_SIZE as u64, mask: LONG_MASK }; + let aliases = match self.part { + Part::Bridge => vec![CpuWindow { space: Space::Mem, cpu: PIO32_ALIAS as u64, pci: 0, len: PIO32_ALIAS_SIZE as u64, mask: LONG_MASK }], + Part::XBridge => XBRIDGE_ALIASES.iter().map(|&(cpu, len, space)| CpuWindow { space, cpu, pci: 0, len, mask: u64::MAX }).collect(), + }; (0..DEVICES as u32) .map(|s| { let d = st.r(reg::device(s)); @@ -609,7 +765,7 @@ impl Bridge { mask: WINDOW_MASK as u64, } }) - .chain(std::iter::once(alias)) + .chain(aliases) .collect() } @@ -620,14 +776,23 @@ impl Bridge { /// programmed swap does for a window, and the device counts as 32-bit. fn order(&self, window: usize) -> ByteOrder { let st = self.st.lock(); - if window == ALIAS_WINDOW { - return Self::alias_order(st.r(reg::CONTROL)); + if window >= ALIAS_WINDOW { + return Self::alias_order(st.r(reg::CONTROL), Self::alias_space(window)); } Self::order_of(st.r(reg::device(window as u32))) } - fn alias_order(control: u32) -> ByteOrder { - if control & CONTROL_MEM_SWAP != 0 { ByteOrder::Stream } else { ByteOrder::values(4) } + /// The space alias window `window` opens onto: the XBridge's third alias + /// is I/O, every other memory. + fn alias_space(window: usize) -> Space { + if window == ALIAS_WINDOW + 2 { Space::Io } else { Space::Mem } + } + + /// Control's memory swap bit steers the memory aliases, its I/O swap bit + /// the I/O alias. + fn alias_order(control: u32, space: Space) -> ByteOrder { + let swap = if space == Space::Io { CONTROL_IO_SWAP } else { CONTROL_MEM_SWAP }; + if control & swap != 0 { ByteOrder::Stream } else { ByteOrder::values(4) } } fn order_of(device: u32) -> ByteOrder { @@ -695,6 +860,15 @@ impl Bridge { st.set_r(reg::IDENTIFICATION, (id & !(0xf << ID_REVISION_SHIFT)) | ((revision & 0xf) << ID_REVISION_SHIFT)); } + /// Strap Control bits the guest cannot write (the XBridge head's PCI + /// bus speed, bits 7:4). They survive a reset, as the board's wiring does. + pub fn set_control_strap(&self, bits: u32) { + self.control_strap.store(bits, Ordering::Release); + let mut st = self.st.lock(); + let c = st.r(reg::CONTROL); + st.set_r(reg::CONTROL, c | bits); + } + /// Tell the Bridge which widgets the Crossbow has a live link to. A DMA /// request to any other widget is dropped by the Crossbow, which the /// Bridge's request timer reports. The default is the host chip alone. @@ -735,7 +909,8 @@ impl Bridge { pub fn reset(&self) { let strap = self.flash_strap.load(Ordering::Acquire); let revision = self.revision.load(Ordering::Acquire); - self.st.lock().reset_core(strap, revision); + let control = self.control_strap.load(Ordering::Acquire); + self.st.lock().reset_core(self.part, strap, revision, control); self.bus.set_host_windows(&|| self.windows()); } @@ -796,7 +971,7 @@ impl Bridge { /// queued for `irq_locked` to send once the state lock is dropped. fn send_interrupt(&self, st: &State, p: &mut Packets, pin: usize, set: bool) { let host = st.r(reg::int_host(pin as u32)); - if host == 0 { + if host == 0 && self.part == Part::Bridge { // An unprogrammed destination is an interrupt nobody routed. log::debug!("bridge: interrupt pin {pin} changed with no destination programmed"); return; @@ -810,7 +985,7 @@ impl Bridge { match self.up.get(dest.widget) { Some(h) if h.host.accepts(dest.widget, address, vector) => { crate::dlog!(LogModule::Bridge, "bridge: pin {pin} {} vector {vector} of widget {}", if set { "sets" } else { "clears" }, dest.widget); - p.push((dest.widget, vector, set)); + p.push(Packet { source: source(st), widget: dest.widget, address, vector, set }); } Some(_) => log::debug!("bridge: interrupt packet to widget {} address {address:#x} vector {vector} is not one its host takes; dropped", dest.widget), None => log::debug!("bridge: interrupt packet to widget {}, which is not a host; dropped", dest.widget), @@ -823,7 +998,9 @@ impl Bridge { let dest = int_dest(st.r(reg::INT_DEST_UPPER), st.r(reg::INT_DEST_LOWER)); // An unrouted error is one nobody asked for. match self.up.get(dest.widget) { - Some(h) if h.host.accepts(dest.widget, dest.address, number) => p.push((dest.widget, number, true)), + Some(h) if h.host.accepts(dest.widget, dest.address, number) => { + p.push(Packet { source: source(st), widget: dest.widget, address: dest.address, vector: number, set: true }) + } _ => log::debug!("bridge: error interrupt with no usable destination; dropped"), } } @@ -869,12 +1046,23 @@ impl Bridge { /// An undefined address in the block posts and flags rather than /// faulting: a firmware self-test provokes one and fails on any /// synchronous exception. - fn invalid_address(&self, st: &mut State, p: &mut Packets, off: u32, write: bool) { + fn invalid_address(&self, st: &mut State, p: &mut Packets, off: u64, write: bool) { log::debug!( "bridge: {} of undefined register {off:#x}; flagging an invalid-address error", if write { "write" } else { "read" } ); + let first = st.int_error & INT_INVALID_ADDR == 0; st.int_error |= INT_INVALID_ADDR; + if self.part == Part::XBridge { + // The XBridge captures the first such address and interrupts + // for it alone. + st.multiple |= INT_INVALID_ADDR; + if !first { + return; + } + st.set_r(reg::ERR_UPPER, (off >> 32) as u32); + st.set_r(reg::ERR_LOWER, off as u32); + } if st.r(reg::INT_ENABLE) & INT_INVALID_ADDR != 0 { self.raise_host_error(st, p); } @@ -882,15 +1070,16 @@ impl Bridge { fn reg_read(&self, off: u32, size: usize) -> Option { // Not implemented: no answer rather than a made-up one. - if off == RESERVED_DATA_HALF || size != 4 { + if (off == RESERVED_DATA_HALF && self.part == Part::Bridge) || size != 4 { return None; } self.irq_locked(|st, p| { - let Some(info) = (if off % 8 == SLOT_DATA { reg_info(off - SLOT_DATA) } else { None }) else { - self.invalid_address(st, p, off, false); + let Some(info) = (if off % 8 == SLOT_DATA { reg_info(self.part, off - SLOT_DATA) } else { None }) else { + self.invalid_address(st, p, off as u64, false); return Some(0xffff_ffff); }; Some(match info.access { + Access::Live if off - SLOT_DATA == reg::INT_MULTIPLE => st.multiple as u64, Access::Live => (st.pin_level as u32 & INT_DEVICE_PINS | st.int_error) as u64, Access::Wo => info.reset as u64, _ => st.r(off - SLOT_DATA) as u64, @@ -899,7 +1088,7 @@ impl Bridge { } fn reg_write(&self, off: u32, size: usize, val: u64) -> bool { - if off == RESERVED_DATA_HALF { + if off == RESERVED_DATA_HALF && self.part == Part::Bridge { return size == 4 && val == 0; } if size != 4 { @@ -908,8 +1097,8 @@ impl Bridge { let val = val as u32; let mut asserted = 0u32; let ok = self.irq_locked(|st, p| { - let Some(info) = (if off % 8 == SLOT_DATA { reg_info(off - SLOT_DATA) } else { None }) else { - self.invalid_address(st, p, off, true); + let Some(info) = (if off % 8 == SLOT_DATA { reg_info(self.part, off - SLOT_DATA) } else { None }) else { + self.invalid_address(st, p, off as u64, true); return true; }; let base = off - SLOT_DATA; @@ -920,27 +1109,44 @@ impl Bridge { Access::Wo => { if base == reg::INT_RESET { // A group's strobe clears that group's errors and no other's. + let mut clear = 0; if val & INT_RESET_PCI_GRP != 0 { - st.int_error &= !INT_PCI_GROUP; + clear |= INT_PCI_GROUP; + } + // Only the XBridge raises the group's bits. + if val & INT_RESET_SSRAM_GRP != 0 && self.part == Part::XBridge { + clear |= INT_SSRAM_GROUP; } if val & INT_RESET_REQ_DSP_GRP != 0 { - st.int_error &= !INT_INVALID_ADDR; + clear |= INT_INVALID_ADDR; } if val & INT_RESET_RESP_BUF_GRP != 0 { - st.int_error &= !INT_RESP_BUF_GROUP; + clear |= INT_RESP_BUF_GROUP; } if val & INT_RESET_MULTI_ERR != 0 { - st.int_error &= !INT_MULTI_ERR; + clear |= INT_MULTI_ERR; + } + st.int_error &= !clear; + st.multiple = if val & INT_RESET_MULTI_ERR != 0 { 0 } else { st.multiple & !clear }; + } else if (reg::FORCE_ALWAYS..=reg::FORCE_PIN_LAST).contains(&base) { + let pin = ((base - reg::FORCE_ALWAYS) / 8 % PINS as u32) as usize; + if st.r(reg::INT_ENABLE) >> pin & 1 == 0 { + crate::dlog!(LogModule::Bridge, "bridge: force of masked pin {pin} ignored"); + } else if base < reg::FORCE_PIN || st.pin_level >> pin & 1 != 0 { + self.send_interrupt(st, p, pin, true); } } } + Access::Rw if base == reg::NIC && self.part == Part::XBridge && self.microlan.lock().is_none() => { + crate::dlog!(LogModule::Bridge, "bridge: XBridge NIC has no MicroLAN fitted; write of {val:#x} ignored"); + } Access::Rw => { let old = st.r(base); let new = (old & !info.writable) | (val & info.writable); st.set_r(base, new); // Control 27:24 going from 1 to 0 asserts RST# (see // CONTROL_RST_PINS). - if base == reg::CONTROL { + if base == reg::CONTROL && self.part == Part::Bridge { asserted = (old & !new & CONTROL_RST_PINS) >> 24; } if base == reg::NIC { @@ -1158,6 +1364,9 @@ impl Bridge { } }); } + if self.part == Part::XBridge { + return Some(self.xbridge_unclaimed(space, base.wrapping_add(woff), size, val)); + } // The cycle's PCI address is where the byte order sends it: a byte stream as // given, word values with the access moved to the device's own byte position. let width = if dev_reg & DEV_64BIT_DEVICE != 0 { 8 } else { 4 }; @@ -1177,9 +1386,9 @@ impl Bridge { /// A processor access to PCI memory address `pci` through the 32-bit /// alias. As through a window, nothing driving device select is a /// master time-out: a read fails and a write posts. - fn alias_access(&self, pci: u32, size: usize, val: Option) -> Option { - let order = Self::alias_order(self.st.lock().r(reg::CONTROL)); - if let Some((f, bar, off)) = self.bus.decode(Space::Mem, pci) { + fn alias_access(&self, space: Space, pci: u32, size: usize, val: Option) -> Option { + let order = Self::alias_order(self.st.lock().r(reg::CONTROL), space); + if let Some((f, bar, off)) = self.bus.decode(space, pci) { return Some(match val { None => order.read(f, bar, off, size), Some(v) => { @@ -1188,6 +1397,9 @@ impl Bridge { } }); } + if self.part == Part::XBridge { + return Some(self.xbridge_unclaimed(space, pci, size, val)); + } let aim = |a: u32, n: usize| if order == ByteOrder::Stream { a } else { a ^ (4 - n.min(4)) as u32 }; self.config_error(0, aim(pci, size) as u64); val.map(|_| 0) @@ -1244,7 +1456,75 @@ impl Bridge { /// Map a PCI address of slot `slot` to the XIO address it reaches and the /// placement of its bytes. `None`: not memory this Bridge reaches. - fn translate(&self, st: &State, slot: usize, addr: u64) -> Option<(u64, Placement, u8)> { + fn translate(&self, st: &mut State, slot: usize, addr: u64) -> Option { + match self.part { + Part::Bridge => Self::translate_bridge(st, slot, addr).map(|(xio, p, w)| Route::Host(xio, p, w)), + Part::XBridge => Self::translate_xbridge(st, slot, addr), + } + } + + /// The XBridge head's translation. Its table maps 512 pages at + /// 0x4000_0000 and the same pages with bytes swapped at 0x6000_0000; an + /// entry's own swap bit swaps too, and an entry that is not valid or not + /// held latches a page fault. PCI address bit 55 swaps a 64-bit direct + /// transfer, the Device(x) direct swap bit a 32-bit one. The 64 KB below + /// the mapped window is the flush target. + fn translate_xbridge(st: &mut State, slot: usize, addr: u64) -> Option { + let dev = st.r(reg::device(slot as u32)); + let wide = st.dma_64bit >> slot & 1 != 0; + let placement = |swap: bool| match (swap, wide) { + (true, _) => Placement::Identity, + (false, false) => Placement::Word, + (false, true) => Placement::Doubleword, + }; + if addr >> 32 != 0 { + let widget = ((addr >> 60) & 0xf) as u8; + if widget == 0 { + log::debug!("bridge: 64-bit DMA names widget 0, which the crossbar cannot answer"); + return None; + } + return Some(Route::Host(addr & ((1 << 48) - 1), placement(addr & XBRIDGE_DMA64_SWAP != 0), widget)); + } + let addr = addr as u32; + if XBRIDGE_FLUSH.contains(&addr) { + return Some(Route::Flush); + } + let window = addr & !XBRIDGE_MAPPED_SWAP; + if (PCI_TRANSLATED..PCI_TRANSLATED + XBRIDGE_MAPPED_SPAN).contains(&window) { + let page_shift = if st.r(reg::CONTROL) & CONTROL_PAGE_16K != 0 { 14 } else { 12 }; + let page_mask = (1u64 << page_shift) - 1; + let index = ((window - PCI_TRANSLATED) >> page_shift) as usize; + if index >= XBRIDGE_ATE_MAPPED || st.ate[index] & ATE_VALID == 0 { + st.int_error |= INT_PAGE_FAULT; + st.multiple |= INT_PAGE_FAULT; + crate::dlog!(LogModule::Bridge, "bridge: XBridge DMA to {addr:#x} faults on translation entry {index}"); + return None; + } + let ate = st.ate[index]; + let widget = ((ate >> ATE_TARGET_WIDGET_SHIFT) & 0xf) as u8; + let swap = ate & XBRIDGE_ATE_SWAP != 0 || addr & XBRIDGE_MAPPED_SWAP != 0; + return Some(Route::Host(((ate & ATE_ADDRESS) & !page_mask) | (addr as u64 & page_mask), placement(swap), widget)); + } + if addr >= PCI_DIRECT { + let (xio, widget) = Self::direct(st, addr); + return (widget != 0).then_some(Route::Host(xio, placement(dev & DEV_SWAP_DIRECT != 0), widget)); + } + crate::dlog!(LogModule::Bridge, "bridge: XBridge DMA to {addr:#x} is in no window"); + None + } + + /// The 31-bit direct window's XIO address and widget for `addr`. + fn direct(st: &State, addr: u32) -> (u64, u8) { + let direct = st.r(reg::DIRECT_MAP); + let mut offset = (((direct & DIRECT_OFFSET) as u64) << 31) | (addr & 0x7fff_ffff) as u64; + // Applies only with a zero offset field; the bit resets set. + if direct & DIRECT_OFFSET == 0 && direct & DIRECT_ADD_512MB != 0 { + offset += DIRECT_512MB; + } + (offset, ((direct & DIRECT_WIDGET) >> DIRECT_WIDGET_SHIFT) as u8) + } + + fn translate_bridge(st: &State, slot: usize, addr: u64) -> Option<(u64, Placement, u8)> { let dev = st.r(reg::device(slot as u32)); let wide = st.dma_64bit >> slot & 1 != 0; let placement_for = |swap_bit: u32| match (dev & swap_bit != 0, wide) { @@ -1274,13 +1554,7 @@ impl Bridge { let page_mask = (1u64 << page_shift) - 1; let placement = placement_for(if addr < PCI_DIRECT { DEV_SWAP_TRANSLATED } else { DEV_SWAP_DIRECT }); if addr >= PCI_DIRECT { - let direct = st.r(reg::DIRECT_MAP); - let mut offset = (((direct & DIRECT_OFFSET) as u64) << 31) | (addr & 0x7fff_ffff) as u64; - // Applies only with a zero offset field; the bit resets set. - if direct & DIRECT_OFFSET == 0 && direct & DIRECT_ADD_512MB != 0 { - offset += DIRECT_512MB; - } - let widget = ((direct & DIRECT_WIDGET) >> DIRECT_WIDGET_SHIFT) as u8; + let (offset, widget) = Self::direct(st, addr); if widget == 0 { return None; } @@ -1318,7 +1592,7 @@ impl Bridge { let mut st = self.st.lock(); st.dma_pci_address = a; st.dma_device = slot as u32; - self.translate(&st, slot, a) + self.translate(&mut st, slot, a) }; // Only the host chip is memory. The Crossbow drops a request to a widget // it has no live link to (1 to 7 are illegal, 15 is this Bridge's @@ -1328,7 +1602,13 @@ impl Bridge { let mut dropped = false; let good = match target { None => false, - Some((xio, p, w)) => match self.up.get(w) { + Some(Route::Flush) => { + if let DmaBuf::Read(b) = &mut buf { + b[done..done + step].fill(0); + } + true + } + Some(Route::Host(xio, p, w)) => match self.up.get(w) { Some(h) if !h.link_up() => { // The Crossbow drops it; a dropped read is timed out. log::debug!("bridge: transfer to widget {w}, whose link is down; dropped"); @@ -1352,7 +1632,7 @@ impl Bridge { if !good { crate::dlog!(LogModule::Bridge, "bridge: slot {slot} {} of {step} bytes at PCI {a:#x} reached no memory ({})", if matches!(buf, DmaBuf::Read(_)) { "read" } else { "write" }, - match target { None => "no translation".to_string(), Some((x, _, w)) => format!("widget {w:#x} address {x:#x}") }); + match target { Some(Route::Host(x, _, w)) => format!("widget {w:#x} address {x:#x}"), _ => "no translation".to_string() }); ok = false; if let DmaBuf::Read(b) = &mut buf { b[done..done + step].fill(0); @@ -1510,6 +1790,145 @@ impl Bridge { } } +/// What the XBridge head's own address space holds outside the Bridge's +/// register and window decode: registers it defers (they read zero and +/// take writes), with no flash, SSRAM or diagnostic engine behind them. +fn xbridge_deferred(off: u64) -> bool { + (0x300..0x600).contains(&off) || (0x3_0000..0x3_0008).contains(&off) || (FLASH_BASE as u64..WIDGET_SMALL_SIZE as u64).contains(&off) +} + +impl Bridge { + /// What the machine connects at the XBridge head's crossbar port: the + /// whole of the head's XIO space, the 16 MB small window and the three + /// aliases of PCI space past it. An undefined address answers zero and + /// latches an invalid-address error; nothing faults. + pub fn head_window(self: &Arc) -> Arc { + assert_eq!(self.part, Part::XBridge, "only an XBridge head has a head window"); + Arc::new(HeadWindow(self.clone())) + } + + /// An XBridge processor access nothing on the bus claims: a read finds + /// ones and a write is dropped, with no error latched. + fn xbridge_unclaimed(&self, space: Space, pci: u32, size: usize, val: Option) -> u64 { + crate::dlog!(LogModule::Bridge, "bridge: XBridge widget {} has no PCI BAR at {space:?} {pci:#x}", self.widget.load(Ordering::Acquire)); + if val.is_some() { 0 } else { u64::MAX >> (64 - 8 * size) } + } + + /// The XBridge head's configuration apertures. A doubleword is two + /// configuration words, the lower address in the upper half. A type 1 + /// cycle for bus 0 reaches the head's own bus. A cycle nothing claims + /// reads as ones and notes a master time-out in the multiple-error + /// register. + fn xbridge_config(&self, off: u32, size: usize, val: Option) -> u64 { + if size == 8 { + let hi = self.xbridge_config(off, 4, val.map(|v| v >> 32)); + return (hi << 32) | self.xbridge_config(off + 4, 4, val.map(|v| v & 0xffff_ffff)); + } + let reg = Self::lane_correct(off & 0xff, size) as usize; + let function = ((off >> 8) & 7) as u8; + let (bus, device) = if off < TYPE1_BASE { + (Some(self.bus.clone()), ((off - CFG0_BASE) >> 12) as u8) + } else { + let sel = self.st.lock().r(reg::PCI_TYPE1_CONFIG); + (self.bus.find_bus((sel >> 16) as u8), ((sel >> 11) & 0x1f) as u8) + }; + let at = DevFn { device, function }; + let mask = u32::MAX >> (32 - size * 8); + if let Some(v) = val { + if let Some(bus) = bus { + bus.config_write(at, reg, size, v as u32 & mask); + } + return 0; + } + match bus.and_then(|b| b.config_read(at, reg, size)) { + Some(v) => v as u64, + None => { + self.st.lock().multiple |= INT_PCI_MASTER_TIMEOUT; + mask as u64 + } + } + } + + /// A processor access to the XBridge head's space at `off`; `val` is + /// `Some` for a write. Registers are 32 bits in the data half of their + /// slot, translation entries are written whole and read a half at a + /// time, and anything else undefined latches an invalid-address error + /// and reads zero. + fn xbridge_access(&self, off: u64, size: usize, val: Option) -> u64 { + let small = off < WIDGET_SMALL_SIZE as u64; + let o = off as u32; + if small && (CFG0_BASE..TYPE1_BASE + TYPE1_SIZE).contains(&o) { + return self.xbridge_config(o, size, val); + } + if small && (DEVICE_WINDOW_BASE..FLASH_BASE).contains(&o) { + if let Some(slot) = window_slot(o) { + return self.window_access(slot as usize, o - device_window_offset(slot), size, val).unwrap_or(0); + } + } + if let Some(i) = XBRIDGE_ALIASES.iter().position(|&(base, len, _)| (base..base + len).contains(&off)) { + let (base, _, space) = XBRIDGE_ALIASES[i]; + return self.alias_access(space, (off - base) as u32, size, val).unwrap_or(0); + } + if small && size == 8 && o % 8 == 0 && (ATE_BASE..XBRIDGE_ATE_LOW_BASE).contains(&o) { + if let Some(v) = val { + crate::dlog!(LogModule::Bridge, "bridge: write translation entry {} {v:#x}", (o - ATE_BASE) / 8); + self.st.lock().ate[((o - ATE_BASE) / 8) as usize] = v & XBRIDGE_ATE_KEPT; + return 0; + } + } + if small && size == 4 && o % 8 == SLOT_DATA { + if val.is_none() && (ATE_BASE..XBRIDGE_ATE_END).contains(&o) { + let st = self.st.lock(); + return if o < XBRIDGE_ATE_LOW_BASE { + st.ate[((o - ATE_BASE) / 8) as usize] >> 32 + } else { + st.ate[((o - XBRIDGE_ATE_LOW_BASE) / 8) as usize] & 0xffff_ffff + }; + } + if o < REGS_DEFINED && reg_info(Part::XBridge, o - SLOT_DATA).is_some() { + crate::dlog!(LogModule::Bridge, "bridge: XBridge {} {} /{size}{}", if val.is_some() { "write" } else { "read" }, + reg_name(o - SLOT_DATA), val.map_or(String::new(), |v| format!(" {v:#x}"))); + return match val { + Some(v) => { + self.reg_write(o, size, v); + 0 + } + None => self.reg_read(o, size).unwrap_or(0), + }; + } + } + if xbridge_deferred(off) { + crate::dlog!(LogModule::Bridge, "bridge: XBridge widget {} deferred access at {off:#x}", self.widget.load(Ordering::Acquire)); + } else { + self.irq_locked(|st, p| self.invalid_address(st, p, off, val.is_some())); + } + 0 + } +} + +/// The XBridge head's XIO space as a processor reaches it: every access +/// completes. +struct HeadWindow(Arc); + +macro_rules! head_access { + ($r:ident, $w:ident, $t:ty, $out:ident, $n:expr) => { + fn $r(&self, a: u64) -> $out { + $out::ok(self.0.xbridge_access(a, $n, None) as $t) + } + fn $w(&self, a: u64, v: $t) -> u32 { + self.0.xbridge_access(a, $n, Some(v as u64)); + BUS_OK + } + }; +} + +impl BusDevice for HeadWindow { + head_access!(read8, write8, u8, BusRead8, 1); + head_access!(read16, write16, u16, BusRead16, 2); + head_access!(read32, write32, u32, BusRead32, 4); + head_access!(read64, write64, u64, BusRead64, 8); +} + impl Bridge { /// The interrupt destination the guest programmed: (target widget, /// address), for the trace. @@ -1573,7 +1992,7 @@ impl LongWindow { fn read(&self, addr: u64, size: usize) -> Option { match (addr & LONG_MASK) as u32 { off @ 0..WIDGET_SMALL_SIZE => self.0.read(off, size), - off @ PIO32_ALIAS.. => self.0.alias_access(off - PIO32_ALIAS, size, None), + off @ PIO32_ALIAS.. => self.0.alias_access(Space::Mem, off - PIO32_ALIAS, size, None), // Approximation: the specification calls 16 MB to 1 GB reserved // (p. 52) and does not say what an access there does; a bus // error here. @@ -1585,7 +2004,7 @@ impl LongWindow { match (addr & LONG_MASK) as u32 { off @ 0..WIDGET_SMALL_SIZE => self.0.write(off, size, val), off @ PIO32_ALIAS.. => { - self.0.alias_access(off - PIO32_ALIAS, size, Some(val)); + self.0.alias_access(Space::Mem, off - PIO32_ALIAS, size, Some(val)); BUS_OK } _ => BUS_ERR, @@ -1727,17 +2146,21 @@ impl Saveable for Bridge { fn save_state(&self) -> toml::Value { let microlan = self.microlan.lock().as_ref().map_or(toml::Value::Boolean(false), |bus| bus.save_state()); let st = self.st.lock(); - StateTable::new() + let mut table = StateTable::new(); + table .u32s("reg", &st.reg) - .u64s("ate", &st.ate) + .u64s("ate", &st.ate[..self.part.ate_entries()]) .u32("int_error", st.int_error) .u8s("pins", &[st.pin_level, st.dma_64bit]) .u64("dma_pci_address", st.dma_pci_address) .u32("dma_device", st.dma_device) .value("bus", self.bus.save_state()) .value("microlan", microlan) - .u32("revision", self.revision.load(Ordering::Acquire)) - .build() + .u32("revision", self.revision.load(Ordering::Acquire)); + if self.part == Part::XBridge { + table.u32("multiple", st.multiple); + } + table.build() } fn load_state(&self, v: &toml::Value) -> Result<(), String> { @@ -1746,8 +2169,9 @@ impl Saveable for Bridge { return Err("the saved Bridge is another revision".into()); } let ate = field_u64_vec(v, "ate")?; - if ate.len() != ATE_ENTRIES { - return Err(format!("ate: {} entries, expected {ATE_ENTRIES}", ate.len())); + let entries = self.part.ate_entries(); + if ate.len() != entries { + return Err(format!("ate: {} entries, expected {entries}", ate.len())); } self.bus.load_state(field(v, "bus")?)?; let saved = field(v, "microlan")?; @@ -1760,7 +2184,10 @@ impl Saveable for Bridge { let [pin_level, dma_64bit] = field_u8s::<2>(v, "pins")?; let mut st = self.st.lock(); st.reg = field_u32s(v, "reg")?; - st.ate.copy_from_slice(&ate); + st.ate[..entries].copy_from_slice(&ate); + if self.part == Part::XBridge { + st.multiple = field_u32(v, "multiple")?; + } st.int_error = field_u32(v, "int_error")?; st.pin_level = pin_level; st.dma_64bit = dma_64bit; diff --git a/src/dev/bridge_tests.rs b/src/dev/bridge_tests.rs index b510998b6..c01a15070 100644 --- a/src/dev/bridge_tests.rs +++ b/src/dev/bridge_tests.rs @@ -1364,6 +1364,40 @@ fn a_host_with_no_packet_address_takes_any_address_and_over_64_vectors() { assert_eq!(r.sink.take(), [(100, true)]); } +/// A host that decodes the packet itself, as Bedrock's two processor +/// interfaces do, gets the whole packet: the widget ID the guest gave the +/// chip, and the destination address with the pin's host field in bits +/// 47:38. A host that refuses it costs the Bridge nothing. +#[test] +fn a_host_that_decodes_packets_sees_their_source_and_address() { + #[derive(Default)] + struct Decoder(Mutex>); + impl InterruptController for Decoder { + fn set_line(&self, _: u32, _: bool) { + unreachable!("a packet host takes packets"); + } + fn packet(&self, source: u8, address: u64, line: u32, level: bool) -> bool { + self.0.lock().push((source, address, line, level)); + false + } + } + let host = XtalkHost { widget: 0xa, int_packet: 0, int_packet_mask: 0, vectors: 256 }; + let decoder = Arc::new(Decoder::default()); + let r = Rig { bridge: Bridge::new(Upstream::single(host, Arc::new(Mem::new()), decoder.clone()), &crate::ip30::PCI0), + sink: Arc::new(Sink::default()), mem: Arc::new(Mem::new()) }; + let c = r.card(2, PCI_INTA); + r.set(reg::CONTROL, (r.get(reg::CONTROL) & !0xf) | 0xd); + r.set(reg::INT_DEST_UPPER, 0xa << 16 | 0x1); + r.set(reg::INT_DEST_LOWER, 0x0180_0090); + r.set(reg::int_host(2), 0x155 << 8 | 0x42); + r.set(reg::INT_MODE, 1 << 2); + r.set(reg::INT_ENABLE, 1 << 2); + c.raise(true); + c.raise(false); + let address = (0x1_0180_0090u64 & !(0x3ff << 38)) | 0x155 << 38; + assert_eq!(*decoder.0.lock(), [(0xd, address, 0x42, true), (0xd, address, 0x42, false)]); +} + /// Behind a Crossbow with two hosts, the widget the guest programs picks the /// host: the direct map moves DMA and the interrupt destination moves /// packets, and a host whose link is down hears neither. @@ -1840,3 +1874,236 @@ fn nvme_ready_follows_enable_through_every_lane_mode_and_width() { let _ = std::fs::remove_file(img); } } + +// ---- the XBridge head ---- +// +// Cases follow tests/qtest/sn1-xbridge-test.c (page-dma, interrupt-packets, +// access-errors) in jamesbraid/qemu-sgi-origami. + +/// Bedrock's port as an XBridge head sees it: widget 10, packets taken at +/// any address. +const BEDROCK_HOST: XtalkHost = XtalkHost { widget: 0xa, int_packet: 0, int_packet_mask: 0, vectors: 256 }; + +/// One XBridge head at crossbar port 15, strapped for a 66 MHz bus, with +/// memory and an interrupt sink behind widget 10. +struct Head { + bridge: Arc, + win: Arc, + sink: Arc, + mem: Arc, +} + +impl Head { + fn new() -> Self { + let sink = Arc::new(Sink::default()); + let mem = Arc::new(Mem::new()); + let bridge = Bridge::with_part(Part::XBridge, Upstream::single(BEDROCK_HOST, mem.clone(), sink.clone()), &crate::ip30::PCI0); + bridge.set_revision(2); + bridge.set_widget(15); + bridge.set_control_strap(1 << 4); + let win = bridge.head_window(); + Head { bridge, win, sink, mem } + } + + fn set(&self, off: u32, val: u32) { + assert_eq!(self.win.write32(u64::from(off + 4), val), BUS_OK); + } + + fn get(&self, off: u32) -> u32 { + self.win.read32(u64::from(off + 4)).data + } + + /// A card in `slot` as [`Rig::card`] fits one: memory BAR at PCI + /// 0x1300_0000, I/O BAR at 0x2000, the slot's window on memory. + fn card(&self, slot: u8, pin: u8) -> Arc { + let c = Card::new(pin); + self.bridge.bus().attach(DevFn { device: slot, function: 0 }, c.clone()); + let cfg = |reg: u32, size: usize, v: u64| { + let off = u64::from(Rig::cfg(slot as u32, reg) ^ (4 - size as u32) & 3); + match size { + 2 => assert_eq!(self.win.write16(off, v as u16), BUS_OK), + _ => assert_eq!(self.win.write32(off, v as u32), BUS_OK), + } + }; + cfg(0x10, 4, 0x2000); + cfg(0x14, 4, 0x1300_0000); + cfg(0x04, 2, u64::from(PCI_COMMAND_IO | PCI_COMMAND_MEMORY)); + self.set(reg::device(slot as u32), DEV_MEMORY_SPACE | 0x130); + c + } +} + +#[test] +fn an_xbridge_head_identifies_itself_and_keeps_its_speed_strap() { + let h = Head::new(); + assert_eq!(h.get(reg::IDENTIFICATION), 0x2d00_2049); + assert_eq!(h.get(reg::CONTROL), 0x0f00_321f); + h.set(reg::CONTROL, 0); + assert_eq!(h.get(reg::CONTROL), 0x10, "the speed strap is not the guest's to write"); + h.bridge.reset(); + assert_eq!(h.get(reg::CONTROL), 0x0f00_321f); + for (off, val, kept) in [ + (reg::DIRECT_MAP, u32::MAX, 0xf3_ffff), + (reg::BUS_TIMEOUT, u32::MAX, 0x1f_03ff), + (reg::INT_DEST_UPPER, u32::MAX, 0xf_ffff), + (reg::TEST_PIN_CONTROL, u32::MAX, 0xff_ffff), + (reg::device(2), u32::MAX, 0x1eef_ffff), + ] { + h.set(off, val); + assert_eq!(h.get(off), kept, "{off:#x}"); + } +} + +/// The head's table holds 1024 entries, written whole and read a half at a +/// time; 512 of them map a window that repeats with bytes swapped at bit +/// 29, and an entry's own swap bit swaps too. A transfer through an entry +/// that is not valid, or past the mapped ones, latches a page fault, which +/// the SSRAM group's reset clears. The 64 KB below the window takes +/// transfers and drops them. +#[test] +fn an_xbridge_head_maps_dma_through_its_larger_table() { + let h = Head::new(); + let c = h.card(2, PCI_INTA); + h.win.write64(u64::from(ATE_BASE) + 800 * 8, 0xffff_1234_5678_9abc); + assert_eq!(h.win.read32(u64::from(ATE_BASE) + 800 * 8 + 4).data, 0x1234, "the entry keeps 48 bits"); + assert_eq!(h.win.read32(u64::from(XBRIDGE_ATE_LOW_BASE) + 800 * 8 + 4).data, 0x5678_9abc); + h.win.write64(u64::from(ATE_BASE), 0x2000 | 0xa << 8 | 1); + h.win.write64(u64::from(ATE_BASE) + 8, 0x3000 | 0xa << 8 | 0x21); + let data = [0, 1, 2, 3, 4, 5, 6, 7]; + for (pci, at, placed) in [ + (0x4000_0010u64, 0x2010, [3, 2, 1, 0, 7, 6, 5, 4]), + (0x6000_0020, 0x2020, data), + (0x4000_1030, 0x3030, data), + ] { + assert_eq!(c.dma().dma_write(pci, &data), PciDmaStatus::Ok, "{pci:#x}"); + assert_eq!(h.mem.peek(at, 8), placed, "{pci:#x}"); + let mut back = [0; 8]; + assert_eq!(c.dma().dma_read(pci, &mut back), PciDmaStatus::Ok); + assert_eq!(back, data); + } + assert_eq!(h.get(reg::INT_STATUS) & INT_PAGE_FAULT, 0); + for pci in [0x4000_2000, 0x4020_0000] { + let mut bytes = [0xff; 4]; + assert_eq!(c.dma().dma_read(pci, &mut bytes), PciDmaStatus::MasterAbort, "{pci:#x}"); + assert_eq!(bytes, [0; 4]); + assert_ne!(h.get(reg::INT_STATUS) & INT_PAGE_FAULT, 0, "{pci:#x}"); + assert_ne!(h.get(reg::INT_MULTIPLE) & INT_PAGE_FAULT, 0); + h.set(reg::INT_RESET, 2); + assert_eq!(h.get(reg::INT_STATUS) & INT_PAGE_FAULT, 0); + } + let mut bytes = [0xff; 4]; + assert_eq!(c.dma().dma_read(0x3fff_8000, &mut bytes), PciDmaStatus::Ok); + assert_eq!(bytes, [0; 4]); + assert_eq!(c.dma().dma_write(0x3fff_8000, &data), PciDmaStatus::Ok); + assert_eq!(c.dma().dma_read(0x4080_0000, &mut bytes), PciDmaStatus::MasterAbort, "past the window"); +} + +/// A 64-bit transfer names its widget in bits 63:60 and asks for its bytes +/// unswapped with bit 55; the 31-bit direct window follows Device(x). +#[test] +fn an_xbridge_head_swaps_64_bit_dma_by_its_address() { + let h = Head::new(); + let c = h.card(2, PCI_INTA); + let data = [0, 1, 2, 3, 4, 5, 6, 7]; + assert_eq!(c.dma().dma_write(0xa << 60 | 1 << 55 | 0x2000, &data), PciDmaStatus::Ok); + assert_eq!(h.mem.peek(0x2000, 8), data); + assert_eq!(c.dma().dma_write(0xa << 60 | 0x2010, &data), PciDmaStatus::Ok); + assert_eq!(h.mem.peek(0x2010, 8), [3, 2, 1, 0, 7, 6, 5, 4]); + h.set(reg::DIRECT_MAP, 0xa << DIRECT_WIDGET_SHIFT); + h.set(reg::device(2), DEV_MEMORY_SPACE | DEV_SWAP_DIRECT | 0x130); + assert_eq!(c.dma().dma_write(0x8000_2020, &data), PciDmaStatus::Ok); + assert_eq!(h.mem.peek(0x2020, 8), data); +} + +/// The head reaches PCI memory through two aliases and I/O through a third, +/// each byte order from Control; nothing claiming an address reads ones, +/// and takes a write, with no error. +#[test] +fn an_xbridge_head_reaches_pci_space_through_its_three_aliases() { + let h = Head::new(); + let _c = h.card(2, PCI_INTA); + for at in [0x4000_0000u64 + 0x1300_0020, 0x8000_0000 + 0x1300_0020, 0x1_0000_0000 + 0x2020] { + assert_eq!(h.win.read32(at).data, 0x1234_5678, "{at:#x}"); + } + h.set(reg::CONTROL, CONTROL_IO_SWAP); + assert_eq!(h.win.read32(0x1_0000_2020).data, 0x7856_3412, "the I/O alias swaps on its own bit"); + assert_eq!(h.win.read32(0x4000_0000 + 0x1300_0020).data, 0x1234_5678); + h.set(reg::INT_ENABLE, INT_PCI_MASTER_TIMEOUT); + assert_eq!(h.win.read32(0x4800_0000).data, u32::MAX); + assert_eq!(h.win.write32(0x4800_0000, 0), BUS_OK); + assert_eq!(h.win.read32(u64::from(device_window_offset(3))).data, u32::MAX); + assert_eq!(h.get(reg::INT_STATUS) & !INT_DEVICE_PINS, 0); +} + +/// An empty slot reads as ones and notes the master time-out in the +/// multiple-error register; a doubleword is two words, the lower address +/// in the upper half; a type 1 cycle for bus 0 reaches the head's own bus. +#[test] +fn an_xbridge_head_answers_configuration_cycles_as_its_bus_does() { + let h = Head::new(); + let _c = h.card(2, PCI_INTA); + assert_eq!(h.win.read32(u64::from(Rig::cfg(5, 0))).data, u32::MAX); + assert_ne!(h.get(reg::INT_MULTIPLE) & INT_PCI_MASTER_TIMEOUT, 0); + assert_eq!(h.get(reg::INT_STATUS) & INT_PCI_MASTER_TIMEOUT, 0); + let id = u64::from(CARD_DEVICE << 16 | CARD_VENDOR); + let command = u64::from(h.win.read32(u64::from(Rig::cfg(2, 4))).data); + assert_eq!(h.win.read64(u64::from(Rig::cfg(2, 0))).data, id << 32 | command); + h.set(reg::PCI_TYPE1_CONFIG, 2 << 11); + assert_eq!(h.win.read32(u64::from(TYPE1_BASE)).data as u64, id); +} + +/// An undefined address reads zero and latches an invalid-address error, +/// capturing the first address and interrupting for it once; the deferred +/// registers read zero with no error. +#[test] +fn an_xbridge_head_captures_its_first_invalid_address() { + let h = Head::new(); + h.set(reg::INT_DEST_UPPER, 0xa << 16); + h.set(reg::INT_DEST_LOWER, 0x80); + h.set(reg::HOST_ERR_FIELD, 73); + h.set(reg::INT_ENABLE, INT_INVALID_ADDR); + assert_eq!(h.win.read32(0x300).data, 0, "deferred"); + assert!(h.sink.take().is_empty()); + assert_eq!(h.win.read32(0x100_0004).data, 0); + assert_eq!(h.win.read32(0x200_0004).data, 0); + assert_eq!(h.get(reg::ERR_LOWER), 0x100_0004); + assert_eq!(h.sink.take(), [(73, true)]); + assert_ne!(h.get(reg::INT_MULTIPLE) & INT_INVALID_ADDR, 0); + h.set(reg::INT_RESET, 8); + assert_eq!(h.get(reg::INT_STATUS) & INT_INVALID_ADDR, 0); +} + +/// A pin's packet goes out even to an unprogrammed host field, and the +/// force registers send a pin's packet: always, or only while it is high. +#[test] +fn an_xbridge_head_forces_its_interrupts() { + let h = Head::new(); + let c = h.card(2, PCI_INTA); + h.set(reg::INT_DEST_UPPER, 0xa << 16); + h.set(reg::INT_ENABLE, 1 << 2); + c.raise(true); + assert_eq!(h.sink.take(), [(0, true)]); + h.set(reg::int_host(2), 0x42); + h.set(reg::FORCE_ALWAYS + 2 * 8, 1); + h.set(reg::FORCE_PIN + 2 * 8, 1); + assert_eq!(h.sink.take(), [(0x42, true), (0x42, true)]); + c.raise(false); + h.set(reg::FORCE_PIN + 2 * 8, 1); + h.set(reg::FORCE_ALWAYS + 3 * 8, 1); + assert!(h.sink.take().is_empty(), "pin 2 is low and pin 3 is masked"); + assert_eq!(h.get(reg::FORCE_ALWAYS + 2 * 8), 0); +} + +/// A snapshot keeps all 1024 entries and the multiple-error register. +#[test] +fn an_xbridge_head_snapshot_keeps_its_table_and_errors() { + let src = Head::new(); + src.win.write64(u64::from(ATE_BASE) + 1000 * 8, 0x5000 | 0xa << 8 | 1); + let _ = src.win.read32(u64::from(Rig::cfg(5, 0))); + let v = src.bridge.save_state(); + let dst = Head::new(); + dst.bridge.load_state(&v).expect("load_state"); + assert_eq!(v, dst.bridge.save_state()); + assert_eq!(dst.win.read32(u64::from(XBRIDGE_ATE_LOW_BASE) + 1000 * 8 + 4).data, 0x5a01); + assert!(Rig::new().bridge.load_state(&v).is_err(), "a Bridge holds 128 entries"); +} diff --git a/src/dev/deadline_timers.rs b/src/dev/deadline_timers.rs new file mode 100644 index 000000000..6331fd08f --- /dev/null +++ b/src/dev/deadline_timers.rs @@ -0,0 +1,200 @@ +//! One-shot timers for the host-time deadlines a device keeps. +//! +//! A device evaluates its deadlines lazily, on every access, and arms one +//! timer per deadline so the work still happens when the guest is idle. The +//! IOC3 and IOC4 do this for their serial ring timers and transmit delays, +//! and the IOC3 for its interrupt output. This holds the part they share: +//! which timer serves which deadline, replacing one whose deadline moved, +//! and taking them all away on stop. +//! +//! QEMU has no counterpart: its timers belong to the virtual clock and are +//! armed where they are set. + +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, OnceLock, Weak}; +use std::time::{Duration, Instant}; + +use parking_lot::Mutex; + +use crate::hptimer::{TimerId, TimerManager, TimerReturn}; + +/// The timers of a device with `N` kinds of deadline. +pub struct DeadlineTimers { + manager: OnceLock>, + /// The timer pending for each kind, and the deadline it serves. + armed: Mutex<[Option<(TimerId, Instant)>; N]>, +} + +impl DeadlineTimers { + pub fn new() -> Self { + Self { manager: OnceLock::new(), armed: Mutex::new([None; N]) } + } + + /// The timer manager to arm on. Set it before the device starts. + pub fn set_manager(&self, tm: Arc) { + let _ = self.manager.set(tm); + } + + /// Make the pending timers match `want`: one one-shot per deadline. A + /// timer whose deadline moved is replaced. With no manager set, or the + /// device not `running`, deadlines are only evaluated lazily, by the next + /// access. `floor` is the least delay for a kind, for a deadline that can + /// recur faster than a timer should fire; `fire` runs on the timer thread + /// with the device and the kind and deadline it was armed for. + pub fn arm( + &self, + want: &[Option; N], + running: &AtomicBool, + device: &Weak, + floor: fn(usize) -> Duration, + fire: fn(&Weak, usize, Instant) -> TimerReturn, + ) { + let Some(tm) = self.manager.get() else { return }; + let mut armed = self.armed.lock(); + // Checked under the lock `cancel_all` clears the slots with: a pass + // that sees `running` set has its timers removed by that stop. + if !running.load(Ordering::SeqCst) { + return; + } + for (kind, w) in want.iter().enumerate() { + if armed[kind].map(|(_, d)| d) == *w { + continue; + } + if let Some((id, _)) = armed[kind].take() { + tm.remove(id); + } + let Some(due) = *w else { continue }; + let delay = due.saturating_duration_since(Instant::now()).max(floor(kind)); + let id = tm.add_one_shot(delay, device.clone(), move |d| fire(d, kind, due)); + armed[kind] = Some((id, due)); + } + } + + /// A timer for `kind` and `due` has fired: forget it, unless a newer + /// deadline has replaced it. + pub fn fired(&self, kind: usize, due: Instant) { + let mut armed = self.armed.lock(); + if armed[kind].map(|(_, d)| d) == Some(due) { + armed[kind] = None; + } + } + + /// Whether a timer is pending for `kind`. + #[cfg(test)] + pub fn is_armed(&self, kind: usize) -> bool { + self.armed.lock()[kind].is_some() + } + + /// Whether no timer is pending. + #[cfg(test)] + pub fn none_armed(&self) -> bool { + self.armed.lock().iter().all(|a| a.is_none()) + } + + /// Remove every pending timer. + pub fn cancel_all(&self) { + let mut armed = self.armed.lock(); + for slot in armed.iter_mut() { + if let (Some((id, _)), Some(tm)) = (slot.take(), self.manager.get()) { + tm.remove(id); + } + } + } +} + +impl Default for DeadlineTimers { + fn default() -> Self { + Self::new() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::atomic::AtomicUsize; + + struct Device { + timers: DeadlineTimers<2>, + running: AtomicBool, + fired: [AtomicUsize; 2], + } + + fn fire(me: &Weak, kind: usize, due: Instant) -> TimerReturn { + if let Some(d) = me.upgrade() { + d.timers.fired(kind, due); + d.fired[kind].fetch_add(1, Ordering::SeqCst); + } + TimerReturn::Delete + } + + fn device() -> Arc { + let d = Arc::new(Device { + timers: DeadlineTimers::new(), + running: AtomicBool::new(true), + fired: Default::default(), + }); + d.timers.set_manager(Arc::new(TimerManager::new())); + d + } + + fn wait_for(f: impl Fn() -> bool) -> bool { + for _ in 0..500 { + if f() { + return true; + } + std::thread::sleep(Duration::from_millis(2)); + } + false + } + + #[test] + fn a_deadline_fires_once_and_its_slot_empties() { + let d = device(); + let due = Instant::now() + Duration::from_millis(5); + d.timers.arm(&[Some(due), None], &d.running, &Arc::downgrade(&d), |_| Duration::ZERO, fire); + assert!(d.timers.is_armed(0)); + assert!(!d.timers.is_armed(1)); + assert!(wait_for(|| d.fired[0].load(Ordering::SeqCst) == 1)); + assert!(d.timers.none_armed()); + assert_eq!(d.fired[1].load(Ordering::SeqCst), 0); + } + + #[test] + fn a_moved_deadline_replaces_its_timer() { + let d = device(); + let me = Arc::downgrade(&d); + let far = Instant::now() + Duration::from_secs(3600); + d.timers.arm(&[Some(far), None], &d.running, &me, |_| Duration::ZERO, fire); + let near = Instant::now() + Duration::from_millis(5); + d.timers.arm(&[Some(near), None], &d.running, &me, |_| Duration::ZERO, fire); + assert!(wait_for(|| d.fired[0].load(Ordering::SeqCst) == 1)); + assert_eq!(d.fired[0].load(Ordering::SeqCst), 1, "the far timer was removed, not left to fire"); + } + + #[test] + fn a_stopped_device_arms_nothing_and_cancel_clears_what_was_armed() { + let d = device(); + let me = Arc::downgrade(&d); + let far = Some(Instant::now() + Duration::from_secs(3600)); + d.running.store(false, Ordering::SeqCst); + d.timers.arm(&[far, far], &d.running, &me, |_| Duration::ZERO, fire); + assert!(d.timers.none_armed()); + d.running.store(true, Ordering::SeqCst); + d.timers.arm(&[far, far], &d.running, &me, |_| Duration::ZERO, fire); + assert!(d.timers.is_armed(0) && d.timers.is_armed(1)); + d.timers.cancel_all(); + assert!(d.timers.none_armed()); + } + + #[test] + fn the_floor_delays_a_kind_that_recurs_quickly() { + let d = device(); + let now = Instant::now(); + d.timers.arm(&[Some(now), Some(now)], &d.running, &Arc::downgrade(&d), |k| { + if k == 0 { Duration::from_millis(40) } else { Duration::ZERO } + }, fire); + assert!(wait_for(|| d.fired[1].load(Ordering::SeqCst) == 1)); + assert_eq!(d.fired[0].load(Ordering::SeqCst), 0, "kind 0 is held back by its floor"); + assert!(wait_for(|| d.fired[0].load(Ordering::SeqCst) == 1)); + } +} diff --git a/src/dev/ds1744.rs b/src/dev/ds1744.rs new file mode 100644 index 000000000..e5e1c30bd --- /dev/null +++ b/src/dev/ds1744.rs @@ -0,0 +1,450 @@ +//! DS1744-compatible timekeeping SRAM: 32 KiB of battery-backed bytes whose +//! last eight are a BCD calendar. +//! +//! The Tezro's IO9 maps one in the IOC4's BAR (byte-bus +//! select 0 of the IOC4). The part reads like the Fuel's M48T35 [`Timekeeper`] at a +//! glance, and has the same geometry and clock offsets, but differs where +//! software looks: the control register carries the century beside the +//! write and read bits, the day register holds a weekday that advances +//! with the calendar, an oscillator-stop bit rides in the seconds, a commit +//! with an impossible date is refused, and a test-mode bit makes the +//! seconds' low bit run at 1024 Hz. Those are a different calendar rather +//! than a flag on the M48T35's, so this is its own model. +//! +//! The guest's write bit stages a calendar image and the clearing of it +//! commits it; the read bit freezes the visible calendar while the real one +//! runs on. +//! +//! # What is approximated here +//! +//! * The fitted IO9 part and its physical address wiring are unknown. The +//! 32 KiB geometry is the profile's choice. +//! * With a fresh-year base of 1968 the guest's year digits count from that +//! year, as its driver expects: the host date is shifted into 2000 to +//! 2099 and the weekday comes from the unshifted date, so the weekday +//! follows the host's real calendar while the digits follow the shifted +//! one. +//! * Host time and a backing file stand in for the oscillator and battery. +//! A restart restores a completed calendar and lets host time pass; an +//! unfinished write or read transaction is not restored. +//! +//! Ported from hw/rtc/ds1742.c in jamesbraid/qemu-sgi-origami. + +use std::path::{Path, PathBuf}; +use std::sync::Arc; +use std::time::{Instant, SystemTime, UNIX_EPOCH}; + +use parking_lot::Mutex; + +use crate::traits::{BusDevice, BusRead8, BUS_ERR, BUS_OK}; + +pub const SIZE: usize = 0x8000; +/// The control register, first of the eight calendar bytes. +pub const CONTROL: usize = 0x7ff8; +const WRITE: u8 = 0x80; +const READ: u8 = 0x40; +/// Seconds bit 7: the oscillator is stopped. +const OSCILLATOR_STOP: u8 = 0x80; +/// Day bit 6: test mode; bit 7: the battery flag, read-only and always set. +const TEST_MODE: u8 = 0x40; +const BATTERY_FLAG: u8 = 0x80; +const CLOCK_REF_BYTES: usize = 16; +const CLOCK_REF_MAGIC: &[u8; 8] = b"DS1742R1"; +/// Test mode flips the seconds' low bit every 976562 ns. +const TEST_BIT_NS: i64 = 976_562; + +/// How the guest's year digits relate to the host's calendar. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum YearBase { + /// The host's own year. + Gregorian, + /// The year digits count from this year: the host date is shifted into + /// 2000 to 2099. Only 1968 is accepted. + From(u32), +} + +fn bcd(v: i64) -> u8 { + ((v / 10) * 16 + v % 10) as u8 +} + +fn from_bcd(v: u8) -> i64 { + i64::from((v >> 4) * 10 + (v & 15)) +} + +fn bcd_in(v: u8, lo: i64, hi: i64) -> bool { + v & 15 < 10 && v >> 4 < 10 && (lo..=hi).contains(&from_bcd(v)) +} + +fn is_leap(year: i64) -> bool { + year % 4 == 0 && (year % 100 != 0 || year % 400 == 0) +} + +fn month_days(year: i64, month: i64) -> i64 { + match month { + 2 => 28 + i64::from(is_leap(year)), + 4 | 6 | 9 | 11 => 30, + _ => 31, + } +} + +/// Days since 1970-01-01 of a Gregorian date. +fn days_from_civil(year: i64, month: i64, day: i64) -> i64 { + let y = if month <= 2 { year - 1 } else { year }; + let era = y.div_euclid(400); + let yoe = y - era * 400; + let doy = (153 * ((month + 9) % 12) + 2) / 5 + day - 1; + let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; + era * 146_097 + doe - 719_468 +} + +/// The Gregorian date of a day count since 1970-01-01. +fn civil_from_days(days: i64) -> (i64, i64, i64) { + let z = days + 719_468; + let era = z.div_euclid(146_097); + let doe = z - era * 146_097; + let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365; + let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); + let mp = (5 * doy + 2) / 153; + let day = doy - (153 * mp + 2) / 5 + 1; + let month = if mp < 10 { mp + 3 } else { mp - 9 }; + let year = yoe + era * 400 + i64::from(month <= 2); + (year, month, day) +} + +/// Day number of `seconds`, rounding down. +fn day_index(seconds: i64) -> i64 { + seconds.div_euclid(86_400) +} + +/// The 1-based weekday (Sunday is 1) of a count of days since 1970-01-01, +/// a Thursday. +fn weekday_of_day(days: i64) -> u8 { + (days + 4).rem_euclid(7) as u8 + 1 +} + +struct State { + ram: Vec, + control: u8, + /// Test-mode flag and weekday as last written (FT and bits 2:0). + weekday: u8, + staged: [u8; 8], + frozen: [u8; 8], + frozen_valid: bool, + /// Unix seconds of the calendar when it was last set. + epoch: i64, + /// Clock reading (ns) when it was last set. + reference_ns: i64, + stopped: bool, + file: Option, + /// A backing write failed; the next equal write retries it. + storage_failed: bool, +} + +/// The nanosecond clock the calendar runs on. +type Clock = Arc i64 + Send + Sync>; + +pub struct Ds1744 { + st: Mutex, + clock: Clock, +} + +impl Ds1744 { + /// A part whose calendar starts at the host's date and time. + pub fn new(base: YearBase) -> Result { + let host = SystemTime::now().duration_since(UNIX_EPOCH).map_err(|e| e.to_string())?.as_secs() as i64; + let born = Instant::now(); + Ds1744::with_clock(host, base, Arc::new(move || born.elapsed().as_nanos() as i64)) + } + + /// A part seeded with `host_unix_secs` and running on `clock`, a count of + /// nanoseconds. Tests supply a clock they step. + pub fn with_clock(host_unix_secs: i64, base: YearBase, clock: Clock) -> Result { + let mut epoch = host_unix_secs; + let (year, month, day) = civil_from_days(day_index(host_unix_secs)); + let mut control = bcd(year / 100); + let weekday = weekday_of_day(day_index(host_unix_secs)); + match base { + YearBase::Gregorian => {} + YearBase::From(1968) => { + let base = 1968; + if !(base..=base + 99).contains(&year) { + return Err(format!("ds1744: fresh-year-base={base} cannot seed host year {year}")); + } + let shifted = 2000 + (year - base) % 100; + // A leap day in a shifted year without one is the next day. + epoch = days_from_civil(shifted, month, day) * 86_400 + host_unix_secs.rem_euclid(86_400); + control = bcd(shifted / 100); + } + YearBase::From(other) => return Err(format!("ds1744: fresh-year-base must be 0 or 1968, not {other}")), + } + let now = clock(); + Ok(Ds1744 { + st: Mutex::new(State { + ram: vec![0; CONTROL], + control, + weekday, + staged: [0; 8], + frozen: [0; 8], + frozen_valid: false, + epoch, + reference_ns: now, + stopped: false, + file: None, + storage_failed: false, + }), + clock, + }) + } + + fn now_ns(&self) -> i64 { + (self.clock)() + } + + fn seconds(&self, st: &State) -> i64 { + st.epoch + if st.stopped { 0 } else { (self.now_ns() - st.reference_ns) / 1_000_000_000 } + } + + /// The calendar bytes as the part would show them now. + fn image(&self, st: &State) -> [u8; 8] { + let now = self.seconds(st); + let (year, month, day) = civil_from_days(day_index(now)); + let sec = now.rem_euclid(86_400); + let weekday = ((i64::from(st.weekday & 7) - 1 + day_index(now) - day_index(st.epoch)).rem_euclid(7)) as u8; + [ + (st.control & 0xc0) | bcd(year / 100), + bcd(sec % 60) | if st.stopped { OSCILLATOR_STOP } else { 0 }, + bcd(sec / 60 % 60), + bcd(sec / 3600), + BATTERY_FLAG | (st.weekday & TEST_MODE) | (weekday + 1), + bcd(day), + bcd(month), + bcd(year % 100), + ] + } + + /// Make the staged image the calendar, if it is a date. A refusal leaves + /// the calendar as it was. + fn commit(&self, st: &mut State, century: u8) -> bool { + let s = st.staged; + let valid = bcd_in(century, 0, 39) + && bcd_in(s[1] & 0x7f, 0, 59) + && bcd_in(s[2], 0, 59) + && bcd_in(s[3], 0, 23) + && bcd_in(s[4] & 7, 1, 7) + && bcd_in(s[5], 1, 31) + && bcd_in(s[6], 1, 12) + && bcd_in(s[7], 0, 99); + if !valid { + crate::dlog!(crate::devlog::LogModule::Rtc, "DS1744: invalid calendar write"); + return false; + } + let year = from_bcd(century) * 100 + from_bcd(s[7]); + let (month, day) = (from_bcd(s[6]), from_bcd(s[5])); + if day > month_days(year, month) { + crate::dlog!(crate::devlog::LogModule::Rtc, "DS1744: unsupported calendar date {year:04}-{month:02}-{day:02}"); + return false; + } + st.epoch = days_from_civil(year, month, day) * 86_400 + + from_bcd(s[3]) * 3600 + + from_bcd(s[2]) * 60 + + from_bcd(s[1] & 0x7f); + st.reference_ns = self.now_ns(); + st.weekday = s[4] & (TEST_MODE | 7); + st.stopped = s[1] & OSCILLATOR_STOP != 0; + true + } + + // ----- backing file ----- + + /// Back the part with a 32 KiB file and a 16-byte clock reference beside + /// it (`.clock`). A missing file starts blank and is created; a file + /// of another size is refused. A saved calendar is restored and lets the + /// host time since it was saved pass; a blank one keeps the seed. + pub fn set_nvram_file(&self, path: &Path) -> Result<(), String> { + let clock_path = PathBuf::from(format!("{}.clock", path.display())); + let image = match std::fs::read(path) { + Ok(v) if v.len() == SIZE => Some(v), + Ok(_) => return Err("DS1744: backing image must contain exactly 32768 bytes".into()), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, + Err(e) => return Err(e.to_string()), + }; + let reference = match std::fs::read(&clock_path) { + Ok(v) if v.len() >= CLOCK_REF_BYTES => Some(v), + Ok(_) => return Err("DS1744: clock reference is shorter than 16 bytes".into()), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, + Err(e) => return Err(e.to_string()), + }; + let mut st = self.st.lock(); + if let Some(image) = image { + st.ram.copy_from_slice(&image[..CONTROL]); + let cal: [u8; 8] = image[CONTROL..].try_into().expect("eight calendar bytes"); + if [cal[1], cal[2], cal[3], cal[5], cal[6], cal[7]].iter().any(|&b| b != 0) { + self.restore_clock(&mut st, cal, reference.as_deref()); + } + } + st.file = Some(path.to_path_buf()); + self.save_clock(&mut st); + Ok(()) + } + + fn restore_clock(&self, st: &mut State, cal: [u8; 8], reference: Option<&[u8]>) { + st.staged = cal; + if !self.commit(st, cal[0] & 0x3f) { + crate::dlog!(crate::devlog::LogModule::Rtc, "DS1744: invalid saved calendar; using the seed"); + return; + } + st.control = cal[0] & 0x3f; + let mut elapsed = 0; + if !st.stopped { + if let Some(r) = reference.filter(|r| &r[..8] == CLOCK_REF_MAGIC) { + let saved = i64::from_le_bytes(r[8..16].try_into().expect("eight bytes")); + elapsed = (unix_now() - saved).max(0); + } + } + if elapsed != 0 { + let days = day_index(st.epoch + elapsed) - day_index(st.epoch); + st.weekday = (st.weekday & TEST_MODE) | (((i64::from(st.weekday & 7) - 1 + days % 7) % 7) as u8 + 1); + st.epoch += elapsed; + } + } + + /// Write the whole image and the clock reference. Errors are reported + /// and remembered so an equal write retries them. + fn save_clock(&self, st: &mut State) { + let Some(path) = st.file.clone() else { return }; + let mut image = st.ram.clone(); + image.extend_from_slice(&self.image(st)); + let mut reference = CLOCK_REF_MAGIC.to_vec(); + reference.extend_from_slice(&unix_now().to_le_bytes()); + let clock_path = format!("{}.clock", path.display()); + let result = std::fs::write(&path, &image).and_then(|()| std::fs::write(clock_path, &reference)); + st.storage_failed = result.is_err(); + if let Err(e) = result { + log::error!("DS1744: could not save {}: {e}", path.display()); + } + } + + /// Save the calendar now, as a stop or shutdown does. + pub fn flush(&self) { + let mut st = self.st.lock(); + self.save_clock(&mut st); + } + + fn read(&self, a: usize) -> u8 { + let st = self.st.lock(); + if a < CONTROL { + return st.ram[a]; + } + let image = self.image(&st); + if a == CONTROL { + return if st.control & WRITE != 0 { + st.control + } else if st.frozen_valid { + (st.control & 0xc0) | (st.frozen[0] & 0x3f) + } else { + image[0] + }; + } + let i = a - CONTROL; + let mut value = if st.control & WRITE != 0 { + st.staged[i] + } else if st.frozen_valid { + st.frozen[i] + } else { + image[i] + }; + if i == 4 { + value |= BATTERY_FLAG; + } + if i == 1 && st.weekday & TEST_MODE != 0 && !st.stopped { + value = (value & !1) | ((self.now_ns() / TEST_BIT_NS) & 1) as u8; + } + value + } + + fn write(&self, a: usize, value: u8) { + let mut st = self.st.lock(); + if a < CONTROL { + if st.ram[a] == value && !st.storage_failed { + return; + } + st.ram[a] = value; + if let Some(path) = &st.file { + use std::io::{Seek, SeekFrom, Write}; + let result = std::fs::OpenOptions::new().write(true).open(path).and_then(|mut f| { + f.seek(SeekFrom::Start(a as u64))?; + f.write_all(&[value]) + }); + st.storage_failed = result.is_err(); + if let Err(e) = result { + log::error!("DS1744: could not save SRAM byte at {a:#x}: {e}"); + } + } + return; + } + if a != CONTROL { + if st.control & WRITE != 0 { + st.staged[a - CONTROL] = value; + } else { + crate::dlog!(crate::devlog::LogModule::Rtc, "DS1744: calendar write with W clear at {a:#x}"); + } + return; + } + if st.control & (WRITE | READ) == 0 { + st.control = self.image(&st)[0]; + } + let old = st.control; + if value & READ != 0 && old & READ == 0 { + st.frozen = self.image(&st); + st.frozen_valid = true; + } else if value & READ == 0 { + st.frozen_valid = false; + } + if value & WRITE != 0 && old & WRITE == 0 { + st.staged = self.image(&st); + } + if value & WRITE == 0 && old & WRITE != 0 { + let valid = self.commit(&mut st, value & 0x3f); + st.control = (value & 0xc0) | if valid { value & 0x3f } else { old & 0x3f }; + if valid { + self.save_clock(&mut st); + } + return; + } + // The control rules preserve the century while W or R is set. + st.control = (value & 0xc0) | (old & 0x3f); + } +} + +fn unix_now() -> i64 { + SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |d| d.as_secs() as i64) +} + +impl BusDevice for Ds1744 { + fn read8(&self, addr: u64) -> BusRead8 { + match usize::try_from(addr) { + Ok(a) if a < SIZE => BusRead8 { status: BUS_OK, data: self.read(a) }, + _ => BusRead8::err(), + } + } + + fn write8(&self, addr: u64, val: u8) -> u32 { + match usize::try_from(addr) { + Ok(a) if a < SIZE => { + self.write(a, val); + BUS_OK + } + _ => BUS_ERR, + } + } +} + +impl Drop for Ds1744 { + fn drop(&mut self) { + self.flush(); + } +} + +#[cfg(test)] +#[path = "ds1744_tests.rs"] +mod tests; diff --git a/src/dev/ds1744_tests.rs b/src/dev/ds1744_tests.rs new file mode 100644 index 000000000..4ca13cca8 --- /dev/null +++ b/src/dev/ds1744_tests.rs @@ -0,0 +1,305 @@ +//! DS1744 register-contract tests, through the byte bus with a clock the +//! test steps. +//! +//! Ported from tests/qtest/ds1742-test.c in jamesbraid/qemu-sgi-origami. + +use super::*; +use std::sync::atomic::{AtomicI64, Ordering}; + +const SECOND: i64 = 1_000_000_000; + +fn unix(y: i64, mo: i64, d: i64, h: i64, mi: i64, s: i64) -> i64 { + days_from_civil(y, mo, d) * 86_400 + h * 3600 + mi * 60 + s +} + +struct Rig { + d: Ds1744, + ns: Arc, +} + +impl Rig { + fn at(host: i64, base: YearBase) -> Rig { + let ns = Arc::new(AtomicI64::new(0)); + let n = ns.clone(); + let d = Ds1744::with_clock(host, base, Arc::new(move || n.load(Ordering::SeqCst))).unwrap(); + Rig { d, ns } + } + + /// The qtest's default: January 1st 2024, on the host's calendar. + fn new() -> Rig { + Rig::at(unix(2024, 1, 1, 0, 0, 0), YearBase::Gregorian) + } + + fn step(&self, ns: i64) { + self.ns.fetch_add(ns, Ordering::SeqCst); + } + + fn rd(&self, a: usize) -> u8 { + let r = self.d.read8(a as u64); + assert_eq!(r.status, BUS_OK); + r.data + } + + fn wr(&self, a: usize, v: u8) { + assert_eq!(self.d.write8(a as u64, v), BUS_OK); + } + + /// Set the calendar through the write bit, with a weekday. + #[allow(clippy::too_many_arguments)] + fn set_time_weekday(&self, sec: u8, min: u8, hour: u8, date: u8, month: u8, year: u8, century: u8, weekday: u8) { + self.wr(CONTROL, 0x80); + for (i, v) in [sec, min, hour, weekday, date, month, year].into_iter().enumerate() { + self.wr(CONTROL + 1 + i, v); + } + self.wr(CONTROL, century); + } + + #[allow(clippy::too_many_arguments)] + fn set_time(&self, sec: u8, min: u8, hour: u8, date: u8, month: u8, year: u8, century: u8) { + self.set_time_weekday(sec, min, hour, date, month, year, century, 0x02); + } + + /// The guest-visible calendar: control, seconds, minutes, hours, weekday, + /// date, month, year. + fn calendar(&self) -> [u8; 8] { + std::array::from_fn(|i| self.rd(CONTROL + i)) + } +} + +/// Ports test_fresh_seed: with the 1968 base the guest's year digits count +/// from 1968 and the weekday follows the host's calendar, across a day. +#[test] +fn a_fresh_seed_counts_years_from_1968() { + let r = Rig::at(unix(2024, 2, 28, 23, 59, 59), YearBase::From(1968)); + assert_eq!(r.rd(CONTROL), 0x20); + assert_eq!((r.rd(CONTROL + 7), r.rd(CONTROL + 6), r.rd(CONTROL + 5)), (0x56, 0x02, 0x28)); + assert_eq!(r.rd(CONTROL + 4) & 7, 4); + r.step(2 * SECOND); + assert_eq!(r.rd(CONTROL) & 0x3f, 0x20); + assert_eq!((r.rd(CONTROL + 7), r.rd(CONTROL + 6), r.rd(CONTROL + 5)), (0x56, 0x02, 0x29)); + assert_eq!(r.rd(CONTROL + 1), 0x01); + assert_eq!(r.rd(CONTROL + 4) & 7, 5); + r.step(86_400 * SECOND); + assert_eq!((r.rd(CONTROL + 7), r.rd(CONTROL + 6), r.rd(CONTROL + 5)), (0x56, 0x03, 0x01)); + assert_eq!(r.rd(CONTROL + 4) & 7, 6); +} + +/// Ports test_fresh_seed_default: with no base the guest sees the host's year. +#[test] +fn without_a_base_the_guest_sees_the_host_year() { + let r = Rig::at(unix(2024, 2, 28, 23, 59, 59), YearBase::Gregorian); + assert_eq!(r.rd(CONTROL), 0x20); + assert_eq!((r.rd(CONTROL + 7), r.rd(CONTROL + 6), r.rd(CONTROL + 5)), (0x24, 0x02, 0x28)); + assert_eq!(r.rd(CONTROL + 4) & 7, 4); +} + +/// Ports test_fresh_seed_year_boundary: 2067 maps to 2099, the century +/// register steps to 21 and the shifted 2100 is not a leap year. +#[test] +fn the_shifted_century_rolls_and_2100_is_not_a_leap_year() { + let r = Rig::at(unix(2067, 12, 31, 23, 59, 59), YearBase::From(1968)); + assert_eq!(r.rd(CONTROL), 0x20); + assert_eq!((r.rd(CONTROL + 7), r.rd(CONTROL + 6), r.rd(CONTROL + 5)), (0x99, 0x12, 0x31)); + assert_eq!(r.rd(CONTROL + 4) & 7, 7); + r.step(SECOND); + assert_eq!(r.rd(CONTROL), 0x21); + assert_eq!((r.rd(CONTROL + 7), r.rd(CONTROL + 6), r.rd(CONTROL + 5)), (0x00, 0x01, 0x01)); + assert_eq!(r.rd(CONTROL + 4) & 7, 1); + r.step((58 * 86_400 + 86_399) * SECOND); + assert_eq!((r.rd(CONTROL), r.rd(CONTROL + 7), r.rd(CONTROL + 6), r.rd(CONTROL + 5)), (0x21, 0x00, 0x02, 0x28)); + assert_eq!(r.rd(CONTROL + 4) & 7, 3); + r.step(SECOND); + assert_eq!((r.rd(CONTROL + 6), r.rd(CONTROL + 5)), (0x03, 0x01)); + assert_eq!(r.rd(CONTROL + 4) & 7, 4); +} + +/// Ports test_ds1744_geometry: RAM below the calendar, including its last +/// byte, and an address past the part faults. +#[test] +fn ram_fills_the_part_below_the_calendar() { + let r = Rig::new(); + for (a, v) in [(0, 0x12), (0x800, 0x34), (0x7ff7, 0x56)] { + r.wr(a, v); + } + assert_eq!((r.rd(0), r.rd(0x800), r.rd(0x7ff7)), (0x12, 0x34, 0x56)); + r.set_time(0x12, 0x23, 0x10, 0x25, 0x09, 0x26, 0x20); + assert_eq!(r.rd(CONTROL), 0x20); + assert_eq!((r.rd(CONTROL + 1), r.rd(CONTROL + 2), r.rd(CONTROL + 3)), (0x12, 0x23, 0x10)); + assert_eq!((r.rd(CONTROL + 5), r.rd(CONTROL + 6), r.rd(CONTROL + 7)), (0x25, 0x09, 0x26)); + assert_eq!(r.rd(CONTROL + 4) & 7, 2); + assert_eq!(r.d.read8(SIZE as u64).status, BUS_ERR); + assert_eq!(r.d.write8(SIZE as u64, 1), BUS_ERR); +} + +/// Ports test_calendar: the control register shows the century beside the +/// write and read bits, the battery flag and weekday advance, and a +/// century carries. +#[test] +fn control_shows_the_century_and_the_calendar_advances() { + let r = Rig::new(); + assert_eq!(r.rd(CONTROL), 0x20); + r.wr(CONTROL, 0x40); + assert_eq!(r.rd(CONTROL), 0x60); + r.wr(CONTROL, 0x00); + assert_eq!(r.rd(CONTROL), 0x20); + r.wr(CONTROL, 0x80); + assert_eq!(r.rd(CONTROL), 0xa0); + r.wr(CONTROL, 0x20); + r.set_time(0x59, 0x59, 0x23, 0x31, 0x12, 0x24, 0x20); + assert_eq!(r.rd(CONTROL + 1), 0x59); + assert_eq!(r.rd(CONTROL + 4) & 0x80, 0x80, "battery flag"); + assert_eq!(r.rd(CONTROL + 4) & 7, 2); + r.step(SECOND); + assert_eq!(r.rd(CONTROL + 4) & 7, 3, "the weekday advances with the date"); + assert_eq!((r.rd(CONTROL + 1), r.rd(CONTROL + 5), r.rd(CONTROL + 6), r.rd(CONTROL + 7)), (0, 1, 1, 0x25)); + r.set_time(0x59, 0x59, 0x23, 0x31, 0x12, 0x99, 0x20); + r.step(SECOND); + assert_eq!(r.rd(CONTROL) & 0x3f, 0x21); + assert_eq!(r.rd(CONTROL + 7), 0); + // A commit that is a date is taken, with a different century. + r.wr(CONTROL, 0x80); + r.wr(CONTROL + 6, 0); + r.wr(CONTROL, 0x19); + assert_eq!(r.rd(CONTROL), 0x21, "month 0 is no date, so the century stays"); + assert_eq!(r.rd(CONTROL + 6), 1); + // The read bit freezes what the guest sees while the clock runs. + r.wr(CONTROL, 0x40); + r.step(2 * SECOND); + assert_eq!(r.rd(CONTROL + 1), 0); + r.wr(CONTROL, 0); + assert_eq!(r.rd(CONTROL + 1), 2); +} + +/// Ports test_invalid_dates: a commit that is not a date leaves the calendar +/// alone, a valid leap day is taken, and the stop bit rides in the seconds. +#[test] +fn a_commit_that_is_not_a_date_is_refused() { + let r = Rig::new(); + r.set_time(0x80 | 0x12, 0x34, 0x05, 0x15, 0x01, 0x24, 0x20); + for (date, month, year) in [(0x30, 0x02, 0x24), (0x29, 0x02, 0x23)] { + r.set_time(0x59, 0x59, 0x23, date, month, year, 0x20); + assert_eq!(r.rd(CONTROL + 1), 0x92, "stopped at the earlier time"); + assert_eq!((r.rd(CONTROL + 5), r.rd(CONTROL + 6), r.rd(CONTROL + 7)), (0x15, 0x01, 0x24)); + } + r.set_time(0x59, 0x59, 0x23, 0x29, 0x02, 0x24, 0x20); + r.step(SECOND); + assert_eq!((r.rd(CONTROL + 1), r.rd(CONTROL + 5), r.rd(CONTROL + 6), r.rd(CONTROL + 7)), (0, 1, 3, 0x24)); +} + +/// Ports test_stop_and_test_mode: a stopped oscillator holds the seconds, +/// the battery flag is read-only, and the test-mode bit flips the seconds' +/// low bit every 976562 ns unless the oscillator is stopped. +#[test] +fn test_mode_flips_the_low_second_bit() { + let r = Rig::new(); + r.set_time(0x80 | 0x12, 0x34, 0x05, 0x06, 0x07, 0x26, 0x20); + r.step(3 * SECOND); + assert_eq!(r.rd(CONTROL + 1), 0x92); + r.wr(CONTROL, 0x80); + r.wr(CONTROL + 1, 0x12); + r.wr(CONTROL + 4, 0xc2); + r.wr(CONTROL, 0x20); + assert_eq!(r.rd(CONTROL + 4), 0xc2); + assert_eq!(r.rd(CONTROL + 1) & 0xfe, 0x12); + let bit = r.rd(CONTROL + 1) & 1; + r.step(976_562); + assert_eq!(r.rd(CONTROL + 1) & 1, bit ^ 1); + r.wr(CONTROL, 0x80); + r.wr(CONTROL + 4, 0x02); + r.wr(CONTROL, 0x20); + let bit = r.rd(CONTROL + 1) & 1; + r.step(976_562); + assert_eq!(r.rd(CONTROL + 1) & 1, bit, "test mode off"); + r.wr(CONTROL, 0x80); + r.wr(CONTROL + 4, 0x42); + r.wr(CONTROL + 1, 0x92); + r.wr(CONTROL, 0x20); + let bit = r.rd(CONTROL + 1) & 1; + r.step(3 * 976_562); + assert_eq!(r.rd(CONTROL + 1) & 1, bit, "stopped, so it holds"); +} + +/// Ports test_restart_persistence: RAM and a completed calendar come back +/// from the file, the elapsed host time passes, a stopped clock does not +/// advance, and a file of the wrong size is refused. +#[test] +fn a_backing_file_restores_ram_and_the_calendar() { + let dir = std::env::temp_dir().join(format!("ds1744-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("clock"); + { + let r = Rig::new(); + r.d.set_nvram_file(&path).unwrap(); + r.wr(0x100, 0xa5); + r.set_time(0x30, 0x15, 0x08, 0x15, 0x06, 0x30, 0x20); + r.d.flush(); + } + let reopened = Rig::at(unix(2024, 1, 1, 0, 0, 0), YearBase::Gregorian); + reopened.d.set_nvram_file(&path).unwrap(); + assert_eq!(reopened.rd(0x100), 0xa5); + let c = reopened.calendar(); + assert_eq!((c[2], c[3], c[5], c[6], c[7]), (0x15, 0x08, 0x15, 0x06, 0x30)); + assert_eq!(c[0], 0x20); + + let short = dir.join("short"); + std::fs::write(&short, [0u8; 16]).unwrap(); + assert!(Rig::new().d.set_nvram_file(&short).is_err()); + std::fs::remove_dir_all(&dir).unwrap(); +} + +/// One step of a register sequence: write, read and expect, or let time pass. +enum Step { + W(usize, u8), + R(usize, u8), + Advance(i64), +} + +/// The register sequences QEMU's ds1742-test asserts, as one table so the +/// two models can be held to the same bytes. Offsets are from the 32 KiB +/// part's calendar (`CONTROL`); the seed is 2024-01-01T00:00:00. +/// +/// Ports test_calendar, test_invalid_dates and test_stop_and_test_mode +/// from tests/qtest/ds1742-test.c in jamesbraid/qemu-sgi-origami. The +/// model's answers come from the QEMU file's assertions, not from running +/// QEMU here; the end-to-end comparison with a live QEMU is not done. +#[test] +fn the_qemu_register_sequences_read_back_the_same() { + use Step::{Advance, R, W}; + const C: usize = CONTROL; + let set = |s, mi, h, wd, d, mo, y, cent| { + vec![W(C, 0x80), W(C + 1, s), W(C + 2, mi), W(C + 3, h), W(C + 4, wd), W(C + 5, d), W(C + 6, mo), W(C + 7, y), W(C, cent)] + }; + let mut steps = vec![R(C, 0x20), W(C, 0x40), R(C, 0x60), W(C, 0x00), R(C, 0x20), W(C, 0x80), R(C, 0xa0), W(C, 0x20)]; + // A rollover into 2025. + steps.extend(set(0x59, 0x59, 0x23, 0x02, 0x31, 0x12, 0x24, 0x20)); + steps.extend([R(C + 1, 0x59), R(C + 4, 0x82), Advance(1_000_000_000)]); + steps.extend([R(C + 4, 0x83), R(C + 1, 0x00), R(C + 5, 0x01), R(C + 6, 0x01), R(C + 7, 0x25)]); + // A century carry. + steps.extend(set(0x59, 0x59, 0x23, 0x02, 0x31, 0x12, 0x99, 0x20)); + steps.extend([Advance(1_000_000_000), R(C, 0x21), R(C + 7, 0x00)]); + // An impossible month leaves the calendar and its century alone. + steps.extend([W(C, 0x80), W(C + 6, 0x00), W(C, 0x19), R(C, 0x21), R(C + 6, 0x01)]); + // The read bit freezes the display; the clock runs on. + steps.extend([W(C, 0x40), Advance(2_000_000_000), R(C + 1, 0x00), W(C, 0x00), R(C + 1, 0x02)]); + // An impossible date is refused; a leap day is taken. + steps.extend(set(0x92, 0x34, 0x05, 0x02, 0x15, 0x01, 0x24, 0x20)); + steps.extend(set(0x59, 0x59, 0x23, 0x02, 0x30, 0x02, 0x24, 0x20)); + steps.extend([R(C + 1, 0x92), R(C + 5, 0x15), R(C + 6, 0x01), R(C + 7, 0x24)]); + steps.extend(set(0x59, 0x59, 0x23, 0x02, 0x29, 0x02, 0x23, 0x20)); + steps.extend([R(C + 1, 0x92), R(C + 5, 0x15)]); + steps.extend(set(0x59, 0x59, 0x23, 0x02, 0x29, 0x02, 0x24, 0x20)); + steps.extend([Advance(1_000_000_000), R(C + 1, 0x00), R(C + 5, 0x01), R(C + 6, 0x03), R(C + 7, 0x24)]); + // The stop bit holds the seconds; test mode makes the low bit run. + steps.extend(set(0x80 | 0x12, 0x34, 0x05, 0x02, 0x06, 0x07, 0x26, 0x20)); + steps.extend([Advance(3_000_000_000), R(C + 1, 0x92)]); + steps.extend([W(C, 0x80), W(C + 1, 0x12), W(C + 4, 0xc2), W(C, 0x20), R(C + 4, 0xc2)]); + + let r = Rig::new(); + for (i, step) in steps.iter().enumerate() { + match *step { + W(a, v) => r.wr(a, v), + R(a, want) => assert_eq!(r.rd(a), want, "step {i}: read of {a:#x}"), + Advance(ns) => r.step(ns), + } + } +} diff --git a/src/dev/ioc3.rs b/src/dev/ioc3.rs index d8557c9bd..89422a064 100644 --- a/src/dev/ioc3.rs +++ b/src/dev/ioc3.rs @@ -2856,23 +2856,23 @@ impl PciFunction for Ioc3 { // was waiting out. Deadlines are encoded as nanoseconds left plus one, // zero for none. -fn until(due: Option, now: Instant) -> u64 { +pub(crate) fn until(due: Option, now: Instant) -> u64 { due.map_or(0, |d| d.saturating_duration_since(now).as_nanos() as u64 + 1) } -fn due(left: u64, now: Instant) -> Option { +pub(crate) fn due(left: u64, now: Instant) -> Option { (left != 0).then(|| now + Duration::from_nanos(left - 1)) } -fn span(d: Option) -> u64 { +pub(crate) fn span(d: Option) -> u64 { d.map_or(0, |d| d.as_nanos() as u64 + 1) } -fn unspan(n: u64) -> Option { +pub(crate) fn unspan(n: u64) -> Option { (n != 0).then(|| Duration::from_nanos(n - 1)) } -fn usize_field(v: &toml::Value, key: &str, limit: usize) -> Result { +pub(crate) fn usize_field(v: &toml::Value, key: &str, limit: usize) -> Result { let n = field_u64(v, key)? as usize; if n > limit { return Err(format!("{key}: {n} is more than {limit}")); diff --git a/src/dev/ioc3_kbc.rs b/src/dev/ioc3_kbc.rs new file mode 100644 index 000000000..e8459ebc0 --- /dev/null +++ b/src/dev/ioc3_kbc.rs @@ -0,0 +1,340 @@ +//! SGI IOC3 PS/2 keyboard and mouse controller. +//! +//! A block of the IOC3 that the IOC4 carries too, at another place in its +//! BAR. It is a bit-level PS/2 line engine, not an 8042: five 32-bit +//! registers, a control and status register, a receive register per port +//! and a transmit register per port. Each port holds up to three received +//! characters; the clamp bits in the control register stop a port's device +//! once it has sent one or three, and with no clamp armed a fourth +//! character is lost and the receive register reports the overflow. +//! +//! The host's keyboard and mouse model is a [`Ps2Controller`], a byte queue +//! shared by both ports. This block drains it into the per-port holdings. +//! The chip around it owns the interrupt: every call that can move a +//! character in says so, and the chip latches its own source bit. +//! +//! # What is approximated here +//! +//! * The data-available line is asserted for every character that reaches a +//! holding, not once per drain. The silicon may pulse per character or hold +//! a level over the holding, and no public driver tells them apart. +//! * A device stopped by a clamp resumes inside the read that empties the +//! holding; the part takes a character time. A guest that reads and only +//! then acknowledges can clear the assertion those resumed characters +//! made. Linux acknowledges before it reads and OpenBSD reads until the +//! register comes back empty, so both avoid it. +//! * The control register's timeout bits (16, 17) and the hardware status +//! bits read clear: no stalled line is simulated, and a transmit completes +//! inside the write. +//! * Reset values and the register's endianness are the emulator's choice. +//! +//! Ported from hw/input/sgi-ioc3-kbc.c in jamesbraid/qemu-sgi-origami. QEMU's +//! two `ps2` devices are the shared [`Ps2Controller`] here, and its +//! re-entrancy guard around a fill is the owner's poke flag. + +use crate::dev::ioc3::usize_field; +use crate::snapshot::{field, field_bool, field_u32, field_u8s, StateTable}; +use std::sync::Weak; + +use crate::dev::ps2::Ps2Controller; +use crate::irq::InterruptController; + +/// Register offsets within the controller's window. +pub mod reg { + /// Control and status. + pub const CONTROL: u32 = 0x00; + /// Keyboard receive (read-only). + pub const KBD_RECEIVE: u32 = 0x04; + /// Mouse receive (read-only). + pub const AUX_RECEIVE: u32 = 0x08; + /// Keyboard transmit (write-only). + pub const KBD_TRANSMIT: u32 = 0x0c; + /// Mouse transmit (write-only). + pub const AUX_TRANSMIT: u32 = 0x10; + pub const MMIO_SIZE: u32 = AUX_TRANSMIT + 4; +} + +/// Characters a port holds. +pub const RX_SLOTS: usize = 3; +pub const PORT_KBD: usize = 0; +pub const PORT_AUX: usize = 1; + +/// Control bits, from the public Linux header. +pub mod ctrl { + pub const EMULATION: u32 = 1 << 12; + pub const FAST_CLOCK: u32 = 1 << 13; + pub const K_TO_EN: u32 = 1 << 18; + pub const M_TO_EN: u32 = 1 << 19; + pub const K_CLAMP_1: u32 = 1 << 20; + pub const M_CLAMP_1: u32 = 1 << 21; + pub const K_CLAMP_3: u32 = 1 << 22; + pub const M_CLAMP_3: u32 = 1 << 23; + pub const CLAMP_MASK: u32 = 0x00f0_0000; + /// Writable bits: the low half, the timeout enables and the clamps. + pub const MASK: u32 = 0x0000_ffff | K_TO_EN | M_TO_EN | CLAMP_MASK; +} + +/// Receive register: bit 27 says the characters came from the mouse, bit 28 +/// that one arrived with the slots full and was lost. +pub const RX_SOURCE_AUX: u32 = 1 << 27; +pub const RX_OVERFLOW: u32 = 1 << 28; + +/// The valid flag of receive slot `slot` (2 is the oldest character). +pub const fn rx_valid(slot: usize) -> u32 { + 1 << (29 + slot) +} + +/// What one port holds: the characters it has taken, oldest first, and +/// whether one was lost for want of room. +#[derive(Default, Clone, Copy)] +struct Holding { + data: [u8; RX_SLOTS], + count: usize, + overflow: bool, +} + +/// A chip that owns a [`Ioc3Kbc`] and wants to know when the host's PS/2 +/// model has a character. +pub trait KbcOwner: Send + Sync { + /// A device raised its line. If an access holds the chip (possibly this + /// very thread, from inside a transmit), that access picks the character + /// up when it finishes. + fn kbc_poke(&self); +} + +/// The line a PS/2 model drives when it has something to send. +pub struct KbcLine(pub Weak); + +impl InterruptController for KbcLine { + fn set_line(&self, _line: u32, _level: bool) { + if let Some(owner) = self.0.upgrade() { + owner.kbc_poke(); + } + } +} + +pub struct Ioc3Kbc { + /// Board wiring: whether the connectors are fitted. + ports_present: bool, + control: u32, + port: [Holding; 2], +} + +impl Ioc3Kbc { + pub fn new(ports_present: bool) -> Ioc3Kbc { + Ioc3Kbc { ports_present, control: 0, port: [Holding::default(); 2] } + } + + /// Reset: the control register and both holdings clear. + pub fn reset(&mut self) { + self.control = 0; + self.port = [Holding::default(); 2]; + } + + /// How many characters this port may take before its clock line is held + /// low. Zero means no clamp is armed, and then the controller has no way + /// to stop the device. Arming both clamps is not a combination any driver + /// writes, so the tighter one is taken. + fn clamp(&self, port: usize) -> usize { + let (one, three) = if port == PORT_AUX { + (ctrl::M_CLAMP_1, ctrl::M_CLAMP_3) + } else { + (ctrl::K_CLAMP_1, ctrl::K_CLAMP_3) + }; + if self.control & one != 0 { + 1 + } else if self.control & three != 0 { + RX_SLOTS + } else { + 0 + } + } + + /// Move what the device has sent into this port's holding, up to what + /// the clamp allows. A clamped port stops with its clock held low and the + /// rest still in the device; an unclamped one loses the surplus and says + /// so through the overflow flag. Returns whether a character arrived, + /// which is when the chip latches its source. + pub fn fill(&mut self, port: usize, ps2: &Ps2Controller) -> bool { + if !self.ports_present { + return false; + } + let clamp = self.clamp(port); + let mut arrived = false; + loop { + if clamp != 0 && self.port[port].count >= clamp { + break; + } + let Some(byte) = ps2.pop_from(port == PORT_AUX) else { break }; + let p = &mut self.port[port]; + if p.count >= RX_SLOTS { + p.overflow = true; + continue; + } + p.data[p.count] = byte; + p.count += 1; + arrived = true; + } + arrived + } + + /// Fill both ports. + pub fn fill_all(&mut self, ps2: &Ps2Controller) -> bool { + let kbd = self.fill(PORT_KBD, ps2); + let aux = self.fill(PORT_AUX, ps2); + kbd || aux + } + + /// Report what the port holds, oldest character in the highest slot, and + /// empty the holding: the register packs the characters since the last + /// read, so the read clears them, their flags and the overflow flag + /// together. The clamp releases with it. Returns the value and whether + /// the device's resumed characters arrived. + fn receive(&mut self, port: usize, ps2: &Ps2Controller) -> (u32, bool) { + let p = self.port[port]; + let mut val = 0u32; + for (i, &b) in p.data.iter().enumerate().take(p.count) { + let slot = (RX_SLOTS - 1) - i; + val |= u32::from(b) << (8 * slot); + val |= rx_valid(slot); + // The per-character error flag (bit 24 + slot) stays clear: the + // PS/2 layer delivers clean bytes. + } + if p.count != 0 && port == PORT_AUX { + val |= RX_SOURCE_AUX; + } + if p.overflow { + val |= RX_OVERFLOW; + } + self.port[port] = Holding::default(); + let arrived = self.fill(port, ps2); + (val, arrived) + } + + /// A 32-bit read at `off`. Returns the value, and whether a character + /// arrived as a result. + pub fn read(&mut self, off: u32, ps2: &Ps2Controller) -> (u32, bool) { + match off { + // Write-pending and the other status bits read clear: a + // synchronous transmit completes within the register write. + reg::CONTROL => (self.control, false), + reg::KBD_RECEIVE => self.receive(PORT_KBD, ps2), + reg::AUX_RECEIVE => self.receive(PORT_AUX, ps2), + _ => { + crate::dlog!(crate::devlog::LogModule::Ioc, "ioc3-kbc: read of write-only register {off:#x}"); + (0, false) + } + } + } + + /// A 32-bit write at `off`. Returns whether a character arrived. + pub fn write(&mut self, off: u32, val: u32, ps2: &Ps2Controller) -> bool { + match off { + reg::CONTROL => { + self.control = val & ctrl::MASK; + if val & (ctrl::EMULATION | ctrl::FAST_CLOCK) != 0 { + crate::dlog!(crate::devlog::LogModule::Ioc, "ioc3-kbc: emulation/fast-clock effects are not modeled"); + } + // Releasing a clamp lets a stopped device send again. + self.fill_all(ps2) + } + reg::KBD_TRANSMIT | reg::AUX_TRANSMIT => { + if !self.ports_present { + crate::dlog!(crate::devlog::LogModule::Ioc, "ioc3-kbc: transmit on an unwired port"); + return false; + } + ps2.write_to(val as u8, off == reg::AUX_TRANSMIT); + self.fill_all(ps2) + } + _ => { + crate::dlog!(crate::devlog::LogModule::Ioc, "ioc3-kbc: write of {val:#x} to read-only register {off:#x}"); + false + } + } + } +} + +// The controller's state for a snapshot of the IOC3 that carries it: the +// control register and each port's held characters. Whether the connectors +// are fitted is the board's. + +impl Holding { + fn save_state(&self) -> toml::Value { + StateTable::new() + .u8s("data", &self.data) + .u64("count", self.count as u64) + .bool("overflow", self.overflow) + .build() + } + fn load_state(v: &toml::Value) -> Result { + Ok(Self { + data: field_u8s(v, "data")?, + count: usize_field(v, "count", RX_SLOTS)?, + overflow: field_bool(v, "overflow")?, + }) + } +} + +impl Ioc3Kbc { + pub fn save_state(&self) -> toml::Value { + StateTable::new() + .u32("control", self.control) + .value("port0", self.port[0].save_state()) + .value("port1", self.port[1].save_state()) + .build() + } + + /// The controller `save_state` described, on this board's connectors. + pub fn loaded(&self, v: &toml::Value) -> Result { + Ok(Ioc3Kbc { + ports_present: self.ports_present, + control: field_u32(v, "control")?, + port: [Holding::load_state(field(v, "port0")?)?, Holding::load_state(field(v, "port1")?)?], + }) + } +} + +#[cfg(test)] +mod tests { + // Cases follow the sgi-ioc3-kbc qtests in jamesbraid/qemu-sgi-origami: + // clamp-after-one-splits-a-reply and the unclamped overflow. + use super::*; + + fn kbc() -> (Ioc3Kbc, Ps2Controller) { + (Ioc3Kbc::new(true), Ps2Controller::new(None)) + } + + /// Clamping after one character splits a reset reply: the acknowledge + /// first and the self-test byte next, each alone in the oldest slot, and + /// each read lets the stopped keyboard send the next. + #[test] + fn clamp_after_one_splits_a_reset_reply() { + let (mut k, ps2) = kbc(); + assert!(!k.write(reg::CONTROL, ctrl::K_CLAMP_1, &ps2)); + assert!(k.write(reg::KBD_TRANSMIT, 0xff, &ps2), "the acknowledge arrives"); + assert_eq!(k.read(reg::KBD_RECEIVE, &ps2), (0x80fa_0000, true), "and the self-test byte follows the read"); + assert_eq!(k.read(reg::KBD_RECEIVE, &ps2), (0x80aa_0000, false)); + assert_eq!(k.read(reg::KBD_RECEIVE, &ps2), (0, false)); + } + + /// With no clamp a port holds three characters, oldest in the highest + /// slot; a fourth is lost and the read reports the overflow, which the + /// read clears. + #[test] + fn an_unclamped_port_holds_three_and_reports_a_lost_fourth() { + let (mut k, ps2) = kbc(); + k.write(reg::KBD_TRANSMIT, 0xff, &ps2); + k.write(reg::KBD_TRANSMIT, 0xff, &ps2); + let all = rx_valid(0) | rx_valid(1) | rx_valid(2); + assert_eq!(k.read(reg::KBD_RECEIVE, &ps2).0, all | RX_OVERFLOW | 0x00fa_aafa); + assert_eq!(k.read(reg::KBD_RECEIVE, &ps2).0, 0, "the read cleared the holding and the flag"); + } + + /// A board without the connectors takes no transmit and holds nothing. + #[test] + fn unwired_ports_hold_nothing() { + let (mut k, ps2) = (Ioc3Kbc::new(false), Ps2Controller::new(None)); + assert!(!k.write(reg::KBD_TRANSMIT, 0xff, &ps2)); + assert_eq!(k.read(reg::KBD_RECEIVE, &ps2), (0, false)); + } +} diff --git a/src/dev/ioc4.rs b/src/dev/ioc4.rs new file mode 100644 index 000000000..6447b5667 --- /dev/null +++ b/src/dev/ioc4.rs @@ -0,0 +1,1010 @@ +//! SGI IOC4 base I/O controller, revision 0x53, as the IO9 board fits it. +//! +//! One PCI function with a 1 MiB memory BAR. Behind it: four 16550-compatible +//! serial ports with a ring-DMA engine each, a local interrupt controller for +//! them, the registers that capture a failed ring transfer, the UART clock +//! select, the ATA channel's DMA readback word the PROM's diagnostic probes, +//! and the timekeeping SRAM on byte-bus select 0 when the board fits one +//! ([`Ds1744`]), and the PS/2 keyboard and mouse controller the IO9 always +//! carries ([`Ioc3Kbc`], the block the IOC3 has too). The ATA channel, the +//! GPIO pins and the other byte-bus selects are not modelled; an access to +//! them reads zero or is dropped, and is logged. +//! +//! Behaviour follows the public Linux and OpenBSD drivers; where the model +//! approximates, the comment says so. +//! +//! # One interrupt output +//! +//! Everything the chip reports ORs into the PCI interrupt pin: the serial +//! status word (eight bits a port) and the other word (ATA, DMA errors, +//! keyboard), each behind its own enable. UART pass-through, transmit-ring- +//! empty, receive-ring-full and receive-high-water follow their source's level and ignore a +//! write-one-to-clear; the rest latch until acknowledged. +//! +//! # Byte order +//! +//! IOC4 orders its own bytes. Its slot is self-ordered: the bridge carries the +//! processor's bytes in address order and leaves the address alone, so a +//! byte access to UART register `r` of port `n` arrives at `0x380 + 8n + r`. +//! A PCI function's BAR value is its bytes read little-endian, and IOC4 reads +//! them big-endian, so the chip takes a 32-bit value byte-reversed: the +//! processor's value is the register's value, as QEMU's big-endian IOC4 +//! region reads it. Control registers and the +//! keyboard block take 32-bit accesses only; a UART window or the clock takes +//! bytes only. Anything else is refused (reads zero, writes dropped). Which widths the silicon accepts is not +//! known. +//! +//! # What is approximated here +//! +//! * The reset values, the BAR size and the PCI class are the emulator's +//! choices. +//! * The serial ring engine is the one the IOC3 uses (`ioc_serial_dma.rs`) and +//! carries its approximations: timers run on host time and receive-ring-full +//! is a level that follows a held entry. After a failed ring transfer the port +//! halts until a channel reset. +//! * Register 0x20 reads the DMA-idle bit unless a transfer is under way. +//! Transfers complete inside the access that starts them, so software +//! always finds the engine idle. +//! * The keyboard block's status bits (write-pending, line control, line +//! levels and idle flags, mask `0xc33f`) read clear and drop writes, as the +//! QEMU adapter does. Its port order, write-pending timing, and whether +//! the interrupt is a pulse per character or a level are not established, +//! and the model follows QEMU: a latch per arriving character, which an +//! acknowledge racing an arrival can lose. The IOC3-only receive flags are +//! masked out of the receive registers. The ports are always fitted, as the +//! IO9's are in QEMU. +//! * The ATA readback word at 0x148 is a plain 32-bit latch with no ATA +//! behind it. Only boards that ask for it have it. +//! * The PCI status register reads 0x0220 (66 MHz capable, medium DEVSEL) on +//! boards that ask for it: a value taken from another revision of the chip, +//! not measured on this one. +//! +//! Names follow QEMU's: `sgi-ioc4.c` has the same registers under the same +//! names, its `sgi_ioc4_sdma_*` callbacks are the chip's `SerialDmaHost`, and +//! the ring engine is `ioc_serial_dma.rs` (QEMU's `sgi-ioc-serial-dma.c`). +//! +//! Ported from hw/misc/sgi-ioc4.c in jamesbraid/qemu-sgi-origami. + +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, OnceLock, Weak}; +use std::time::{Duration, Instant}; + +use parking_lot::Mutex; + +use crate::dev::ds1744::Ds1744; +use crate::dev::ioc3_kbc::{self, Ioc3Kbc, KbcLine, KbcOwner}; +use crate::dev::ps2::Ps2Controller; +use crate::dev::ioc_serial_dma::{self, off as sdma_off, SerialDmaUart, SerialDmaHost, SerialDmaParams, SerialDma, SerialDmaSource}; +use crate::dev::ns16550::Ns16550; +use crate::dev::serial_host::SerialBackend; +use crate::dev::deadline_timers::DeadlineTimers; +use crate::hptimer::{TimerManager, TimerReturn}; +use crate::irq::IrqLine; +use crate::pci::{Bar, PciConfig, PciConfigPolicy, PciDma, PciFunction, PciFunctionState, PCI_INTA}; +use crate::traits::{BusDevice, Device, Resettable}; + +pub const VENDOR_ID: u16 = 0x10a9; +pub const DEVICE_ID: u16 = 0x100a; +pub const REVISION: u8 = 0x53; +/// PCI class, subclass and interface: "other", an emulator default. +const PCI_CLASS: u32 = 0x00_ff_00; +pub const BAR_SIZE: u32 = 1 << 20; +/// Status of a 66 MHz capable function with medium DEVSEL timing. +const PCI_STATUS_COMPAT: u16 = 0x0220; +pub const PORTS: usize = 4; + +/// Register offsets within the BAR. +pub mod reg { + /// Captured address of the first failed ring transfer, low and high. + pub const ERROR_LOW: u32 = 0x00; + pub const ERROR_HIGH: u32 = 0x04; + pub const SERIAL_PENDING: u32 = 0x08; + pub const OTHER_PENDING: u32 = 0x0c; + pub const SERIAL_ENABLE_SET: u32 = 0x10; + pub const OTHER_ENABLE_SET: u32 = 0x14; + pub const SERIAL_ENABLE_CLR: u32 = 0x18; + pub const OTHER_ENABLE_CLR: u32 = 0x1c; + pub const SERIAL_CONTROL: u32 = 0x20; + pub const MISC_CONTROL: u32 = 0x24; + /// A word of the ATA channel's DMA block the PROM writes and reads back. + pub const ATA_READBACK: u32 = 0x148; + /// Ring bases, a low and a high word for each pair of ports. + pub const SDMA_BASE: u32 = 0x300; + pub const SDMA_PORT: u32 = 0x310; + pub const SDMA_PORT_STRIDE: u32 = super::sdma_off::REGS * 4; + /// The first 16550 window; eight bytes a port. + pub const UART: u32 = 0x380; + pub const UART_STRIDE: u32 = 8; + /// Byte-bus select 0: the timekeeping SRAM, whose last eight bytes are + /// the clock (at 0x87ff8). + pub const CLOCK: u32 = 0x80000; + pub const CLOCK_SIZE: u32 = crate::dev::ds1744::SIZE as u32; + /// The PS/2 controller's five registers. + pub const KBC: u32 = 0x200; +} + +/// Bits of the serial status word, eight to a port. +pub mod int { + pub const PORT_STRIDE: u32 = 8; + pub const PASSTHROUGH: u32 = 6; + pub const fn serial(port: usize, src: u32) -> u32 { + 1 << (port as u32 * PORT_STRIDE + src) + } + pub const fn pass(port: usize) -> u32 { + serial(port, PASSTHROUGH) + } + /// The other word: a serial DMA error, one bit a port from bit 2. + pub const fn dma_error(port: usize) -> u32 { + 1 << (port as u32 + 2) + } +} + +/// Bits of the captured error address. +mod err { + pub const VALID: u32 = 1; + pub const MULTIPLE: u32 = 1 << 5; + pub const REQUESTER_SHIFT: u32 = 1; + /// The low six address bits are not captured. + pub const ADDRESS_MASK: u64 = 0xffff_ffc0; +} + +/// Other word bit 6: a keyboard or mouse character arrived. +const OTHER_KEYBOARD: u32 = 0x40; +/// Keyboard control bits that report state on the controller: write-pending +/// (0, 1), the last write's line-control bit (2, 3), the data and clock line +/// levels (4, 5, 8, 9) and the state machines' idle flags (14, 15). The model +/// does not simulate the lines, so they all read clear and a write to them is +/// dropped. +const KBC_STATUS: u32 = 0x0000_c33f; +/// The receive registers carry no IOC3-only flags: bits 24 to 28 (per-slot +/// errors, source and overflow) read clear. +const KBC_RX_MASK: u32 = 0xe0ff_ffff; + +/// Serial control bits the model acts on: the threshold, drain, enable, +/// pause and reset. The rest are stored and read back. +const SSCR_MODELLED: u32 = 0x1ff | (1 << 27) | (1 << 28) | (1 << 29) | (1 << 31); +/// The receive consume index also holds the timer arm in bit 31. +const SRCIR_TIMER_ARM: u32 = 1 << 31; +/// The ring size select in a pair base's low word. +const RING_4K: u32 = 1; + +/// Register 0x20: the DMA engines are idle. +const SERIAL_CONTROL_IDLE: u32 = 0x80; + +/// IOC4's engine halts a port on a failed transfer and does not define the +/// packer's held count. +const IOC4_SDMA_PARAMS: SerialDmaParams = SerialDmaParams { held_counts: false, halt_on_dma_error: true }; + +/// The UART input clock the board feeds the chip. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Ioc4Clock { + Mhz33, + Mhz66, +} + +impl Ioc4Clock { + pub fn hz(self) -> u32 { + match self { + Ioc4Clock::Mhz33 => 33_333_333, + Ioc4Clock::Mhz66 => 66_666_667, + } + } +} + +/// What the board makes of the chip. +#[derive(Clone, Copy, Debug)] +pub struct Ioc4Config { + pub clock: Ioc4Clock, + /// Report PCI status 0x0220. + pub pci_status_compat: bool, + /// Latch the ATA DMA readback word. + pub ata_readback: bool, +} + +impl Ioc4Config { + /// The chip with nothing a board asks for beyond its clock. + pub fn bare(clock: Ioc4Clock) -> Self { + Self { clock, pci_status_compat: false, ata_readback: false } + } + + /// The Tezro's IO9: a 66.67 MHz clock, and the status and ATA readback + /// words its PROM expects. + pub fn tezro() -> Self { + Self { clock: Ioc4Clock::Mhz66, pci_status_compat: true, ata_readback: true } + } +} + +/// A 32-bit register in the ring-engine block. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum SdmaReg { + Base { pair: usize, high: bool }, + Port { port: usize, off: u32 }, +} + +/// Ring bases sit at 0x300 (low then high word, a pair each), then seven +/// registers a port from 0x310. +fn sdma_decode(a: u32) -> Option { + if a & 3 != 0 { + return None; + } + if (reg::SDMA_BASE..reg::SDMA_PORT).contains(&a) { + let n = a - reg::SDMA_BASE; + return Some(SdmaReg::Base { pair: (n / 8) as usize, high: n & 4 != 0 }); + } + let end = reg::SDMA_PORT + PORTS as u32 * reg::SDMA_PORT_STRIDE; + if (reg::SDMA_PORT..end).contains(&a) { + let n = a - reg::SDMA_PORT; + return Some(SdmaReg::Port { port: (n / reg::SDMA_PORT_STRIDE) as usize, off: n % reg::SDMA_PORT_STRIDE }); + } + None +} + +/// Where a byte access to the UART windows lands. +fn uart_decode(a: u32) -> Option<(usize, u32)> { + let end = reg::UART + PORTS as u32 * reg::UART_STRIDE; + (reg::UART..end).contains(&a).then(|| (((a - reg::UART) / reg::UART_STRIDE) as usize, (a - reg::UART) % reg::UART_STRIDE)) +} + +fn in_kbc(a: u32) -> bool { + (reg::KBC..reg::KBC + ioc3_kbc::reg::MMIO_SIZE).contains(&a) +} + +fn in_clock(a: u32) -> bool { + (reg::CLOCK..reg::CLOCK + reg::CLOCK_SIZE).contains(&a) +} + +struct St { + cfg: Ioc4Config, + serial_pending: u32, + serial_enable: u32, + other_pending: u32, + other_enable: u32, + /// Sources that follow a level, in the serial word's layout. + level: u32, + error_low: u32, + error_high: u32, + ata_readback: u32, + uart: [Ns16550; PORTS], + sdma: SerialDma, + kbc: Ioc3Kbc, + ps2: Arc, + dma: Option>, +} + +impl St { + fn new(cfg: Ioc4Config, ps2: Arc) -> St { + St { + cfg, + serial_pending: 0, + serial_enable: 0, + other_pending: 0, + other_enable: 0, + level: 0, + error_low: 0, + error_high: 0, + ata_readback: 0, + uart: std::array::from_fn(|_| Ns16550::new(cfg.clock.hz() / 16)), + sdma: SerialDma::new(PORTS, IOC4_SDMA_PARAMS), + // Every IO9 brings the keyboard and mouse out. + kbc: Ioc3Kbc::new(true), + ps2, + dma: None, + } + } + + /// A PCI reset: the interrupt state, the ports and the engines. + fn reset(&mut self) { + self.serial_pending = 0; + self.serial_enable = 0; + self.other_pending = 0; + self.other_enable = 0; + self.level = 0; + self.error_low = 0; + self.error_high = 0; + self.ata_readback = 0; + for u in &mut self.uart { + u.reset(); + } + self.sdma.reset(); + self.kbc.reset(); + self.ps2.power_on(); + } + + /// Move what the host's PS/2 model has sent into the holdings, and latch + /// the keyboard source if a character arrived. + fn kbc_fill(&mut self) { + let ps2 = self.ps2.clone(); + if self.kbc.fill_all(&ps2) { + self.other_pending |= OTHER_KEYBOARD; + } + } + + /// A 32-bit read of the keyboard block at `off` in its window, as the + /// adapter presents it: no write-pending or other status, and none of + /// the IOC3-only receive flags. + fn kbc_read(&mut self, off: u32) -> u32 { + let ps2 = self.ps2.clone(); + let (raw, arrived) = self.kbc.read(off, &ps2); + if arrived { + self.other_pending |= OTHER_KEYBOARD; + } + if off == ioc3_kbc::reg::CONTROL { raw & !KBC_STATUS } else { raw & KBC_RX_MASK } + } + + fn kbc_write(&mut self, off: u32, val: u32) { + let mut val = val; + if off == ioc3_kbc::reg::CONTROL { + // The status bits read clear; a write that differs is a guest + // error and is dropped. + if val & KBC_STATUS != 0 { + crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 write to read-only KBC status bits {:#x} ignored", val & KBC_STATUS); + } + val &= !KBC_STATUS; + } + let ps2 = self.ps2.clone(); + if self.kbc.write(off, val, &ps2) { + self.other_pending |= OTHER_KEYBOARD; + } + } + + /// The PCI interrupt pin: any enabled source. + fn line_level(&self) -> bool { + (self.serial_pending | self.level) & self.serial_enable != 0 || self.other_pending & self.other_enable != 0 + } + + /// The host-time deadlines the engines hold, for the timers. + fn deadlines(&self) -> Deadlines { + let mut d = [None; KINDS]; + for p in 0..PORTS { + d[K_RX + p] = self.sdma.rx_deadline(p); + d[K_TX_DELAY + p] = self.sdma.tx_delay_deadline(p); + d[K_LOOP + p] = self.sdma.loop_deadline(p); + } + d + } + + fn sdma_engine_split(&mut self) -> (&mut SerialDma, Ioc4SerialDmaHost<'_>) { + let host = Ioc4SerialDmaHost { + uart: &mut self.uart, + serial_pending: &mut self.serial_pending, + level: &mut self.level, + other_pending: &mut self.other_pending, + error_low: &mut self.error_low, + error_high: &mut self.error_high, + dma: self.dma.as_deref(), + }; + (&mut self.sdma, host) + } + + fn sdma_run(&mut self, p: usize) { + let (sdma, mut host) = self.sdma_engine_split(); + sdma.run(p, &mut host); + } + + fn sdma_catch_up(&mut self, now: Instant) { + let (sdma, mut host) = self.sdma_engine_split(); + sdma.catch_up(now, &mut host); + } + + // ----- the 16550s ----- + + /// Keep the pass-through source on the port's interrupt level, and let + /// the engine move: a ring driver receives with the 16550's own + /// interrupts disabled. + fn uart_changed(&mut self, p: usize) { + let level = self.uart[p].irq(); + set_level_bit(&mut self.level, int::pass(p), level); + self.sdma_run(p); + } + + /// Move host input into the port and let the receive timeout run out. + /// Returns whether anything changed. + fn uart_pump(&mut self, p: usize) -> bool { + let u = &mut self.uart[p]; + let moved = u.pump(); + u.check_timeout(Instant::now()) || moved + } + + fn uart_read(&mut self, p: usize, r: u32) -> u8 { + let v = self.uart[p].read(r); + self.uart_changed(p); + v + } + + fn uart_write(&mut self, p: usize, r: u32, v: u8) { + if r == 0 && !self.uart[p].dlab() && self.sdma.control(p) & ioc_serial_dma::ctl::ENABLE != 0 { + crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 PIO transmit on port {p} while DMA is enabled"); + } + self.uart[p].write(r, v); + self.uart_changed(p); + } + + // ----- mode-switch diagnostics ----- + + /// Report the DMA mode switches and the writes the model takes but does + /// not act on. They are reported, not refused: the UART orders PIO and + /// ring DMA itself, and the write goes ahead. + fn check_port_write(&self, p: usize, off: u32, v: u32) { + let log = |what: std::fmt::Arguments| crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 {what}"); + match off { + sdma_off::CONTROL => { + if v & !SSCR_MODELLED != 0 { + log(format_args!("serial control bits {:#010x} on port {p} are stored but not modelled", v & !SSCR_MODELLED)); + } + self.control_check(p, v); + } + sdma_off::TX_PRODUCE | sdma_off::RX_CONSUME => { + let size = if self.sdma.base_read(p / 2, false) & RING_4K != 0 { 0x1000 } else { 0x400 }; + let keep = (size - 8) | if off == sdma_off::RX_CONSUME { SRCIR_TIMER_ARM } else { 0 }; + if v & !keep != 0 { + let what = if off == sdma_off::TX_PRODUCE { "TX producer" } else { "RX consumer" }; + log(format_args!("{what} {v:#010x} on port {p} has bits outside the pointer field")); + } + } + sdma_off::RX_TIMER if v & !0xfff != 0 => { + log(format_args!("unsupported RX timer bits {:#010x} on port {p}", v & !0xfff)); + } + _ => {} + } + } + + /// Enabling DMA over a base misaligned for its rings, or over PIO data in + /// the UART's FIFOs, and disabling it with ring work pending. + fn control_check(&self, p: usize, v: u32) { + use ioc_serial_dma::ctl; + let old_on = self.sdma.control(p) & ctl::ENABLE != 0; + let want_on = v & ctl::ENABLE != 0; + if v & ctl::RESET != 0 || old_on == want_on { + return; + } + let log = |what: std::fmt::Arguments| crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 {what}"); + let pair = p / 2; + if want_on { + let base = u64::from(self.sdma.base_read(pair, true)) << 32 | u64::from(self.sdma.base_read(pair, false) & !RING_4K); + let size: u64 = if self.sdma.base_read(pair, false) & RING_4K != 0 { 0x1000 } else { 0x400 }; + if base & (size * 4 - 1) != 0 { + log(format_args!("DMA enabled on port {p} with pair base {base:#x} misaligned for its rings")); + } + if self.uart[p].rx_pending() != 0 { + log(format_args!("DMA enabled on port {p} with PIO data in the UART's FIFOs")); + } + } else if !self.sdma.port_quiescent(p) { + log(format_args!("DMA disabled on port {p} with ring work pending")); + } + } + + // ----- control registers ----- + + fn read32(&mut self, a: u32) -> u32 { + self.sdma_catch_up(Instant::now()); + if let Some(r) = sdma_decode(a) { + let (sdma, mut host) = self.sdma_engine_split(); + return match r { + SdmaReg::Base { pair, high } => sdma.base_read(pair, high), + SdmaReg::Port { port, off } => sdma.port_read(port, off, &mut host), + }; + } + match a { + reg::ERROR_LOW => self.error_low, + reg::ERROR_HIGH => self.error_high, + reg::SERIAL_PENDING => self.serial_pending | self.level, + reg::OTHER_PENDING => self.other_pending, + reg::SERIAL_ENABLE_SET => self.serial_enable, + reg::OTHER_ENABLE_SET => self.other_enable, + reg::SERIAL_CONTROL => if self.sdma.idle() { SERIAL_CONTROL_IDLE } else { 0 }, + reg::MISC_CONTROL => u32::from(self.cfg.clock == Ioc4Clock::Mhz66), + reg::ATA_READBACK if self.cfg.ata_readback => self.ata_readback, + _ => { + crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 read of unsupported register {a:#x}"); + 0 + } + } + } + + fn write32(&mut self, a: u32, v: u32) { + self.sdma_catch_up(Instant::now()); + if let Some(r) = sdma_decode(a) { + match r { + SdmaReg::Base { pair, high } => { + if !self.sdma.pair_quiescent(pair) { + crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 DMA base of pair {pair} changed while it is active"); + } + self.sdma.base_write(pair, high, v); + } + SdmaReg::Port { port, off } => { + self.check_port_write(port, off, v); + let (sdma, mut host) = self.sdma_engine_split(); + sdma.port_write(port, off, v, &mut host); + } + } + return; + } + match a { + // Acknowledge: write one to clear. Level sources ignore it. + reg::SERIAL_PENDING => self.serial_pending &= !v, + reg::OTHER_PENDING => self.other_pending &= !v, + reg::SERIAL_ENABLE_SET => self.serial_enable |= v, + reg::OTHER_ENABLE_SET => self.other_enable |= v, + reg::SERIAL_ENABLE_CLR => self.serial_enable &= !v, + reg::OTHER_ENABLE_CLR => self.other_enable &= !v, + reg::ATA_READBACK if self.cfg.ata_readback => self.ata_readback = v, + reg::ERROR_LOW | reg::ERROR_HIGH | reg::SERIAL_CONTROL | reg::MISC_CONTROL => { + crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 write {v:#x} to read-only register {a:#x}"); + } + _ => crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 write {v:#x} to unsupported register {a:#x}"), + } + } +} + +fn set_level_bit(word: &mut u32, mask: u32, level: bool) { + if level { + *word |= mask; + } else { + *word &= !mask; + } +} + +/// IOC4 as the ring engine sees it: four 16550s, the serial status word, +/// the error capture registers and the bus master. +struct Ioc4SerialDmaHost<'a> { + uart: &'a mut [Ns16550; PORTS], + serial_pending: &'a mut u32, + level: &'a mut u32, + other_pending: &'a mut u32, + error_low: &'a mut u32, + error_high: &'a mut u32, + dma: Option<&'a dyn PciDma>, +} + +impl SerialDmaHost for Ioc4SerialDmaHost<'_> { + fn uart(&mut self, port: usize) -> &mut dyn SerialDmaUart { + &mut self.uart[port] + } + + fn raise(&mut self, port: usize, src: SerialDmaSource) { + *self.serial_pending |= int::serial(port, src as u32); + } + + fn set_level(&mut self, port: usize, src: SerialDmaSource, level: bool) { + set_level_bit(self.level, int::serial(port, src as u32), level); + } + + /// The engines always reach their ports. + fn reachable(&self, _port: usize) -> bool { + true + } + + fn uart_line_changed(&mut self, port: usize) { + let level = self.uart[port].irq(); + set_level_bit(self.level, int::pass(port), level); + } + + fn dma_read(&self, addr: u64, buf: &mut [u8]) -> bool { + self.dma.is_some_and(|d| d.dma_read(addr, buf).is_ok()) + } + + fn dma_write(&self, addr: u64, data: &[u8]) -> bool { + self.dma.is_some_and(|d| d.dma_write(addr, data).is_ok()) + } + + /// The first failure captures its address and its requester: twice the + /// port for a transmit fetch, one more for a receive post. Later ones + /// only set the multiple-error bit. The port's source is in the other + /// word. + fn dma_error(&mut self, port: usize, rx: bool, addr: u64) { + if *self.error_low & err::VALID == 0 { + let requester = (2 * port + usize::from(rx)) as u32; + *self.error_low = (addr & err::ADDRESS_MASK) as u32 | requester << err::REQUESTER_SHIFT | err::VALID; + *self.error_high = (addr >> 32) as u32; + } else { + *self.error_low |= err::MULTIPLE; + } + *self.other_pending |= int::dma_error(port); + crate::dlog!( + crate::devlog::LogModule::Ioc, + "IOC4 serial DMA {} failed for port {port} at {addr:#x}", + if rx { "write" } else { "read" } + ); + } +} + +// --------------------------------------------------------------------------- +// The device +// --------------------------------------------------------------------------- + +/// How long a serial receive thread sleeps when its backend has nothing: +/// `SerialBackend` has no blocking read to wait on. +const RX_IDLE: Duration = Duration::from_millis(2); + +/// The deadlines the chip keeps on host time, one timer slot each. +const K_RX: usize = 0; +const K_TX_DELAY: usize = PORTS; +const K_LOOP: usize = 2 * PORTS; +const KINDS: usize = 3 * PORTS; +type Deadlines = [Option; KINDS]; + +/// Lock order: `gate`, then `line_order`, then `st`; `armed` and `line` are +/// leaves. The engines move their rings with `st` held: the DMA goes down +/// through the bridge, which never calls back into the chip. +struct Shared { + pci: PciFunctionState, + st: Mutex, + /// PCI interrupt pin A. + line: Mutex>, + /// What the line was last driven to. + line_level: AtomicBool, + /// Serialises driving the line; see [`Shared::sync_line`]. + line_order: Mutex<()>, + me: Weak, + timers: DeadlineTimers, + /// A PS/2 character arrived while an access held the chip. + kbc_dirty: AtomicBool, + /// Between `start` and `stop`. + running: AtomicBool, + /// Held by a timer callback while it runs, so `stop` can wait one out. + gate: Mutex<()>, +} + +impl KbcOwner for Shared { + /// A device raised its line. If an access holds the chip (possibly this + /// very thread, from inside a transmit), that access picks the character + /// up when it finishes: the flag is set before the lock is tried, so + /// either this thread gets the lock or the holder sees the flag. + fn kbc_poke(&self) { + self.kbc_dirty.store(true, Ordering::SeqCst); + if let Some(mut st) = self.st.try_lock() { + if self.kbc_dirty.swap(false, Ordering::SeqCst) { + st.kbc_fill(); + } + drop(st); + self.sync_line(); + } + } +} + +impl Shared { + /// Put the interrupt line at the level the chip's state gives, then pick + /// up any PS/2 character that arrived while the chip was held (a poke + /// sets the flag before it tries the lock, so whoever holds the lock sees + /// it here, after releasing). + fn sync_line(&self) { + loop { + self.drive_line(); + if !self.kbc_dirty.swap(false, Ordering::SeqCst) { + break; + } + self.st.lock().kbc_fill(); + } + } + + /// The level is read after taking `line_order`, so whichever thread + /// drives last has seen the latest state. `st` is never held when this + /// is called. + fn drive_line(&self) { + let _order = self.line_order.lock(); + let level = self.st.lock().line_level(); + if self.line_level.swap(level, Ordering::SeqCst) != level { + if let Some(l) = self.line.lock().as_ref() { + l.set(level); + } + } + } + + /// Run `f` with the chip locked, then arm the timers for the deadlines it + /// left and drive the line from its state. + fn with_st(&self, f: impl FnOnce(&mut St) -> R) -> R { + let (r, want) = { + let mut st = self.st.lock(); + let r = f(&mut st); + (r, st.deadlines()) + }; + self.arm(want); + self.sync_line(); + r + } + + /// Arm a timer for each deadline the chip holds. + fn arm(&self, want: Deadlines) { + self.timers.arm(&want, &self.running, &self.me, |_| Duration::ZERO, Shared::fire); + } + + /// A deadline passed with no guest access to notice it. Runs on the timer + /// thread: short, no waiting beyond the chip's own locks. + fn fire(me: &Weak, kind: usize, due: Instant) -> TimerReturn { + let Some(sh) = me.upgrade() else { return TimerReturn::Delete }; + let _gate = sh.gate.lock(); + if !sh.running.load(Ordering::SeqCst) { + return TimerReturn::Delete; + } + sh.timers.fired(kind, due); + sh.with_st(|st| st.sdma_catch_up(Instant::now())); + TimerReturn::Delete + } + + /// Feed host input from serial backend `port` into its FIFO until the + /// chip stops. + fn rx_loop(me: Weak, port: usize) { + loop { + { + let Some(sh) = me.upgrade() else { return }; + if !sh.running.load(Ordering::SeqCst) { + return; + } + sh.with_st(|st| { + if st.uart_pump(port) { + st.uart_changed(port); + } + }); + } + std::thread::park_timeout(RX_IDLE); + } + } + + /// One synchronous pass of what the receive threads and timers do. + #[cfg(test)] + fn service(&self) { + self.with_st(|st| { + for p in 0..PORTS { + st.uart_pump(p); + st.uart_changed(p); + } + st.sdma_catch_up(Instant::now()); + for p in 0..PORTS { + st.sdma_run(p); + } + }); + } +} + +/// The IOC4 PCI function. +pub struct Ioc4 { + sh: Arc, + /// The DS1744 on byte-bus select 0, when the board fits one. + clock: OnceLock>, + /// The serial receive threads, joined in `stop`. + threads: Mutex>>, +} + +impl Ioc4 { + pub fn new(cfg: Ioc4Config) -> Ioc4 { + let sh = Arc::new_cyclic(|weak: &Weak| { + let line = IrqLine::new(Arc::new(KbcLine(weak.clone())), 0); + let ps2 = Arc::new(Ps2Controller::new(Some(line))); + // Host key and mouse events are accepted from the start (this + // only sets a flag); `stop` and `start` follow the chip's. + Device::start(&*ps2); + let bars = [Bar::Mem(BAR_SIZE), Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused]; + let policy = PciConfigPolicy { + status: if cfg.pci_status_compat { PCI_STATUS_COMPAT } else { 0 }, + status_w1c_mask: 0, + command_mask: 0xffff, + ..Default::default() + }; + Shared { + pci: PciFunctionState::new(PciConfig::new_with_policy(VENDOR_ID, DEVICE_ID, PCI_CLASS, REVISION, bars, PCI_INTA, policy)), + st: Mutex::new(St::new(cfg, ps2)), + line: Mutex::new(None), + line_level: AtomicBool::new(false), + line_order: Mutex::new(()), + kbc_dirty: AtomicBool::new(false), + me: weak.clone(), + timers: DeadlineTimers::new(), + running: AtomicBool::new(false), + gate: Mutex::new(()), + } + }); + Ioc4 { sh, clock: OnceLock::new(), threads: Mutex::new(Vec::new()) } + } + + /// Fit the timekeeping SRAM on byte-bus select 0. Its 32 KiB fill the + /// BAR from 0x80000 and its calendar sits in the last eight bytes. The + /// chip's reset does not touch it: it is memory with a battery. + pub fn attach_clock(&self, clock: Arc) { + assert!(self.clock.set(clock).is_ok(), "IOC4 clock attached more than once"); + } + + /// The timer manager for device deadlines. Set it before start. + pub fn set_timer_manager(&self, tm: Arc) { + self.sh.timers.set_manager(tm); + } + + /// Wire the host end of serial port `port` (0 is the console): a TCP + /// listener, a log tee, or nothing. + pub fn set_serial_backend(&self, port: usize, backend: Arc) { + self.sh.st.lock().uart[port].set_backend(backend); + } + + /// The PS/2 keyboard and mouse model, for the host side to push keys and + /// pointer motion into. + pub fn ps2(&self) -> Arc { + self.sh.st.lock().ps2.clone() + } + + /// A PCI reset of the chip. + pub fn reset(&self) { + self.sh.pci.config().reset(); + self.sh.with_st(St::reset); + } + + /// One synchronous pass of the work the receive threads and the timers + /// do between guest accesses. Tests use it to settle the chip without + /// waiting on a thread. + #[cfg(test)] + pub fn service(&self) { + self.sh.service(); + } + + /// Name the chip's long-lived locks in the lock registry, which holds + /// only a weak reference to them. + pub fn register_locks(&self) { + for (what, probe) in [ + ("st", (|s: &Shared| s.st.is_locked()) as fn(&Shared) -> bool), + ("line_order", |s| s.line_order.is_locked()), + ] { + let sh = Arc::downgrade(&self.sh); + crate::locks::register_lock(format!("ioc4::{what}"), move || sh.upgrade().is_some_and(|s| probe(&s))); + } + } + + /// Say that an access the chip does not take was refused. + fn refuse(what: &str, width: usize, off: u32) { + crate::dlog!(crate::devlog::LogModule::Ioc, "IOC4 {what} of width {width} at {off:#x} refused"); + } +} + +impl Device for Ioc4 { + fn step(&self, _cycles: u64) {} + + /// Start the serial receive threads and arm the timers for any deadline + /// the chip already holds. + fn start(&self) { + if self.sh.running.swap(true, Ordering::SeqCst) { + return; + } + Device::start(&*self.sh.st.lock().ps2); + let mut threads = self.threads.lock(); + for p in 0..PORTS { + let me = self.sh.me.clone(); + threads.push( + std::thread::Builder::new() + .name(format!("IOC4-RX-{p}")) + .spawn(move || Shared::rx_loop(me, p)) + .expect("spawn ioc4 receive thread"), + ); + } + drop(threads); + let want = self.sh.st.lock().deadlines(); + self.sh.arm(want); + } + + /// Remove the timers, wait out a callback in flight and join the threads. + /// Nothing touches memory or the line afterwards, until `start`. + fn stop(&self) { + if !self.sh.running.swap(false, Ordering::SeqCst) { + return; + } + self.sh.timers.cancel_all(); + drop(self.sh.gate.lock()); + let mut threads = self.threads.lock(); + for t in threads.iter() { + t.thread().unpark(); + } + for t in threads.drain(..) { + let _ = t.join(); + } + drop(threads); + Device::stop(&*self.sh.st.lock().ps2); + } + + fn is_running(&self) -> bool { + self.sh.running.load(Ordering::SeqCst) + } + + fn get_clock(&self) -> u64 { + 0 + } +} + +impl Resettable for Ioc4 { + fn power_on(&self) { + self.reset(); + } +} + +impl Drop for Ioc4 { + fn drop(&mut self) { + Device::stop(&*self); + } +} + +impl PciFunction for Ioc4 { + fn state(&self) -> &PciFunctionState { + &self.sh.pci + } + + fn bar_read(&self, bar: usize, off: u32, width: usize) -> u32 { + if bar != 0 || off >= BAR_SIZE { + return 0; + } + if let Some((p, r)) = uart_decode(off) { + if width != 1 { + Ioc4::refuse("UART read", width, off); + return 0; + } + return u32::from(self.sh.with_st(|st| st.uart_read(p, r))); + } + if let Some(clock) = self.clock.get().filter(|_| in_clock(off)) { + if width != 1 { + Ioc4::refuse("clock read", width, off); + return 0; + } + return u32::from(clock.read8(u64::from(off - reg::CLOCK)).data); + } + if in_kbc(off) { + if width != 4 || off & 3 != 0 { + Ioc4::refuse("KBC read", width, off); + return 0; + } + return self.sh.with_st(|st| st.kbc_read(off - reg::KBC)).swap_bytes(); + } + if width != 4 || off & 3 != 0 { + Ioc4::refuse("read", width, off); + return 0; + } + self.sh.with_st(|st| st.read32(off)).swap_bytes() + } + + fn bar_write(&self, bar: usize, off: u32, val: u32, width: usize) { + if bar != 0 || off >= BAR_SIZE { + return; + } + if let Some((p, r)) = uart_decode(off) { + if width != 1 { + Ioc4::refuse("UART write", width, off); + return; + } + self.sh.with_st(|st| st.uart_write(p, r, val as u8)); + return; + } + if let Some(clock) = self.clock.get().filter(|_| in_clock(off)) { + if width != 1 { + Ioc4::refuse("clock write", width, off); + return; + } + clock.write8(u64::from(off - reg::CLOCK), val as u8); + return; + } + if in_kbc(off) { + if width != 4 || off & 3 != 0 { + Ioc4::refuse("KBC write", width, off); + return; + } + self.sh.with_st(|st| st.kbc_write(off - reg::KBC, val.swap_bytes())); + return; + } + if width != 4 || off & 3 != 0 { + Ioc4::refuse("write", width, off); + return; + } + self.sh.with_st(|st| st.write32(off, val.swap_bytes())); + } + + /// The chip's DMA, and the PCI interrupt pin: any enabled source. + fn attached(&self) { + if let Some(dma) = self.sh.pci.dma() { + self.sh.st.lock().dma = Some(dma.clone()); + } + if let Some(line) = self.sh.pci.irq() { + { + let _order = self.sh.line_order.lock(); + line.set(self.sh.line_level.load(Ordering::SeqCst)); + *self.sh.line.lock() = Some(line.clone()); + } + self.sh.sync_line(); + } + } +} + +#[cfg(test)] +#[path = "ioc4_tests.rs"] +mod tests; diff --git a/src/dev/ioc4_tests.rs b/src/dev/ioc4_tests.rs new file mode 100644 index 000000000..c2530d0dd --- /dev/null +++ b/src/dev/ioc4_tests.rs @@ -0,0 +1,1682 @@ +//! IOC4 register-contract tests. +//! +//! Accesses go through `PciFunction` the way the bridge makes them: IOC4 +//! orders its own bytes, so a register is at its address and a UART register +//! is a byte access at its window. Each test builds only the chip and a +//! block of bus memory for the rings. +//! +//! Ported from tests/qtest/sgi-ioc4-test.c in jamesbraid/qemu-sgi-origami. +//! That file drives the chip through an XBridge with chardev sockets and the +//! virtual clock; here the sockets are a `Wire` per port and time is the host +//! clock, so timers are given a few milliseconds and `service` plays the part +//! of the main loop. + +use super::*; +use crate::dev::ds1744::{self, Ds1744, YearBase}; +use crate::irq::LevelProbe; +use std::sync::atomic::{AtomicI64, Ordering as AtomicOrdering}; +use crate::pci::PciDmaStatus; +use std::collections::VecDeque; + +const MEM_SIZE: usize = 0x10000; +/// Where the rings sit in bus memory. +const RINGS: u64 = 0x4000; +/// An address no memory answers. +const BAD_BASE: u64 = 0x20_0000; +const TX_VALID: u8 = 0x40; +const TX_EXPLICIT: u8 = 0x20; +const TX_MCR: u8 = 0x80; +const TX_DELAY: u8 = 0xc0; +/// Received status from the line: data and modem valid, CTS and DCD up. +const RX_FROM_LINE: u8 = 0xf0; +/// Received status looped back: RTS returns as CTS, and OUT2 clear is no DCD. +const RX_LOOPBACK: u8 = 0xd0; +const ENABLE: u32 = 1 << 28; +const RESET: u32 = 1 << 31; +const THRESHOLD: u32 = 1; +const ARM: u32 = 1 << 31; +const DRAIN: u32 = 1 << 27; +const SHADOW_OUTPUTS: u32 = 0x1f00_0000; +const ERROR_VALID: u32 = 1; +const ERROR_MULTIPLE: u32 = 1 << 5; + +/// A sink for a serial line: what the guest sent, and what the host types. +#[derive(Default)] +struct Wire { + out: std::sync::Mutex>, + inp: std::sync::Mutex>, +} + +impl SerialBackend for Wire { + fn send_byte(&self, b: u8) { + self.out.lock().unwrap().push(b); + } + fn recv_byte(&self) -> std::io::Result { + self.inp + .lock() + .unwrap() + .pop_front() + .ok_or_else(|| std::io::Error::new(std::io::ErrorKind::WouldBlock, "empty")) + } +} + +/// Bus memory, as bytes at bus addresses. Addresses past its end abort. +struct TestMem(std::sync::Mutex>); + +impl PciDma for TestMem { + fn dma_read(&self, addr: u64, buf: &mut [u8]) -> PciDmaStatus { + let m = self.0.lock().unwrap(); + let a = addr as usize; + if addr > u64::from(u32::MAX) || a + buf.len() > m.len() { + buf.fill(0); + return PciDmaStatus::MasterAbort; + } + buf.copy_from_slice(&m[a..a + buf.len()]); + PciDmaStatus::Ok + } + fn dma_write(&self, addr: u64, data: &[u8]) -> PciDmaStatus { + let mut m = self.0.lock().unwrap(); + let a = addr as usize; + if addr > u64::from(u32::MAX) || a + data.len() > m.len() { + return PciDmaStatus::MasterAbort; + } + m[a..a + data.len()].copy_from_slice(data); + PciDmaStatus::Ok + } +} + +struct Rig { + c: Ioc4, + mem: Arc, + wire: [Arc; PORTS], + pin: Arc, +} + +impl Rig { + fn new() -> Rig { + Rig::with(Ioc4Config::bare(Ioc4Clock::Mhz33)) + } + + fn with(cfg: Ioc4Config) -> Rig { + let c = Ioc4::new(cfg); + let mem = Arc::new(TestMem(std::sync::Mutex::new(vec![0; MEM_SIZE]))); + c.state().connect_dma(mem.clone()); + let wire: [Arc; PORTS] = std::array::from_fn(|_| Arc::new(Wire::default())); + for (p, w) in wire.iter().enumerate() { + c.set_serial_backend(p, w.clone()); + } + let (pin, line) = LevelProbe::line(); + c.state().connect_irq(line); + c.attached(); + Rig { c, mem, wire, pin } + } + + /// A 32-bit register as the processor sees it through the self-ordered + /// slot: the BAR value is the address-order bytes read little-endian. + fn r32(&self, a: u32) -> u32 { + self.c.bar_read(0, a, 4).swap_bytes() + } + fn w32(&self, a: u32, v: u32) { + self.c.bar_write(0, a, v.swap_bytes(), 4) + } + fn r8(&self, a: u32) -> u8 { + self.c.bar_read(0, a, 1) as u8 + } + fn w8(&self, a: u32, v: u8) { + self.c.bar_write(0, a, u32::from(v), 1) + } + + /// UART register `r` of port `p`. + fn ur(&self, p: usize, r: u32) -> u8 { + self.r8(reg::UART + reg::UART_STRIDE * p as u32 + r) + } + fn uw(&self, p: usize, r: u32, v: u8) { + self.w8(reg::UART + reg::UART_STRIDE * p as u32 + r, v) + } + + /// A port's DMA register. + fn sr(&self, p: usize, off: u32) -> u32 { + self.r32(reg::SDMA_PORT + reg::SDMA_PORT_STRIDE * p as u32 + off) + } + fn sw(&self, p: usize, off: u32, v: u32) { + self.w32(reg::SDMA_PORT + reg::SDMA_PORT_STRIDE * p as u32 + off, v) + } + + /// Point a pair's rings at `base` with `size` byte rings. + fn pair_base(&self, pair: u32, base: u64, size: u32) { + let lo = reg::SDMA_BASE + 8 * pair; + self.w32(lo + 4, (base >> 32) as u32); + self.w32(lo, base as u32 | u32::from(size == 0x1000)); + } + + /// Where port `p`'s ring for `dir` (0 transmit, 1 receive) is, with its + /// pair based at `RINGS`. + fn ring(&self, p: usize, dir: u64, size: u64) -> usize { + (RINGS + ((p as u64 & 1) * 2 + dir) * size) as usize + } + + fn put(&self, at: usize, bytes: &[u8]) { + self.mem.0.lock().unwrap()[at..at + bytes.len()].copy_from_slice(bytes); + } + + fn get(&self, at: usize, n: usize) -> Vec { + self.mem.0.lock().unwrap()[at..at + n].to_vec() + } + + /// One transmit entry in port `p`'s ring, `at` bytes in. + fn tx_entry(&self, p: usize, size: u64, at: usize, data: [u8; 4], control: [u8; 4]) { + let ring = self.ring(p, 0, size) + at; + self.put(ring, &data); + self.put(ring + 4, &control); + } + + /// Wait out `ms` of host time, then play the timers. + fn settle(&self, ms: u64) { + std::thread::sleep(Duration::from_millis(ms)); + self.c.service(); + } + + /// Play the timers until `done`, a millisecond of host time apart. + fn until(&self, what: &str, done: impl Fn(&Rig) -> bool) { + for _ in 0..3000 { + self.c.service(); + if done(self) { + return; + } + std::thread::sleep(Duration::from_millis(1)); + } + panic!("timed out waiting for {what}"); + } + + /// Divisor one: a character crosses the line in about five microseconds + /// instead of a millisecond, so a long loopback run stays short. + fn fast_line(&self, p: usize) { + self.uw(p, 3, 0x80); + self.uw(p, 0, 1); + self.uw(p, 1, 0); + self.uw(p, 3, 0x03); + } + + fn sent(&self, p: usize) -> Vec { + self.wire[p].out.lock().unwrap().clone() + } + + fn type_in(&self, p: usize, bytes: &[u8]) { + self.wire[p].inp.lock().unwrap().extend(bytes.iter().copied()); + self.c.service(); + } +} + +// --------------------------------------------------------------------------- +// PCI function +// --------------------------------------------------------------------------- + +/// Ports test_identity_and_reset: vendor, device, revision 0x53, the +/// emulator's class and interrupt pin, and a 1 MiB BAR. +#[test] +fn identity_and_bar() { + let r = Rig::new(); + let cfg = r.c.state().config(); + assert_eq!(cfg.read(0), 0x100a_10a9); + assert_eq!(cfg.read(2), 0x00ff_0053); + assert_eq!(cfg.interrupt_pin(), PCI_INTA); + cfg.write(4, 0xffff_ffff); + assert_eq!(cfg.read(4), 0xfff0_0000, "one megabyte of memory space"); +} + +/// The IP53 status policy: 0x0220 survives every write and a reset, and an +/// instance without the policy keeps its own value. +#[test] +fn status_follows_the_board() { + let r = Rig::with(Ioc4Config::tezro()); + let cfg = r.c.state().config(); + assert_eq!(cfg.read(1) >> 16, 0x0220); + cfg.write(1, 0x0000_0006); + assert_eq!(cfg.read(1), 0x0220_0006); + cfg.write(1, 0xffff_0000); + assert_eq!(cfg.read(1) >> 16, 0x0220, "a status write of ones leaves the read-only fields"); + r.c.reset(); + assert_eq!(cfg.read(1), 0x0220_0000); + + let bare = Rig::new(); + assert_eq!(bare.c.state().config().read(1) >> 16, 0); +} + +/// Ports the reset half of test_identity_and_reset: the driver's +/// disable-and-acknowledge is not a reset, but a reset clears what it left. +#[test] +fn reset_clears_the_interrupt_state_and_the_ports() { + let r = Rig::new(); + r.w32(reg::SERIAL_ENABLE_SET, int::pass(0)); + r.w32(reg::OTHER_ENABLE_SET, 0x40); + r.uw(0, 3, 0x80); + r.uw(0, 0, 0x34); + r.uw(0, 3, 0x03); + r.uw(0, 1, 0x02); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(0), int::pass(0)); + assert!(r.pin.level()); + + r.c.reset(); + assert_eq!(r.r32(reg::SERIAL_ENABLE_SET), 0); + assert_eq!(r.r32(reg::OTHER_ENABLE_SET), 0); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(0), 0); + assert!(!r.pin.level()); + assert_eq!(r.ur(0, 3), 0, "line control"); + assert_eq!(r.ur(0, 1), 0, "interrupt enable"); + assert_eq!(r.ur(0, 2) & 0xc0, 0, "FIFOs off"); + // The driver's own initialisation: raw pending, separate from enabled. + r.uw(0, 1, 2); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(0), int::pass(0)); + assert!(!r.pin.level(), "pending but not enabled"); +} + +/// The pin follows the enabled sources of either word. Ports the IRQ half +/// of test_identity_and_reset. +#[test] +fn the_pin_follows_enabled_sources() { + let r = Rig::new(); + r.uw(0, 1, 0x02); + assert!(!r.pin.level()); + r.w32(reg::SERIAL_ENABLE_SET, int::pass(0)); + assert!(r.pin.level()); + r.w32(reg::SERIAL_ENABLE_CLR, int::pass(0)); + assert!(!r.pin.level()); + r.w32(reg::OTHER_ENABLE_SET, int::dma_error(1)); + assert!(!r.pin.level(), "nothing pending in the other word"); +} + +// --------------------------------------------------------------------------- +// The four 16550s and the local interrupt controller +// --------------------------------------------------------------------------- + +/// Ports test_uart_and_clock: the misc-control bit names the board's clock, +/// writes to it are ignored, and each port's input is the clock over 16. +#[test] +fn clock_select_reads_back_and_sets_the_baud_base() { + for (clock, bit, hz) in [(Ioc4Clock::Mhz33, 0, 33_333_333u32), (Ioc4Clock::Mhz66, 1, 66_666_667)] { + let r = Rig::with(Ioc4Config::bare(clock)); + assert_eq!(r.r32(reg::MISC_CONTROL) & 1, bit); + r.w32(reg::MISC_CONTROL, u32::from(bit == 0)); + assert_eq!(r.r32(reg::MISC_CONTROL) & 1, bit, "the clock is the board's"); + for p in 0..PORTS { + assert_eq!(r.c.sh.st.lock().uart[p].baudbase(), hz / 16); + } + } +} + +/// Ports test_uart_and_clock's per-port register loop. +#[test] +fn every_port_has_its_own_registers() { + let r = Rig::new(); + for n in 0..PORTS { + let n8 = n as u8; + assert_eq!(r.ur(n, 5) & 0x60, 0x60, "transmitter empty after reset"); + r.uw(n, 3, 0x80); + r.uw(n, 0, 0x31 + n8); + r.uw(n, 1, 0x12 + n8); + assert_eq!(r.ur(n, 0), 0x31 + n8, "divisor low"); + assert_eq!(r.ur(n, 1), 0x12 + n8, "divisor high"); + r.uw(n, 3, 3); + r.uw(n, 2, 1); + assert_eq!(r.ur(n, 2) & 0xc0, 0xc0, "FIFOs enabled"); + r.uw(n, 2, 7); + } + for n in 0..PORTS { + assert_eq!(r.ur(n, 3), 3, "no port inherited another's line control"); + } +} + +/// Ports test_interrupts: the enable-set and enable-clear aliases, raw +/// pending against enabled, and one port's source isolated from the others'. +#[test] +fn serial_interrupts_mask_pending_and_acknowledge() { + let r = Rig::new(); + r.w32(reg::SERIAL_ENABLE_CLR, u32::MAX); + assert_eq!(r.r32(reg::SERIAL_ENABLE_SET), 0); + r.w32(reg::OTHER_ENABLE_CLR, u32::MAX); + assert_eq!(r.r32(reg::OTHER_ENABLE_SET), 0); + r.w32(reg::SERIAL_PENDING, u32::MAX); + r.w32(reg::OTHER_PENDING, u32::MAX); + for n in 0..PORTS { + r.uw(n, 1, 2); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(n), int::pass(n)); + assert!(!r.pin.level(), "pending but masked"); + r.w32(reg::SERIAL_ENABLE_SET, int::pass(n)); + assert_eq!(r.r32(reg::SERIAL_ENABLE_SET), int::pass(n)); + assert!(r.pin.level()); + r.w32(reg::SERIAL_PENDING, int::pass(n)); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(n), int::pass(n), "a level ignores the acknowledge"); + assert!(r.pin.level()); + assert_eq!(r.ur(n, 2) & 0xf, 2, "transmit-empty cause"); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(n), 0, "the identification read quieted it"); + assert!(!r.pin.level()); + r.w32(reg::SERIAL_ENABLE_CLR, int::pass(n)); + assert_eq!(r.r32(reg::SERIAL_ENABLE_SET), 0); + r.uw(n, 1, 0); + } + for n in 0..PORTS { + r.uw(n, 1, 2); + r.w32(reg::SERIAL_ENABLE_SET, int::pass(n)); + } + let all = (0..PORTS).fold(0, |m, n| m | int::pass(n)); + assert_eq!(r.r32(reg::SERIAL_PENDING) & all, all); + r.ur(0, 2); + assert_eq!(r.r32(reg::SERIAL_PENDING) & all, all & !int::pass(0), "only port 0 quieted"); + assert!(r.pin.level()); + assert_eq!(r.r32(reg::OTHER_PENDING), 0, "no ATA or error source"); + r.w32(reg::OTHER_ENABLE_SET, 0x40); + assert_eq!(r.r32(reg::OTHER_ENABLE_SET), 0x40); + r.w32(reg::OTHER_ENABLE_CLR, 0x40); + assert_eq!(r.r32(reg::OTHER_ENABLE_SET), 0); +} + +/// Ports test_uart_pass_through_is_a_level: a write cannot clear a UART's +/// source, only the UART can. +#[test] +fn uart_pass_through_follows_the_uart() { + let r = Rig::new(); + r.w32(reg::SERIAL_PENDING, u32::MAX); + r.uw(0, 1, 2); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(0), int::pass(0)); + r.w32(reg::SERIAL_PENDING, int::pass(0)); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(0), int::pass(0)); + r.uw(1, 1, 2); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(1), int::pass(1)); + assert_eq!(r.ur(1, 2) & 0xf, 2); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(1), 0, "quieted by the UART alone, it falls"); +} + +/// Ports test_pio_fifo: a loopback burst fills the sixteen-byte FIFO and +/// comes back in order, and a receive FIFO reset drops what is held. +#[test] +fn programmed_io_goes_through_the_fifo() { + let r = Rig::new(); + for n in 0..PORTS { + r.uw(n, 3, 3); + r.uw(n, 2, 1); + r.uw(n, 4, 0x10); + for i in 0..16 { + r.uw(n, 0, 0x40 + i); + } + assert_eq!(r.ur(n, 5) & 1, 1); + for i in 0..16 { + assert_eq!(r.ur(n, 0), 0x40 + i); + } + assert_eq!(r.ur(n, 5) & 1, 0); + r.uw(n, 0, 0xaa); + assert_eq!(r.ur(n, 5) & 1, 1); + r.uw(n, 2, 3); + assert_eq!(r.ur(n, 5) & 1, 0, "a receive FIFO reset drops it"); + } +} + +/// Ports test_uart_rejects_aligned_wide_write: a wider access to a UART +/// window is refused whole and reaches no register. +#[test] +fn uart_windows_take_bytes_only() { + let r = Rig::new(); + r.uw(0, 4, 0x10); + assert_eq!(r.ur(0, 4), 0x10); + r.c.bar_write(0, reg::UART, 0x5a5a_5a5a, 4); + assert_eq!(r.ur(0, 5) & 1, 0, "no character arrived"); + r.c.bar_write(0, reg::UART, 0x5a5a, 2); + assert_eq!(r.ur(0, 5) & 1, 0); + assert_eq!(r.c.bar_read(0, reg::UART, 4), 0); + r.uw(0, 0, 0x5a); + assert_eq!(r.ur(0, 5) & 1, 1); + assert_eq!(r.ur(0, 0), 0x5a); +} + +/// Ports test_ata_readback_rejects_width: the control registers take 32-bit +/// accesses and nothing narrower, and the words around the latch are not it. +#[test] +fn control_registers_take_words_only() { + let r = Rig::with(Ioc4Config::tezro()); + r.w32(reg::ATA_READBACK, 0x1234_5678); + assert_eq!(r.c.bar_read(0, reg::ATA_READBACK, 1), 0); + assert_eq!(r.c.bar_read(0, reg::ATA_READBACK, 2), 0); + r.c.bar_write(0, reg::ATA_READBACK, 0xaa, 1); + r.c.bar_write(0, reg::ATA_READBACK, 0xbbcc, 2); + assert_eq!(r.r32(reg::ATA_READBACK), 0x1234_5678); + assert_eq!(r.r32(reg::ATA_READBACK - 4), 0); + assert_eq!(r.r32(reg::ATA_READBACK + 4), 0); + r.w32(reg::ATA_READBACK - 4, 0xaabb_ccdd); + r.w32(reg::ATA_READBACK + 4, 0x5566_7788); + assert_eq!(r.r32(reg::ATA_READBACK), 0x1234_5678); +} + +/// Ports test_ata_readback_latch and test_ata_readback_default_unsupported: +/// every one-hot value and all ones read back, a reset clears the word, and +/// a chip whose board did not ask for it reads zero. +#[test] +fn ata_readback_is_a_latch_where_the_board_asks_for_it() { + let r = Rig::with(Ioc4Config::tezro()); + let before = r.get(RINGS as usize, 8); + for bit in 0..32 { + r.w32(reg::ATA_READBACK, 1 << bit); + assert_eq!(r.r32(reg::ATA_READBACK), 1 << bit); + } + r.w32(reg::ATA_READBACK, u32::MAX); + assert_eq!(r.r32(reg::ATA_READBACK), u32::MAX); + assert_eq!(r.get(RINGS as usize, 8), before, "no DMA behind it"); + assert!(!r.pin.level()); + r.c.reset(); + assert_eq!(r.r32(reg::ATA_READBACK), 0); + + let bare = Rig::new(); + bare.w32(reg::ATA_READBACK, u32::MAX); + assert_eq!(bare.r32(reg::ATA_READBACK), 0); +} + +/// Ports test_serial_idle_status_default_pio: with every transfer finishing +/// inside the access that starts it, the DMA status reads idle, read-only. +#[test] +fn serial_control_reads_idle() { + let r = Rig::new(); + assert_eq!(r.r32(reg::SERIAL_CONTROL), 0x80); + r.w32(reg::SERIAL_CONTROL, 0); + assert_eq!(r.r32(reg::SERIAL_CONTROL), 0x80); +} + +// --------------------------------------------------------------------------- +// Serial ring DMA +// --------------------------------------------------------------------------- + +/// Ports test_dma_tx_control_entries: a modem-control slot writes the 16550's +/// register, a delay slot holds the walk for its milliseconds, and an +/// explicit-interrupt flag raises the port's completion bit. +#[test] +fn tx_control_entries_drive_the_uart() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.tx_entry(0, 0x1000, 0, [0x1f, 3, 0, 0], [TX_MCR | TX_EXPLICIT, TX_DELAY | TX_EXPLICIT, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.sr(0, sdma_off::SHADOW) & SHADOW_OUTPUTS, SHADOW_OUTPUTS, "the modem control reached the UART"); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::serial(0, 7), int::serial(0, 7), "the first slot's completion"); + r.w32(reg::SERIAL_PENDING, int::serial(0, 7)); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 0, "the delay holds the entry"); + assert_eq!(r.sr(0, sdma_off::CONTROL) & (1 << 16), 1 << 16, "and reads busy"); + r.settle(10); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 8); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::serial(0, 7), int::serial(0, 7), "the delay's completion"); +} + +/// Ports test_dma_tx_wrap_1k: the consumer wraps at the 1 KiB ring's last +/// entry, and the empty source comes back with it. +#[test] +fn tx_ring_wraps_at_one_kilobyte() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::TX_PRODUCE, 0x3f8); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 0x3f8, "empty entries are skipped"); + r.tx_entry(0, 0x400, 0x3f8, [0x5e, 0, 0, 0], [TX_VALID, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 0); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 0, "wrapped"); + assert_eq!(r.sent(0), [0x5e]); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::serial(0, 0), int::serial(0, 0), "transmit ring empty"); +} + +/// Ports test_dma_tx_empty_follows_the_ring: transmit-ring-empty is a level +/// that is clear while an entry is in the engine and deaf to a write. +#[test] +fn tx_empty_follows_the_ring() { + let r = Rig::new(); + let empty = int::serial(0, 0); + r.pair_base(0, RINGS, 0x1000); + assert_eq!(r.r32(reg::SERIAL_PENDING) & empty, 0, "off with ring DMA off"); + r.sw(0, sdma_off::CONTROL, ENABLE); + assert_eq!(r.r32(reg::SERIAL_PENDING) & empty, empty); + r.w32(reg::SERIAL_PENDING, empty); + assert_eq!(r.r32(reg::SERIAL_PENDING) & empty, empty); + r.tx_entry(0, 0x1000, 0, [3, 0, 0, 0], [TX_DELAY, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.r32(reg::SERIAL_PENDING) & empty, 0); + r.settle(10); + assert_eq!(r.r32(reg::SERIAL_PENDING) & empty, empty); +} + +/// Ports test_dma_rx_drain_and_threshold: characters wait in the packer +/// until a drain posts them, with the line's modem state, and the high-water +/// source follows the entries waiting. +#[test] +fn rx_drain_posts_the_packer_and_raises_the_threshold() { + let r = Rig::new(); + let high = int::serial(0, 2); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD); + // A long reload, so the drain and not the timer posts the entry. + r.sw(0, sdma_off::RX_TIMER, 0xfff); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.type_in(0, b"abc"); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE) & 0xff8, 0); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD | DRAIN); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE) & 0xff8, 8); + let entry = r.get(r.ring(0, 1, 0x1000), 8); + assert_eq!(entry, [b'a', b'b', b'c', 0, RX_FROM_LINE, RX_FROM_LINE, RX_FROM_LINE, 0]); + assert_eq!(r.r32(reg::SERIAL_PENDING) & high, high); + r.w32(reg::SERIAL_PENDING, high); + assert_eq!(r.r32(reg::SERIAL_PENDING) & high, high, "a level"); + r.sw(0, sdma_off::RX_CONSUME, 8); + assert_eq!(r.r32(reg::SERIAL_PENDING) & high, 0, "the consumer made room"); +} + +/// Ports test_dma_rx_timer_expiry: the timer posts a lone character when +/// its reload runs out and raises the expiry and threshold sources. +#[test] +fn rx_timer_posts_a_lone_character() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD); + r.sw(0, sdma_off::RX_TIMER, 2); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.type_in(0, &[0x6d]); + r.settle(10); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE) & 0xff8, 8); + let entry = r.get(r.ring(0, 1, 0x1000), 8); + assert_eq!((entry[0], entry[4]), (0x6d, RX_FROM_LINE)); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::serial(0, 3), int::serial(0, 3), "timer expired"); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::serial(0, 2), int::serial(0, 2), "threshold"); +} + +/// Ports test_dma_threshold_is_a_level. +#[test] +fn threshold_is_a_level() { + let r = Rig::new(); + let high = int::serial(0, 2); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, THRESHOLD); + r.sw(0, sdma_off::RX_PRODUCE, 8); + assert_eq!(r.r32(reg::SERIAL_PENDING) & high, high); + r.w32(reg::SERIAL_PENDING, high); + assert_eq!(r.r32(reg::SERIAL_PENDING) & high, high); + r.sw(0, sdma_off::RX_CONSUME, 8); + assert_eq!(r.r32(reg::SERIAL_PENDING) & high, 0); +} + +/// Ports test_dma_channel_reset_keeps_settings: a channel reset zeroes the +/// four indices and keeps the timer reload, the UART's registers and what +/// the write that reset it stored. +#[test] +fn channel_reset_keeps_settings() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::RX_TIMER, 0x140); + r.uw(0, 4, 0x03); + r.sw(0, sdma_off::TX_PRODUCE, 0x40); + r.sw(0, sdma_off::CONTROL, RESET | 5); + assert_eq!(r.sr(0, sdma_off::TX_PRODUCE), 0); + assert_eq!(r.sr(0, sdma_off::RX_TIMER) & 0xfff, 0x140); + assert_eq!(r.sr(0, sdma_off::SHADOW) & SHADOW_OUTPUTS, 0x0300_0000); + assert_eq!(r.sr(0, sdma_off::CONTROL), 5); +} + +/// Ports test_dma_full_reset_clears_shadow: the shadow mirrors the UART and +/// ignores a write, and a chip reset puts it back to the UART's reset +/// modem control. +#[test] +fn shadow_mirrors_the_uart() { + let r = Rig::new(); + r.uw(0, 4, 0x13); + assert_eq!(r.sr(0, sdma_off::SHADOW) & SHADOW_OUTPUTS, 0x1300_0000); + r.sw(0, sdma_off::SHADOW, 0x1f00_0000); + assert_eq!(r.sr(0, sdma_off::SHADOW) & SHADOW_OUTPUTS, 0x1300_0000); + r.c.reset(); + let mcr = u32::from(r.ur(0, 4)); + assert_eq!(r.sr(0, sdma_off::SHADOW) & SHADOW_OUTPUTS, mcr << 24); +} + +/// Ports test_dma_loopback_tx_mcr in miniature: with the UART looped back +/// the engine's transmitted characters return through the receive ring with +/// the looped modem state (DTR and RTS set, so RTS comes back as CTS), and +/// none reach the line. +#[test] +fn loopback_returns_through_the_rings() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.fast_line(0); + r.uw(0, 4, 0x13); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD); + r.tx_entry(0, 0x400, 0, *b"wxyz", [TX_VALID; 4]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + r.until("the receive entry", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 8); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 8); + assert_eq!(r.get(r.ring(0, 1, 0x400), 8), [b'w', b'x', b'y', b'z', RX_LOOPBACK, RX_LOOPBACK, RX_LOOPBACK, RX_LOOPBACK]); + assert!(r.sent(0).is_empty(), "nothing reached the line"); +} + +/// A looped-back character takes a character time to reach the receiver, and +/// the walk waits for it: at 9600 baud the second character of an entry is +/// not sent for a millisecond. Ports the "still on the line" case of +/// test_dma_loopback_tx_mcr. +#[test] +fn a_looped_back_character_takes_a_character_time() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.uw(0, 4, 0x13); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD); + r.tx_entry(0, 0x400, 0, *b"abcd", [TX_VALID; 4]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert!(r.c.sh.st.lock().sdma.loop_deadline(0).is_some(), "the first character is on the line"); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 0, "the entry is not done"); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8, 0, "and nothing has been posted"); + r.until("the entry", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 8); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 8); +} + +/// Ports test_dma_loopback_pio_mcr's loopback half: a full 127-entry transmit +/// ring crosses the line and fills the receive ring, entry for entry. +#[test] +fn loopback_fills_the_receive_ring() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.fast_line(0); + r.uw(0, 4, 0x13); + assert_eq!(r.ur(0, 4), 0x13); + assert_eq!(r.sr(0, sdma_off::SHADOW) & SHADOW_OUTPUTS, 0x1300_0000); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD); + let expected = fill_loopback_ring(&r); + r.until("the transmit ring", |r| r.sr(0, sdma_off::TX_CONSUME) & 0x3f8 == 0x3f8); + r.until("the receive ring", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 0x3f8); + assert_eq!((r.sr(0, sdma_off::TX_PRODUCE), r.sr(0, sdma_off::TX_CONSUME)), (0x3f8, 0x3f8)); + assert_eq!(r.sr(0, sdma_off::RX_CONSUME), 0); + assert_eq!(r.get(r.ring(0, 1, 0x400), 0x3f8), expected); + assert!(r.sent(0).is_empty()); +} + +/// Ports test_dma_loopback_tx_mcr: a modem-control slot ahead of the +/// characters writes the UART's own register, and its characters then loop +/// back, entry by entry, the first holding three. +#[test] +fn a_modem_control_slot_leads_a_looped_back_run() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.fast_line(0); + r.uw(0, 4, 0); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD); + let mut expected = vec![0u8; 0x3f8]; + let mut count = 0usize; + for i in 0..127usize { + let (data, control): ([u8; 4], [u8; 4]) = if i == 0 { + ([0x13, 0, 1, 2], [TX_MCR, TX_VALID, TX_VALID, TX_VALID]) + } else { + (std::array::from_fn(|j| (i * 4 + j - 1) as u8), [TX_VALID; 4]) + }; + for &byte in &data[usize::from(i == 0)..] { + expected[count / 4 * 8 + count % 4] = byte; + expected[count / 4 * 8 + 4 + count % 4] = RX_LOOPBACK; + count += 1; + } + r.tx_entry(0, 0x400, i * 8, data, control); + } + assert_eq!(count, 507); + r.sw(0, sdma_off::TX_PRODUCE, 0x3f8); + r.until("the transmit ring", |r| r.sr(0, sdma_off::TX_CONSUME) & 0x3f8 == 0x3f8); + r.until("the last full entry", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 0x3f0); + assert_eq!(r.ur(0, 4), 0x13, "the modem-control slot wrote the UART's register"); + assert_eq!(r.sr(0, sdma_off::SHADOW) & SHADOW_OUTPUTS, 0x1300_0000); + // Three characters wait in the packer for a drain. + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD | DRAIN); + r.until("the drain", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 0x3f8); + assert_eq!(r.get(r.ring(0, 1, 0x400), 0x3f8), expected); + assert!(r.sent(0).is_empty()); +} + +/// Ports test_dma_loopback_shadow_reset: a write to the shadow changes +/// nothing, and a channel reset leaves the outputs as they were. +#[test] +fn the_shadow_ignores_writes_and_a_channel_reset() { + let r = Rig::new(); + r.uw(0, 4, 0); + r.sw(0, sdma_off::SHADOW, SHADOW_OUTPUTS); + assert_eq!(r.ur(0, 4), 0); + r.uw(0, 4, 0x13); + r.sw(0, sdma_off::SHADOW, 0x1300_0000); + assert_eq!(r.ur(0, 4), 0x13); + r.sw(0, sdma_off::CONTROL, RESET); + assert_eq!(r.sr(0, sdma_off::SHADOW) & SHADOW_OUTPUTS, 0x1300_0000); + assert_eq!(r.ur(0, 4), 0x13); +} + +/// A full 127-entry transmit ring of valid characters; the receive ring it +/// should fill. +fn fill_loopback_ring(r: &Rig) -> Vec { + let mut expected = vec![0u8; 0x3f8]; + for i in 0..127usize { + let data: [u8; 4] = std::array::from_fn(|j| (i * 4 + j) as u8); + expected[i * 8..i * 8 + 4].copy_from_slice(&data); + expected[i * 8 + 4..i * 8 + 8].fill(RX_LOOPBACK); + r.tx_entry(0, 0x400, i * 8, data, [TX_VALID; 4]); + } + r.sw(0, sdma_off::TX_PRODUCE, 0x3f8); + expected +} + +/// Ports test_dma_requester_ids' use of both pairs: ports 2 and 3 take their +/// rings from the second pair's base, apart from the first pair's. +#[test] +fn second_pair_has_its_own_base() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.pair_base(1, RINGS + 0x2000, 0x400); + assert_eq!(r.r32(reg::SDMA_BASE), RINGS as u32); + assert_eq!(r.r32(reg::SDMA_BASE + 8), (RINGS + 0x2000) as u32); + for p in [0usize, 2, 3] { + r.sw(p, sdma_off::CONTROL, ENABLE); + } + let at = |pair: u64, p: u64| (RINGS + pair * 0x2000 + (p & 1) * 2 * 0x400) as usize; + r.put(at(1, 2), &[b'c', 0, 0, 0, TX_VALID, 0, 0, 0]); + r.put(at(1, 3), &[b'd', 0, 0, 0, TX_VALID, 0, 0, 0]); + r.put(at(0, 0), &[b'a', 0, 0, 0, TX_VALID, 0, 0, 0]); + r.sw(3, sdma_off::TX_PRODUCE, 8); + r.sw(2, sdma_off::TX_PRODUCE, 8); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_eq!((r.sent(0), r.sent(1), r.sent(2), r.sent(3)), (b"a".to_vec(), vec![], b"c".to_vec(), b"d".to_vec())); +} + +/// Ports test_dma_error_capture_and_reset: the first failed transfer +/// captures its address, a second sets the multiple bit, the port's source +/// is in the other word, and a channel reset recovers the port but leaves +/// the capture and the source for software to acknowledge. +#[test] +fn a_failed_ring_transfer_is_captured_and_halts_the_port() { + let r = Rig::new(); + let bad = BAD_BASE + (1 << 32); + let low = BAD_BASE as u32 | ERROR_VALID; + r.pair_base(0, bad, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 0, "the fetch failed"); + assert_eq!(r.r32(reg::ERROR_LOW), low); + assert_eq!(r.r32(reg::ERROR_HIGH), 1); + assert_eq!(r.r32(reg::OTHER_PENDING) & int::dma_error(0), int::dma_error(0)); + + r.sw(0, sdma_off::CONTROL, RESET); + assert_eq!(r.r32(reg::ERROR_LOW), low, "a channel reset leaves the capture"); + assert_eq!(r.r32(reg::OTHER_PENDING) & int::dma_error(0), int::dma_error(0)); + r.w32(reg::OTHER_PENDING, int::dma_error(0)); + assert_eq!(r.r32(reg::OTHER_PENDING) & int::dma_error(0), 0); + r.w32(reg::ERROR_LOW, 0); + assert_eq!(r.r32(reg::ERROR_LOW), low, "the capture is read-only"); + + r.sw(0, sdma_off::CONTROL, ENABLE); + r.type_in(0, &[0x73]); + r.sw(0, sdma_off::CONTROL, ENABLE | DRAIN); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE) & 0xff8, 0, "the post failed"); + assert_eq!(r.r32(reg::ERROR_LOW), low | ERROR_MULTIPLE); + assert_eq!(r.r32(reg::OTHER_PENDING) & int::dma_error(0), int::dma_error(0)); + r.c.reset(); + assert_eq!((r.r32(reg::ERROR_LOW), r.r32(reg::ERROR_HIGH), r.r32(reg::OTHER_PENDING)), (0, 0, 0)); +} + +/// Ports test_dma_requester_ids: a transmit fetch names twice the port and a +/// receive post one more, on every port. +#[test] +fn the_error_capture_names_the_requester() { + for p in 0..PORTS { + let r = Rig::new(); + r.pair_base(p as u32 / 2, BAD_BASE, 0x400); + r.sw(p, sdma_off::CONTROL, ENABLE); + r.sw(p, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.r32(reg::OTHER_PENDING), int::dma_error(p), "only this port's bit"); + assert_eq!(r.r32(reg::ERROR_LOW) >> 1 & 0xf, 2 * p as u32); + assert_eq!(r.r32(reg::ERROR_LOW) & ERROR_VALID, ERROR_VALID); + + r.c.reset(); + r.pair_base(p as u32 / 2, BAD_BASE, 0x400); + r.sw(p, sdma_off::CONTROL, ENABLE); + r.type_in(p, &[1, 2, 3, 4]); + assert_eq!(r.r32(reg::ERROR_LOW) >> 1 & 0xf, 2 * p as u32 + 1); + assert_eq!(r.sr(p, sdma_off::RX_PRODUCE), 0); + } +} + +/// Ports test_dma_port_fault_isolation_and_recovery: a fault on one port +/// leaves its pair's other port running, and the faulted port runs again +/// once reset. +#[test] +fn a_fault_halts_only_its_own_port() { + let r = Rig::new(); + r.pair_base(0, BAD_BASE, 0x400); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(1, sdma_off::CONTROL, ENABLE); + r.sw(1, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.r32(reg::ERROR_LOW) >> 1 & 0xf, 2); + assert_eq!(r.r32(reg::OTHER_PENDING), int::dma_error(1)); + + // The base is shared, so quiet both ports before changing it. + r.sw(0, sdma_off::CONTROL, 0); + r.sw(1, sdma_off::CONTROL, RESET); + assert_eq!((r.sr(1, sdma_off::TX_PRODUCE), r.sr(1, sdma_off::TX_CONSUME)), (0, 0)); + r.pair_base(0, RINGS, 0x400); + assert_eq!(r.r32(reg::SDMA_BASE), RINGS as u32); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.tx_entry(0, 0x400, 0, [b'p', 0, 0, 0], [TX_VALID, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.sent(0), b"p"); + assert_eq!(r.sr(1, sdma_off::TX_CONSUME), 0); + + r.sw(1, sdma_off::CONTROL, ENABLE); + r.tx_entry(1, 0x400, 0, [0x15, 0, 0, 0], [TX_MCR, 0, 0, 0]); + r.sw(1, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.sr(1, sdma_off::TX_CONSUME), 8, "the reset port runs"); + assert_eq!(r.sr(1, sdma_off::SHADOW) & SHADOW_OUTPUTS, 0x1500_0000); +} + +// --------------------------------------------------------------------------- +// The clock on byte-bus select 0 +// --------------------------------------------------------------------------- + +const CLOCK_CONTROL: u32 = reg::CLOCK + ds1744::CONTROL as u32; + +/// A chip with a clock on a stepped nanosecond counter, seeded on +/// 2024-01-01. +fn with_clock() -> (Rig, Arc) { + let r = Rig::new(); + let ns = Arc::new(AtomicI64::new(0)); + let n = ns.clone(); + let seed = 19_723 * 86_400; // 2024-01-01 + let clock = Ds1744::with_clock(seed, YearBase::Gregorian, Arc::new(move || n.load(AtomicOrdering::SeqCst))); + r.c.attach_clock(Arc::new(clock.unwrap())); + (r, ns) +} + +/// Set the calendar through the write bit: seconds, minutes, hours, date, +/// month and year, then the century that commits it. +fn set_time(r: &Rig, [sec, min, hour, date, month, year]: [u8; 6], century: u8) { + r.w8(CLOCK_CONTROL, 0x80); + for (i, v) in [sec, min, hour, 0x02, date, month, year].into_iter().enumerate() { + r.w8(CLOCK_CONTROL + 1 + i as u32, v); + } + r.w8(CLOCK_CONTROL, century); +} + +/// Ports test_rtc_absent: with no clock fitted the window reads zero and +/// drops a write. +#[test] +fn an_unfitted_clock_reads_zero() { + let r = Rig::new(); + assert_eq!(r.r8(CLOCK_CONTROL), 0); + r.w8(CLOCK_CONTROL, 0x55); + assert_eq!(r.r8(CLOCK_CONTROL), 0); + for a in [reg::CLOCK + 0x7ff7, reg::CLOCK + 0x8000, reg::CLOCK] { + assert_eq!(r.r8(a), 0); + } +} + +/// Ports test_rtc_alias and test_nvram_window: the SRAM fills the BAR from +/// 0x80000 to the calendar at 0x87ff8, the frozen calendar reads through it, +/// and neither side of the window answers. +#[test] +fn the_clock_window_holds_ram_and_the_calendar() { + let (r, ns) = with_clock(); + let frozen = [0x60u8, 0x59, 0x59, 0x23, 0x82, 0x31, 0x12, 0x24]; + r.w8(reg::CLOCK, 0xa5); + r.w8(reg::CLOCK + 0x7ff7, 0x5a); + for (i, a) in [0x0000e, 0x004bc, 0x07fcb, 0x07fd0].into_iter().enumerate() { + r.w8(reg::CLOCK + a, 0x31 + i as u8); + } + assert_eq!((r.r8(reg::CLOCK), r.r8(reg::CLOCK + 0x7ff7)), (0xa5, 0x5a)); + for (i, a) in [0x0000e, 0x004bc, 0x07fcb, 0x07fd0].into_iter().enumerate() { + assert_eq!(r.r8(reg::CLOCK + a), 0x31 + i as u8); + } + + set_time(&r, [0x59, 0x59, 0x23, 0x31, 0x12, 0x24], 0x20); + r.w8(CLOCK_CONTROL, 0x40); + ns.fetch_add(2_000_000_000, AtomicOrdering::SeqCst); + for (i, want) in frozen.into_iter().enumerate() { + assert_eq!(r.r8(CLOCK_CONTROL + i as u32), want, "calendar byte {i}"); + } + assert!(!r.pin.level()); + r.w8(CLOCK_CONTROL, 0); + assert_eq!((r.r8(CLOCK_CONTROL + 1), r.r8(CLOCK_CONTROL + 5), r.r8(CLOCK_CONTROL + 6), r.r8(CLOCK_CONTROL + 7)), (1, 1, 1, 0x25)); + + // Past the part on either side nothing answers. + assert_eq!(r.r8(reg::CLOCK + 0x8000), 0); + assert_eq!(r.r8(reg::CLOCK + 0x8008), 0); + assert_eq!(r.r8(reg::CLOCK - 1), 0); + + // A chip reset leaves the SRAM and the calendar alone. + r.w8(CLOCK_CONTROL, 0x40); + r.c.reset(); + assert_eq!((r.r8(reg::CLOCK), r.r8(reg::CLOCK + 0x7ff7)), (0xa5, 0x5a)); + assert_eq!(r.r8(CLOCK_CONTROL), 0x60, "even a read in progress"); +} + +/// Ports test_rtc_rejects_wide_access: a halfword or word to the clock is +/// refused whole and reaches no register. +#[test] +fn the_clock_takes_bytes_only() { + let (r, _) = with_clock(); + set_time(&r, [0x12, 0x34, 0x05, 0x15, 0x01, 0x24], 0x20); + r.w8(CLOCK_CONTROL, 0x80); + r.w8(CLOCK_CONTROL + 1, 0x11); + r.w8(CLOCK_CONTROL + 2, 0x22); + r.c.bar_write(0, CLOCK_CONTROL + 2, 0x4433, 2); + r.c.bar_write(0, CLOCK_CONTROL, 0x2000_0000, 4); + assert_eq!(r.c.bar_read(0, CLOCK_CONTROL + 2, 2), 0); + assert_eq!(r.c.bar_read(0, CLOCK_CONTROL, 4), 0); + assert_eq!((r.r8(CLOCK_CONTROL + 2), r.r8(CLOCK_CONTROL + 3)), (0x22, 0x05)); + r.w8(CLOCK_CONTROL, 0x20); + assert_eq!((r.r8(CLOCK_CONTROL + 2), r.r8(CLOCK_CONTROL + 3)), (0x22, 0x05)); +} + +// --------------------------------------------------------------------------- +// A receive ring with no room +// --------------------------------------------------------------------------- + +const RX_FULL: u32 = int::serial(0, 1); + +/// Ports test_dma_rx_full_and_staged_recovery: a character the full ring +/// cannot take waits, receive-ring-full is a level that stays while an +/// entry waits for room and ignores an acknowledge, and the entries post as +/// the consumer makes room. Filled exactly with nothing waiting, it stays +/// down. +#[test] +fn receive_ring_full_is_a_level_while_an_entry_waits() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.sw(0, sdma_off::CONTROL, ENABLE); + for i in 0..127u32 { + r.type_in(0, &[0x31, 0x32, 0x33, 0x34]); + r.until("the entry", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == ((i + 1) * 8) & 0x3f8); + } + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE), 0x3f8); + assert_eq!(r.r32(reg::SERIAL_PENDING) & RX_FULL, 0, "full, but nothing waits for room"); + + r.sw(0, sdma_off::RX_TIMER, 1); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.type_in(0, &[0xa1, 0xa2, 0xa3, 0xa4, 0xb5]); + r.until("the source", |r| r.r32(reg::SERIAL_PENDING) & RX_FULL != 0); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE), 0x3f8); + r.w32(reg::SERIAL_PENDING, RX_FULL); + assert_eq!(r.r32(reg::SERIAL_PENDING) & RX_FULL, RX_FULL, "a level ignores the acknowledge"); + + r.sw(0, sdma_off::RX_CONSUME, 16 | ARM); + r.until("room", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 0); + let entry = r.get(r.ring(0, 1, 0x400) + 0x3f8, 8); + assert_eq!((&entry[..4], entry[4], entry[7]), (&[0xa1, 0xa2, 0xa3, 0xa4][..], RX_FROM_LINE, RX_FROM_LINE)); + assert_eq!(r.r32(reg::SERIAL_PENDING) & RX_FULL, 0, "it fell once the consumer made room"); + + r.until("the last character", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 8); + let entry = r.get(r.ring(0, 1, 0x400), 8); + assert_eq!((entry[0], entry[1], entry[4], entry[5]), (0xb5, 0, RX_FROM_LINE, 0)); + assert_eq!(r.r32(reg::SERIAL_PENDING) & RX_FULL, 0, "full again, but nothing waits for room"); +} + +/// Ports test_dma_loopback_full_ring_recovery: a full receive ring does not +/// stall the transmit walk in loopback. Both entries go out and the eight +/// characters wait in the packer and the 16550's FIFO, then land as the +/// consumer frees the ring. +#[test] +fn a_full_ring_does_not_stall_a_loopback_walk() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.fast_line(0); + r.uw(0, 2, 0x01); + r.uw(0, 4, 0x13); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD); + let expected = fill_loopback_ring(&r); + r.until("the transmit ring", |r| r.sr(0, sdma_off::TX_CONSUME) & 0x3f8 == 0x3f8); + r.until("the receive ring", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 0x3f8); + assert_eq!(r.get(r.ring(0, 1, 0x400), 0x3f8), expected); + + let looped = |c: [u8; 4]| [c[0], c[1], c[2], c[3], RX_LOOPBACK, RX_LOOPBACK, RX_LOOPBACK, RX_LOOPBACK]; + r.tx_entry(0, 0x400, 0x3f8, [0xa1, 0xa2, 0xa3, 0xa4], [TX_VALID; 4]); + r.tx_entry(0, 0x400, 0, [0xb1, 0xb2, 0xb3, 0xb4], [TX_VALID; 4]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + r.until("both entries sent", |r| r.sr(0, sdma_off::TX_CONSUME) & 0x3f8 == 8); + r.until("the source", |r| r.r32(reg::SERIAL_PENDING) & RX_FULL != 0); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE), 0x3f8); + + r.sw(0, sdma_off::RX_CONSUME, 8); + r.until("the first", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 0); + assert_eq!(r.get(r.ring(0, 1, 0x400) + 0x3f8, 8), looped([0xa1, 0xa2, 0xa3, 0xa4])); + r.sw(0, sdma_off::RX_CONSUME, 16); + r.until("the second", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8 == 8); + assert_eq!(r.r32(reg::SERIAL_PENDING) & RX_FULL, 0, "filled exactly with nothing waiting"); + assert_eq!(r.get(r.ring(0, 1, 0x400), 8), looped([0xb1, 0xb2, 0xb3, 0xb4])); + assert!(r.sent(0).is_empty()); +} + +/// Ports test_dma_loopback_overrun_reaches_status: with the receive ring +/// out of room the characters that do not fit are lost to the 16550's +/// overrun, which shows in the status byte of the next character read. +#[test] +fn a_loopback_overrun_reaches_the_status_byte() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x400); + r.fast_line(0); + r.uw(0, 2, 0x01); + r.uw(0, 4, 0x13); + r.sw(0, sdma_off::CONTROL, ENABLE); + // Room for one entry before the ring is full. + r.sw(0, sdma_off::RX_CONSUME, 16); + for i in 0..8usize { + r.tx_entry(0, 0x400, 8 * i, std::array::from_fn(|j| (4 * i + j) as u8), [TX_VALID; 4]); + } + r.sw(0, sdma_off::TX_PRODUCE, 64); + r.until("the walk", |r| r.sr(0, sdma_off::TX_CONSUME) & 0x3f8 == 64); + r.settle(5); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8, 8); + + r.sw(0, sdma_off::RX_CONSUME, 8); + r.sw(0, sdma_off::CONTROL, ENABLE | DRAIN); + r.until("the drain", |r| r.sr(0, sdma_off::CONTROL) & DRAIN == 0); + let posted = (r.sr(0, sdma_off::RX_PRODUCE) & 0x3f8) / 8; + assert!(posted > 1); + let overrun = (1..posted as usize).any(|i| r.get(r.ring(0, 1, 0x400) + 8 * i + 4, 4).iter().any(|s| s & 1 != 0)); + assert!(overrun, "the loss is marked on a later character"); + assert!(posted * 4 < 32, "and not everything came back"); +} + +/// Ports test_serial_idle_status: the serial control word reads the idle bit +/// set after a write, a pause, a reset, PIO and a ring run, and the writes to +/// it and to the unsupported ATA window disturb no interrupt word. +#[test] +fn serial_idle_status_survives_writes_and_ring_runs() { + let r = Rig::new(); + assert_eq!(r.r32(reg::SERIAL_CONTROL), 0x80); + r.w32(reg::SERIAL_CONTROL, 0x80); + r.c.bar_read(0, reg::SERIAL_CONTROL, 1); + r.c.bar_write(0, reg::SERIAL_CONTROL, 0x80, 1); + assert_eq!(r.r32(reg::SERIAL_CONTROL), 0x80); + assert_eq!(r.r32(0x100), 0); + r.w32(0x100, u32::MAX); + assert_eq!(r.r32(reg::SERIAL_CONTROL), 0x80); + + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE | (1 << 29)); + r.tx_entry(0, 0x1000, 0, [0x5a, 0, 0, 0], [TX_VALID, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_ne!(r.sr(0, sdma_off::CONTROL) & (1 << 30), 0, "paused"); + assert_eq!(r.r32(reg::SERIAL_CONTROL), 0x80); + + r.c.reset(); + r.uw(0, 0, 0x5a); + assert_eq!(r.sent(0), [0x5a]); + assert_eq!(r.r32(reg::SERIAL_CONTROL), 0x80); + + r.pair_base(0, RINGS, 0x1000); + r.tx_entry(1, 0x1000, 0, [0x15, 0, 0, 0], [TX_MCR | TX_EXPLICIT, 0, 0, 0]); + r.sw(1, sdma_off::CONTROL, ENABLE); + r.sw(1, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.sr(1, sdma_off::TX_CONSUME), 8); + let (serial, other) = (r.r32(reg::SERIAL_PENDING), r.r32(reg::OTHER_PENDING)); + assert_ne!(serial, 0); + r.w32(reg::SERIAL_CONTROL, 0x80); + r.w32(0x100, u32::MAX); + assert_eq!((r.r32(reg::SERIAL_PENDING), r.r32(reg::OTHER_PENDING)), (serial, other)); + assert_eq!(r.r32(reg::SERIAL_CONTROL), 0x80); +} + +// --------------------------------------------------------------------------- +// The host end of a port +// --------------------------------------------------------------------------- + +/// Ports test_dma_loopback_discards_external_input: in loopback the UART's +/// serial input is disconnected, as on a PC16550D. Input from outside is +/// discarded on arrival, not kept for later, and new input arrives as usual +/// once the loopback is off. +#[test] +fn loopback_discards_input_from_outside() { + let r = Rig::new(); + let input = [0xa1, 0xa2, 0xa3, 0xa4]; + r.pair_base(0, RINGS, 0x400); + r.uw(0, 4, 0x10); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD); + r.type_in(0, &input); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD | DRAIN); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE), 0); + + // Out of loopback nothing comes back: it was discarded, not held. + r.uw(0, 4, 0); + r.c.service(); + r.sw(0, sdma_off::CONTROL, ENABLE | THRESHOLD | DRAIN); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE), 0); + + r.type_in(0, &input); + r.until("the entry", |r| r.sr(0, sdma_off::RX_PRODUCE) == 8); + assert_eq!(&r.get(r.ring(0, 1, 0x400), 4), &input); +} + +/// Ports test_dma_chardev_reconnect: a port's backend can be replaced while +/// the machine runs, and both PIO and the ring engine then use the new one. +#[test] +fn replacing_the_backend_keeps_the_engine_working() { + let r = Rig::new(); + let new = Arc::new(Wire::default()); + r.c.set_serial_backend(0, new.clone()); + r.c.set_serial_backend(0, new.clone()); + r.uw(0, 0, b'p'); + assert_eq!(*new.out.lock().unwrap(), b"p"); + assert!(r.sent(0).is_empty(), "the old backend saw nothing"); + + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.tx_entry(0, 0x1000, 0, [0x61, 0x62, 0, 0], [TX_VALID, TX_VALID, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 8); + assert_eq!(*new.out.lock().unwrap(), b"pab"); + new.inp.lock().unwrap().push_back(0x7a); + r.c.service(); + r.sw(0, sdma_off::CONTROL, ENABLE | DRAIN); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE) & 0xff8, 8, "and its input arrives"); + assert_eq!(r.get(r.ring(0, 1, 0x1000), 1), [0x7a]); +} + +// --------------------------------------------------------------------------- +// The rest of the ring engine's register cases +// --------------------------------------------------------------------------- + +const TIMER_EXPIRED: u32 = int::serial(0, 3); +const TX_COMPLETION: u32 = int::serial(0, 7); + +/// Ports test_dma_tx_wrap_4k. +#[test] +fn tx_ring_wraps_at_four_kilobytes() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::TX_PRODUCE, 0xff8); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 0xff8); + r.tx_entry(0, 0x1000, 0xff8, [0x6e, 0, 0, 0], [TX_VALID, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 0); + assert_eq!(r.sent(0), [0x6e]); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 0); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::serial(0, 0), int::serial(0, 0)); +} + +/// Ports test_dma_control_stores_its_bits: the control register keeps every +/// writable bit as written, including ones the model does not act on. +#[test] +fn control_stores_the_bits_it_does_not_act_on() { + let r = Rig::new(); + let written = (1 << 17) | (1 << 20) | 0x1ff; + r.sw(0, sdma_off::CONTROL, written); + assert_eq!(r.sr(0, sdma_off::CONTROL), written); +} + +/// Ports test_dma_engine_indices_take_writes and +/// test_dma_host_index_masks_its_pointer: the two indices the engine moves +/// still take a write, and every index keeps only its pointer field. +#[test] +fn indices_keep_their_pointer_field() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::TX_CONSUME, u32::MAX); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 0xff8); + r.sw(0, sdma_off::RX_PRODUCE, 0x18); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE), 0x18); + r.sw(0, sdma_off::TX_PRODUCE, 0x10014); + assert_eq!(r.sr(0, sdma_off::TX_PRODUCE), 0x10); + r.sw(0, sdma_off::RX_CONSUME, 0x2c); + assert_eq!(r.sr(0, sdma_off::RX_CONSUME) & 0xff8, 0x28); +} + +/// Ports test_dma_empty_slot_ends_the_entry: an empty slot ends its entry +/// wherever it sits, so the slots behind it are not sent. +#[test] +fn an_empty_slot_ends_the_entry() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.tx_entry(0, 0x1000, 0, *b"abcd", [TX_VALID, 0, TX_VALID, TX_VALID]); + r.tx_entry(0, 0x1000, 8, [b'e', 0, 0, 0], [TX_VALID, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 16); + assert_eq!(r.sent(0), b"ae"); +} + +/// Ports test_dma_tx_control_decodes_type_and_bit5: a control byte is +/// decoded by its type field and bit 5, undefined low bits do not stop the +/// port, and a slot of type none raises nothing even with bit 5 set. +#[test] +fn a_control_byte_is_decoded_by_type_and_bit_five() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.tx_entry(0, 0x1000, 0, [b'a', b'b', b'c', 0], [TX_VALID | 0x10, TX_EXPLICIT, TX_VALID, 0]); + r.tx_entry(0, 0x1000, 8, [b'd', 0, 0, 0], [TX_VALID, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 16); + assert_eq!(r.sent(0), b"ad"); + assert_eq!(r.sr(0, sdma_off::TX_CONSUME), 16); + assert_eq!(r.r32(reg::SERIAL_PENDING) & TX_COMPLETION, 0); +} + +/// Ports test_dma_tx_delay_sets_the_busy_bit: control bit 16 reads set while +/// a delay entry runs and clear once it has ended. +#[test] +fn a_delay_entry_sets_the_busy_bit() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.tx_entry(0, 0x1000, 0, [3, 0, 0, 0], [TX_DELAY, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.sr(0, sdma_off::CONTROL) & (1 << 16), 1 << 16); + r.settle(10); + assert_eq!(r.sr(0, sdma_off::CONTROL) & (1 << 16), 0); +} + +/// Ports test_dma_rx_timer_due_before_access: an expiry already due +/// happens before a register access, with no timer thread to run it. +#[test] +fn a_due_receive_timer_expires_before_an_access() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::RX_TIMER, 2); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.type_in(0, &[0x64]); + std::thread::sleep(Duration::from_millis(5)); + assert_eq!(r.r32(reg::SERIAL_PENDING) & TIMER_EXPIRED, TIMER_EXPIRED); +} + +/// Ports test_dma_rx_timer_count_reads_back: a reload write loads the live +/// count too and an idle timer holds it; a running one counts down. +#[test] +fn the_receive_timer_count_reads_back() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::RX_TIMER, 0xfff); + assert_eq!(r.sr(0, sdma_off::RX_TIMER) & 0x0fff_0fff, 0x0fff_0fff); + std::thread::sleep(Duration::from_millis(2)); + assert_eq!(r.sr(0, sdma_off::RX_TIMER) & 0x0fff_0fff, 0x0fff_0fff, "idle, it holds"); + r.sw(0, sdma_off::RX_TIMER, 0x140); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.type_in(0, &[0x63]); + let count = r.sr(0, sdma_off::RX_TIMER) >> 16 & 0xfff; + assert!((1..=0x140).contains(&count), "running: {count:#x} of 0x140 left"); +} + +/// Ports test_dma_consume_without_arm_keeps_it: a consume write with bit 31 +/// clear leaves the arm, which reads back and still starts the timer. +#[test] +fn a_consume_write_without_the_arm_keeps_it() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::RX_TIMER, 2); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.sw(0, sdma_off::RX_CONSUME, 0); + assert_eq!(r.sr(0, sdma_off::RX_CONSUME), ARM); + r.type_in(0, &[0x6b]); + r.until("the expiry", |r| r.r32(reg::SERIAL_PENDING) & TIMER_EXPIRED != 0); + assert_eq!(r.sr(0, sdma_off::RX_CONSUME), 0, "and the expiry spent it"); +} + +/// Ports test_dma_zero_reload_expires_at_once. +#[test] +fn a_zero_reload_expires_at_once() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::RX_TIMER, 0); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.type_in(0, &[0x7a]); + r.until("the expiry", |r| r.r32(reg::SERIAL_PENDING) & TIMER_EXPIRED != 0); + assert_eq!(r.sr(0, sdma_off::RX_PRODUCE), 8); +} + +/// Ports test_dma_rx_status_carries_modem_state: a character from the line +/// carries the live CTS and DCD levels, as the UART's modem status shows. +#[test] +fn a_received_character_carries_the_modem_state() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.type_in(0, &[0x6e]); + r.sw(0, sdma_off::CONTROL, ENABLE | DRAIN); + let entry = r.get(r.ring(0, 1, 0x1000), 8); + assert_eq!((entry[0], entry[4]), (0x6e, RX_FROM_LINE)); + assert_eq!(r.ur(0, 6) & 0xb0, 0xb0); +} + +/// Ports test_dma_rx_timer_expiry_after_full_entry: a full entry posts at +/// once, the timer still raises its source when it runs out, and once +/// acknowledged it does not come back without another arm. +#[test] +fn the_timer_still_expires_after_a_full_entry() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::RX_TIMER, 2); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.type_in(0, &[0x31, 0x32, 0x33, 0x34]); + r.until("the entry", |r| r.sr(0, sdma_off::RX_PRODUCE) & 0xff8 == 8); + let entry = r.get(r.ring(0, 1, 0x1000), 8); + assert_eq!((&entry[..4], entry[4], entry[7]), (&[0x31, 0x32, 0x33, 0x34][..], RX_FROM_LINE, RX_FROM_LINE)); + r.until("the expiry", |r| r.r32(reg::SERIAL_PENDING) & TIMER_EXPIRED != 0); + r.w32(reg::SERIAL_PENDING, TIMER_EXPIRED); + r.settle(5); + assert_eq!(r.r32(reg::SERIAL_PENDING) & TIMER_EXPIRED, 0); +} + +/// Ports test_dma_rx_disable_stops_timer_keeps_arm: clearing the enable stops +/// a running receive timer and discards its time, and the arm stays so the +/// first character after re-enabling starts it afresh. +#[test] +fn disabling_the_ring_stops_the_timer_and_keeps_the_arm() { + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.sw(0, sdma_off::RX_TIMER, 0x100); + r.sw(0, sdma_off::RX_CONSUME, ARM); + r.type_in(0, &[0x31]); + assert!(r.c.sh.st.lock().sdma.rx_deadline(0).is_some(), "running"); + r.sw(0, sdma_off::CONTROL, 0); + assert!(r.c.sh.st.lock().sdma.rx_deadline(0).is_none(), "stopped"); + r.settle(20); + assert_eq!(r.r32(reg::SERIAL_PENDING) & TIMER_EXPIRED, 0); + r.sw(0, sdma_off::CONTROL, ENABLE); + assert_eq!(r.sr(0, sdma_off::RX_CONSUME) & ARM, ARM, "the arm stays"); + r.type_in(0, &[0x32]); + assert!(r.c.sh.st.lock().sdma.rx_deadline(0).is_some(), "the next character starts it afresh"); +} + +// --------------------------------------------------------------------------- +// The PS/2 keyboard and mouse controller +// --------------------------------------------------------------------------- + +const KBC_CONTROL: u32 = reg::KBC; +const KBC_KBD_RECEIVE: u32 = reg::KBC + 0x04; +const KBC_AUX_RECEIVE: u32 = reg::KBC + 0x08; +const KBC_KBD_TRANSMIT: u32 = reg::KBC + 0x0c; +const KBC_AUX_TRANSMIT: u32 = reg::KBC + 0x10; +const KBC_IRQ: u32 = 0x40; +const KBC_WRITE_PENDING: u32 = 0x03; + +fn key(r: &Rig, code: winit::keyboard::KeyCode, down: bool) { + r.c.ps2().push_kb(code, down); +} + +/// Ports test_ps2_adapter (QEMU with the read-only status +/// bits): the five registers through the adapter, reply bytes in BE32 +/// presentation, write-pending never set, the keyboard source in the other +/// word with its own enable and acknowledge, isolation from the serial +/// words, a mouse report, and reset. +#[test] +fn the_ps2_adapter_presents_the_controller() { + use winit::keyboard::KeyCode::KeyA; + let r = Rig::new(); + assert_eq!((r.r32(KBC_CONTROL), r.r32(KBC_KBD_RECEIVE), r.r32(KBC_AUX_RECEIVE)), (0, 0, 0)); + r.w32(KBC_CONTROL, u32::MAX); + assert_eq!(r.r32(KBC_CONTROL), 0x00fc_3cc0, "the status bits read clear and drop the write"); + r.w32(KBC_CONTROL, 0); + r.c.bar_read(0, KBC_CONTROL, 1); + r.c.bar_write(0, KBC_KBD_TRANSMIT, 0xff, 2); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0, "a narrow write reached nothing"); + + // 0xff000000 is a zero byte to the keyboard: it only acknowledges. + r.w32(KBC_KBD_TRANSMIT, 0xff00_0000); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x80fa_0000); + r.w32(KBC_KBD_TRANSMIT, 0xff); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0xc0fa_aa00); + assert_eq!(r.r32(KBC_CONTROL) & KBC_WRITE_PENDING, 0); + // QEMU's keyboard scans again after the reset command; this core waits + // for the driver's enable, as the O2's and Octane's PROMs give it. + r.w32(KBC_KBD_TRANSMIT, 0xf4); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x80fa_0000); + r.w32(reg::OTHER_PENDING, KBC_IRQ); + + key(&r, KeyA, true); + assert_eq!(r.r32(reg::OTHER_PENDING) & KBC_IRQ, KBC_IRQ); + assert!(!r.pin.level(), "pending but not enabled"); + r.w32(reg::OTHER_ENABLE_SET, KBC_IRQ); + assert!(r.pin.level()); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x801c_0000); + assert_eq!(r.r32(KBC_KBD_RECEIVE) & 0x1f00_0000, 0); + assert_eq!(r.r32(reg::OTHER_PENDING) & KBC_IRQ, KBC_IRQ, "a drain does not acknowledge"); + + // A serial source holds the pin up when the keyboard's goes. + r.uw(0, 3, 0x80); + r.uw(0, 0, 0x34); + r.uw(0, 1, 0x12); + r.uw(0, 3, 0x03); + r.uw(0, 2, 0x01); + r.uw(0, 1, 0x02); + r.w32(reg::SERIAL_ENABLE_SET, int::pass(0)); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(0), int::pass(0)); + assert!(r.pin.level()); + r.w32(reg::OTHER_ENABLE_CLR, KBC_IRQ); + assert!(r.pin.level()); + r.w32(reg::OTHER_ENABLE_SET, KBC_IRQ); + r.w32(reg::OTHER_PENDING, KBC_IRQ); + assert_eq!(r.r32(reg::OTHER_PENDING) & KBC_IRQ, 0); + assert_eq!(r.r32(reg::SERIAL_PENDING) & int::pass(0), int::pass(0), "the serial source is another word"); + assert!(r.pin.level()); + r.uw(0, 1, 0); + assert!(!r.pin.level()); + + r.w32(KBC_AUX_TRANSMIT, 0xff); + assert_eq!(r.r32(KBC_AUX_RECEIVE), 0xe0fa_aa00); + assert_eq!(r.r32(KBC_CONTROL) & KBC_WRITE_PENDING, 0); + r.w32(reg::OTHER_PENDING, KBC_IRQ); + r.w32(KBC_AUX_TRANSMIT, 0xf4); + assert_eq!(r.r32(KBC_AUX_RECEIVE), 0x80fa_0000); + r.w32(reg::OTHER_PENDING, KBC_IRQ); + r.c.ps2().push_mouse_input(1, 4, 0, 0); + assert_eq!(r.r32(KBC_AUX_RECEIVE), 0xe009_0400); + + r.c.reset(); + assert_eq!((r.r32(KBC_CONTROL), r.r32(KBC_KBD_RECEIVE), r.r32(KBC_AUX_RECEIVE)), (0, 0, 0)); + assert_eq!((r.r32(reg::OTHER_PENDING) & KBC_IRQ, r.r32(reg::OTHER_ENABLE_SET) & KBC_IRQ), (0, 0)); + r.w32(KBC_KBD_TRANSMIT, 0xff); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0xc0fa_aa00); +} + +/// Ports test_ps2_ports_present: every IOC4 has both ports; there is no +/// switch to leave them unplugged. +#[test] +fn both_ps2_ports_are_always_fitted() { + let r = Rig::with(Ioc4Config::bare(Ioc4Clock::Mhz33)); + r.w32(KBC_KBD_TRANSMIT, 0xff); + r.w32(KBC_AUX_TRANSMIT, 0xff); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0xc0fa_aa00); + assert_eq!(r.r32(KBC_AUX_RECEIVE), 0xe0fa_aa00); +} + +/// Ports test_ps2_driver_sequence, the attach and interrupt sequence of +/// OpenBSD's public IOC4 keyboard driver: read-modify-write the control +/// register to clamp after one character, transmit after the write-pending +/// poll, then on each interrupt mask the source, read each receive register +/// until no valid flag is left, acknowledge and unmask. Also the +/// clamp-after-three delivery, and an unclamped port losing a fourth +/// character without showing the IOC3-only overflow flag. +#[test] +fn the_ps2_driver_sequence() { + use winit::keyboard::KeyCode::{KeyA, KeyB, KeyC, KeyD}; + let r = Rig::new(); + let mut csr = r.r32(KBC_CONTROL); + csr &= !0x00c0_0ccc; + csr |= 0x0030_0000; + r.w32(KBC_CONTROL, csr); + assert_eq!(r.r32(KBC_CONTROL), 0x0030_0000); + // Status bits are not storage: a write to them is dropped and logged. + r.w32(KBC_CONTROL, 0x0030_0000 | 0xc33f); + assert_eq!(r.r32(KBC_CONTROL), 0x0030_0000); + r.w32(KBC_CONTROL, 0x0030_0000 | 0x3cc0); + assert_eq!(r.r32(KBC_CONTROL), 0x0030_3cc0, "the writable bits among them stick"); + r.w32(KBC_CONTROL, 0x0030_0000); + r.w32(reg::OTHER_ENABLE_SET, KBC_IRQ); + + // A reset reply is two characters, so the clamp holds the second back. + assert_eq!(r.r32(KBC_CONTROL) & KBC_WRITE_PENDING, 0); + r.w32(KBC_KBD_TRANSMIT, 0xff); + assert!(r.pin.level()); + assert_eq!(r.r32(reg::OTHER_PENDING) & KBC_IRQ, KBC_IRQ); + r.w32(reg::OTHER_ENABLE_CLR, KBC_IRQ); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x80fa_0000); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x80aa_0000); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0); + assert_eq!(r.r32(KBC_AUX_RECEIVE), 0); + // As in the adapter test: this keyboard model needs the driver's enable. + r.w32(KBC_KBD_TRANSMIT, 0xf4); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x80fa_0000); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0); + r.w32(reg::OTHER_PENDING, KBC_IRQ); + r.w32(reg::OTHER_ENABLE_SET, KBC_IRQ); + assert!(!r.pin.level()); + + // Press and release: three characters, one per read under the clamp. + key(&r, KeyA, true); + key(&r, KeyA, false); + assert!(r.pin.level()); + r.w32(reg::OTHER_ENABLE_CLR, KBC_IRQ); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x801c_0000); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x80f0_0000); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0x801c_0000); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0); + r.w32(reg::OTHER_PENDING, KBC_IRQ); + r.w32(reg::OTHER_ENABLE_SET, KBC_IRQ); + assert!(!r.pin.level()); + + // Clamp after three: the same press and release arrive in one read. + r.w32(KBC_CONTROL, 0x0040_0000); + key(&r, KeyA, true); + key(&r, KeyA, false); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0xe01c_f01c); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0); + + // No clamp: a fourth character is lost and the flag is not reported. + r.w32(KBC_CONTROL, 0); + for k in [KeyA, KeyB, KeyC, KeyD] { + key(&r, k, true); + } + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0xe01c_3221); + assert_eq!(r.r32(KBC_KBD_RECEIVE), 0); +} + +/// Ports the width rule of test_ps2_adapter: the keyboard block takes +/// aligned 32-bit accesses only, and a refused one reaches no register. +#[test] +fn the_ps2_block_takes_aligned_words_only() { + let r = Rig::new(); + r.w32(KBC_CONTROL, 0x0030_0000); + assert_eq!(r.c.bar_read(0, KBC_CONTROL, 1), 0); + assert_eq!(r.c.bar_read(0, KBC_CONTROL, 2), 0); + r.c.bar_write(0, KBC_CONTROL, 0, 2); + r.c.bar_write(0, KBC_CONTROL + 1, 0, 1); + assert_eq!(r.r32(KBC_CONTROL), 0x0030_0000); + assert_eq!(r.c.bar_read(0, KBC_CONTROL + 2, 4), 0, "unaligned"); + // Beyond the five registers is not the controller. + assert_eq!(r.r32(reg::KBC + 0x14), 0); +} + +/// A key pressed on the host reaches the pin with no guest access, from the +/// host's thread, as the IOC3's does. +#[test] +fn a_host_key_raises_the_pin_without_a_guest_access() { + let r = Rig::new(); + r.w32(reg::OTHER_ENABLE_SET, KBC_IRQ); + r.w32(KBC_KBD_TRANSMIT, 0xf4); + let _ = r.r32(KBC_KBD_RECEIVE); + r.w32(reg::OTHER_PENDING, KBC_IRQ); + assert!(!r.pin.level()); + key(&r, winit::keyboard::KeyCode::KeyA, true); + assert!(r.pin.level()); +} + +// --------------------------------------------------------------------------- +// Reported, not refused +// --------------------------------------------------------------------------- + +/// Everything the `ioc` log module has said in this process. Tests run side +/// by side and share it, so a check looks for its own message text. +fn device_log() -> String { + use std::io::Write; + struct Sink(Arc>>); + impl Write for Sink { + fn write(&mut self, b: &[u8]) -> std::io::Result { + self.0.lock().unwrap().extend_from_slice(b); + Ok(b.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } + } + static LOG: std::sync::OnceLock>>> = std::sync::OnceLock::new(); + let log = LOG.get_or_init(|| { + let buf = Arc::new(std::sync::Mutex::new(Vec::new())); + let dl = crate::devlog::DEVLOG.get().cloned().unwrap_or_else(crate::devlog::init_devlog); + dl.add_sink(Arc::new(parking_lot::Mutex::new(Sink(buf.clone())))); + dl.enable(crate::devlog::LogModule::Ioc); + buf + }); + String::from_utf8_lossy(&log.lock().unwrap()).into_owned() +} + +/// Ports test_dma_reports_bad_base_and_pointer: a misaligned pair base and a +/// base change on a busy pair are reported, not refused; the enable and the +/// new base both take effect, and an index keeps its pointer field while the +/// stray bits are reported. +#[test] +fn a_bad_base_and_a_stray_pointer_bit_are_reported_not_refused() { + let _ = device_log(); + let r = Rig::new(); + r.pair_base(0, RINGS + 8, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + assert_eq!(r.sr(0, sdma_off::CONTROL) & ENABLE, ENABLE); + r.sw(0, sdma_off::CONTROL, 0); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.pair_base(0, RINGS + 0x4000, 0x1000); + assert_eq!(r.r32(reg::SDMA_BASE), (RINGS + 0x4000) as u32 | 1); + r.sw(0, sdma_off::CONTROL, RESET); + r.sw(0, sdma_off::TX_PRODUCE, 4); + assert_eq!(r.sr(0, sdma_off::TX_PRODUCE), 0); + r.sw(0, sdma_off::RX_TIMER, 0x1_0002); + r.sw(0, sdma_off::CONTROL, 1 << 20); + let log = device_log(); + for what in [ + "misaligned for its rings", + "changed while it is active", + "TX producer 0x00000004 on port 0 has bits outside the pointer field", + "unsupported RX timer bits 0x00010000 on port 0", + "serial control bits 0x00100000 on port 0 are stored but not modelled", + ] { + assert!(log.contains(what), "no report of: {what}"); + } +} + +/// Ports test_dma_mode_switch_over_pio_is_reported and +/// test_dma_tx_disable_reports_pending_delay: the UART orders PIO and ring +/// DMA itself, so enabling DMA over queued PIO data, a PIO transmit while DMA +/// is on, and clearing the enable over a running delay are reported and go +/// ahead. +#[test] +fn a_mode_switch_over_pio_is_reported_and_goes_ahead() { + let _ = device_log(); + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.uw(0, 2, 0x07); + r.type_in(0, &[0x5a]); + assert_eq!(r.ur(0, 5) & 1, 1); + r.sw(0, sdma_off::CONTROL, ENABLE); + assert_eq!(r.sr(0, sdma_off::CONTROL) & ENABLE, ENABLE, "the enable took effect"); + r.uw(0, 0, 0xa5); + assert_eq!(r.sent(0), [0xa5], "and so did the PIO byte"); + + r.tx_entry(0, 0x1000, 0, [30, 0, 0, 0], [TX_DELAY, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + r.sw(0, sdma_off::CONTROL, 0); + assert_eq!(r.sr(0, sdma_off::CONTROL) & ENABLE, 0); + let log = device_log(); + assert!(log.contains("with PIO data in the UART's FIFOs")); + assert!(log.contains("PIO transmit on port 0 while DMA is enabled")); + assert!(log.contains("DMA disabled on port 0 with ring work pending")); +} + +/// Ports the undefined-bits check of test_dma_tx_control_decodes_type_and_bit5. +#[test] +fn undefined_transmit_control_bits_are_reported() { + let _ = device_log(); + let r = Rig::new(); + r.pair_base(0, RINGS, 0x1000); + r.sw(0, sdma_off::CONTROL, ENABLE); + r.tx_entry(0, 0x1000, 0, [b'q', 0, 0, 0], [TX_VALID | 0x10, 0, 0, 0]); + r.sw(0, sdma_off::TX_PRODUCE, 8); + assert_eq!(r.sent(0), b"q", "the slot is obeyed by its type"); + assert!(device_log().contains("undefined TX control bits 0x10 in an entry on port 0")); +} diff --git a/src/dev/ioc_serial_dma.rs b/src/dev/ioc_serial_dma.rs new file mode 100644 index 000000000..31c276a33 --- /dev/null +++ b/src/dev/ioc_serial_dma.rs @@ -0,0 +1,1124 @@ +//! The serial ring-DMA engine of the SGI IOC3 and IOC4. +//! +//! Both chips put the same engine in front of each of their 16550-compatible +//! ports: seven 32-bit registers per port, a ring of eight-byte entries in +//! host memory for each direction, and a 64-bit ring base shared by a pair of +//! ports. The transmit walk writes the port's 16550; the receive side drains +//! its FIFO into a packer that becomes one ring entry. A ring's address is +//! derived from its port and direction, and no register reads it back. +//! +//! The chip around the engine owns its register window, its interrupt status +//! words and the 16550s. The engine reaches them through [`SerialDmaHost`], so the same +//! walk serves the IOC3 (two ports, nine interrupt bits each) and the IOC4 +//! (four ports, eight bits each, DMA errors reported elsewhere). +//! +//! # What differs between the chips +//! +//! [`SerialDmaParams`] holds what the two do differently and nothing else: +//! +//! * IOC3 reports the packer's held-character count in the receive producer +//! index. IOC4's register map does not define the field, so whether the +//! chip has it is not known. +//! * IOC4 stops a port after a failed transfer until a channel reset (its +//! error sources are named "serial halted by mem err"). Whether an IOC3 +//! port halts is not known. +//! +//! # What is approximated here +//! +//! * The receive timer and a transmit delay run on host time. A pause freezes +//! both where they stand and a resume lets each finish the time it had +//! left. No source says what the part does. +//! * Receive-ring-full is a level that follows a held entry; what the part +//! does is not known. +//! +//! # Mapping to QEMU +//! +//! The split is QEMU's: this file is the shared engine, `ioc3.rs` and +//! `ioc4.rs` are the chips that use it. +//! +//! | QEMU (`hw/char/sgi-ioc-serial-dma.c`) | here | +//! |---|---| +//! | `SGIIOCSerialDMA`, `SGIIOCSerialDMAPort` | [`SerialDma`], [`SerialDmaPort`] | +//! | `SGIIOCSerialDMAParams` | [`SerialDmaParams`] | +//! | `SGIIOCSerialDMAOps` and its `opaque` | [`SerialDmaHost`], implemented by the chip | +//! | `SGIIOCSerialDMASource` | [`SerialDmaSource`] | +//! | the port's `SerialState *uart` | [`SerialDmaUart`] | +//! | `sgi_ioc_sdma_port_read`, `_port_write`, `_kick`, `_catch_up`, `_reset` | `port_read`, `port_write`, `run`, `catch_up`, `reset` | +//! | `sgi_ioc_sdma_base_read`, `_base_write` | `base_read`, `base_write` | +//! +//! Ported from hw/char/sgi-ioc-serial-dma.c in jamesbraid/qemu-sgi-origami, +//! in the form the IOC3 had before the engine was shared. + +use std::time::{Duration, Instant}; + +use crate::dev::ioc3::{due, span, unspan, until, usize_field}; +use crate::snapshot::{field, field_bool, field_u32, field_u32s, field_u64, field_u8s, StateTable}; + +use crate::dev::ns16550::Ns16550; + +use crate::dev::ns16550::com_reg::{ + LSR_BI, LSR_DR, LSR_FE, LSR_OE, LSR_PE, MCR_DTR, MCR_LOOP, MCR_OUT1, MCR_OUT2, MCR_RTS, MSR_CTS, MSR_DCD, + MSR_DCTS, MSR_DDCD, +}; +use crate::dev::ns16550::com_reg::{LSR as UART_LSR, MCR as UART_MCR, MSR as UART_MSR, THR as UART_THR}; + +/// Port DMA register offsets (bytes, from the port's first register). +pub mod off { + pub const CONTROL: u32 = 0x00; + pub const TX_PRODUCE: u32 = 0x04; + pub const TX_CONSUME: u32 = 0x08; + pub const RX_PRODUCE: u32 = 0x0c; + pub const RX_CONSUME: u32 = 0x10; + pub const RX_TIMER: u32 = 0x14; + pub const SHADOW: u32 = 0x18; + /// Registers per port. + pub const REGS: u32 = 7; +} + +/// Bits of the port's control register. +pub mod ctl { + pub const HIGH_WATER: u32 = 0x1ff; + pub const TX_TIMER_RUN: u32 = 1 << 16; + pub const FLOW_CONTROL: u32 = 1 << 17; + pub const RX_TO_MEMORY: u32 = 1 << 27; + pub const ENABLE: u32 = 1 << 28; + pub const PAUSE: u32 = 1 << 29; + pub const PAUSED: u32 = 1 << 30; + pub const RESET: u32 = 1 << 31; + /// Status and commands: these never read back as written. + pub const UNSTORED: u32 = RESET | RX_TO_MEMORY | PAUSED | TX_TIMER_RUN; +} + +/// The most ports one engine drives. +pub const MAX_PORTS: usize = 4; +/// Ports sharing one ring base. +const PAIR: usize = 2; + +pub const ENTRY: usize = 8; +pub const ENTRY_CHARS: usize = 4; +/// Bit 31 of the receive consumer index arms the receive timer. +pub const ARM_RX_TIMER: u32 = 1 << 31; +/// The packer's held count, in the receive producer index (IOC3 only). +pub const HELD_SHIFT: u32 = 24; +const TIMER_RELOAD: u32 = 0xfff; +const TIMER_COUNT_SHIFT: u32 = 16; +pub const TIMER_TICK_NS: u64 = 62_500; // 16 kHz +const RING_4K: u32 = 1; +const RING_ADDRESS: u32 = 0xffff_f000; +const RING_1K_SIZE: u32 = 1024; +const RING_4K_SIZE: u32 = 4096; +const TX_TYPE: u8 = 0xc0; +const TX_CHARACTER: u8 = 0x40; +const TX_MODEM: u8 = 0x80; +pub const TX_DELAY: u8 = 0xc0; +pub const TX_INTERRUPT: u8 = 0x20; +const TX_UNDEFINED: u8 = 0x1f; +const RX_OVERRUN: u8 = 0x01; +const RX_PARITY: u8 = 0x02; +const RX_FRAMING: u8 = 0x04; +const RX_BREAK: u8 = 0x08; +const RX_CTS: u8 = 0x10; +const RX_CD: u8 = 0x20; +const RX_MODEM_OK: u8 = 0x40; +const RX_DATA_OK: u8 = 0x80; + +/// A port's interrupt sources, numbered as the IOC3 lays them out in its +/// status word. The IOC4 places all but the DMA error the same way. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum SerialDmaSource { + TxEmpty = 0, + RxFull = 1, + RxHighWater = 2, + RxTimer = 3, + TxExplicit = 7, + DmaError = 8, +} + +/// What differs between the chips, fixed per chip. +#[derive(Clone, Copy, Debug)] +pub struct SerialDmaParams { + /// The receive producer index reports the packer's held count. + pub held_counts: bool, + /// A failed transfer stops the port until a channel reset. + pub halt_on_dma_error: bool, +} + +/// A 16550 as the engine drives it. +pub trait SerialDmaUart { + fn loopback(&self) -> bool; + /// The port can take a character from the engine. The walk waits while + /// it cannot, and goes on when the chip next runs the engine. + fn tx_ready(&self) -> bool; + /// How long a character takes on the line at the programmed rate. + fn char_time(&self) -> Duration; + /// A read with the chip's side effects left out: the 16550 proper. + fn read_core(&mut self, reg: u32) -> u8; + fn write(&mut self, reg: u32, val: u8); + fn lsr(&self) -> u8; + fn mcr(&self) -> u8; + fn msr(&self) -> u8; +} + +/// The chip around the engine. +pub trait SerialDmaHost { + fn uart(&mut self, port: usize) -> &mut dyn SerialDmaUart; + /// Latch a source; the chip's write-one-to-clear acknowledges it. + fn raise(&mut self, port: usize, src: SerialDmaSource); + /// Drive a source that follows a level and ignores acknowledgement. + fn set_level(&mut self, port: usize, src: SerialDmaSource, level: bool); + /// The port's 16550 is fitted and the engine is aimed at it. + fn reachable(&self, port: usize) -> bool; + /// The 16550's interrupt line may have moved: reflect it in the chip's + /// pass-through source. + fn uart_line_changed(&mut self, port: usize); + fn dma_read(&self, addr: u64, buf: &mut [u8]) -> bool; + fn dma_write(&self, addr: u64, data: &[u8]) -> bool; + /// A transfer failed at `addr`: a receive post or a transmit fetch. + fn dma_error(&mut self, port: usize, _rx: bool, _addr: u64) { + self.raise(port, SerialDmaSource::DmaError); + } +} + +#[derive(Default)] +pub struct SerialDmaPort { + reg: [u32; off::REGS as usize], + packer: [u8; ENTRY], + packed: usize, + tx_entry: [u8; ENTRY], + tx_fetched: bool, + tx_slot: usize, + armed: bool, + drain_pending: bool, + busy: bool, + halted: bool, + /// The last post was turned away for lack of ring room. + post_refused: bool, + paused: bool, + pub rx_deadline: Option, + rx_idle_count: u32, + /// Time a frozen timer had left when a pause began. + rx_frozen: Option, + tx_delay_deadline: Option, + tx_delay_frozen: Option, + tx_delay_irq: bool, + /// A looped-back character still on the line: when it reaches the + /// receiver. The 16550 core loops it back the moment it is written, and + /// the engine does not look at the receiver until this time. + loop_due: Option, +} + +pub struct SerialDma { + params: SerialDmaParams, + base_hi: [u32; MAX_PORTS / PAIR], + base_lo: [u32; MAX_PORTS / PAIR], + /// Transfers under way, for the chip's idle status. Nonzero only inside + /// a call: a chip cannot read its own status mid-walk, so software sees + /// idle unless a bus master it reaches re-enters the chip. + in_flight: u32, + pub port: Vec, +} + +impl SerialDma { + pub fn new(ports: usize, params: SerialDmaParams) -> SerialDma { + assert!((1..=MAX_PORTS).contains(&ports)); + SerialDma { + params, + base_hi: [0; MAX_PORTS / PAIR], + base_lo: [0; MAX_PORTS / PAIR], + in_flight: 0, + port: (0..ports).map(|_| SerialDmaPort::default()).collect(), + } + } + + /// Back to the power-on state: the ring bases and every port. + pub fn reset(&mut self) { + *self = SerialDma::new(self.port.len(), self.params); + } + + pub fn base_read(&self, pair: usize, high: bool) -> u32 { + if high { self.base_hi[pair] } else { self.base_lo[pair] } + } + + pub fn base_write(&mut self, pair: usize, high: bool, val: u32) { + if high { + self.base_hi[pair] = val; + } else { + self.base_lo[pair] = val; + } + } + + /// A port's control register as stored. + pub fn control(&self, p: usize) -> u32 { + self.port[p].reg[0] + } + + /// A register as stored, without the status the read adds. + pub fn stored(&self, p: usize, offset: u32) -> u32 { + self.port[p].reg[(offset / 4) as usize] + } + + /// When the port's receive timer runs out, if it is running. + pub fn rx_deadline(&self, p: usize) -> Option { + self.port[p].rx_deadline + } + + /// When the port's transmit delay ends. A paused port keeps it frozen. + pub fn tx_delay_deadline(&self, p: usize) -> Option { + let port = &self.port[p]; + if port.paused { None } else { port.tx_delay_deadline } + } + + /// No transfer is under way: what a chip's DMA idle status reports. + pub fn idle(&self) -> bool { + self.in_flight == 0 + } + + /// A ring read, counted as in flight while the bus master has it. + fn dma_read(&mut self, host: &mut dyn SerialDmaHost, addr: u64, buf: &mut [u8]) -> bool { + self.in_flight += 1; + let ok = host.dma_read(addr, buf); + self.in_flight -= 1; + ok + } + + /// A ring write, counted as in flight while the bus master has it. + fn dma_write(&mut self, host: &mut dyn SerialDmaHost, addr: u64, data: &[u8]) -> bool { + self.in_flight += 1; + let ok = host.dma_write(addr, data); + self.in_flight -= 1; + ok + } + + /// When a looped-back character reaches the receiver, if one is on the + /// line. + pub fn loop_deadline(&self, p: usize) -> Option { + self.port[p].loop_due + } + + /// Nothing queued, held or pending on the port: what a chip that wants + /// the rings left alone while they move checks before it lets a guest + /// change a mode. + pub fn port_quiescent(&self, p: usize) -> bool { + let port = &self.port[p]; + port.reg[1] == port.reg[2] + && port.reg[3] == port.reg[4] + && !port.tx_fetched + && port.tx_delay_deadline.is_none() + && port.packed == 0 + && !port.armed + && !port.drain_pending + } + + /// Both ports of a pair are off, with their indices at zero and nothing + /// pending. + pub fn pair_quiescent(&self, pair: usize) -> bool { + (pair * PAIR..pair * PAIR + PAIR).all(|p| { + self.control(p) & (ctl::ENABLE | ctl::PAUSE) == 0 + && self.port[p].reg[1..=4].iter().all(|&r| r == 0) + && self.port_quiescent(p) + }) + } + + /// Whether the port is stopped by a failed transfer. + pub fn halted(&self, p: usize) -> bool { + self.port[p].halted + } + + /// Looped back with the engine's hardware flow control on, the port + /// reports clear-to-send asserted even with request-to-send clear. + pub fn flow_cts(&self, p: usize, loopback: bool) -> bool { + loopback && self.control(p) & ctl::FLOW_CONTROL != 0 + } + + fn ring_size(&self, p: usize) -> u32 { + if self.base_lo[p / PAIR] & RING_4K != 0 { RING_4K_SIZE } else { RING_1K_SIZE } + } + + /// The selected ring size is also every index register's pointer width. + fn mask(&self, p: usize) -> u32 { + (self.ring_size(p) - 1) & !(ENTRY as u32 - 1) + } + + /// The size-select bit is not address. + fn ring(&self, p: usize, dir: u32) -> u64 { + let pair = p / PAIR; + let base = (u64::from(self.base_hi[pair]) << 32) | u64::from(self.base_lo[pair] & RING_ADDRESS); + base.saturating_add(((p % PAIR) as u64 * 2 + u64::from(dir)) * u64::from(self.ring_size(p))) + } + + fn running(&self, p: usize) -> bool { + let c = self.control(p); + !self.port[p].halted && c & ctl::ENABLE != 0 && c & ctl::PAUSE == 0 + } + + /// A transfer the fabric refused: the chip reports it, and may halt. + fn dma_failed(&mut self, p: usize, rx: bool, addr: u64, host: &mut dyn SerialDmaHost) { + if self.params.halt_on_dma_error { + self.port[p].halted = true; + } + host.dma_error(p, rx, addr); + } + + /// Transmit-ring-empty is a level: ring DMA on, nothing left in the ring + /// or held by the engine. + fn update_tx_empty(&self, p: usize, host: &mut dyn SerialDmaHost) { + let mask = self.mask(p); + let port = &self.port[p]; + let empty = self.control(p) & ctl::ENABLE != 0 + && (port.reg[1] ^ port.reg[2]) & mask == 0 + && !port.tx_fetched + && port.tx_delay_deadline.is_none(); + host.set_level(p, SerialDmaSource::TxEmpty, empty); + } + + /// The receive high-water mark counts ring entries; a level. So is + /// receive-ring-full: set while ring DMA is on and an entry waits, turned + /// away for lack of ring room. It does not come back with every refused + /// retry after software has acknowledged it, and it falls once the + /// consumer makes room, whether or not the held entry has posted yet. A + /// ring filled exactly, with nothing waiting, does not raise it. A model + /// choice; what the part does is not known. + fn update_high_water(&mut self, p: usize, host: &mut dyn SerialDmaHost) { + let mask = self.mask(p); + let hw = self.control(p) & ctl::HIGH_WATER; + let enabled = self.control(p) & ctl::ENABLE != 0; + let port = &mut self.port[p]; + let waiting = port.reg[3].wrapping_sub(port.reg[4]) & mask; + host.set_level(p, SerialDmaSource::RxHighWater, hw != 0 && waiting / ENTRY as u32 >= hw); + if (port.reg[3] + ENTRY as u32) & mask != port.reg[4] & mask { + port.post_refused = false; + } + host.set_level(p, SerialDmaSource::RxFull, enabled && port.post_refused); + } + + /// Post the packer as one entry. Fails, keeping the characters, when the + /// ring is full or the write is refused. + fn post(&mut self, p: usize, host: &mut dyn SerialDmaHost) -> bool { + let mask = self.mask(p); + let ring = self.ring(p, 1); + let (produce, consume, packer) = { + let port = &self.port[p]; + (port.reg[3], port.reg[4], port.packer) + }; + let next = (produce + ENTRY as u32) & mask; + if next == consume & mask { + // Full: refuse. The characters keep in the packer and the FIFO. + self.port[p].post_refused = true; + return false; + } + let at = ring.saturating_add(u64::from(produce & mask)); + if !self.dma_write(host, at, &packer) { + self.dma_failed(p, true, at, host); + return false; + } + let port = &mut self.port[p]; + port.reg[3] = next; + port.post_refused = false; + port.packed = 0; + port.packer = [0; ENTRY]; + true + } + + fn rx_timer_start(&mut self, p: usize, since: Instant) { + let port = &mut self.port[p]; + let reload = port.reg[5] & TIMER_RELOAD; + port.rx_idle_count = 0; + let dur = Duration::from_nanos(u64::from(reload) * TIMER_TICK_NS); + if port.paused { + // Started while paused, it waits frozen at its full count. + port.rx_frozen = Some(dur); + } else { + port.rx_deadline = Some(since + dur); + } + } + + fn rx_timer_expired(&mut self, p: usize, host: &mut dyn SerialDmaHost) { + self.port[p].rx_idle_count = 0; + self.port[p].rx_deadline = None; + if self.port[p].packed != 0 { + self.post(p, host); + } + // Raised at every expiry, even over an empty ring; the arm is spent. + if self.port[p].armed { + host.raise(p, SerialDmaSource::RxTimer); + } + self.port[p].armed = false; + self.update_high_water(p, host); + } + + /// Expiries already due happen before an access. + pub fn catch_up(&mut self, now: Instant, host: &mut dyn SerialDmaHost) { + for p in 0..self.port.len() { + if let Some(d) = self.port[p].tx_delay_deadline { + if !self.port[p].paused && d <= now { + self.tx_delay_ended(p, d, host); + } + } + // Each arrival can send the next character; the ring bounds it. + while let Some(d) = self.port[p].loop_due.filter(|&d| d <= now) { + self.loop_arrived(p, d, host); + } + if let Some(d) = self.port[p].rx_deadline { + if d <= now { + self.rx_timer_expired(p, host); + } + } + } + } + + /// A looped-back character reaches the receiver; the walk goes on from + /// then. + fn loop_arrived(&mut self, p: usize, when: Instant, host: &mut dyn SerialDmaHost) { + self.port[p].loop_due = None; + self.run_from(p, when, host); + } + + fn tx_delay_ended(&mut self, p: usize, when: Instant, host: &mut dyn SerialDmaHost) { + let irq = { + let port = &mut self.port[p]; + port.tx_delay_deadline = None; + std::mem::take(&mut port.tx_delay_irq) + }; + if irq { + host.raise(p, SerialDmaSource::TxExplicit); + } + self.run_from(p, when, host); + } + + fn receive(&mut self, p: usize, since: Instant, host: &mut dyn SerialDmaHost) { + // The 16550 core loops a character back the moment it is written; on + // the part it takes a character time on the line to arrive. + if self.port[p].loop_due.is_some() { + return; + } + while self.running(p) { + if self.port[p].packed == ENTRY_CHARS && !self.post(p, host) { + break; + } + // Line status before the character: reading it clears the latches. + let uart = host.uart(p); + let lsr = uart.read_core(UART_LSR); + if lsr & LSR_DR == 0 { + break; + } + let data = uart.read_core(UART_THR); + let mut msr = uart.read_core(UART_MSR); + let looped = uart.loopback(); + if self.flow_cts(p, looped) { + msr |= MSR_CTS; + } + let mut status = RX_DATA_OK | RX_MODEM_OK; + for (bit, flag) in [(LSR_OE, RX_OVERRUN), (LSR_PE, RX_PARITY), (LSR_FE, RX_FRAMING), (LSR_BI, RX_BREAK)] { + if lsr & bit != 0 { + status |= flag; + } + } + if msr & MSR_CTS != 0 { + status |= RX_CTS; + } + if msr & MSR_DCD != 0 { + status |= RX_CD; + } + let start_timer = { + let port = &mut self.port[p]; + port.packer[port.packed] = data; + port.packer[ENTRY_CHARS + port.packed] = status; + port.packed += 1; + port.armed && port.rx_deadline.is_none() && port.rx_frozen.is_none() + }; + // Started by the first character after the arm; later ones leave + // a running timer alone. + if start_timer { + self.rx_timer_start(p, since); + } + // A fourth character completes the entry; fewer wait for the + // timer or a drain. + if self.port[p].packed == ENTRY_CHARS { + self.post(p, host); + } + } + host.uart_line_changed(p); + } + + /// The type field and bit 5 decide what a slot does; the low five bits + /// have no defined meaning and are reported, not obeyed. Report them once, + /// when the entry is fetched, for the slots up to the one that ends it. + fn check_entry(p: usize, entry: &[u8; ENTRY]) { + let mut undefined = 0; + for &control in &entry[ENTRY_CHARS..] { + undefined |= control & TX_UNDEFINED; + if control & TX_TYPE == 0 { + break; + } + } + if undefined != 0 { + crate::dlog!( + crate::devlog::LogModule::Ioc, + "serial DMA: undefined TX control bits {undefined:#04x} in an entry on port {p}" + ); + } + } + + /// Each control byte types its data byte; the empty type ends the entry, + /// wherever it sits. + fn transmit(&mut self, p: usize, since: Instant, host: &mut dyn SerialDmaHost) { + let mask = self.mask(p); + let ring = self.ring(p, 0); + if self.port[p].tx_delay_deadline.is_some() { + return; + } + loop { + let (produce, consume) = { + let port = &self.port[p]; + (port.reg[1], port.reg[2]) + }; + if !(self.running(p) && consume & mask != produce & mask) { + break; + } + let was_fetched = self.port[p].tx_fetched; + if !was_fetched { + let mut entry = [0u8; ENTRY]; + let at = ring.saturating_add(u64::from(consume & mask)); + if !self.dma_read(host, at, &mut entry) { + self.dma_failed(p, false, at, host); + break; + } + let port = &mut self.port[p]; + port.tx_entry = entry; + port.tx_fetched = true; + } + let entry = self.port[p].tx_entry; + if !was_fetched { + Self::check_entry(p, &entry); + } + let mut slot = self.port[p].tx_slot; + while slot < ENTRY_CHARS { + let control = entry[ENTRY_CHARS + slot]; + let ty = control & TX_TYPE; + if ty == 0 { + break; + } + match ty { + TX_CHARACTER => { + // In loopback the line carries one character at a time. + if !host.uart(p).tx_ready() || self.port[p].loop_due.is_some() { + return; + } + let looped = host.uart(p).loopback(); + // What has already arrived goes ahead of this one. + if looped { + self.receive(p, since, host); + } + host.uart(p).write(UART_THR, entry[slot]); + if looped { + self.port[p].loop_due = Some(since + host.uart(p).char_time()); + } + // Drain per character or loopback overruns the FIFO. + self.receive(p, since, host); + } + TX_MODEM => host.uart(p).write(UART_MCR, entry[slot]), + TX_DELAY => { + // The data byte is milliseconds of quiet. + if entry[slot] != 0 { + let port = &mut self.port[p]; + port.tx_slot = slot + 1; + port.tx_delay_deadline = Some(since + Duration::from_millis(u64::from(entry[slot]))); + port.tx_delay_irq = control & TX_INTERRUPT != 0; + return; + } + } + _ => unreachable!("the type field has two bits"), + } + slot += 1; + self.port[p].tx_slot = slot; + if control & TX_INTERRUPT != 0 { + host.raise(p, SerialDmaSource::TxExplicit); + } + } + let port = &mut self.port[p]; + port.tx_slot = 0; + port.tx_fetched = false; + port.reg[2] = (port.reg[2] + ENTRY as u32) & mask; + } + host.uart_line_changed(p); + } + + /// The drain posts what the packer holds and reads back set until it has; + /// it waits while ring DMA is off or paused and while the ring is full. + fn drain(&mut self, p: usize, host: &mut dyn SerialDmaHost) { + if !self.port[p].drain_pending || !self.running(p) { + return; + } + if self.port[p].packed != 0 && !self.post(p, host) { + return; + } + self.port[p].drain_pending = false; + } + + /// Walk the port's rings from `since`: where a transmit delay ended, or + /// now. + pub fn run_from(&mut self, p: usize, since: Instant, host: &mut dyn SerialDmaHost) { + if !host.reachable(p) || self.port[p].busy { + return; + } + self.port[p].busy = true; + self.transmit(p, since, host); + self.receive(p, since, host); + self.drain(p, host); + self.update_high_water(p, host); + self.update_tx_empty(p, host); + self.port[p].busy = false; + } + + pub fn run(&mut self, p: usize, host: &mut dyn SerialDmaHost) { + self.run_from(p, Instant::now(), host); + } + + /// A passive mirror of the 16550's lines; no read side effects. + fn shadow(&self, p: usize, host: &mut dyn SerialDmaHost) -> u32 { + let u = host.uart(p); + let (lsr, mcr, mut msr) = (u.lsr(), u.mcr(), u.msr()); + if self.flow_cts(p, u.loopback()) { + msr |= MSR_CTS; + } + let mut s = u32::from(lsr); + for (m, bit) in [(MSR_DCTS, 16), (MSR_DDCD, 19), (MSR_CTS, 20), (MSR_DCD, 23)] { + if msr & m != 0 { + s |= 1 << bit; + } + } + for (m, bit) in [(MCR_DTR, 24), (MCR_RTS, 25), (MCR_OUT1, 26), (MCR_OUT2, 27), (MCR_LOOP, 28)] { + if mcr & m != 0 { + s |= 1 << bit; + } + } + s + } + + pub fn port_read(&mut self, p: usize, offset: u32, host: &mut dyn SerialDmaHost) -> u32 { + let now = Instant::now(); + self.catch_up(now, host); + if offset == off::SHADOW { + return self.shadow(p, host); + } + let port = &self.port[p]; + let mut v = port.reg[(offset / 4) as usize]; + match offset { + off::CONTROL => { + if v & ctl::PAUSE != 0 { + v |= ctl::PAUSED; + } + if port.tx_delay_deadline.is_some() { + v |= ctl::TX_TIMER_RUN; + } + if port.drain_pending { + v |= ctl::RX_TO_MEMORY; + } + } + off::RX_CONSUME => { + if port.armed { + v |= ARM_RX_TIMER; + } + } + off::RX_PRODUCE if self.params.held_counts => v |= (port.packed as u32) << HELD_SHIFT, + off::RX_TIMER => { + // Ticks left while running or frozen; idle, whatever a reload + // write last loaded. + let ticks_in = |d: Duration| (d.as_nanos() as u64 / TIMER_TICK_NS).min(u64::from(TIMER_RELOAD)) as u32; + let mut ticks = port.rx_idle_count; + if let (true, Some(left)) = (port.paused, port.rx_frozen) { + ticks = ticks_in(left); + } else if let Some(d) = port.rx_deadline { + ticks = ticks_in(d.saturating_duration_since(now)); + } + v |= ticks << TIMER_COUNT_SHIFT; + } + _ => {} + } + v + } + + /// Resets the four indices and the work in progress; the settings + /// survive, and a port a failed transfer stopped runs again. + fn reset_port(&mut self, p: usize) { + let port = &mut self.port[p]; + port.halted = false; + port.reg[1..=4].fill(0); + port.armed = false; + port.drain_pending = false; + port.post_refused = false; + port.packed = 0; + port.packer = [0; ENTRY]; + port.tx_slot = 0; + port.tx_fetched = false; + port.tx_delay_deadline = None; + port.tx_delay_irq = false; + port.rx_deadline = None; + port.rx_frozen = None; + port.rx_idle_count = 0; + } + + pub fn port_write(&mut self, p: usize, offset: u32, val: u32, host: &mut dyn SerialDmaHost) { + let now = Instant::now(); + self.catch_up(now, host); + let mask = self.mask(p); + match offset { + off::CONTROL => { + self.port[p].reg[0] = val & !ctl::UNSTORED; + if val & ctl::RESET != 0 { + self.reset_port(p); + } + if val & ctl::RX_TO_MEMORY != 0 { + self.port[p].drain_pending = true; + } + let port = &mut self.port[p]; + // A pause freezes the receive timer and a transmit delay; + // resuming lets each finish the time it had left. A model + // choice; no source says what the part does. + if val & ctl::PAUSE != 0 && !port.paused { + port.paused = true; + port.rx_frozen = port.rx_deadline.take().map(|d| d.saturating_duration_since(now)); + port.tx_delay_frozen = port.tx_delay_deadline.take().map(|d| d.saturating_duration_since(now)); + } else if val & ctl::PAUSE == 0 && port.paused { + port.paused = false; + port.rx_deadline = port.rx_frozen.take().map(|l| now + l); + port.tx_delay_deadline = port.tx_delay_frozen.take().map(|l| now + l); + } + if val & ctl::ENABLE == 0 { + port.rx_deadline = None; + port.rx_frozen = None; + } + self.run(p, host); + } + off::TX_PRODUCE | off::TX_CONSUME | off::RX_PRODUCE => { + self.port[p].reg[(offset / 4) as usize] = val & mask; + self.run(p, host); + } + off::RX_CONSUME => { + // Bit 31 sets the arm; a write with it clear leaves it. + self.port[p].reg[4] = val & mask; + if val & ARM_RX_TIMER != 0 { + self.port[p].armed = true; + } + self.run(p, host); + } + off::RX_TIMER => { + let port = &mut self.port[p]; + port.reg[5] = val & TIMER_RELOAD; + port.rx_idle_count = val & TIMER_RELOAD; + } + // The shadow discards writes silently. + _ => {} + } + self.update_tx_empty(p, host); + self.update_high_water(p, host); + } +} + +impl SerialDmaUart for Ns16550 { + fn loopback(&self) -> bool { + Ns16550::loopback(self) + } + + fn tx_ready(&self) -> bool { + Ns16550::tx_ready(self) + } + + fn char_time(&self) -> Duration { + Ns16550::char_time(self) + } + + fn read_core(&mut self, r: u32) -> u8 { + self.read(r) + } + + fn write(&mut self, r: u32, v: u8) { + Ns16550::write(self, r, v); + } + + fn lsr(&self) -> u8 { + Ns16550::lsr(self) + } + + fn mcr(&self) -> u8 { + Ns16550::mcr(self) + } + + fn msr(&self) -> u8 { + Ns16550::msr(self) + } +} + +#[cfg(test)] +mod tests { + //! The engine against a stand-in chip. The IOC3 and IOC4 register tests + //! cover the engine through each chip; these cover what only the second + //! chip's parameters change. + + use super::*; + + #[derive(Default)] + struct FakeUart { + sent: Vec, + rx: std::collections::VecDeque, + } + + impl SerialDmaUart for FakeUart { + fn loopback(&self) -> bool { + false + } + fn tx_ready(&self) -> bool { + true + } + fn char_time(&self) -> Duration { + Duration::ZERO + } + fn read_core(&mut self, reg: u32) -> u8 { + match reg { + UART_LSR => u8::from(!self.rx.is_empty()), + UART_THR => self.rx.pop_front().unwrap_or(0), + _ => 0, + } + } + fn write(&mut self, reg: u32, val: u8) { + if reg == UART_THR { + self.sent.push(val); + } + } + fn lsr(&self) -> u8 { + 0 + } + fn mcr(&self) -> u8 { + 0 + } + fn msr(&self) -> u8 { + 0 + } + } + + struct Chip { + uart: [FakeUart; MAX_PORTS], + mem: Vec, + /// Transfers below this address are refused. + refuse_below: u64, + raised: Vec<(usize, SerialDmaSource)>, + errors: Vec<(usize, bool, u64)>, + /// Whether the chip reports errors itself, as IOC4 does. + reports_errors: bool, + } + + impl Chip { + fn new() -> Chip { + Chip { + uart: Default::default(), + mem: vec![0; 0x4000], + refuse_below: 0, + raised: Vec::new(), + errors: Vec::new(), + reports_errors: true, + } + } + } + + impl SerialDmaHost for Chip { + fn uart(&mut self, port: usize) -> &mut dyn SerialDmaUart { + &mut self.uart[port] + } + fn raise(&mut self, port: usize, src: SerialDmaSource) { + self.raised.push((port, src)); + } + fn set_level(&mut self, _port: usize, _src: SerialDmaSource, _level: bool) {} + fn reachable(&self, _port: usize) -> bool { + true + } + fn uart_line_changed(&mut self, _port: usize) {} + fn dma_read(&self, addr: u64, buf: &mut [u8]) -> bool { + if addr < self.refuse_below { + return false; + } + buf.copy_from_slice(&self.mem[addr as usize..addr as usize + buf.len()]); + true + } + fn dma_write(&self, _addr: u64, _data: &[u8]) -> bool { + true + } + fn dma_error(&mut self, port: usize, rx: bool, addr: u64) { + if self.reports_errors { + self.errors.push((port, rx, addr)); + } else { + self.raise(port, SerialDmaSource::DmaError); + } + } + } + + const FOUR: SerialDmaParams = SerialDmaParams { held_counts: false, halt_on_dma_error: true }; + const TWO: SerialDmaParams = SerialDmaParams { held_counts: true, halt_on_dma_error: false }; + + /// One transmit entry holding `byte`, in the port's transmit ring. + fn queue_char(sdma: &mut SerialDma, chip: &mut Chip, p: usize, byte: u8) { + let at = sdma.ring(p, 0) as usize; + chip.mem[at..at + ENTRY].copy_from_slice(&[byte, 0, 0, 0, TX_CHARACTER, 0, 0, 0]); + let produce = sdma.stored(p, off::TX_PRODUCE); + sdma.port_write(p, off::TX_PRODUCE, produce + ENTRY as u32, chip); + } + + fn enable(sdma: &mut SerialDma, chip: &mut Chip, p: usize) { + sdma.port_write(p, off::CONTROL, ctl::ENABLE, chip); + } + + #[test] + fn a_port_in_the_second_pair_uses_the_second_ring_base() { + let mut sdma = SerialDma::new(4, FOUR); + let mut chip = Chip::new(); + sdma.base_write(0, false, 0x1000); + sdma.base_write(1, false, 0x2000); + assert_eq!(sdma.ring(0, 0), 0x1000); + assert_eq!(sdma.ring(1, 0), 0x1000 + 2 * 1024); + assert_eq!(sdma.ring(2, 0), 0x2000); + assert_eq!(sdma.ring(3, 1), 0x2000 + 3 * 1024); + enable(&mut sdma, &mut chip, 3); + queue_char(&mut sdma, &mut chip, 3, b'x'); + assert_eq!(chip.uart[3].sent, b"x"); + assert!(chip.uart[0].sent.is_empty(), "the other ports saw nothing"); + } + + #[test] + fn a_failed_fetch_halts_the_port_until_a_channel_reset() { + let mut sdma = SerialDma::new(4, FOUR); + let mut chip = Chip::new(); + sdma.base_write(0, false, 0x1000); + enable(&mut sdma, &mut chip, 0); + chip.refuse_below = 0x2000; + queue_char(&mut sdma, &mut chip, 0, b'a'); + assert_eq!(chip.errors, [(0, false, 0x1000)], "the chip is told where the fetch failed"); + assert!(sdma.halted(0)); + chip.refuse_below = 0; + sdma.run(0, &mut chip); + assert!(chip.uart[0].sent.is_empty(), "a halted port does not resume by itself"); + sdma.port_write(0, off::CONTROL, ctl::ENABLE | ctl::RESET, &mut chip); + assert!(!sdma.halted(0)); + assert_eq!(sdma.stored(0, off::TX_PRODUCE), 0, "the reset cleared the indices"); + queue_char(&mut sdma, &mut chip, 0, b'b'); + assert_eq!(chip.uart[0].sent, b"b"); + } + + #[test] + fn a_failed_fetch_leaves_a_chip_that_does_not_halt_running() { + let mut sdma = SerialDma::new(2, TWO); + let mut chip = Chip::new(); + chip.reports_errors = false; + sdma.base_write(0, false, 0x1000); + enable(&mut sdma, &mut chip, 0); + chip.refuse_below = 0x2000; + queue_char(&mut sdma, &mut chip, 0, b'a'); + assert_eq!(chip.raised.last(), Some(&(0, SerialDmaSource::DmaError)), "the default report is the port's own source"); + assert!(!sdma.halted(0)); + chip.refuse_below = 0; + sdma.run(0, &mut chip); + assert_eq!(chip.uart[0].sent, b"a", "the walk retries once the transfer works"); + } + + #[test] + fn only_a_chip_that_reports_held_counts_shows_them() { + for (params, shown) in [(TWO, true), (FOUR, false)] { + let mut sdma = SerialDma::new(2, params); + let mut chip = Chip::new(); + sdma.base_write(0, false, 0x1000); + enable(&mut sdma, &mut chip, 0); + chip.uart[0].rx.extend([1, 2]); + sdma.run(0, &mut chip); + let produce = sdma.port_read(0, off::RX_PRODUCE, &mut chip); + assert_eq!(produce >> HELD_SHIFT & 7, if shown { 2 } else { 0 }); + } + } + + #[test] + fn the_engine_is_idle_between_calls_whether_a_transfer_worked_or_not() { + let mut sdma = SerialDma::new(4, FOUR); + let mut chip = Chip::new(); + assert!(sdma.idle()); + sdma.base_write(0, false, 0x1000); + enable(&mut sdma, &mut chip, 0); + queue_char(&mut sdma, &mut chip, 0, b'a'); + assert!(sdma.idle(), "after a good transfer"); + chip.refuse_below = 0x2000; + queue_char(&mut sdma, &mut chip, 0, b'b'); + assert!(sdma.halted(0)); + assert!(sdma.idle(), "after a failed one"); + } +} + +// The engine's state for a snapshot of the chip that carries it. Each timer +// is saved as the time it had left (see the IOC3's snapshot helpers). + +impl SerialDmaPort { + fn save_state(&self, now: Instant) -> toml::Value { + StateTable::new() + .u32s("reg", &self.reg) + .u8s("packer", &self.packer) + .u64("packed", self.packed as u64) + .u8s("tx_entry", &self.tx_entry) + .bool("tx_fetched", self.tx_fetched) + .u64("tx_slot", self.tx_slot as u64) + .bool("armed", self.armed) + .bool("drain_pending", self.drain_pending) + .bool("busy", self.busy) + .bool("paused", self.paused) + .u64("rx_due", until(self.rx_deadline, now)) + .u32("rx_idle_count", self.rx_idle_count) + .u64("rx_frozen", span(self.rx_frozen)) + .u64("tx_delay_due", until(self.tx_delay_deadline, now)) + .u64("tx_delay_frozen", span(self.tx_delay_frozen)) + .bool("tx_delay_irq", self.tx_delay_irq) + .bool("post_refused", self.post_refused) + .bool("halted", self.halted) + .u64("loop_due", until(self.loop_due, now)) + .build() + } + fn load_state(v: &toml::Value, now: Instant) -> Result { + Ok(Self { + reg: field_u32s(v, "reg")?, + packer: field_u8s(v, "packer")?, + packed: usize_field(v, "packed", ENTRY)?, + tx_entry: field_u8s(v, "tx_entry")?, + tx_fetched: field_bool(v, "tx_fetched")?, + tx_slot: usize_field(v, "tx_slot", ENTRY)?, + armed: field_bool(v, "armed")?, + drain_pending: field_bool(v, "drain_pending")?, + busy: field_bool(v, "busy")?, + paused: field_bool(v, "paused")?, + rx_deadline: due(field_u64(v, "rx_due")?, now), + rx_idle_count: field_u32(v, "rx_idle_count")?, + rx_frozen: unspan(field_u64(v, "rx_frozen")?), + tx_delay_deadline: due(field_u64(v, "tx_delay_due")?, now), + tx_delay_frozen: unspan(field_u64(v, "tx_delay_frozen")?), + tx_delay_irq: field_bool(v, "tx_delay_irq")?, + post_refused: field_bool(v, "post_refused")?, + halted: field_bool(v, "halted")?, + loop_due: due(field_u64(v, "loop_due")?, now), + }) + } +} + +impl SerialDma { + /// The ring bases and each port's state, timers as the time left after + /// `now`. + pub fn save_state(&self, now: Instant) -> toml::Value { + let mut t = StateTable::new(); + t.u32s("base_hi", &self.base_hi).u32s("base_lo", &self.base_lo); + for (i, port) in self.port.iter().enumerate() { + t.value(&format!("port{i}"), port.save_state(now)); + } + t.build() + } + + /// The engine `save_state` described, with this engine's parameters + /// and port count, timers set that far after `now`. + pub fn loaded(&self, v: &toml::Value, now: Instant) -> Result { + let port = (0..self.port.len()) + .map(|i| SerialDmaPort::load_state(field(v, &format!("port{i}"))?, now)) + .collect::, String>>()?; + // Transfers are under way only inside a call, never at a save. + Ok(SerialDma { params: self.params, base_hi: field_u32s(v, "base_hi")?, base_lo: field_u32s(v, "base_lo")?, in_flight: 0, port }) + } +} diff --git a/src/dev/isp1040.rs b/src/dev/isp1040.rs index aeb90dd41..aa88acbe5 100644 --- a/src/dev/isp1040.rs +++ b/src/dev/isp1040.rs @@ -50,8 +50,9 @@ //! again when the hold ends: an approximation, a chip that had fetched it //! would plausibly move on. A parked entry the guest rewrites meanwhile is //! answered incomplete. A snapshot carries each bus's remaining hold and -//! the parked positions. The ISP12160 would widen `BUSES` and decode the -//! bus from mailbox 2 and its request entries; the per-bus state is there. +//! the parked positions. Only the ISP1040's bus reset starts a hold-off: +//! the ISP12160 (two buses, per-bus state is there) keeps its own bus-reset +//! handling, and its hold-off is not modelled. //! * Only logical unit 0 of a present target exists. A command for another //! unit is passed to the target with that unit number, which answers as //! SCSI says (INQUIRY: no device at this unit; anything else: check @@ -81,7 +82,7 @@ use parking_lot::{Condvar, Mutex, MutexGuard}; use crate::devlog::LogModule; use crate::irq::IrqLine; -use crate::pci::{Bar, PciConfig, PciDma, PciFunction, PciFunctionState, PCI_INTA}; +use crate::pci::{Bar, PciConfig, PciConfigPolicy, PciDma, PciFunction, PciFunctionState, PciRegisterPolicy, PCI_INTA}; use crate::scsi::{scsi_cmd, ScsiBus, ScsiDataLength, ScsiIdentities, ScsiRequest, CDROM_IDENTITY, DISK_IDENTITY}; use crate::snapshot::{get_field, hex_u16, hex_u32, load_u16_slice, toml_u16, toml_u32, toml_u8, u16_slice_to_toml, u8_slice_to_toml}; @@ -115,7 +116,7 @@ pub mod reg { pub const WINDOW_SIZE: u32 = 0x100; } -const MAILBOX_COUNT: usize = 8; +pub(crate) const MAILBOX_COUNT: usize = 8; const MAILBOX_END: u32 = reg::MAILBOX_BASE + MAILBOX_COUNT as u32 * 2; const BANK_END: u32 = 0xc0; const BANK_WORDS: usize = ((BANK_END - reg::BANK_BASE) / 2) as usize; @@ -163,6 +164,8 @@ pub mod mbox { pub const INIT_REQ_QUEUE: u16 = 0x0010; pub const INIT_RSP_QUEUE: u16 = 0x0011; pub const ABORT_COMMAND: u16 = 0x0015; + pub const ABORT_DEVICE: u16 = 0x0016; + pub const ABORT_TARGET: u16 = 0x0017; pub const BUS_RESET: u16 = 0x0018; pub const GET_FIRMWARE_STATUS: u16 = 0x001f; pub const GET_TARGET_PARAMETERS: u16 = 0x0028; @@ -187,13 +190,14 @@ pub mod mbox { pub const INVALID: u16 = 0x4001; pub const HOST_INTERFACE_ERROR: u16 = 0x4002; pub const TEST_FAILED: u16 = 0x4003; + pub const COMMAND_ERROR: u16 = 0x4005; pub const PARAM_ERROR: u16 = 0x4006; } const RAM_WORDS: usize = 0x10000; /// The command budget the adapter announces in its firmware status. pub const MAX_REQUESTS: u16 = 512; -const FW_VERSION: [u16; 3] = [1, 0, 0]; +pub(crate) const FW_VERSION: [u16; 3] = [1, 0, 0]; pub const HOST_CMD_SHIFT: u32 = 12; pub const RISC_FLAG_HOST_INT: u16 = 0x0080; @@ -229,6 +233,7 @@ pub const SCSI_STATUS_SENSE_VALID: u16 = 0x0200; pub const IOCB_COMPLETE: u16 = 0x0000; pub const IOCB_INCOMPLETE: u16 = 0x0001; pub const IOCB_RESET: u16 = 0x0004; +pub const IOCB_ABORTED: u16 = 0x0005; pub const IOCB_DATA_UNDERRUN: u16 = 0x0015; pub const STATE_GOT_BUS: u16 = 0x0100; @@ -256,13 +261,14 @@ const DEFAULT_SELECTION_TIMEOUT: u16 = 250; /// source for a device, so it runs on the adapter's clock, host wall time /// less the time the machine was stopped, like the selection timeout. const BUS_RESET_DELAY_UNIT: Duration = Duration::from_secs(1); -/// The SCSI buses this part drives. The hold-off, the parked commands and the -/// selection are all kept per bus, so a two-bus part widens this and decodes -/// the bus from its own request entries and from mailbox 2. -const BUSES: usize = 1; +/// The most SCSI buses any part drives (the ISP12160's two; the ISP1040 has +/// one, `IspPart::channels`). The hold-off, the parked commands and the +/// selection are all kept per bus. Only the ISP1040's bus reset starts a +/// hold-off: the ISP12160 keeps its own bus-reset handling. +const BUSES: usize = 2; const CHECK_CONDITION: u8 = 0x02; -const MAX_TARGET: u8 = 15; +pub(crate) const MAX_TARGET: u8 = 15; const MAX_LUN: u8 = 7; /// A completion held for response-ring room. @@ -374,6 +380,52 @@ enum Prepared { Run(Job), } +/// Part-specific register and firmware behavior; the ring engine owns the state. +/// Defaults preserve the ISP1040 contract, including wrapping firmware RAM. +/// The engine is built once per part, so a part's choices are compiled in +/// rather than looked up on each register access. +pub(crate) trait IspPart: Send + Sync + Sized + 'static { + fn name(&self) -> &'static str; + fn channels(&self) -> usize { 1 } + fn pci_config(&self, bars: [Bar; 6]) -> PciConfig { + PciConfig::new(PCI_VENDOR_QLOGIC, PCI_DEVICE_ISP1040, PCI_CLASS_SCSI, PCI_REVISION_ISP1040, bars, PCI_INTA) + } + fn firmware_version(&self) -> [u16; 3] { FW_VERSION } + fn active_bank(&self, cfg1: u16) -> Bank { + if cfg1 & CFG1_BANK_SELECT != 0 { Bank::Sxp } else { Bank::Risc } + } + fn checks_ram_bounds(&self) -> bool { false } + fn has_abort_commands(&self) -> bool { false } + fn target_on(&self, _channel: u8, target: u8) -> u8 { target } + fn decode_target(&self, target: u8) -> (usize, u8) { (0, target) } + fn id_high(&self) -> u16 { PCI_DEVICE_ISP1040 } + /// The ISP1040 register revision runs one ahead of its PCI revision. + fn bus_cfg0(&self) -> u16 { PCI_REVISION_ISP1040 as u16 + 1 } + fn cfg1_write_mask(&self) -> u16 { CFG1_WRITE_MASK } + fn unmodelled_dma_register(&self, _off: u32) -> bool { false } + fn completion_phases(&self, _moved: u64, _total: u64) -> u16 { STATE_ALL_PHASES & !STATE_GOT_SENSE } + fn mailbox_ready(&self, _semaphore: u16) -> bool { true } + /// Extra target-parameter results, beyond the common asynchronous/narrow reply. + fn clear_target_parameters(&self, _mailbox: &mut [u16; MAILBOX_COUNT]) {} + fn set_reset_delay(&self, st: &mut State) { st.reset_delay = st.mailbox[1]; } + /// Queue target notifications for the worker; the CPU never locks a bus. + fn reset_targets(&self, _st: &mut State) -> bool { false } + /// Handle extra mailbox commands, leaving result signalling to the engine. + fn execute_mailbox(&self, _core: &Core, _st: &mut MutexGuard<'_, State>) -> bool { false } + fn save_extra(&self, _st: &State, _t: &mut toml::map::Map) {} + fn load_extra(&self, _st: &mut State, _v: &toml::Value) {} +} + +/// The Octane's and the Origin 200's single-bus part. +pub struct Isp1040Part; + +impl IspPart for Isp1040Part { + fn name(&self) -> &'static str { "ISP1040" } +} + +/// The register window selected by BUS_CFG1, independent of its encoding. +pub(crate) enum Bank { Risc, Sxp, Sxp1, Dma } + /// A command waiting out its bus's post-reset hold-off. Only the ring /// position is kept: the entry, its data segments included, is read again /// from the guest's ring when the hold ends. @@ -392,7 +444,15 @@ struct Parked { started: bool, } -struct State { +pub(crate) struct State { + pub(crate) reset_targets: bool, + pub(crate) pending_target_resets: VecDeque<(u8, Option)>, + pub(crate) active: Option<(u8, u8, u32)>, + cancelled: bool, + pub(crate) outstanding: Vec<(u8, u8, u32)>, + pub(crate) sxp1_bank: [u16; BANK_WORDS], + pub(crate) dma_bank: [u16; BANK_WORDS], + pub(crate) reset_delay1: u16, ctl: Ctl, bus_cfg1: u16, bus_ctrl: u16, @@ -402,11 +462,11 @@ struct State { risc_flags: u16, dma_cmd_cfg: u16, dma_data_cfg: u16, - mailbox: [u16; MAILBOX_COUNT], + pub(crate) mailbox: [u16; MAILBOX_COUNT], risc_bank: [u16; BANK_WORDS], sxp_bank: [u16; BANK_WORDS], risc_ram: Vec, - reset_delay: u16, + pub(crate) reset_delay: u16, /// Mailbox command 0x0031's value, in [`SELECTION_TIMEOUT_UNIT`]s. selection_timeout: u16, rqst_base: u64, @@ -434,6 +494,14 @@ struct State { impl State { fn new() -> Self { let mut s = State { + reset_targets: false, + pending_target_resets: VecDeque::new(), + active: None, + cancelled: false, + outstanding: Vec::new(), + sxp1_bank: [0; BANK_WORDS], + dma_bank: [0; BANK_WORDS], + reset_delay1: 0, ctl: Ctl::default(), bus_cfg1: 0, bus_ctrl: 0, @@ -485,6 +553,12 @@ impl State { self.mailbox = [0, SIG1, SIG2, SIG3, SIG4, 0, 0, 0]; self.risc_bank = [0; BANK_WORDS]; self.sxp_bank = [0; BANK_WORDS]; + self.sxp1_bank = [0; BANK_WORDS]; + self.dma_bank = [0; BANK_WORDS]; + self.reset_delay1 = 0; + self.active = None; + self.cancelled = false; + self.outstanding.clear(); // The driver re-downloads firmware after every reset. self.risc_ram.iter_mut().for_each(|w| *w = 0); self.reset_delay = 0; @@ -499,6 +573,8 @@ impl State { self.resp_seen = 0; self.resp_cons = 0; self.pending.clear(); + self.reset_targets = false; + self.pending_target_resets.clear(); self.quiet_until = [None; BUSES]; self.parked.clear(); } @@ -524,29 +600,41 @@ impl State { self.resp_len != 0 && (u32::from(self.resp_prod) + 1) % u32::from(self.resp_len) != u32::from(self.resp_cons) } - fn active_bank(&mut self) -> &mut [u16; BANK_WORDS] { - if self.bus_cfg1 & CFG1_BANK_SELECT != 0 { &mut self.sxp_bank } else { &mut self.risc_bank } + fn active_bank(&mut self, bank: Bank) -> &mut [u16; BANK_WORDS] { + match bank { + Bank::Risc => &mut self.risc_bank, + Bank::Sxp => &mut self.sxp_bank, + Bank::Sxp1 => &mut self.sxp1_bank, + Bank::Dma => &mut self.dma_bank, + } } + } /// The adapter. Its state lives in a [`Core`] shared with the thread that /// runs the request ring. -pub struct Isp1040 { - core: Arc, +#[allow(private_bounds)] +pub struct Isp { + core: Arc>, worker: Mutex>>, } -impl std::ops::Deref for Isp1040 { - type Target = Core; - fn deref(&self) -> &Core { &self.core } +/// The ISP1040. +pub type Isp1040 = Isp; + +#[allow(private_bounds)] +impl std::ops::Deref for Isp

{ + type Target = Core

; + fn deref(&self) -> &Core

{ &self.core } } /// Lock order: `line_order`, then `st`. Nothing else is taken with `st` /// held except the leaf locks (`notes`, `dma_misses`, `odd_accesses`): DMA, -/// the targets and `fence` are reached with it released. `fence` is held -/// across one guest-memory store at a time, and `targets` across one +/// either bus and `fence` are reached with it released. `fence` is held +/// across one guest-memory store at a time, and either bus across one /// target command; neither is held with the other. -pub struct Core { +#[allow(private_bounds)] +pub struct Core { st: Mutex, /// `st.ctl.gen`, readable without `st`: what a store into guest memory /// checks under `fence`. @@ -560,13 +648,18 @@ pub struct Core { wake: Condvar, /// Serialises driving the interrupt line; see [`Core::sync_irq`]. line_order: Mutex<()>, - /// The targets on the bus, by SCSI id. - bus: ScsiBus, - /// A bus reset the targets have not yet seen. `st` and `bus` are never - /// held together, so the reset (taken under `st`) is passed on by the - /// next command to take `bus`, before it runs: no command can see the - /// targets as they were. See [`Core::scsi`]. - targets_reset: AtomicBool, + /// Generation for commands cancelled by aborts without resetting the chip. + cancel: AtomicU64, + /// Part policy, shared by register, worker and snapshot paths. + part: P, + /// The controller's one bus, or both channels of the ISP12160: one + /// per `IspPart::channels`. + buses: Box<[ScsiBus]>, + /// A bus reset the targets on that bus have not yet seen. `st` and the + /// buses are never held together, so the reset (taken under `st`) is + /// passed on by the next command to take the bus, before it runs: no + /// command can see the targets as they were. See [`Core::scsi`]. + targets_reset: [AtomicBool; 2], /// Which of the Octane's two controllers this is (0 internal, 1 /// external): the monitor's `scsi`/`cow` address the first, /// `scsi1`/`cow1` the second. @@ -601,7 +694,24 @@ impl Isp1040 { /// The controller at `index` (0 internal, 1 external) on `bus`. pub fn new_at(index: u8, bus: ScsiBus) -> Self { - Self { core: Arc::new(Core::new_at(index, bus)), worker: Mutex::new(None) } + Self::with_part(index, Isp1040Part, Box::new([bus])) + } +} + +/// The two-channel part on the engine directly, for the tests. +#[cfg(test)] +impl Isp { + pub(crate) fn new_12160(bus0: ScsiBus, bus1: ScsiBus) -> Self { + Self::with_part(0, crate::dev::isp12160::Isp12160Part, Box::new([bus0, bus1])) + } +} + +#[allow(private_bounds)] +impl Isp

{ + /// The controller at `index` with one bus per channel of `part`. + pub(crate) fn with_part(index: u8, part: P, buses: Box<[ScsiBus]>) -> Self { + assert_eq!(buses.len(), part.channels(), "one bus per channel"); + Self { core: Arc::new(Core::new_at(index, part, buses)), worker: Mutex::new(None) } } /// Start the ring thread (idempotent) and let it take whatever was @@ -615,7 +725,7 @@ impl Isp1040 { let c = self.core.clone(); *worker = Some( std::thread::Builder::new() - .name(format!("ISP1040-{}", self.index)) + .name(format!("{}-{}", self.core.model_name(), self.index)) .spawn(move || c.run_ring()) .expect("spawn isp1040 thread"), ); @@ -635,50 +745,57 @@ impl Isp1040 { } } -impl Isp1040 { +#[allow(private_bounds)] +impl Isp

{ /// Name the adapter's long-lived locks in the lock registry, which /// holds only a weak reference to them. pub fn register_locks(&self) { - for (what, probe) in [ - ("st", (|c: &Core| c.st.is_locked()) as fn(&Core) -> bool), + let mut locks: Vec<(&str, fn(&Core

) -> bool)> = vec![ + ("st", (|c: &Core

| c.st.is_locked()) as fn(&Core

) -> bool), ("fence", |c| c.fence.is_locked()), - ("targets", |c| c.bus.is_locked()), - ] { + ("targets", |c| c.buses[0].is_locked()), + ]; + if self.core.part.channels() > 1 { + locks.push(("targets1", |c| c.buses[1].is_locked())); + } + for (what, probe) in locks { let core = Arc::downgrade(&self.core); - crate::locks::register_lock(format!("isp1040-{}::{what}", self.index), move || { + crate::locks::register_lock(format!("{}-{}::{what}", self.core.model_name().to_ascii_lowercase(), self.index), move || { core.upgrade().is_some_and(|c| probe(&c)) }); } } } -impl Drop for Isp1040 { +#[allow(private_bounds)] +impl Drop for Isp

{ fn drop(&mut self) { // Backstop for a card dropped while started; the machine stops it. self.stop_ring(); } } -impl Core { - fn new_at(index: u8, bus: ScsiBus) -> Self { +#[allow(private_bounds)] +impl Core

{ + fn new_at(index: u8, part: P, buses: Box<[ScsiBus]>) -> Self { let st = State::new(); let gen = st.ctl.gen; + let bars = [Bar::Io(reg::WINDOW_SIZE), Bar::Mem(reg::WINDOW_SIZE), Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused]; + let pci = part.pci_config(bars); Self { + part, + cancel: AtomicU64::new(0), st: Mutex::new(st), gen: AtomicU64::new(gen), fence: Mutex::new(()), wake: Condvar::new(), line_order: Mutex::new(()), - bus, - targets_reset: AtomicBool::new(false), + buses, + targets_reset: [AtomicBool::new(false), AtomicBool::new(false)], index, // BAR 0 is the I/O window and BAR 1 its memory-space alias, one // register file. - pci: PciFunctionState::new(PciConfig::new( - PCI_VENDOR_QLOGIC, PCI_DEVICE_ISP1040, PCI_CLASS_SCSI, PCI_REVISION_ISP1040, - [Bar::Io(reg::WINDOW_SIZE), Bar::Mem(reg::WINDOW_SIZE), Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused], - PCI_INTA, - )), + pci: PciFunctionState::new(pci), notes: Mutex::new(Vec::new()), dma_misses: Mutex::new(std::collections::BTreeSet::new()), odd_accesses: Mutex::new(0), @@ -688,6 +805,8 @@ impl Core { } } + fn model_name(&self) -> &'static str { self.part.name() } + pub fn index(&self) -> u8 { self.index } /// A machine reset: the chip as at power-on (no firmware, no queues), @@ -698,6 +817,7 @@ impl Core { { let mut st = self.st.lock(); st.reset(); + if self.part.reset_targets(&mut st) { self.kick(&mut st); } self.publish_gen(&st); self.wake.notify_all(); } @@ -919,7 +1039,7 @@ impl Core { } /// Tell the ring thread the ring may have moved. - fn kick(&self, st: &mut State) { + pub(crate) fn kick(&self, st: &mut State) { st.ctl.work = true; self.wake.notify_all(); } @@ -947,11 +1067,16 @@ impl Core { /// The SCSI bus: attach targets to it and ask what is on it. pub fn bus(&self) -> &ScsiBus { - &self.bus + &self.buses[0] + } + + pub(crate) fn bus_for_channel(&self, channel: u8) -> Option<&ScsiBus> { + ((channel as usize) < self.part.channels()).then(|| &self.buses[channel as usize]) } - fn has_target(&self, id: u8) -> bool { - self.bus.has_target(id) + fn bus_for_target(&self, target: u8) -> (&ScsiBus, u8) { + let (channel, id) = self.part.decode_target(target); + (&self.buses[channel], id) } // ---- DMA --------------------------------------------------------- @@ -1035,12 +1160,18 @@ impl Core { /// A RISC RAM load or dump of up to 128 KiB, with the state unlocked /// for the DMA. The whole transfer is read before a word of it is kept, /// so a refused DMA leaves the array as it was. Words travel - /// little-endian. This part has no edge to its array: a transfer past - /// the end folds to the start. - fn ram_transfer(&self, st: &mut MutexGuard<'_, State>, load: bool) -> u16 { + /// little-endian. The ISP1040 wraps past the end; the ISP12160 rejects + /// transfers that would cross it. + pub(crate) fn ram_transfer(&self, st: &mut MutexGuard<'_, State>, load: bool) -> u16 { let base = st.mailbox[1]; let count = st.mailbox[4] as usize; - let host = Self::mbox_host_addr(&st.mailbox); + if self.part.checks_ram_bounds() && base as usize + count > RAM_WORDS { + return mbox::PARAM_ERROR; + } + let mut host = Self::mbox_host_addr(&st.mailbox); + if st.mailbox[0] == mbox::LOAD_RAM_A64 { + host |= ((st.mailbox[7] as u64) << 32) | ((st.mailbox[6] as u64) << 48); + } if count == 0 { return mbox::COMPLETE; } @@ -1067,6 +1198,20 @@ impl Core { /// Synchronous: the protocol has no way to leave a command pending. fn mailbox_command(&self, st: &mut MutexGuard<'_, State>) { let cmd = st.mailbox[0]; + if self.part.has_abort_commands() && matches!(cmd, mbox::ABORT_COMMAND | mbox::ABORT_DEVICE | mbox::ABORT_TARGET | mbox::BUS_RESET) { + self.with_dispatch_held(st, |st| self.execute_mailbox_command(st)); + } else { + self.execute_mailbox_command(st); + } + } + + fn execute_mailbox_command(&self, st: &mut MutexGuard<'_, State>) { + let cmd = st.mailbox[0]; + if self.part.execute_mailbox(self, st) { + st.semaphore = SEMAPHORE_LOCK; + st.bus_status |= STATUS_INT_REQ | STATUS_RISC_INT_REQ; + return; + } match cmd { mbox::NOP => st.mailbox[0] = mbox::COMPLETE, mbox::WRITE_RAM_WORD => { @@ -1088,7 +1233,19 @@ impl Core { // zero. mbox::VERIFY_CHECKSUM => { let start = st.mailbox[1]; + if self.part.checks_ram_bounds() && start as usize + 4 > RAM_WORDS { + st.mailbox[0] = mbox::PARAM_ERROR; + st.semaphore = SEMAPHORE_LOCK; + st.bus_status |= STATUS_INT_REQ | STATUS_RISC_INT_REQ; + return; + } let len = st.risc_ram[start.wrapping_add(3) as usize]; + if self.part.checks_ram_bounds() && start as usize + len as usize > RAM_WORDS { + st.mailbox[0] = mbox::PARAM_ERROR; + st.semaphore = SEMAPHORE_LOCK; + st.bus_status |= STATUS_INT_REQ | STATUS_RISC_INT_REQ; + return; + } let mut sum = 0u16; for i in 0..len { sum = sum.wrapping_add(st.risc_ram[start.wrapping_add(i) as usize]); @@ -1101,9 +1258,7 @@ impl Core { // Accepted, never executed. mbox::EXEC_FIRMWARE => st.mailbox[0] = mbox::COMPLETE, mbox::ABOUT_FIRMWARE => { - st.mailbox[1] = FW_VERSION[0]; - st.mailbox[2] = FW_VERSION[1]; - st.mailbox[3] = FW_VERSION[2]; + st.mailbox[1..4].copy_from_slice(&self.part.firmware_version()); st.mailbox[6] = 0; st.mailbox[0] = mbox::COMPLETE; } @@ -1118,6 +1273,7 @@ impl Core { mbox::GET_TARGET_PARAMETERS => { st.mailbox[2] = 0; st.mailbox[3] = 0; + self.part.clear_target_parameters(&mut st.mailbox); st.mailbox[0] = mbox::COMPLETE; } // Setters for state nothing reads back; answering invalid is a @@ -1139,7 +1295,7 @@ impl Core { st.mailbox[0] = mbox::COMPLETE; } mbox::SET_RESET_DELAY_PARAMS => { - st.reset_delay = st.mailbox[1]; + self.part.set_reset_delay(st); st.mailbox[0] = mbox::COMPLETE; } mbox::INIT_REQ_QUEUE | mbox::INIT_REQ_QUEUE_A64 | mbox::INIT_RSP_QUEUE | mbox::INIT_RSP_QUEUE_A64 => { @@ -1194,7 +1350,7 @@ impl Core { mbox::BUS_RESET => { let now = self.now_ns(st); let bus = usize::from(st.mailbox[2]); - if bus >= BUSES { + if bus >= self.part.channels() { // A bus this part does not have: nothing to quiet or // retire, and the datasheet's per-bus hold leaves the // real bus alone. Noted so a driver that leaves mailbox 2 @@ -1218,7 +1374,7 @@ impl Core { st.pending.push_back(Pending { handle: p.handle, completion: IOCB_RESET, ..Default::default() }); } } - self.targets_reset.store(true, Ordering::Release); + self.targets_reset[bus].store(true, Ordering::Release); let delay = BUS_RESET_DELAY_UNIT * u32::from(st.mailbox[1]); st.quiet_until[bus] = (!delay.is_zero()).then(|| now.saturating_add(delay.as_nanos() as i64)); self.kick(st); @@ -1235,6 +1391,67 @@ impl Core { st.bus_status |= STATUS_INT_REQ | STATUS_RISC_INT_REQ; } + /// Prevent the worker from taking another entry while a mailbox drops `st`. + fn with_dispatch_held(&self, st: &mut MutexGuard<'_, State>, f: impl FnOnce(&mut MutexGuard<'_, State>) -> R) -> R { + struct Hold<'a, 'b, P: IspPart> { + core: &'a Core

, + st: &'a mut MutexGuard<'b, State>, + } + impl Drop for Hold<'_, '_, P> { + fn drop(&mut self) { + self.st.ctl.hold -= 1; + self.core.kick(self.st); + } + } + st.ctl.hold += 1; + let held = Hold { core: self, st }; + f(held.st) + } + + /// A reset of `channel`'s bus: each disk on it puts its mode parameters back before the next + /// command, see [`ScsiTarget::reset_mode_parameters`]. A channel the part lacks is ignored. + pub(crate) fn note_bus_reset(&self, channel: usize) { + if let Some(flag) = self.targets_reset.get(channel) { flag.store(true, Ordering::Release); } + } + + /// Notify targets with `st` released; a blocked target must not stall register access. + fn reset_targets(&self, st: &mut MutexGuard<'_, State>, channel: u8, target: Option) { + let Some(bus) = self.bus_for_channel(channel) else { return }; + self.with_dispatch_held(st, |st| MutexGuard::unlocked(st, || bus.for_each(|id, dev| { + if target.is_none_or(|target| target as usize == id) { + dev.bus_reset(); + } + }))); + } + + pub(crate) fn retire_outstanding(&self, st: &mut State, matches: impl Fn((u8, u8, u32)) -> bool, completion: u16) -> usize { + let mut retired = 0; + let mut keep = Vec::new(); + for item in st.outstanding.drain(..) { + if matches(item) { + st.pending.push_back(Pending { handle: item.2, completion, ..Pending::default() }); + retired += 1; + } else { + keep.push(item); + } + } + st.outstanding = keep; + if retired != 0 { self.kick(st); } + retired + } + + /// Queue one cancellation reply, then fence its remaining guest-memory stores. + pub(crate) fn cancel_active(&self, st: &mut MutexGuard<'_, State>, completion: u16) { + if st.cancelled { return; } + if let Some((_, _, handle)) = st.active { + self.cancel.fetch_add(1, Ordering::AcqRel); + st.cancelled = true; + st.pending.push_back(Pending { handle, completion, ..Pending::default() }); + self.kick(st); + MutexGuard::unlocked(st, || self.retire_stores()); + } + } + fn host_command(&self, st: &mut MutexGuard<'_, State>, val: u16) { match val >> HOST_CMD_SHIFT { 0x0 => {} @@ -1245,6 +1462,10 @@ impl Core { 0x3 => st.risc_flags &= !(RISC_FLAG_PAUSED | RISC_FLAG_RESET), // "A mailbox command is ready": the trigger to process one. 0x5 => { + if !self.part.mailbox_ready(st.semaphore) { + self.note("mailbox submitted with an unacknowledged result".into()); + return; + } st.risc_flags |= RISC_FLAG_HOST_INT; self.mailbox_command(st); } @@ -1430,17 +1651,19 @@ impl Core { /// Run a CDB against the target; `data` is what a write sends it. None /// when a reset cancelled the command while it waited for the target. - fn scsi(&self, gen: u64, target: u8, cdb: &[u8], data: Option>) -> Option<(Vec, u8)> { - let mut targets = self.bus.lock(); - if self.targets_reset.swap(false, Ordering::AcqRel) { + fn scsi(&self, gen: u64, cancel: u64, target: u8, cdb: &[u8], data: Option>) -> Option<(Vec, u8)> { + let (channel, _) = self.part.decode_target(target); + let (bus, id) = self.bus_for_target(target); + let mut targets = bus.lock(); + if self.targets_reset[channel].swap(false, Ordering::AcqRel) { // The bus reset puts each target's mode parameters back; see // [`ScsiTarget::reset_mode_parameters`]. targets.iter_mut().flatten().for_each(|t| t.reset_mode_parameters()); } - if self.gen.load(Ordering::Acquire) != gen { + if self.gen.load(Ordering::Acquire) != gen || self.cancel.load(Ordering::Acquire) != cancel { return None; } - let Some(dev) = targets.get_mut(target as usize).and_then(|t| t.as_mut()) else { + let Some(dev) = targets.get_mut(id as usize).and_then(|t| t.as_mut()) else { return Some((Vec::new(), CHECK_CONDITION)); }; let req = ScsiRequest { cdb: cdb.to_vec(), data_len: ScsiDataLength::Unlimited, data_in: data }; @@ -1471,14 +1694,15 @@ impl Core { let lun = (device & 0xff) as u8; // A bus this part does not have. - if usize::from(channel) >= BUSES { + if channel as usize >= self.part.channels() { return Prepared::Answered(Self::incomplete(handle, 0, 0)); } // GOT_BUS alone is Linux's bad-target; the timeout bit beside it is // what lets the empty slot be treated as benign. - if target > MAX_TARGET || !self.has_target(target) { + if target > MAX_TARGET || !self.buses[channel as usize].has_target(target) { return Prepared::Absent(channel as u8, Self::incomplete(handle, STATE_GOT_BUS, STATUS_FLAG_TIMEOUT)); } + let id = target; if lun > MAX_LUN { return Prepared::Answered(Self::incomplete(handle, STATE_GOT_BUS, STATUS_FLAG_TIMEOUT)); } @@ -1487,9 +1711,9 @@ impl Core { cdb[1] = (cdb[1] & 0x1f) | (lun << 5); let cdb = &cdb[..(cdb_len as usize).clamp(1, CDB_FIELD_LEN)]; let block = self - .bus + .buses[channel as usize] .lock() - .get(target as usize) + .get(id as usize) .and_then(|t| t.as_ref()) .and_then(|t| t.block_size()) .map(|b| b as usize) @@ -1497,6 +1721,7 @@ impl Core { .unwrap_or(BLOCK_SIZE); let (xfer, to_dev) = Self::cdb_transfer(cdb, block); let (segs, described) = self.read_segments(ring, addr, entry_count, seg_count, xfer as u64, a64); + let target = self.part.target_on(channel as u8, target); Prepared::Run(Job { handle, target, cdb: cdb.to_vec(), xfer, to_dev, segs, described }) } @@ -1512,7 +1737,7 @@ impl Core { /// None too when a reset cancelled the command (`gen` is the generation /// it was read under): what it had not yet stored in host memory is not /// stored, and no completion is built. - fn run_job(&self, job: Job, gen: u64) -> Option { + fn run_job(&self, job: Job, gen: u64, cancel: u64) -> Option { let Job { handle, target, cdb, xfer, to_dev, segs, described } = job; // The bar has lights for SCSI ids 0-5. if let (Some(hb), 0..=5) = (self.heartbeat.get(), target) { @@ -1530,9 +1755,10 @@ impl Core { buf.extend_from_slice(&chunk); } moved = buf.len() as u64; - status = self.scsi(gen, target, &cdb, Some(buf))?.1; + if self.cancel.load(Ordering::Acquire) != cancel { return None; } + status = self.scsi(gen, cancel, target, &cdb, Some(buf))?.1; } else { - let (data, s) = self.scsi(gen, target, &cdb, None)?; + let (data, s) = self.scsi(gen, cancel, target, &cdb, None)?; status = s; // Offer the target's bytes to the descriptors in order. let take = (data.len() as u64).min(described).min(xfer as u64) as usize; @@ -1540,7 +1766,7 @@ impl Core { for &(a, l) in &segs { if off >= take { break; } let n = (l as usize).min(take - off); - if !self.fenced(gen, || self.dma_write(a, &data[off..off + n]))? { + if !self.fenced(gen, || self.cancel.load(Ordering::Acquire) == cancel && self.dma_write(a, &data[off..off + n]))? { self.note(format!("payload DMA for handle {handle:#x} refused by the fabric; command stopped")); return None; } @@ -1551,12 +1777,13 @@ impl Core { // Auto-sense runs only after a CHECK CONDITION. let mut sense = Vec::new(); + if self.cancel.load(Ordering::Acquire) != cancel { return None; } if status == CHECK_CONDITION { - sense = self.scsi(gen, target, &[scsi_cmd::REQUEST_SENSE, 0, 0, 0, SENSE_MAX as u8, 0], None)?.0; + sense = self.scsi(gen, cancel, target, &[scsi_cmd::REQUEST_SENSE, 0, 0, 0, SENSE_MAX as u8, 0], None)?.0; sense.truncate(SENSE_MAX); } let mut scsi_status = status as u16; - let mut state_flags = STATE_ALL_PHASES & !STATE_GOT_SENSE; + let mut state_flags = self.part.completion_phases(moved, described.max(xfer as u64)); if !sense.is_empty() { scsi_status |= SCSI_STATUS_SENSE_VALID; state_flags |= STATE_GOT_SENSE; @@ -1653,6 +1880,17 @@ impl Core { st.ctl.work = true; return st; } + if std::mem::take(&mut st.reset_targets) { + for channel in 0..self.part.channels() { + self.note_bus_reset(channel); + self.reset_targets(&mut st, channel as u8, None); + } + continue; + } + if let Some((channel, target)) = st.pending_target_resets.pop_front() { + self.reset_targets(&mut st, channel, target); + continue; + } // Completions held for room go first, in order, as the driver // frees it; while any is held the request ring waits. if !st.pending.is_empty() { @@ -1794,8 +2032,16 @@ impl Core { } Prepared::Run(job) => { st.total_cmds += 1; + st.active = Some((job.target, job.cdb.get(1).copied().unwrap_or(0) >> 5, job.handle)); + st.cancelled = false; st.ctl.busy = true; - let done = MutexGuard::unlocked(&mut st, || self.run_job(job, ring.gen)); + let cancel = self.cancel.load(Ordering::Acquire); + let mut done = MutexGuard::unlocked(&mut st, || self.run_job(job, ring.gen, cancel)); + if st.cancelled { done = None; } + else if self.part.has_abort_commands() && done.is_none() && st.ctl.gen == ring.gen { + if let Some(active) = st.active { st.outstanding.push(active); } + } + st.active = None; st.ctl.busy = false; self.wake.notify_all(); // A reset cancelled the command. @@ -1828,13 +2074,16 @@ impl Core { return st.mailbox[((off - reg::MAILBOX_BASE) / 2) as usize]; } if (reg::BANK_BASE..BANK_END).contains(&off) { - return st.active_bank()[((off - reg::BANK_BASE) / 2) as usize]; + let bank = self.part.active_bank(st.bus_cfg1); + return st.active_bank(bank)[((off - reg::BANK_BASE) / 2) as usize]; + } + if self.part.unmodelled_dma_register(off) { + return 0; } match off { reg::ID_LOW => PCI_VENDOR_QLOGIC, - reg::ID_HIGH => PCI_DEVICE_ISP1040, - // The revision field runs one ahead of the PCI byte. - reg::BUS_CFG0 => PCI_REVISION_ISP1040 as u16 + 1, + reg::ID_HIGH => self.part.id_high(), + reg::BUS_CFG0 => self.part.bus_cfg0(), reg::BUS_CFG1 => st.bus_cfg1, reg::BUS_CTRL => st.bus_ctrl, reg::BUS_STATUS => st.bus_status, @@ -1852,6 +2101,10 @@ impl Core { } fn reg_write(&self, off: u32, val: u16) { + if self.part.unmodelled_dma_register(off) { + self.note(format!("write to ISP12160 DMA register {off:#04x} is not modelled")); + return; + } let mut st = self.st.lock(); if off == reg::RESPONSE_QUEUE { // The consumer acknowledgement: recorded, never over the @@ -1874,15 +2127,17 @@ impl Core { return; } if (reg::BANK_BASE..BANK_END).contains(&off) { - st.active_bank()[((off - reg::BANK_BASE) / 2) as usize] = val; + let bank = self.part.active_bank(st.bus_cfg1); + st.active_bank(bank)[((off - reg::BANK_BASE) / 2) as usize] = val; return; } match off { reg::BUS_CFG1 => { - if val & !CFG1_WRITE_MASK != 0 { - self.note(format!("bus configuration 1 bits {:#06x} are not modelled", val & !CFG1_WRITE_MASK)); + let mask = self.part.cfg1_write_mask(); + if val & !mask != 0 { + self.note(format!("bus configuration 1 bits {:#06x} are not modelled", val & !mask)); } - st.bus_cfg1 = val & CFG1_WRITE_MASK; + st.bus_cfg1 = val & mask; } // Applied after the enables are stored, so the reset wins. reg::BUS_CTRL => { @@ -1890,6 +2145,7 @@ impl Core { // The command in flight is cancelled, not waited for. if val & CTRL_SOFT_RESET != 0 { st.reset(); + if self.part.reset_targets(&mut st) { self.kick(&mut st); } self.publish_gen(&st); self.wake.notify_all(); drop(st); @@ -1948,7 +2204,8 @@ impl Core { /// from `scsi_monitor` through this. /// The monitor's `scsi`/`cow` commands (`scsi1`/`cow1` for the external /// bus), as the other SCSI controllers have them, plus `scsi notes`. -impl crate::traits::Device for Isp1040 { +#[allow(private_bounds)] +impl crate::traits::Device for Isp

{ fn step(&self, _cycles: u64) {} fn stop(&self) { self.stop_ring() } fn start(&self) { self.start_ring() } @@ -1993,14 +2250,16 @@ impl crate::traits::Device for Isp1040 { } } -impl crate::scsi_monitor::ScsiTargets for Isp1040 { +#[allow(private_bounds)] +impl crate::scsi_monitor::ScsiTargets for Isp

{ fn scsi_bus(&self) -> &ScsiBus { - &self.bus + &self.buses[0] } } /// Both BARs reach the same registers. -impl PciFunction for Isp1040 { +#[allow(private_bounds)] +impl PciFunction for Isp

{ fn state(&self) -> &PciFunctionState { &self.pci } @@ -2023,7 +2282,8 @@ impl PciFunction for Isp1040 { } } -impl crate::traits::Saveable for Isp1040 { +#[allow(private_bounds)] +impl crate::traits::Saveable for Isp

{ fn save_state(&self) -> toml::Value { self.quiesced(|st| { // Absolute deadlines mean nothing on another run's clock. @@ -2032,14 +2292,15 @@ impl crate::traits::Saveable for Isp1040 { // A hold-off already over but not acted on still crosses: the // commands parked behind it start when the load re-arms it at 0. let quiet_left = st.quiet_until.map(|until| until.map(|u| u - now)); - Self::save_locked(st, left, quiet_left) + self.save_locked(st, left, quiet_left) }) } fn load_state(&self, v: &toml::Value) -> Result<(), String> { self.quiesced(|st| { let now = self.now_ns(st); - Self::load_locked(st, v, now); + self.load_locked(st, v, now); + self.cancel.store(0, Ordering::Release); self.publish_gen(st); }); self.sync_irq(); @@ -2047,8 +2308,9 @@ impl crate::traits::Saveable for Isp1040 { } } -impl Isp1040 { - fn save_locked(st: &State, selecting_left: Option, quiet_left: [Option; BUSES]) -> toml::Value { +#[allow(private_bounds)] +impl Isp

{ + fn save_locked(&self, st: &State, selecting_left: Option, quiet_left: [Option; BUSES]) -> toml::Value { let mut t = toml::map::Map::new(); for (k, v) in [ ("bus_cfg1", st.bus_cfg1), ("bus_ctrl", st.bus_ctrl), ("bus_status", st.bus_status), @@ -2121,16 +2383,18 @@ impl Isp1040 { m.insert("sense".into(), u8_slice_to_toml(&sel.done.sense)); t.insert("selecting".into(), toml::Value::Table(m)); } + self.part.save_extra(st, &mut t); toml::Value::Table(t) } /// Unprocessed request entries survive: the ring thread is kicked and /// runs them after the load. - fn load_locked(st: &mut State, v: &toml::Value, now: i64) { + fn load_locked(&self, st: &mut State, v: &toml::Value, now: i64) { { let ctl = std::mem::take(&mut st.ctl); *st = State::new(); st.ctl = ctl; + self.part.load_extra(st, v); st.ctl.gen += 1; st.ctl.rqst_epoch += 1; st.ctl.resp_epoch += 1; diff --git a/src/dev/isp1040/tests.rs b/src/dev/isp1040/tests.rs index 4c17086c2..10407bf22 100644 --- a/src/dev/isp1040/tests.rs +++ b/src/dev/isp1040/tests.rs @@ -2,9 +2,11 @@ //! and ring contract. Host memory is a flat byte array behind a fake `PciDma`; the disk //! is a small raw image. //! -//! Ported from tests/qtest/qlogic-isp-test.c in jamesbraid/qemu-sgi-origami. +//! Ported from tests/qtest/qlogic-isp-test.c and tests/qtest/sn0-isp12160-test.c +//! in jamesbraid/qemu-sgi-origami. use super::*; +use crate::dev::isp12160::Isp12160Part; use crate::irq::LevelProbe; use crate::pci::PciDmaStatus; use crate::traits::{Device, Saveable}; @@ -47,8 +49,8 @@ const RQST: u32 = 0x1000; const RESP: u32 = 0x2000; const BUF: u32 = 0x10000; -struct Rig { - isp: Isp1040, +struct Rig { + isp: Isp

, mem: Arc, } @@ -58,15 +60,18 @@ fn image(name: &str, blocks: usize) -> std::path::PathBuf { p } -impl Rig { - /// A chip on fake memory, no disks. - fn bare() -> Rig { +impl Rig

{ + fn on(isp: Isp

) -> Self { let mem = FakeMem::new(); - let isp = Isp1040::new(); Device::start(&isp); isp.state().connect_dma(mem.clone()); Rig { isp, mem } } +} + +impl Rig { + /// A chip on fake memory, no disks. + fn bare() -> Rig { Rig::on(Isp1040::new()) } /// A chip with a disk of `blocks` blocks at target `id`. fn with_disk(name: &str, id: u8, blocks: usize) -> Rig { @@ -74,7 +79,9 @@ impl Rig { r.isp.bus().attach_image(id, &image(name, blocks), false).expect("attach"); r } +} +impl Rig

{ /// Take the adapter's clock off the host's and set it to `ns`; the /// ring thread has seen the new time when this returns. fn clock(&self, ns: i64) { @@ -629,7 +636,7 @@ fn an_absent_target_is_incomplete_with_a_timeout() { const MS: i64 = 1_000_000; /// Whether the response ring holds a completion for `handle` in slot 0. -fn posted(r: &Rig, handle: u32) -> bool { +fn posted(r: &Rig

, handle: u32) -> bool { g32(&r.status(0), 4) == handle } @@ -700,7 +707,7 @@ fn a_reset_cancels_a_selection_in_progress() { /// An absent-target command (target 3 of a rig whose disk is at 1) queued /// in `slot` and the ring told; the thread is waiting out its selection /// when this returns, because `clock` has synchronised with it. -fn queue_absent(r: &Rig, slot: u32, handle: u32) { +fn queue_absent(r: &Rig

, slot: u32, handle: u32) { r.put_entry(slot, &request(handle, 3, &[0, 0, 0, 0, 0, 0], &[])); r.doorbell(slot as u16 + 1); } @@ -1646,6 +1653,403 @@ fn a_controller_built_on_a_filled_bus_answers_for_its_target() { let _ = std::fs::remove_file(img); } + +struct MarkerTarget(u8); + +impl crate::scsi::ScsiTarget for MarkerTarget { + fn request(&mut self, _req: &crate::scsi::ScsiRequest) -> std::io::Result { + Ok(crate::scsi::ScsiResponse { status: 0, data: vec![self.0] }) + } +} + +/// Target 0 on channel 1 must stay distinct from target 0 on channel 0. +#[test] +fn the_isp12160_routes_ring_requests_to_separate_buses() { + let bus0 = Isp1040::new_bus(); + let bus1 = Isp1040::new_bus(); + bus0.attach(0, Box::new(MarkerTarget(0x10))).unwrap(); + bus1.attach(0, Box::new(MarkerTarget(0x20))).unwrap(); + let r = Rig::on(Isp::new_12160(bus0, bus1)); + r.init_rings(4, 4); + + r.put_entry(0, &request(0x1234, 0x80, &[0x12, 0, 0, 0, 36, 0], &[(BUF, 36)])); + r.doorbell(1); + assert_eq!(r.mem.get(BUF, 1), vec![0x20], "channel 1 reaches its target 0"); + + r.put_entry(1, &request(0x1235, 0, &[0x12, 0, 0, 0, 36, 0], &[(BUF + 64, 36)])); + r.doorbell(2); + assert_eq!(r.mem.get(BUF + 64, 1), vec![0x10], "channel 0 keeps its own target 0"); +} + +struct ResetProbe(Arc); + +impl crate::scsi::ScsiTarget for ResetProbe { + fn request(&mut self, _req: &crate::scsi::ScsiRequest) -> std::io::Result { + Ok(crate::scsi::ScsiResponse { status: 0, data: Vec::new() }) + } + + fn bus_reset(&mut self) { + self.0.fetch_add(1, Ordering::Relaxed); + } +} + +/// A snapshot retains both the chip-wide reset and the selected target reset, +/// and the resumed worker delivers both through the original channel buses. +#[test] +fn the_isp12160_preserves_queued_bus_and_target_resets_across_save_load() { + let src = Isp::new_12160(Isp1040::new_bus(), Isp1040::new_bus()); + { + let mut state = src.st.lock(); + state.reset_targets = true; + state.pending_target_resets.push_back((1, Some(2))); + } + let saved = src.save_state(); + + let resets0 = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let resets1 = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let selected_reset = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let bus0 = Isp1040::new_bus(); + let bus1 = Isp1040::new_bus(); + bus0.attach(1, Box::new(ResetProbe(resets0.clone()))).unwrap(); + bus1.attach(1, Box::new(ResetProbe(resets1.clone()))).unwrap(); + bus1.attach(2, Box::new(ResetProbe(selected_reset.clone()))).unwrap(); + let dst = Isp::new_12160(bus0, bus1); + dst.load_state(&saved).unwrap(); + Device::start(&dst); + dst.wait_idle(); + Device::stop(&dst); + + assert_eq!(resets0.load(Ordering::Relaxed), 1); + assert_eq!(resets1.load(Ordering::Relaxed), 1); + assert_eq!(selected_reset.load(Ordering::Relaxed), 2, "the full reset and queued target reset both run"); + assert_eq!(saved.get("reset_targets").and_then(toml::Value::as_bool), Some(true)); + assert_eq!(saved.get("target_resets").and_then(toml::Value::as_array).map(Vec::len), Some(1)); + let after = dst.save_state(); + assert_eq!(after.get("reset_targets").and_then(toml::Value::as_bool), Some(false)); + assert_eq!(after.get("target_resets").and_then(toml::Value::as_array).map(Vec::len), Some(0)); +} + +// ---- ISP12160 second channel and recovery commands ----------------------- + +fn rig12160() -> Rig { Rig::on(Isp::new_12160(Isp1040::new_bus(), Isp1040::new_bus())) } + +/// A 64-block image whose first block is `fill`, so a read says which disk answered. +fn filled_image(name: &str, fill: u8) -> std::path::PathBuf { + let p = image(name, 64); + let mut data = vec![0u8; 64 * BLOCK_SIZE]; + data[..BLOCK_SIZE].fill(fill); + std::fs::write(&p, data).expect("test image"); + p +} + +/// A 12160 with a different disk at target 1 on each channel. +fn disks_on_both_channels(name: &str) -> Rig { + let r = rig12160(); + for channel in 0..2u8 { + let img = filled_image(&format!("{name}{channel}"), 0xa0 + channel); + r.isp.bus_for_channel(channel).unwrap().attach_image(1, &img, false).expect("attach"); + } + r +} + +/// Issues entries one at a time on a 16-entry ring and returns each completion. +struct Seq<'a> { + r: &'a Rig, + n: u16, +} + +impl<'a> Seq<'a> { + fn new(r: &'a Rig) -> Self { + r.init_rings(16, 16); + Seq { r, n: 0 } + } + fn run(&mut self, handle: u32, target: u8, cdb: &[u8], segs: &[(u32, u32)]) -> [u8; ENTRY_SIZE] { + let slot = self.n; + self.r.put_entry(slot as u32, &request(handle, target, cdb, segs)); + self.n += 1; + self.r.doorbell(self.n); + self.r.status(slot as u32) + } +} + +const TUR: [u8; 6] = [0; 6]; + +/// Counts bus resets and answers any command with a full block pair, so a read +/// whose second buffer segment is unreachable stays outstanding. +struct CountingDisk(Arc); + +impl crate::scsi::ScsiTarget for CountingDisk { + fn request(&mut self, _req: &crate::scsi::ScsiRequest) -> std::io::Result { + Ok(crate::scsi::ScsiResponse { status: 0, data: vec![0; 2 * BLOCK_SIZE] }) + } + fn bus_reset(&mut self) { self.0.fetch_add(1, Ordering::Relaxed); } +} + +struct Held { + r: Rig, + /// Bus resets seen by the counting disks at target 1 on channel 0, target 1 on channel 1, + /// and target 2 on channel 1. + ch0: Arc, + ch1: Arc, + ch1_other: Arc, +} + +impl Held { + fn new() -> Held { + let counters: Vec<_> = (0..3).map(|_| Arc::new(std::sync::atomic::AtomicUsize::new(0))).collect(); + let (bus0, bus1) = (Isp1040::new_bus(), Isp1040::new_bus()); + bus0.attach(1, Box::new(CountingDisk(counters[0].clone()))).unwrap(); + bus1.attach(1, Box::new(CountingDisk(counters[1].clone()))).unwrap(); + bus1.attach(2, Box::new(CountingDisk(counters[2].clone()))).unwrap(); + let r = Rig::on(Isp::new_12160(bus0, bus1)); + r.init_rings(16, 16); + Held { r, ch0: counters[0].clone(), ch1: counters[1].clone(), ch1_other: counters[2].clone() } + } + /// A read whose payload store the fabric refuses: the adapter keeps it outstanding, which + /// is what the abort and reset commands act on. + fn hold(&self, slot: u32, handle: u32, target: u8) { + self.r.put_entry(slot, &request(handle, target, &cdb10(scsi_cmd::READ_10, 0, 2), + &[(BUF, BLOCK_SIZE as u32), (MEM as u32, BLOCK_SIZE as u32)])); + self.r.doorbell(slot as u16 + 1); + assert_eq!(g32(&self.r.status(slot), 4), 0, "handle {handle:#x} is held, not completed"); + } + fn command(&self, regs: &[u16]) -> u16 { + let status = self.r.cmd(regs); + self.r.isp.wait_idle(); + status + } + /// The completion posted in response slot `slot`: (handle, completion code). + fn answer(&self, slot: u32) -> (u32, u16) { + let s = self.r.status(slot); + (g32(&s, 4), g16(&s, 10)) + } +} + +const CH1: u16 = 0x8000; + +/// A driver sizing its queues and recognising the part reads these before any ring exists. +#[test] +fn the_isp12160_reports_its_own_identity_firmware_and_unmapped_registers() { + let r = rig12160(); + assert_eq!(r.r(reg::ID_HIGH), 0x1216); + assert_eq!(r.r(reg::BUS_CFG0), 0, "no revision field on this part"); + r.w(reg::DMA_CMD_CFG, 0xffff); + r.w(reg::DMA_DATA_CFG, 0xffff); + assert_eq!((r.r(reg::DMA_CMD_CFG), r.r(reg::DMA_DATA_CFG)), (0, 0), "the flat ISP1040 DMA blocks are not here"); + + // Mailbox 6 is seeded because an untouched output would also read zero. + r.w(r.mb(6), 0x1234); + r.w(r.mb(0), mbox::ABOUT_FIRMWARE); + r.w(reg::HOST_CMD, 0x5 << HOST_CMD_SHIFT); + assert_eq!(r.r(r.mb(0)), mbox::COMPLETE); + assert_eq!([r.r(r.mb(1)), r.r(r.mb(2))], [10, 4]); + assert_eq!(r.r(r.mb(6)), 0, "the attribute word is cleared, not echoed"); + r.ack(); + r.w(r.mb(0), mbox::GET_FIRMWARE_STATUS); + r.w(reg::HOST_CMD, 0x5 << HOST_CMD_SHIFT); + assert_eq!(r.r(r.mb(0)), mbox::COMPLETE); + assert_eq!([r.r(r.mb(1)), r.r(r.mb(2))], [0, MAX_REQUESTS]); + r.ack(); +} + +/// The driver switches register banks through bus configuration 1 bits 9:8; each of the +/// four selections must show its own words, and no other bit may stick. +#[test] +fn the_isp12160_selects_four_distinct_banks_and_keeps_only_the_bank_field() { + let r = rig12160(); + let selections = [0x0000, 0x0100, 0x0200, 0x0300]; + for sel in selections { + r.w(reg::BUS_CFG1, sel); + r.w(reg::BANK_BASE, 0x1000 + sel); + r.w(reg::BANK_BASE + 0x20, 0x2000 + sel); + } + for sel in selections { + r.w(reg::BUS_CFG1, sel); + assert_eq!(r.r(reg::BANK_BASE), 0x1000 + sel, "bank {sel:#x}"); + assert_eq!(r.r(reg::BANK_BASE + 0x20), 0x2000 + sel, "bank {sel:#x}"); + } + r.w(reg::BUS_CFG1, 0xb2c5); + assert_eq!(r.r(reg::BUS_CFG1), 0x0200); + + // A soft reset hands a re-initialising driver empty banks, not the previous run's words. + r.w(reg::BUS_CTRL, CTRL_SOFT_RESET); + assert_eq!(r.r(reg::BUS_CFG1), 0); + for sel in selections { + r.w(reg::BUS_CFG1, sel); + assert_eq!(r.r(reg::BANK_BASE), 0, "bank {sel:#x} after reset"); + } +} + +/// A second command submitted before the driver acknowledged the first result must not +/// run, or it would overwrite the answer the driver has not read. +#[test] +fn the_isp12160_ignores_a_mailbox_submission_while_the_result_is_unacknowledged() { + let r = rig12160(); + r.w(r.mb(0), mbox::NOP); + r.w(reg::HOST_CMD, 0x5 << HOST_CMD_SHIFT); + assert_eq!(r.r(r.mb(0)), mbox::COMPLETE); + assert_eq!(r.r(reg::SEMAPHORE) & SEMAPHORE_LOCK, SEMAPHORE_LOCK); + + r.w(r.mb(1), 0x9999); + r.w(r.mb(0), mbox::ABOUT_FIRMWARE); + r.w(reg::HOST_CMD, 0x5 << HOST_CMD_SHIFT); + assert_eq!(r.r(r.mb(0)), mbox::ABOUT_FIRMWARE, "the second submission did not run"); + assert_eq!(r.r(r.mb(1)), 0x9999); + + r.ack(); + r.w(reg::HOST_CMD, 0x5 << HOST_CMD_SHIFT); + assert_eq!(r.r(r.mb(0)), mbox::COMPLETE); + assert_eq!(r.r(r.mb(1)), 10, "acknowledged, the same submission runs"); + r.ack(); +} + +/// An empty slot on either channel times out on its own bus; a driver probing channel 1 +/// must see an absent target, not a hang or the channel 0 answer. +#[test] +fn a_missing_target_times_out_on_each_channel() { + let r = rig12160(); + r.init_rings(16, 16); + r.clock(0); + // An absent target answers once its selection timeout has run out on + // the adapter's clock. + let mut now = 0; + for (slot, (handle, target)) in [(0x31, 5), (0x32, 0x80 | 5)].into_iter().enumerate() { + r.put_entry(slot as u32, &request(handle, target, &TUR, &[])); + r.doorbell(slot as u16 + 1); + assert_ne!(g32(&r.status(slot as u32), 4), handle, "nothing before the selection timeout"); + now += DEFAULT_SELECTION_TIMEOUT as i64 * MS; + r.clock(now); + let s = r.status(slot as u32); + assert_eq!(g32(&s, 4), handle); + assert_eq!(g16(&s, 10), IOCB_INCOMPLETE); + assert_eq!(g16(&s, 12), STATE_GOT_BUS); + assert_eq!(g16(&s, 14), STATUS_FLAG_TIMEOUT); + } +} + +/// A command refused on channel 1 must not disturb its neighbour on channel 0, and +/// a write must land on the disk of the channel it named. +#[test] +fn a_refused_command_on_channel_1_leaves_channel_0_and_the_media_alone() { + let r = disks_on_both_channels("refuse"); + let mut seq = Seq::new(&r); + // Collect each disk's power-on unit attention so the reads below see real answers. + seq.run(0xa0, 1, &TUR, &[]); + seq.run(0xa1, 0x81, &TUR, &[]); + + r.mem.put(BUF, &[0x5c; BLOCK_SIZE]); + let beyond_disk = seq.run(0xb1, 0x81, &cdb10(scsi_cmd::READ_10, 1000, 1), &[(BUF, BLOCK_SIZE as u32)]); + assert_eq!(g16(&beyond_disk, 8) & 0xff, CHECK_CONDITION as u16); + assert_eq!(r.mem.get(BUF, BLOCK_SIZE), vec![0x5c; BLOCK_SIZE], "the refused read stored nothing"); + + let good = seq.run(0xb0, 1, &cdb10(scsi_cmd::READ_10, 0, 1), &[(BUF + 0x1000, BLOCK_SIZE as u32)]); + assert_eq!(g16(&good, 10), IOCB_COMPLETE); + assert_eq!(r.mem.get(BUF + 0x1000, BLOCK_SIZE), vec![0xa0; BLOCK_SIZE], "channel 0 still reads its own disk"); + + r.mem.put(BUF + 0x2000, &[0x77; BLOCK_SIZE]); + seq.run(0xc0, 0x81, &cdb10(scsi_cmd::WRITE_10, 3, 1), &[(BUF + 0x2000, BLOCK_SIZE as u32)]); + seq.run(0xc1, 0x81, &cdb10(scsi_cmd::READ_10, 3, 1), &[(BUF + 0x3000, BLOCK_SIZE as u32)]); + seq.run(0xc2, 1, &cdb10(scsi_cmd::READ_10, 3, 1), &[(BUF + 0x4000, BLOCK_SIZE as u32)]); + assert_eq!(r.mem.get(BUF + 0x3000, BLOCK_SIZE), vec![0x77; BLOCK_SIZE], "channel 1 holds what was written"); + assert_eq!(r.mem.get(BUF + 0x4000, BLOCK_SIZE), vec![0; BLOCK_SIZE], "channel 0's block 3 is untouched"); +} + +/// Abort device names a channel in the top bit of its selector; the same target number +/// held on the other bus must stay outstanding. +#[test] +fn abort_device_retires_only_the_named_channels_commands() { + let h = Held::new(); + h.hold(0, 0x41, 1); + h.hold(1, 0x42, 0x81); + assert_eq!(h.command(&[mbox::ABORT_DEVICE, CH1 | 0x0100, 0, 0]), mbox::COMPLETE); + assert_eq!(h.answer(0), (0x42, IOCB_ABORTED)); + assert_eq!(h.answer(1), (0, 0), "channel 0's command is still held"); + // The survivor is still the adapter's to abort. + assert_eq!(h.command(&[mbox::ABORT_DEVICE, 0x0100, 0, 0]), mbox::COMPLETE); + assert_eq!(h.answer(1), (0x41, IOCB_ABORTED)); +} + +/// Abort command names one handle: two commands to the same channel, target and unit differ +/// only by it, and a selector that matches a held handle on the wrong channel is an error. +#[test] +fn abort_command_matches_the_handle_and_the_whole_selector() { + let h = Held::new(); + h.hold(0, 0x51, 0x81); + h.hold(1, 0x52, 0x81); + assert_eq!(h.command(&[mbox::ABORT_COMMAND, CH1 | 0x0100, 0, 0x52]), mbox::COMPLETE); + assert_eq!(h.answer(0), (0x52, IOCB_ABORTED)); + assert_eq!(h.answer(1), (0, 0), "its neighbour on the same target is untouched"); + + assert_eq!(h.command(&[mbox::ABORT_COMMAND, 0x0100, 0, 0x51]), mbox::COMMAND_ERROR, + "a held handle named on the wrong channel is not a match"); + assert_eq!(h.answer(1), (0, 0)); + assert_eq!(h.command(&[mbox::ABORT_COMMAND, CH1 | 0x0100, 0, 0x51]), mbox::COMPLETE); + assert_eq!(h.answer(1), (0x51, IOCB_ABORTED)); +} + +/// A driver clearing a device it already drained must read success, while naming a handle +/// nothing is using must read as an error rather than claim a retirement. +#[test] +fn aborting_nothing_is_complete_but_aborting_an_unknown_handle_is_refused() { + let h = Held::new(); + assert_eq!(h.command(&[mbox::ABORT_DEVICE, CH1 | 0x0100, 0, 0]), mbox::COMPLETE); + assert_eq!(h.command(&[mbox::ABORT_COMMAND, CH1 | 0x0100, 0, 0x77]), mbox::COMMAND_ERROR); + assert_eq!(h.answer(0), (0, 0), "neither posted a completion"); +} + +/// Target reset reaches one target on one channel; the same number on the other channel and +/// the neighbour on the same channel must not see it, and the reset target keeps serving. +#[test] +fn target_reset_is_scoped_to_one_target_on_one_channel() { + let h = Held::new(); + h.hold(0, 0x41, 1); + h.hold(1, 0x42, 0x81); + assert_eq!(h.command(&[mbox::ABORT_TARGET, CH1 | 0x0100, 0, 0]), mbox::COMPLETE); + assert_eq!(h.answer(0), (0x42, IOCB_RESET)); + assert_eq!(h.answer(1), (0, 0)); + assert_eq!([h.ch0.load(Ordering::Relaxed), h.ch1.load(Ordering::Relaxed), h.ch1_other.load(Ordering::Relaxed)], + [0, 1, 0]); + + h.r.put_entry(2, &request(0x43, 0x81, &TUR, &[])); + h.r.doorbell(3); + assert_eq!(h.answer(1), (0x43, IOCB_COMPLETE)); +} + +/// Bus reset takes its channel from mailbox 2; mailbox 1 is the reset delay and here holds a value +/// that is not even a channel. The whole named bus resets, the other one does not. +#[test] +fn bus_reset_takes_its_channel_from_mailbox_two_and_resets_every_target_on_it() { + let h = Held::new(); + h.hold(0, 0x41, 1); + h.hold(1, 0x42, 0x81); + assert_eq!(h.command(&[mbox::BUS_RESET, 3, 1]), mbox::COMPLETE); + assert_eq!(h.answer(0), (0x42, IOCB_RESET)); + assert_eq!(h.answer(1), (0, 0)); + assert_eq!([h.ch0.load(Ordering::Relaxed), h.ch1.load(Ordering::Relaxed), h.ch1_other.load(Ordering::Relaxed)], + [0, 1, 1]); + assert_eq!(h.command(&[mbox::BUS_RESET, 0, 2]), mbox::PARAM_ERROR, "there is no third bus"); +} + +/// A bus reset leaves a unit attention on the disks of that channel only; the driver's +/// first command afterwards is expected to collect it. +#[test] +fn a_bus_reset_leaves_a_unit_attention_on_the_reset_channel_only() { + let r = disks_on_both_channels("uareset"); + let mut seq = Seq::new(&r); + seq.run(0x10, 1, &TUR, &[]); + seq.run(0x11, 0x81, &TUR, &[]); + assert_eq!(g16(&seq.run(0x12, 0x81, &TUR, &[]), 8) & 0xff, 0, "attention was collected"); + + assert_eq!(r.cmd(&[mbox::BUS_RESET, 0, 1]), mbox::COMPLETE); + r.isp.wait_idle(); + let after = seq.run(0x13, 0x81, &TUR, &[]); + assert_eq!(g16(&after, 8) & 0xff, CHECK_CONDITION as u16); + assert_eq!(after[34] & 0x0f, 0x06, "unit attention"); + let other = seq.run(0x14, 1, &TUR, &[]); + assert_eq!(g16(&other, 8) & 0xff, 0, "channel 0 was not reset"); + assert_eq!(g16(&seq.run(0x15, 0x81, &TUR, &[]), 8) & 0xff, 0, "reported once"); +} + // ---- the hold-off after a bus reset --------------------------------------- // // Datasheet, "Bus-reset hold-off (0x0018)": after the reset the chip starts no @@ -1659,7 +2063,7 @@ fn a_controller_built_on_a_filled_bus_answers_for_its_target() { const S: i64 = 1_000_000_000; /// A bus reset of `bus` with `secs` seconds of hold-off. -fn bus_reset(r: &Rig, secs: u16, bus: u16) { +fn bus_reset(r: &Rig

, secs: u16, bus: u16) { assert_eq!(r.cmd(&[mbox::BUS_RESET, secs, bus]), mbox::COMPLETE); } @@ -1681,6 +2085,87 @@ fn a_bus_reset_puts_a_disks_write_cache_back() { assert_eq!(r.mem.get(BUF + 6, 1)[0] & 0x04, 0x04, "WCE is back"); } +/// A 12160 with a cached disk at target 1 on each channel whose guests cleared WCE on both. +fn cleared_write_cache_on_both_channels(name: &str) -> Rig { + let r = rig12160(); + for channel in 0..2u8 { + let bus = r.isp.bus_for_channel(channel).unwrap(); + bus.attach_cached_image(1, &image(&format!("{name}{channel}"), 64)).expect("attach"); + bus.select_write_cache(1, false); + assert!(!bus.write_cache_reported(1), "channel {channel}: the guest cleared it"); + } + r +} + +/// The write cache bit MODE SENSE(6) of the caching page reports for target 1 on `channel`. A TUR +/// goes first: it collects the reset's unit attention, and consumes the reset flag as a guest's +/// first command does, so the mode sense that follows returns its page. +fn wce_after_next_command(seq: &mut Seq, channel: u8, handle: u32) -> bool { + let target = channel << 7 | 1; + seq.run(handle, target, &TUR, &[]); + let done = seq.run(handle + 0x100, target, &[0x1a, 0x08, 0x08, 0, 24, 0], &[(BUF, 24)]); + assert_eq!(g16(&done, 10), IOCB_COMPLETE); + assert_eq!(seq.r.mem.get(BUF + 4, 1)[0], 0x08, "caching page"); + seq.r.mem.get(BUF + 6, 1)[0] & 0x04 != 0 +} + +/// Each channel's disks put their write cache back on that channel's reset and not on the +/// other's, so the flag has to be set for the channel the mailbox names. +#[test] +fn a_12160_bus_reset_of_channel_1_restores_write_cache_on_channel_1_only() { + let r = cleared_write_cache_on_both_channels("wce12160-ch1"); + let mut seq = Seq::new(&r); + assert_eq!(r.cmd(&[mbox::BUS_RESET, 0, 1]), mbox::COMPLETE); + r.isp.wait_idle(); + assert!(!wce_after_next_command(&mut seq, 0, 0x31), "channel 0 was not reset"); + assert!(wce_after_next_command(&mut seq, 1, 0x32), "channel 1 was reset"); +} + +#[test] +fn a_12160_bus_reset_of_channel_0_restores_write_cache_on_channel_0_only() { + let r = cleared_write_cache_on_both_channels("wce12160-ch0"); + let mut seq = Seq::new(&r); + assert_eq!(r.cmd(&[mbox::BUS_RESET, 0, 0]), mbox::COMPLETE); + r.isp.wait_idle(); + assert!(!wce_after_next_command(&mut seq, 1, 0x33), "channel 1 was not reset"); + assert!(wce_after_next_command(&mut seq, 0, 0x34), "channel 0 was reset"); +} + +/// Both reset the chip's SCSI buses, which the disks see as a bus reset on each channel. +#[test] +fn a_12160_chip_reset_restores_write_cache_on_both_channels() { + let r = cleared_write_cache_on_both_channels("wce12160-chip"); + r.isp.reset(); + r.isp.wait_idle(); + let mut seq = Seq::new(&r); + assert!(wce_after_next_command(&mut seq, 0, 0x35), "channel 0"); + assert!(wce_after_next_command(&mut seq, 1, 0x36), "channel 1"); +} + +#[test] +fn a_12160_soft_reset_restores_write_cache_on_both_channels() { + let r = cleared_write_cache_on_both_channels("wce12160-soft"); + r.w(reg::BUS_CTRL, CTRL_SOFT_RESET); + r.isp.wait_idle(); + let mut seq = Seq::new(&r); + assert!(wce_after_next_command(&mut seq, 0, 0x37), "channel 0"); + assert!(wce_after_next_command(&mut seq, 1, 0x38), "channel 1"); +} + +/// The single-bus part has no bus 1: a mailbox that names it is ignored, and the one bus it has +/// keeps its disks' write cache as the guest set it. +#[test] +fn an_isp1040_bus_reset_naming_bus_1_leaves_the_reset_flag_unset() { + let r = Rig::bare(); + r.isp.bus().attach_cached_image(1, &image("wce-bus1", 64)).expect("attach"); + r.init_rings(4, 4); + r.isp.bus().select_write_cache(1, false); + bus_reset(&r, 0, 1); + r.isp.wait_idle(); + assert!(!r.isp.targets_reset[0].load(Ordering::Acquire)); + assert!(!r.isp.targets_reset[1].load(Ordering::Acquire)); +} + /// A TUR for the disk at target 1. fn tur1(handle: u32) -> [u8; ENTRY_SIZE] { request(handle, 1, &[0, 0, 0, 0, 0, 0], &[]) @@ -1701,7 +2186,7 @@ fn hold_rig(name: &str) -> Rig { r } -fn handles(r: &Rig, n: u32) -> Vec { +fn handles(r: &Rig

, n: u32) -> Vec { (0..n).map(|i| g32(&r.status(i), 4)).collect() } @@ -2061,7 +2546,7 @@ fn a_hold_that_ended_unfired_still_crosses_a_snapshot() { // ---- the hold-off on the real clock, and thread timing ---- -fn wait_resp(r: &Rig, n: u16, max: Duration) -> bool { +fn wait_resp(r: &Rig

, n: u16, max: Duration) -> bool { let t = std::time::Instant::now(); while t.elapsed() < max { if r.r(reg::RESPONSE_QUEUE) >= n { return true; } diff --git a/src/dev/isp12160.rs b/src/dev/isp12160.rs new file mode 100644 index 000000000..9ae86968f --- /dev/null +++ b/src/dev/isp12160.rs @@ -0,0 +1,337 @@ +//! The QLogic ISP12160: two SCSI buses behind one PCI function. +//! +//! The ISP12160 shares the ISP1040 ring, DMA, and mailbox mechanism, while +//! selecting targets from either channel. Each channel remains an explicit +//! [`ScsiBus`], including for monitor and run-time disc operations. +//! +//! Ported from hw/scsi/qlogic-isp.c and include/hw/scsi/qlogic-isp.h +//! in jamesbraid/qemu-sgi-origami. + +use std::collections::VecDeque; +use std::ops::Deref; + +use parking_lot::MutexGuard; + +use crate::dev::isp1040::{ + mbox, reg, Bank, Core, Isp, IspPart, State, CFG1_BANK_SELECT, FW_VERSION, IOCB_ABORTED, IOCB_RESET, MAILBOX_COUNT, + MAX_REQUESTS, MAX_TARGET, PCI_CLASS_SCSI, PCI_VENDOR_QLOGIC, SEMAPHORE_LOCK, STATE_ALL_PHASES, +}; +use crate::irq::IrqLine; +use crate::pci::{Bar, PciConfig, PciConfigPolicy, PciFunction, PciFunctionState, PciRegisterPolicy, PCI_INTA}; +use crate::scsi::{ScsiBus, ScsiIdentities, ScsiIdentity}; +use crate::snapshot::{get_field, hex_u16, hex_u32, load_u16_slice, toml_u16, toml_u32, toml_u8, u16_slice_to_toml}; +use crate::traits::{Device, Resettable, Saveable}; +use crate::workstation::DiscChanger; + +const ISP_DISK: ScsiIdentity = ScsiIdentity::new(b"SGI ", b"IRIS EMULATED ", b"1.0 "); +const ISP_CDROM: ScsiIdentity = ISP_DISK; + +pub struct Isp12160Part; + +impl IspPart for Isp12160Part { + fn name(&self) -> &'static str { "ISP12160" } + fn channels(&self) -> usize { 2 } + fn pci_config(&self, bars: [Bar; 6]) -> PciConfig { + let registers = [ + PciRegisterPolicy { reg_dword: 13, initial: 0x44, writable_mask: 0 }, + PciRegisterPolicy { reg_dword: 17, initial: 0x0001_0001, writable_mask: 0 }, + PciRegisterPolicy { reg_dword: 18, initial: 0, writable_mask: 0x100 }, + ]; + PciConfig::new_with_policy( + PCI_VENDOR_QLOGIC, + 0x1216, + PCI_CLASS_SCSI, + 6, + bars, + PCI_INTA, + PciConfigPolicy { + status: 0x02b0, + status_w1c_mask: 0, + command_mask: 0x0156, + latency_mask: 0xff, + cache_line_mask: 0xff, + registers: ®isters, + }, + ) + } + fn firmware_version(&self) -> [u16; 3] { [10, 4, FW_VERSION[2]] } + fn active_bank(&self, cfg1: u16) -> Bank { + match (cfg1 >> 8) & 3 { + 1 => Bank::Sxp, + 2 => Bank::Sxp1, + 3 => Bank::Dma, + _ => Bank::Risc, + } + } + fn checks_ram_bounds(&self) -> bool { true } + fn has_abort_commands(&self) -> bool { true } + fn target_on(&self, channel: u8, target: u8) -> u8 { target | (channel << 4) } + fn decode_target(&self, target: u8) -> (usize, u8) { ((target >> 4) as usize, target & 0x0f) } + fn id_high(&self) -> u16 { 0x1216 } + fn bus_cfg0(&self) -> u16 { 0 } + fn cfg1_write_mask(&self) -> u16 { 0x0300 } + fn unmodelled_dma_register(&self, off: u32) -> bool { + matches!(off, reg::DMA_CMD_CFG | reg::DMA_CMD_CTRL | reg::DMA_DATA_CFG | reg::DMA_DATA_CTRL) + } + fn completion_phases(&self, moved: u64, total: u64) -> u16 { + let mut flags = 0x1700; + if moved != 0 { + flags |= 0x0800; + if moved == total { flags |= 0x4000; } + } + flags + } + fn mailbox_ready(&self, semaphore: u16) -> bool { semaphore & SEMAPHORE_LOCK == 0 } + fn clear_target_parameters(&self, mailbox: &mut [u16; MAILBOX_COUNT]) { mailbox[6] = 0; } + fn set_reset_delay(&self, st: &mut State) { + st.reset_delay = st.mailbox[1]; + st.reset_delay1 = st.mailbox[2]; + } + fn reset_targets(&self, st: &mut State) -> bool { + st.reset_targets = true; + true + } + fn execute_mailbox(&self, core: &Core, st: &mut MutexGuard<'_, State>) -> bool { + let cmd = st.mailbox[0]; + match cmd { + mbox::LOAD_RAM_A64 => st.mailbox[0] = core.ram_transfer(st, true), + mbox::ABORT_COMMAND | mbox::ABORT_DEVICE | mbox::ABORT_TARGET => { + let device = st.mailbox[1]; + let valid_target = (device >> 8) & 0x7f <= MAX_TARGET as u16; + let target = ((device >> 8) & 0x7f) as u8 | (((device >> 15) as u8) << 4); + let lun = device as u8; + let handle = ((st.mailbox[2] as u32) << 16) | st.mailbox[3] as u32; + let hit = valid_target && st.active.is_some_and(|(t, l, h)| t == target + && (cmd == mbox::ABORT_TARGET || l == lun) && (cmd != mbox::ABORT_COMMAND || h == handle)); + let completion = if cmd == mbox::ABORT_TARGET { IOCB_RESET } else { IOCB_ABORTED }; + if hit { core.cancel_active(st, completion); } + let retired = core.retire_outstanding(st, |(t, l, h)| valid_target && t == target + && (cmd == mbox::ABORT_TARGET || l == lun) && (cmd != mbox::ABORT_COMMAND || h == handle), completion); + if cmd == mbox::ABORT_TARGET && valid_target { + st.pending_target_resets.push_back((target >> 4, Some(target & 15))); + } + st.mailbox[0] = if cmd == mbox::ABORT_COMMAND && !hit && retired == 0 { mbox::COMMAND_ERROR } else { mbox::COMPLETE }; + } + // Mailbox 1 is the reset delay; mailbox 2 selects the bus. + mbox::BUS_RESET => { + let channel = st.mailbox[2]; + if channel > 1 { st.mailbox[0] = mbox::PARAM_ERROR; } + else { + if st.active.is_some_and(|(target, _, _)| target >> 4 == channel as u8) { + core.cancel_active(st, IOCB_RESET); + } + core.retire_outstanding(st, |(target, _, _)| target >> 4 == channel as u8, IOCB_RESET); + st.pending_target_resets.push_back((channel as u8, None)); + core.note_bus_reset(usize::from(channel)); + core.kick(st); + st.mailbox[0] = mbox::COMPLETE; + } + } + _ => return false, + } + true + } + fn save_extra(&self, st: &State, t: &mut toml::map::Map) { + t.insert("reset_targets".into(), toml::Value::Boolean(st.reset_targets)); + t.insert("target_resets".into(), toml::Value::Array(st.pending_target_resets.iter().map(|(channel, target)| { + toml::Value::Array(vec![hex_u16(*channel as u16), hex_u16(target.unwrap_or(0xff) as u16)]) + }).collect())); + t.insert("sxp1_bank".into(), u16_slice_to_toml(&st.sxp1_bank)); + t.insert("dma_bank".into(), u16_slice_to_toml(&st.dma_bank)); + t.insert("reset_delay1".into(), hex_u16(st.reset_delay1)); + t.insert("outstanding".into(), toml::Value::Array(st.outstanding.iter().map(|&(target, lun, handle)| { + toml::Value::Array(vec![hex_u16(target as u16), hex_u16(lun as u16), hex_u32(handle)]) + }).collect())); + } + fn load_extra(&self, st: &mut State, v: &toml::Value) { + st.reset_targets = get_field(v, "reset_targets").and_then(|x| x.as_bool()).unwrap_or(false); + if let Some(toml::Value::Array(items)) = get_field(v, "target_resets") { + for item in items.iter().take(64) { + if let toml::Value::Array(fields) = item { + if fields.len() == 2 { + if let (Some(channel), Some(target)) = (toml_u8(&fields[0]), toml_u8(&fields[1])) { + if channel < 2 && (target < 16 || target == 0xff) { + st.pending_target_resets.push_back((channel, (target < 16).then_some(target))); + } + } + } + } + } + } + let u16f = |k: &str| get_field(v, k).and_then(toml_u16).unwrap_or(0); + st.reset_delay1 = u16f("reset_delay1"); + if let Some(m) = get_field(v, "sxp1_bank") { load_u16_slice(m, &mut st.sxp1_bank); } + if let Some(m) = get_field(v, "dma_bank") { load_u16_slice(m, &mut st.dma_bank); } + if let Some(toml::Value::Array(items)) = get_field(v, "outstanding") { + for item in items.iter().take(MAX_REQUESTS as usize) { + if let toml::Value::Array(fields) = item { + if fields.len() == 3 { + if let (Some(target), Some(lun), Some(handle)) = + (toml_u8(&fields[0]), toml_u8(&fields[1]), toml_u32(&fields[2])) { + if target < 32 && lun < 8 { st.outstanding.push((target, lun, handle)); } + } + } + } + } + } + } +} + +/// The two-channel controller. The shared ISP engine owns both buses. +pub struct Isp12160(Isp); + +impl Default for Isp12160 { + fn default() -> Self { Self::new(Self::new_bus(), Self::new_bus()) } +} + +impl Isp12160 { + /// Light the status bar's SCSI activity bits in `heartbeat`. + pub fn set_heartbeat(&self, heartbeat: std::sync::Arc) { + self.0.set_heartbeat(heartbeat); + } + + /// A 16-id bus with the ISP controller's disk and CD-ROM identities. + pub fn new_bus() -> ScsiBus { + ScsiBus::new(16, ScsiIdentities { disk: ISP_DISK, cdrom: ISP_CDROM }) + } + + /// The controller with its two channel buses already populated. + pub fn new(bus0: ScsiBus, bus1: ScsiBus) -> Self { + Self(Isp::with_part(0, Isp12160Part, Box::new([bus0, bus1]))) + } + + /// Borrow one channel for the shared monitor commands. + pub fn channel(&self, channel: u8) -> Option> { + (channel < 2).then_some(Channel { adapter: self, channel }) + } + + pub fn register_locks(&self) { self.0.register_locks(); } +} + +impl Deref for Isp12160 { + type Target = Core; + fn deref(&self) -> &Core { &self.0 } +} + +/// One channel, with SCSI ids 0–15 as the monitor presents them. +pub struct Channel<'a> { + adapter: &'a Isp12160, + channel: u8, +} + +impl crate::scsi_monitor::ScsiTargets for Channel<'_> { + fn scsi_bus(&self) -> &ScsiBus { + self.adapter.bus_for_channel(self.channel).expect("valid ISP12160 channel") + } +} + +/// The machine API flattens the ISP12160's channels to ids 0–31. +/// Monitor commands use a `Channel` instead, so their ids remain 0–15. +impl DiscChanger for Isp12160 { + fn disc_status(&self) -> Vec<(usize, String, Vec, u64, u64)> { + (0..2).flat_map(|channel| { + crate::scsi_monitor::disc_status(&self.channel(channel).expect("valid ISP12160 channel")) + .into_iter() + .map(move |(id, current, discs, physical, logical)| (id + channel as usize * 16, current, discs, physical, logical)) + }).collect() + } + + fn load_disc(&self, id: usize, path: String) -> Result { + let (channel, target) = Self::split_id(id)?; + crate::scsi_monitor::load_disc(&self.channel(channel).unwrap(), target, path) + } + + fn eject_disc(&self, id: usize) -> Result { + let (channel, target) = Self::split_id(id)?; + crate::scsi_monitor::eject_disc(&self.channel(channel).unwrap(), target) + } + + fn eject_to_empty(&self, id: usize) -> Result<(), String> { + let (channel, target) = Self::split_id(id)?; + crate::scsi_monitor::eject_to_empty(&self.channel(channel).unwrap(), target) + } +} + +impl Isp12160 { + fn split_id(id: usize) -> Result<(u8, usize), String> { + if id >= 32 { return Err(format!("SCSI ID {id} is out of range")); } + Ok(((id / 16) as u8, id % 16)) + } +} + +impl Device for Isp12160 { + fn step(&self, cycles: u64) { Device::step(&self.0, cycles) } + fn start(&self) { Device::start(&self.0) } + fn stop(&self) { Device::stop(&self.0) } + fn is_running(&self) -> bool { Device::is_running(&self.0) } + fn get_clock(&self) -> u64 { Device::get_clock(&self.0) } + + fn register_commands(&self) -> Vec<(String, String)> { + let usage = crate::scsi_monitor::SCSI_USAGE; + vec![ + ("scsi".into(), format!("SCSI (QLogic ISP12160, channel 0): {usage} | scsi notes")), + ("scsi0".into(), "Alias for `scsi` (ISP12160 channel 0)".into()), + ("cow".into(), "COW overlays on ISP12160 channel 0: cow status | cow commit [id] | cow reset [id]".into()), + ("scsi1".into(), format!("SCSI (QLogic ISP12160, channel 1): {usage}")), + ("cow1".into(), "COW overlays on ISP12160 channel 1: same subcommands as `cow`".into()), + ] + } + + fn execute_command(&self, cmd: &str, args: &[&str], mut w: Box) -> Result<(), String> { + use std::io::Write as _; + let (channel, cow) = match cmd { + "scsi" | "scsi0" | "cow" => (0, cmd.starts_with("cow")), + "scsi1" | "cow1" => (1, cmd.starts_with("cow")), + _ => return Err("Command not found".into()), + }; + let ctl = self.channel(channel).expect("valid ISP12160 channel"); + if cow { + return crate::scsi_monitor::cow_command(&ctl, args, &mut *w); + } + if let Some(done) = crate::scsi_monitor::scsi_subcommand(&ctl, args, &mut *w) { + return done; + } + if channel == 0 && args.first().copied() == Some("notes") { + let _ = writeln!(w, "QLogic ISP12160: {} commands", self.total_commands()); + for note in self.notes() { let _ = writeln!(w, " note: {note}"); } + return Ok(()); + } + Err(format!("Usage: {}", crate::scsi_monitor::SCSI_USAGE)) + } +} + +impl Resettable for Isp12160 { + fn power_on(&self) { + self.0.reset(); + } +} + +impl Saveable for Isp12160 { + fn save_state(&self) -> toml::Value { Saveable::save_state(&self.0) } + fn load_state(&self, v: &toml::Value) -> Result<(), String> { Saveable::load_state(&self.0, v) } +} + +impl PciFunction for Isp12160 { + fn state(&self) -> &PciFunctionState { PciFunction::state(&self.0) } + fn bar_read(&self, bar: usize, off: u32, width: usize) -> u32 { PciFunction::bar_read(&self.0, bar, off, width) } + fn bar_write(&self, bar: usize, off: u32, val: u32, width: usize) { PciFunction::bar_write(&self.0, bar, off, val, width) } + fn attached(&self) { PciFunction::attached(&self.0) } + fn reset(&self) { PciFunction::reset(&self.0) } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The machine disc API uses target 17 for channel 1, target 1. + #[test] + fn disc_status_flattens_the_second_channels_target_ids() { + let bus0 = Isp12160::new_bus(); + let bus1 = Isp12160::new_bus(); + bus1.attach_device(1, "", true, Vec::new(), false, None, false).unwrap(); + let isp = Isp12160::new(bus0, bus1); + + let status = DiscChanger::disc_status(&isp); + assert_eq!(status.iter().map(|drive| drive.0).collect::>(), vec![17]); + } +} diff --git a/src/dev/l1.rs b/src/dev/l1.rs new file mode 100644 index 000000000..d370f96e8 --- /dev/null +++ b/src/dev/l1.rs @@ -0,0 +1,440 @@ +//! Fuel's L1 system controller and its processor-side UART. +//! +//! The internal L1 connection is separate from IOC3's external serial ports. +//! The Ns16550 owns the UART registers, FIFO, timeout and interrupt state; +//! this device keeps the controller protocol and its eight-byte register stride. +//! It answers the requests the stock PROM and IRIX send and refuses the rest +//! as unknown, with a log line. Power requests take effect at once: power +//! down (0x2005) turns the node off, and a power cycle (0x2006) turns it off +//! and on again, which restarts it. Remote routing and controller +//! diagnostics are absent. A configured router interpreter answers +//! the single-fragment `brick` request for the Chimera brick types; other +//! interpreter commands and multi-fragment messages are refused and logged. +//! +//! Ported from hw/misc/sgi-l1.c in jamesbraid/qemu-sgi-origami. + +use crate::dev::ns16550::{com_reg, Ns16550}; +use crate::dev::serial_host::{null_serial_backend, SerialBackend}; +use crate::devlog::LogModule; +use crate::irq::IrqLine; +use crate::traits::{BusDevice, BusRead64, Device, Resettable, BUS_ERR, BUS_OK}; +use parking_lot::Mutex; +use std::collections::VecDeque; +use std::io; +use std::sync::{ + atomic::{AtomicBool, AtomicU64, Ordering}, + Arc, +}; + +mod model; +use model::State; +pub use model::{parse_revision, BoardIdentity, CpuContext, L1Config, L1Event, DEFAULT_BRICK_SERIAL, DEFAULT_REVISION}; + +fn fcs(mut crc: u16, byte: u8) -> u16 { + crc ^= byte as u16; + for _ in 0..8 { + crc = (crc >> 1) ^ if crc & 1 != 0 { 0x8408 } else { 0 }; + } + crc +} + +fn encode(payload: &[u8]) -> Vec { + let crc = !payload.iter().fold(0xffff, |value, byte| fcs(value, *byte)); + let mut out = vec![0x7e]; + for byte in payload.iter().copied().chain([crc as u8, (crc >> 8) as u8]) { + if matches!(byte, 0x7e | 0x7d | 0x0d) { + out.extend_from_slice(&[0x7d, byte ^ 0x20]); + } else { + out.push(byte); + } + } + out.push(0x7e); + out +} + +const UART_BAUDBASE: u32 = 115200; +const UART_WINDOW_SIZE: u64 = 0x40; + +mod uart_offset { + use super::com_reg; + pub const DATA: u64 = 0x00; + pub const IER: u64 = 0x08; + pub const IIR_FCR: u64 = 0x10; + pub const LCR: u64 = 0x18; + pub const MCR: u64 = 0x20; + pub const LSR: u64 = 0x28; + pub const MSR: u64 = 0x30; + pub const SCR: u64 = 0x38; + + pub fn register(offset: u64) -> u32 { + match offset { + DATA => com_reg::THR, + IER => com_reg::IER, + IIR_FCR => com_reg::IIR_FCR, + LCR => com_reg::LCR, + MCR => com_reg::MCR, + LSR => com_reg::LSR, + MSR => com_reg::MSR, + SCR => com_reg::SCR, + _ => unreachable!("validated L1 UART offset"), + } + } +} + +type EventHandler = Arc; + +#[derive(Default)] +struct Dispatch { + active: bool, + pending: bool, + level: Option, +} + +/// The UART backend does not connect guest L1 commands to a host terminal. +/// Guest transmit bytes are handed to the protocol parser after the UART lock +/// is released; receive bytes come from the protocol reply queue. +struct ProtocolBackend { + guest_tx: Arc>>, + protocol_rx: Arc>>, +} + +impl SerialBackend for ProtocolBackend { + fn send_byte(&self, byte: u8) { + self.guest_tx.lock().push_back(byte); + } + + fn recv_byte(&self) -> io::Result { + self.protocol_rx + .lock() + .pop_front() + .ok_or_else(|| io::ErrorKind::WouldBlock.into()) + } +} + +/// The internal L1 connection is independent of IOC3's external serial ports. +/// Register offsets are relative to the 0x40-byte UART window. Only aligned +/// 64-bit accesses are decoded; register values occupy the low eight bits. +pub struct L1 { + // UART RX takes UART -> protocol queue. Model replies take state -> protocol + // queue. No path holds the protocol queue while acquiring either lock. + state: Arc>, + // Take dispatch before state when draining, release it before the UART or callbacks. + dispatch: Arc>, + uart: Arc>, + guest_tx: Arc>>, + backend: Arc>>, + irq: Arc>, + handler: Arc>>, + running: Arc, + clock: Arc, + worker: Mutex>>, + wake: Arc<(Mutex<()>, parking_lot::Condvar)>, +} +impl L1 { + pub fn new(config: L1Config) -> Result { + if config.rack > 0x3ff || config.bay > 0x3f || config.node_position > 3 { + return Err("L1 rack, bay or node position out of range".into()); + } + if config.port_speed_first > 1 { + return Err("L1 first port-speed value must be zero or one".into()); + } + if config.mac == [0; 6] || config.mac[0] & 1 != 0 { + return Err("L1 MAC must be nonzero unicast".into()); + } + if config + .brick_serial + .as_ref() + .is_some_and(|s| s.len() > 37 || s.as_bytes().contains(&0)) + { + return Err("L1 brick serial must fit a 37-byte text field".into()); + } + + let protocol_rx = Arc::new(Mutex::new(VecDeque::new())); + let state = Arc::new(Mutex::new(State::new(config, protocol_rx.clone()))); + let guest_tx = Arc::new(Mutex::new(VecDeque::new())); + let protocol_backend = Arc::new(ProtocolBackend { + guest_tx: guest_tx.clone(), + protocol_rx, + }); + let mut uart = Ns16550::new(UART_BAUDBASE); + uart.set_backend(protocol_backend); + + Ok(Self { + state, + dispatch: Arc::new(Mutex::new(Dispatch::default())), + uart: Arc::new(Mutex::new(uart)), + guest_tx, + backend: Arc::new(Mutex::new(null_serial_backend())), + irq: Arc::new(Mutex::new(IrqLine::unconnected())), + handler: Arc::new(Mutex::new(None)), + running: Arc::new(AtomicBool::new(false)), + clock: Arc::new(AtomicU64::new(0)), + worker: Mutex::new(None), + wake: Arc::new((Mutex::new(()), parking_lot::Condvar::new())), + }) + } + + /// Install the separate host-side L1 console connection. + pub fn set_serial_backend(&self, backend: Arc) { + *self.backend.lock() = backend; + } + + pub fn set_irq(&self, line: IrqLine) { + *self.irq.lock() = line; + self.dispatch.lock().level = None; + self.flush(); + } + + /// Power/reset requests reach the machine only after the reply is queued + /// and after the device state lock is released. + pub fn set_event_handler(&self, handler: EventHandler) { + *self.handler.lock() = Some(handler); + } + + #[cfg(test)] + pub fn node_powered(&self) -> bool { + self.state.lock().powered + } + + pub fn station_mac(&self) -> [u8; 6] { + self.state.lock().config.mac + } + + #[cfg(test)] + pub fn config(&self) -> L1Config { + self.state.lock().config.clone() + } + + #[cfg(test)] + pub fn record(&self, component: usize, area: u32) -> Option> { + let index = match area { + 1 => 0, + 2 => 1, + 4 => 2, + _ => return None, + }; + self.state.lock().records.get(component).and_then(|r| { + if r[index].is_empty() { + None + } else { + Some(r[index].clone()) + } + }) + } + + pub fn send_event(&self, code: u32, number: u32, text: &str) { + self.state.lock().event(code, number, text); + self.service_protocol_uart(); + self.flush(); + } + + /// Deterministic input path shared with the polled host serial backend. + pub fn receive_console(&self, bytes: &[u8]) { + for &byte in bytes { + self.state.lock().console_input(byte); + self.service_protocol_uart(); + self.flush(); + } + } + + pub fn poll_serial(&self) { + let backend = self.backend.lock().clone(); + for _ in 0..256 { + if self.state.lock().queue.lock().len() > 4088 { + break; + } + match backend.recv_byte() { + Ok(byte) => self.receive_console(&[byte]), + Err(error) => { + if error.kind() != io::ErrorKind::WouldBlock { + log::debug!("L1 serial input: {error}"); + } + break; + } + } + } + } + + /// Let the native UART pull reply bytes from the model's shared bounded queue. + fn service_protocol_uart(&self) { + let mut uart = self.uart.lock(); + uart.pump(); + uart.check_timeout(std::time::Instant::now()); + } + + /// Parse guest transmit only after the UART lock is released. The UART + /// callback only appends to this independent queue. + fn process_guest_transmit(&self) { + let bytes: Vec<_> = self.guest_tx.lock().drain(..).collect(); + if bytes.is_empty() { + return; + } + { + let mut state = self.state.lock(); + for byte in bytes { + state.parse(byte); + } + } + self.service_protocol_uart(); + } + + fn flush(&self) { + let mut dispatch = self.dispatch.lock(); + dispatch.pending = true; + if dispatch.active { + return; + } + dispatch.active = true; + loop { + dispatch.pending = false; + let (output, actions) = { + let mut state = self.state.lock(); + ( + std::mem::take(&mut state.output), + std::mem::take(&mut state.actions), + ) + }; + drop(dispatch); + let level = self.uart.lock().irq(); + + // Reentrant reads request another drain, but must not repeat an + // unchanged IRQ and recursively trigger the same machine callback. + dispatch = self.dispatch.lock(); + let changed = dispatch.level != Some(level); + dispatch.level = Some(level); + drop(dispatch); + + if changed { + let irq = self.irq.lock().clone(); + irq.set(level); + } + let backend = self.backend.lock().clone(); + for byte in output { + backend.send_byte(byte); + } + let handler = self.handler.lock().clone(); + for action in actions { + if let Some(handler) = &handler { + handler(action); + } else { + crate::dlog!(LogModule::L1, "unconnected L1 action {action:?}"); + } + } + + dispatch = self.dispatch.lock(); + if !dispatch.pending { + dispatch.active = false; + break; + } + } + } +} +impl BusDevice for L1 { + fn read64(&self, addr: u64) -> BusRead64 { + if addr >= UART_WINDOW_SIZE || addr & 7 != 0 { + return BusRead64::err(); + } + let value = self.uart.lock().read(uart_offset::register(addr)); + self.flush(); + BusRead64::ok(value as u64) + } + + fn write64(&self, addr: u64, value: u64) -> u32 { + if addr >= UART_WINDOW_SIZE || addr & 7 != 0 { + return BUS_ERR; + } + let register = uart_offset::register(addr); + let value = value as u8; + if register == com_reg::IIR_FCR && value & com_reg::FCR_RFR != 0 { + self.state.lock().queue.lock().clear(); + } + self.uart.lock().write(register, value); + self.process_guest_transmit(); + self.flush(); + BUS_OK + } +} +impl Resettable for L1 { + fn power_on(&self) { + self.state.lock().reset(); + self.guest_tx.lock().clear(); + self.uart.lock().reset(); + self.clock.store(0, Ordering::Relaxed); + self.flush(); + } +} +impl Device for L1 { + fn start(&self) { + let mut worker = self.worker.lock(); + if worker.is_some() { + return; + } + self.running.store(true, Ordering::Release); + let device = Self { + state: self.state.clone(), + dispatch: self.dispatch.clone(), + uart: self.uart.clone(), + guest_tx: self.guest_tx.clone(), + backend: self.backend.clone(), + irq: self.irq.clone(), + handler: self.handler.clone(), + running: self.running.clone(), + clock: self.clock.clone(), + worker: Mutex::new(None), + wake: self.wake.clone(), + }; + *worker = Some( + std::thread::Builder::new() + .name("l1-serial".into()) + .spawn(move || { + while device.is_running() { + device.poll_serial(); + let mut guard = device.wake.0.lock(); + if device.is_running() { + device + .wake + .1 + .wait_for(&mut guard, std::time::Duration::from_millis(2)); + } + } + }) + .expect("start L1 serial worker"), + ); + } + + fn stop(&self) { + let mut slot = self.worker.lock(); + { + let _guard = self.wake.0.lock(); + self.running.store(false, Ordering::Release); + self.wake.1.notify_all(); + } + let worker = slot.take(); + if let Some(worker) = worker { + if worker.thread().id() != std::thread::current().id() { + let _ = worker.join(); + } + } + } + + fn is_running(&self) -> bool { + self.running.load(Ordering::Acquire) + } + + fn get_clock(&self) -> u64 { + self.clock.load(Ordering::Relaxed) + } + + fn step(&self, cycles: u64) { + if self.is_running() { + self.clock.fetch_add(cycles, Ordering::Relaxed); + } + } +} +impl Drop for L1 { + fn drop(&mut self) { + self.stop(); + } +} +#[cfg(test)] +mod tests; diff --git a/src/dev/l1/model.rs b/src/dev/l1/model.rs new file mode 100644 index 000000000..2c24fe175 --- /dev/null +++ b/src/dev/l1/model.rs @@ -0,0 +1,879 @@ +//! Fuel's L1 controller protocol parser and reply model. +//! +//! This state owns packet framing, controller identity records, console +//! commands and power events. The UART register file and receive pacing live +//! in the shared Ns16550 core. +//! +//! Ported from hw/misc/sgi-l1.c in jamesbraid/qemu-sgi-origami. + +use super::*; + +/// Firmware revision the model reports for request 0x0005. +/// +/// 1.48.1 is the revision a real Fuel's L1 prints. The model reproduces +/// that firmware's replies, not its code, so the revision is a label. +pub const DEFAULT_REVISION: [u32; 3] = [1, 48, 1]; + +/// Brick serial the model reports for request 0x0004. +/// +/// A made-up serial, `EMU` for the emulator, system-board letter `N` and +/// instance 0. The real format is unknown, and the value must not pass as +/// a real SGI serial. +pub const DEFAULT_BRICK_SERIAL: &str = "EMUN00"; + +/// Parse a "major.minor.patch" revision string, as a `[fuel] l1_revision` +/// override spells it. Each part is a decimal number. +pub fn parse_revision(text: &str) -> Result<[u32; 3], String> { + let bad = || format!("L1 revision \"{text}\" must be major.minor.patch, each a decimal number"); + let mut parts = text.split('.'); + let mut rev = [0u32; 3]; + for slot in &mut rev { + let part = parts.next().ok_or_else(bad)?; + if part.is_empty() || !part.bytes().all(|b| b.is_ascii_digit()) { + return Err(bad()); + } + *slot = part.parse().map_err(|_| bad())?; + } + if parts.next().is_some() { + return Err(bad()); + } + Ok(rev) +} + +#[derive(Clone, Debug)] +pub struct L1Config { + pub rack: u32, + pub bay: u32, + pub node_position: u8, + pub type_code: u32, + pub revision: Option<[u32; 3]>, + pub mac: [u8; 6], + pub brick_serial: Option, + pub auto_power: bool, + /// First integer of the port-speed reply (request 0xa). The PROM uses it + /// to select a display branch; it is not a link speed. Zero or one. + pub port_speed_first: u32, + /// Answer the router command-interpreter `brick` request. Only the brick + /// types in `router_type_name` have a reply. + pub router: bool, + /// The CPU configuration the node board's record carries, if any. + pub cpu_context: Option, + /// The node board's identity in component 0's board area. Unset, the + /// board is the Fuel's system board, which answers as both the node + /// board and the I/O board, the I/O area carrying the station address. + /// Set, component 0 names this board and no I/O board record is given, + /// as QEMU's L1 does for an IP53 node. + pub node_board: Option, +} + +/// A board's identity as its board area names it: product name, part number +/// and revision. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct BoardIdentity { + pub name: &'static str, + pub part: &'static str, + pub revision: &'static str, +} +impl BoardIdentity { + /// The Fuel's system board as the model names it: a placeholder, not a + /// read record. + pub const FUEL: Self = Self { name: "IP34", part: "030-1707-003", revision: "-G" }; +} + +/// The CPU configuration an IP35 node board's EEPROM record carries beside +/// its identity: the processor mode word and the processor and Hub clocks. +/// At start-up the PROM checks the configuration block in its boot flash +/// against these, and when a board record carries none it falls back to a +/// table row chosen by the board word; a block that matches neither draws +/// "CPU configuration values mismatch". The encoding is the one QEMU's L1 +/// gives the Tezro and Origin 350 boards, which the stock PROM 6.210 +/// accepts; no factory record has been read, so the bytes are a software +/// input, not a capture. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct CpuContext { + /// The full 32-bit mode word, as the configuration block holds it. + pub mode_word: u32, + pub cpu_mhz: u32, + pub hub_mhz: u32, +} +impl Default for L1Config { + fn default() -> Self { + Self { + rack: 1, + bay: 1, + node_position: 0, + type_code: 0x41, + revision: Some(DEFAULT_REVISION), + mac: [8, 0, 0x69, 0x12, 0x34, 0x56], + brick_serial: Some(DEFAULT_BRICK_SERIAL.into()), + auto_power: true, + port_speed_first: 0, + router: false, + cpu_context: None, + node_board: None, + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum L1Event { + Power(bool), + Reset, + Shutdown, +} + +pub(super) struct State { + pub config: L1Config, + pub queue: Arc>>, + frame: Vec, + in_frame: bool, + escape: bool, + overflow: bool, + pub records: [[Vec; 3]; 16], + pub actions: Vec, + pub output: Vec, + pub powered: bool, + console: bool, + escaped: bool, + command: Vec, + port: u8, + events: Option, + power_down_pending: bool, + /// One reassembly per subchannel. + router: [Option; 32], +} + +/// A router message being reassembled from 0x3001, 0x3002 and 0x3003 chunks. +struct RouterAssembly { + fragment_count: u8, + fragments_left: u8, + first_length: usize, + expected: usize, + received: usize, + data: Vec, +} +const ROUTER_MAX_FRAGMENT: usize = 0x400; +const ROUTER_CHUNK_DATA: usize = 113; + +/// Brick types for which the router interpreter reply exists. +fn router_type_name(type_code: u32) -> Option<&'static str> { + match type_code { + 0x37 => Some("Chimera Server"), + 0x39 => Some("Chimera WS"), + 0x51 => Some("Rackmount WS"), + _ => None, + } +} +fn text_field(out: &mut Vec, text: &[u8]) { + out.push(1); + out.extend_from_slice(text); + out.push(0); +} +pub(super) fn ints(values: &[u32]) -> Vec { + let mut out = Vec::new(); + for v in values { + out.push(0); + out.extend_from_slice(&v.to_be_bytes()); + } + out +} +fn word(p: &[u8]) -> u32 { + u32::from_be_bytes(p[..4].try_into().unwrap()) +} +fn arguments(mut p: &[u8], count: u8) -> Option> { + let mut fields = Vec::new(); + for _ in 0..count { + let tag = *p.first()?; + let len = match tag { + 0 => 5, + 1 => p[1..].iter().position(|b| *b == 0)? + 2, + t if t & 0x80 != 0 => 1 + (t & 0x7f) as usize, + _ => return None, + }; + fields.push(p.get(..len)?); + p = &p[len..]; + } + if p.is_empty() { + Some(fields) + } else { + None + } +} +fn area_finish(mut area: Vec) -> Vec { + while (area.len() + 1) % 8 != 0 { + area.push(0); + } + let sum = area.iter().fold(0u8, |s, b| s.wrapping_add(*b)); + area.push(sum.wrapping_neg()); + area +} +fn ia_text(area: &mut Vec, text: &str) { + area.push(0xc0 | text.len() as u8); + area.extend_from_slice(text.as_bytes()); +} +fn board(identity: BoardIdentity, mac: Option<[u8; 6]>, cpu: Option) -> Vec { + let mut b = vec![0; 6]; + for s in ["SYNTHETIC", identity.name, "EMUN00", identity.part] { + ia_text(&mut b, s); + } + b.push(0); + ia_text(&mut b, identity.revision); + b.extend_from_slice(&[1, 0]); + ia_text(&mut b, "00"); + if let Some(cpu) = cpu { + // Format 2 and a one-byte binary 3 select the CPU fields: the Hub + // mode bits, Config0 to Config3, the Hub and CPU clocks in MHz and + // three minimum versions. Only Config0 and the clocks are known; + // the rest are empty fields, not zeros. + b[0] = 2; + b.extend_from_slice(&[1, 3, 0]); + b.push(4); + b.extend_from_slice(&cpu.mode_word.to_be_bytes()); + b.extend_from_slice(&[0, 0, 0]); + for mhz in [cpu.hub_mhz, cpu.cpu_mhz] { + b.push(4); + b.extend_from_slice(&mhz.to_be_bytes()); + } + b.extend_from_slice(&[0, 0, 0]); + } + for _ in 0..3 { + b.extend_from_slice(&[4, 0, 0, 0, 0]); + } + if let Some(mac) = mac { + ia_text( + &mut b, + &mac.iter().map(|v| format!("{v:02X}")).collect::(), + ); + } + b.push(0xc1); + // The count includes the checksum and padding, not a separate header. + b[1] = (b.len() + 1).div_ceil(8) as u8; + area_finish(b) +} +impl State { + pub fn new(config: L1Config, queue: Arc>>) -> Self { + let mut records: [[Vec; 3]; 16] = + std::array::from_fn(|_| std::array::from_fn(|_| Vec::new())); + let chassis = area_finish(vec![0, 2, 0, 0xc2, b'N', b'A', 0xc2, b'N', b'A', 0xc1]); + let identity = config.node_board.unwrap_or(BoardIdentity::FUEL); + records[0] = [chassis.clone(), board(identity, None, config.cpu_context), Vec::new()]; + if config.node_board.is_none() { + records[1] = [chassis, board(identity, Some(config.mac), None), Vec::new()]; + } + let powered = config.auto_power; + let port = config.node_position << 5; + Self { + config, + queue, + frame: Vec::new(), + in_frame: false, + escape: false, + overflow: false, + records, + actions: Vec::new(), + output: Vec::new(), + powered, + console: powered, + escaped: false, + command: Vec::new(), + port, + events: None, + power_down_pending: false, + router: std::array::from_fn(|_| None), + } + } + pub fn reset(&mut self) { + // The event subchannel survives the controller's local boot if it is + // one of the operating system's. The UART and packet parser reset. + let events = self.events.filter(|&channel| channel >= 4); + let records = std::mem::replace( + &mut self.records, + std::array::from_fn(|_| std::array::from_fn(|_| Vec::new())), + ); + let queue = self.queue.clone(); + queue.lock().clear(); + *self = Self::new(self.config.clone(), queue); + self.events = events; + self.records = records; + } + fn emit(&mut self, payload: &[u8]) { + if payload.len() > 128 { + crate::dlog!(LogModule::L1, "L1 reply exceeds frame capacity"); + return; + } + let bytes = encode(payload); + let mut queue = self.queue.lock(); + if queue.len() + bytes.len() <= 4096 { + queue.extend(bytes); + } else { + crate::dlog!(LogModule::L1, "L1 receive queue full"); + } + } + fn reply(&mut self, channel: u8, code: i32, count: u8, fields: &[u8]) { + let mut p = vec![0x20 | channel]; + p.extend_from_slice(&code.to_be_bytes()); + p.push(count); + p.extend_from_slice(fields); + self.emit(&p); + } + pub fn parse(&mut self, mut byte: u8) { + if byte == 0x7e { + if self.in_frame + && !self.escape + && !self.overflow + && self.frame.len() >= 3 + && self.frame.iter().fold(0xffff, |v, b| fcs(v, *b)) == 0xf0b8 + { + let payload = self.frame[..self.frame.len() - 2].to_vec(); + self.request(&payload); + } + self.in_frame = true; + self.escape = false; + self.overflow = false; + self.frame.clear(); + return; + } + if !self.in_frame || self.overflow { + return; + } + if byte == 0x7d { + self.escape = true; + return; + } + if self.escape { + byte ^= 0x20; + self.escape = false; + } + if self.frame.len() == 258 { + self.overflow = true; + } else { + self.frame.push(byte); + } + } + fn eeprom(&mut self, channel: u8, a: &[&[u8]]) { + if a.len() < 4 || a[..4].iter().any(|v| v[0] != 0) { + self.reply(channel, -100, 0, &[]); + return; + } + let component = word(&a[0][1..]) as usize; + let area = word(&a[1][1..]); + let offset = word(&a[2][1..]) as usize; + let requested = word(&a[3][1..]) as usize; + if component >= 16 || self.records[component].iter().all(Vec::is_empty) { + self.reply(channel, -104, 0, &[]); + return; + } + if area > 4 { + self.reply(channel, -105, 0, &[]); + return; + } + let index = match area { + 1 => 0, + 2 => 1, + 4 => 2, + _ => { + self.reply(channel, -104, 0, &[]); + return; + } + }; + let record = &self.records[component][index]; + if record.is_empty() { + self.reply(channel, -104, 0, &[]); + return; + } + let checksum_len = if area == 4 { + record.len().min(64) + } else { + record.len() + }; + if record[..checksum_len] + .iter() + .fold(0u8, |s, b| s.wrapping_add(*b)) + != 0 + { + self.reply(channel, -107, 0, &[]); + return; + } + if offset >= record.len() { + self.reply(channel, -105, 0, &[]); + return; + } + let len = requested.min(record.len() - offset); + if len > 116 { + crate::dlog!(LogModule::L1, "L1 EEPROM reply exceeds frame capacity"); + self.reply(channel, -105, 0, &[]); + return; + } + let mut fields = ints(&[(record.len() - offset - len) as u32]); + if requested != 0 { + fields.push(0x80 | len as u8); + fields.extend_from_slice(&record[offset..offset + len]); + } + self.reply(channel, 0, if requested == 0 { 1 } else { 2 }, &fields); + } + fn request(&mut self, p: &[u8]) { + let channel = p[0] & 31; + if self.config.router + && p.len() >= 7 + && p[0] & 0xe0 == 0 + && word(&p[1..]) == 0 + && p[5] == 0x30 + && matches!(p[6], 1..=3) + { + self.router_chunk(p); + return; + } + if p[0] & 0xe0 == 0x40 { + self.port = (self.port & 0x60) | channel; + if self.console && self.powered { + self.output.extend_from_slice(&p[1..]); + } else { + crate::dlog!(LogModule::L1, "L1 console output while operator is in L1 mode"); + } + return; + } + if p[0] & 0xe0 != 0 || p.len() < 7 { + return; + } + let address = word(&p[1..]); + let cmd = u16::from_be_bytes([p[5], p[6]]); + let key = address >> 12; + if key != 0xffff && key != ((self.config.rack << 6) | self.config.bay) { + crate::dlog!(LogModule::L1, "L1 unsupported destination {address:#x}"); + self.emit(&[0x20 | channel, 255, 255, 255, 255, 0, 0, 0, 8]); + return; + } + let count = if p.len() == 7 { 0 } else { p[7] }; + let raw = if p.len() > 8 { &p[8..] } else { &[] }; + let task = address & 31; + if task != 6 && task != 3 { + crate::dlog!(LogModule::L1, "L1 unsupported task {task}"); + return; + } + let Some(a) = arguments(raw, count) else { + self.reply(channel, -100, 0, &[]); + return; + }; + if task == 3 { + if a.len() != 1 || a[0][0] != 1 { + self.reply(channel, -100, 0, &[]); + return; + } + let text = String::from_utf8_lossy(&a[0][1..a[0].len() - 1]); + let (code, output) = self.run_line(&text, false); + let mut fields = Vec::new(); + let mut n = 0; + for line in output.split("\r\n").filter(|s| !s.is_empty()) { + if fields.len() + line.len() + 2 > 121 { + break; + } + text_field(&mut fields, line.as_bytes()); + n += 1; + } + self.reply(channel, code, n, &fields); + return; + } + let mut values = Vec::new(); + let mut fields = Vec::new(); + let mut field_count = 0; + let mut error = 0; + // Requests that neither the stock PROM nor IRIX sends are refused + // as unknown, and logged. + match cmd { + // No debug switch set. + 1 => values.push(0), + 3 => { + if self.config.type_code == 0 { + error = -104; + } else { + values.extend([ + (self.config.rack << 6) | self.config.bay, + self.config.type_code, + ]); + } + } + 4 => { + if let Some(serial) = &self.config.brick_serial { + text_field(&mut fields, serial.as_bytes()); + field_count = 1; + } else { + error = -104; + } + } + 5 => { + if let Some(rev) = self.config.revision { + values.extend(rev); + } else { + crate::dlog!(LogModule::L1, "L1 revision metadata not configured"); + error = -104; + } + } + 6 => { + self.eeprom(channel, &a); + return; + } + // The system serial is the station address as twelve uppercase + // hexadecimal digits in address order, no separators or OUI + // added: the text the stock PROM converts straight back into the + // Ethernet address. + 8 => { + let mac = self + .config + .mac + .iter() + .map(|b| format!("{b:02X}")) + .collect::(); + text_field(&mut fields, mac.as_bytes()); + field_count = 1; + } + // Not partitioned. + 9 => values.push(0), + 0xa => values.extend([self.config.port_speed_first, 0]), + // Zero: the PROM's partition check goes no further. + 0xb => values.push(0), + 0xc => values.extend([0; 8]), + 0xe => values.push((self.port >> 5) as u32 & 3), + 0x1007 => { + if a.is_empty() { + error = -100; + } else if a[0][0] != 0 { + error = -105; + } else { + self.events = Some(word(&a[0][1..]) as u8); + } + } + 0x1009 => { + if a.len() < 2 { + error = -100; + } else if a[0][0] != 0 || a[1][0] != 0 || word(&a[1][1..]) & 15 > 2 { + error = -105; + } + } + 0x100a => self.events = None, + 0x100c => { + if a.len() != 2 { + error = -100; + } + } + 0x2001 | 0x2005 | 0x2006 => { + self.reply(channel, 0, 0, &[]); + match cmd { + 0x2001 => self.reset_node(), + 0x2005 => self.power_down(true, false), + _ => { + self.power_down(false, false); + self.power_up(true); + } + } + return; + } + // The expander byte, which IRIX's VPro driver asks for, is only + // on router and I/O bricks: refused on this one. + 0x2007 | 0x2008 => error = -108, + _ => { + crate::dlog!(LogModule::L1, "L1 unsupported request {cmd:#x}"); + error = -101; + } + } + if !values.is_empty() { + field_count = values.len() as u8; + fields = ints(&values); + } + self.reply(channel, error, field_count, &fields); + } + /// The router replies are chunked and reassembled with the same three + /// commands; a malformed chunk drops the pending message. + fn router_chunk(&mut self, p: &[u8]) { + let channel = p[0] & 0x1f; + let slot = usize::from(channel); + if p.len() < 10 { + crate::dlog!(LogModule::L1, "L1 malformed router chunk"); + return; + } + let command = u16::from_be_bytes([p[5], p[6]]); + let payload = &p[10..]; + let fragment_length = u16::from_be_bytes([p[8], p[9]]) as usize; + match command { + 0x3001 => { + let fragments = p[7]; + if self.router[slot].take().is_some() { + crate::dlog!(LogModule::L1, "L1 replacing pending router request"); + } + if fragments == 0 + || fragments >= 128 + || fragment_length == 0 + || fragment_length > ROUTER_MAX_FRAGMENT + || payload.len() > fragment_length + { + crate::dlog!(LogModule::L1, "L1 unsupported router message start"); + return; + } + self.router[slot] = Some(RouterAssembly { + fragment_count: fragments, + fragments_left: fragments, + first_length: 0, + expected: fragment_length, + received: payload.len(), + data: payload.to_vec(), + }); + } + 0x3002 => { + let ok = self.router[slot].as_ref().is_some_and(|a| { + a.fragments_left != 0 + && a.received == a.expected + && fragment_length != 0 + && fragment_length <= ROUTER_MAX_FRAGMENT - a.data.len() + && payload.len() <= fragment_length + }); + let Some(a) = self.router[slot].as_mut().filter(|_| ok) else { + self.router[slot] = None; + crate::dlog!(LogModule::L1, "L1 invalid router next fragment"); + return; + }; + a.expected = fragment_length; + a.received = payload.len(); + a.data.extend_from_slice(payload); + } + _ => { + let ok = self.router[slot].as_ref().is_some_and(|a| { + a.received != a.expected + && payload.len() <= a.expected - a.received + && payload.len() <= ROUTER_MAX_FRAGMENT - a.data.len() + }); + let Some(a) = self.router[slot].as_mut().filter(|_| ok) else { + self.router[slot] = None; + crate::dlog!(LogModule::L1, "L1 router continuation without valid message"); + return; + }; + a.received += payload.len(); + a.data.extend_from_slice(payload); + } + } + self.router_complete(channel); + } + fn router_complete(&mut self, channel: u8) { + let slot = usize::from(channel); + let Some(a) = self.router[slot].as_mut() else { return }; + if a.received != a.expected { + return; + } + if a.first_length == 0 { + a.first_length = a.data.len(); + } + a.fragments_left -= 1; + if a.fragments_left != 0 { + return; + } + let a = self.router[slot].take().unwrap(); + // Only the single-fragment layout has a known descriptor mapping. + if a.fragment_count == 1 { + self.router_reply(channel, &a.data[..a.first_length], a.fragment_count); + } else { + crate::dlog!(LogModule::L1, "L1 multi-fragment router dispatch is unsupported"); + } + } + fn router_reply(&mut self, channel: u8, request: &[u8], fragment_count: u8) { + let Some(type_name) = router_type_name(self.config.type_code) else { + crate::dlog!(LogModule::L1, "L1 unsupported or malformed router request"); + return; + }; + let n = request.len(); + // Fixed header, an all-zero scalar block, then the string descriptor + // (kind 0x0a, 1) carrying the interpreter command. + let well_formed = n >= 37 + && usize::from(u16::from_be_bytes([request[0], request[1]])) == n + && request[2] == 2 + && request[3] == 0 + && request[5] == 0x80 | fragment_count + && request[6] == 0 + && request[7] == 0 + && word(&request[8..]) == 0x0fff_f003 + && request[16] == 2 + && request[20..28] == [0; 8] + && request[28] == 0x0a + && request[29] == 1 + && word(&request[32..]) == 52 + && usize::from(u16::from_be_bytes([request[30], request[31]])) == n - 36 + && request[n - 1] == 0; + if !well_formed { + crate::dlog!(LogModule::L1, "L1 unsupported or malformed router request"); + return; + } + // The captured form has one trailing space; the plain spelling is + // accepted too. Anything else is refused. + if !matches!(&request[36..], b"brick\0" | b"brick \0") { + crate::dlog!(LogModule::L1, "L1 router command interpreter operation refused"); + return; + } + // The node is not partitioned (see request 9). + let partition = "none"; + let serial = self.config.brick_serial.as_deref().unwrap_or("(null)"); + let mut text = format!( + "rack: {:03}, slot: {:02}, partition: {partition}, type: {type_name} \ + [?MB flash], serial:{serial}, source: OTHER\n\n", + self.config.rack, self.config.bay + ) + .into_bytes(); + text.push(0); + let total = 36 + text.len(); + let node_id = (self.config.rack << 6) | self.config.bay; + let mut response = vec![0u8; 36]; + response[..2].copy_from_slice(&(total as u16).to_be_bytes()); + response[2] = 2; + response[4] = request[4]; + response[5] = 0x81; + response[6] = 1; + // The reply goes back to the requesting channel on this node. + response[8..12].copy_from_slice(&((node_id << 12) | (u32::from(channel) << 5) | 0x15).to_be_bytes()); + response[12..16].copy_from_slice(&((node_id << 12) | 3).to_be_bytes()); + response[16] = 2; + response[17] = request[17]; + response[18..20].copy_from_slice(&request[18..20]); + response[28] = 0x0a; + response[29] = 1; + response[30..32].copy_from_slice(&(text.len() as u16).to_be_bytes()); + response[32..36].copy_from_slice(&0x34u32.to_be_bytes()); + response.extend_from_slice(&text); + if total > ROUTER_MAX_FRAGMENT { + crate::dlog!(LogModule::L1, "L1 router brick response exceeds fragment limit"); + return; + } + for (i, part) in response.chunks(ROUTER_CHUNK_DATA).enumerate() { + let mut chunk = vec![0x20 | channel, 0, 0, 0, 0, 0x30, if i == 0 { 1 } else { 3 }]; + if i == 0 { + chunk.push(1); + chunk.extend_from_slice(&(total as u16).to_be_bytes()); + } else { + chunk.extend_from_slice(&[0, 0, 0]); + } + chunk.extend_from_slice(part); + self.emit(&chunk); + } + } + fn reset_node(&mut self) { + if self.powered { + self.actions.push(L1Event::Reset); + } + } + fn power_down(&mut self, shutdown: bool, confirm: bool) { + if self.powered { + if confirm && !self.power_down_pending { + self.power_down_pending = true; + self.event( + 0x8000, + 0, + "Power Down: issue 'pwr d' again to immediately power down.", + ); + return; + } + self.power_down_pending = false; + self.powered = false; + self.actions.push(L1Event::Power(false)); + if shutdown { + self.actions.push(L1Event::Shutdown); + } + } + } + fn power_up(&mut self, restart: bool) { + self.powered = true; + self.power_down_pending = false; + self.port &= 0x60; + self.console = true; + self.actions.push(L1Event::Power(true)); + if restart { + self.actions.push(L1Event::Reset); + } + } + pub fn event(&mut self, code: u32, number: u32, text: &str) { + let Some(channel) = self.events else { + return; + }; + let mut p = vec![0x40 | (channel & 31)]; + p.extend_from_slice( + &((self.config.rack << 18) | (self.config.bay << 12) | 4).to_be_bytes(), + ); + p.extend_from_slice(&code.to_be_bytes()); + p.push(2); + p.extend(ints(&[number])); + text_field( + &mut p, + format!( + "\r\n{:03}c{:02} ATTN: {text}\r\n", + self.config.rack, self.config.bay + ) + .as_bytes(), + ); + if p.len() <= 127 { + self.emit(&p); + } + } + fn run_line(&mut self, line: &str, interactive: bool) -> (i32, String) { + let words = line.split_whitespace().collect::>(); + let local = if words.first() == Some(&"*") { + &words[1..] + } else { + &words[..] + }; + match local { + ["pwr" | "power", "d" | "down"] => { + self.power_down(true, interactive); + (0, String::new()) + } + ["pwr" | "power", "u" | "up"] => { + if !self.powered { + self.power_up(true); + } + (0, String::new()) + } + ["reset"] => { + self.reset_node(); + (0, String::new()) + } + ["l1"] => { + self.escaped = false; + (0, String::new()) + } + _ => { + crate::dlog!(LogModule::L1, "L1 unsupported operator command {line:?}"); + (-1, "ERROR: command not found.\r\n".into()) + } + } + } + fn prompt(&mut self) { + self.output.extend_from_slice( + format!("{:03}c{:02}-L1>", self.config.rack, self.config.bay).as_bytes(), + ); + } + pub fn console_input(&mut self, byte: u8) { + if self.console { + if byte == 0x14 { + self.console = false; + self.escaped = true; + self.output + .extend_from_slice(b"escaping to L1 system controller\r\n"); + self.prompt(); + } else if self.powered { + self.emit(&[self.port & 31, byte]); + } else { + crate::dlog!(LogModule::L1, "L1 console input while powered off"); + } + } else if byte == 4 { + self.command.clear(); + self.console = true; + } else if byte == b'\r' || byte == b'\n' { + let line = String::from_utf8_lossy(&self.command).into_owned(); + self.command.clear(); + let (_, output) = self.run_line(&line, true); + self.output.extend_from_slice(output.as_bytes()); + if self.escaped { + self.console = true; + } else { + self.prompt(); + } + } else if byte == 8 || byte == 127 { + self.command.pop(); + } else if self.command.len() < 255 { + self.command.push(byte); + } + } +} diff --git a/src/dev/l1/tests.rs b/src/dev/l1/tests.rs new file mode 100644 index 000000000..d224b0063 --- /dev/null +++ b/src/dev/l1/tests.rs @@ -0,0 +1,645 @@ +// Cases follow tests/qtest/sn1-l1-test.c in jamesbraid/qemu-sgi-origami: +// uart, frames, identity, eeprom, errors and reset, plus the port-speed and +// router brick-reply cases (test_port_speed, test_irouter_*). + +use super::*; + +fn send(l1: &L1, bytes: &[u8]) { + for &b in bytes { + assert_eq!(l1.write64(0, b as u64), BUS_OK); + } +} +fn drain(l1: &L1) -> Vec { + let mut out = Vec::new(); + while l1.read64(0x28).data & 1 != 0 { + out.push(l1.read64(0).data as u8); + } + out +} +fn request(l1: &L1, command: u16, args: &[u8], count: u8) -> Vec { + let mut p = vec![ + 0x10, + 0x0f, + 0xff, + 0xf0, + 6, + (command >> 8) as u8, + command as u8, + count, + ]; + p.extend_from_slice(args); + send(l1, &encode(&p)); + drain(l1) +} +fn integer_reply(values: &[u32]) -> Vec { + let mut p = vec![0x30, 0, 0, 0, 0, values.len() as u8]; + p.extend(model::ints(values)); + encode(&p) +} + +/// The PROM accesses the L1 UART through byte-sized register operations. +#[test] +fn the_l1_uart_frames_bytes_and_prioritizes_interrupts() { + let l1 = L1::new(Default::default()).unwrap(); + assert_eq!(l1.read64(1).status, BUS_ERR); + assert_eq!(l1.read8(0).status, BUS_ERR); + assert_eq!(l1.read32(0).status, BUS_ERR); + assert_eq!(l1.read64(0x28).data, 0x60); + send(&l1, &[0x7e, 0x10, 0x0f, 0xff, 0xf0, 6, 0, 0x0e, 0, 0xf9, 0x72, 0x7e]); + assert_eq!(drain(&l1), [0x7e, 0x30, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0x30, 0x98, 0x7e]); + let (irq, line) = crate::irq::LevelProbe::line(); + l1.set_irq(line); + l1.write64(8, 3); + assert!(irq.level()); + assert_eq!(l1.read64(0x10).data, 2); + assert!(!irq.level()); + l1.write64(0x18, 0x80); + l1.write64(0, 42); + l1.write64(8, 7); + assert_eq!(l1.read64(0).data, 42); + assert_eq!(l1.read64(8).data, 7); + l1.write64(0x18, 3); + send(&l1, &encode(&[0x10, 0x0f, 0xff, 0xf0, 6, 0, 0x0e, 0])); + assert_eq!(l1.read64(0x10).data, 4); + drain(&l1); + assert_eq!(l1.read64(0x10).data, 2); + assert!(!irq.level()); +} + +/// The guest reads controller identity and checks the EEPROM record checksum. +#[test] +fn the_l1_returns_identity_and_validated_eeprom_records() { + let l1 = L1::new(Default::default()).unwrap(); + assert_eq!(request(&l1, 3, &[], 0), integer_reply(&[65, 65])); + let board = l1.record(1, 2).unwrap(); + assert_eq!(board.iter().fold(0u8, |s, x| s.wrapping_add(*x)), 0); + for text in [b"IP34".as_slice(), b"EMUN00", b"080069123456"] { + assert!(board.windows(text.len()).any(|x| x == text)); + } + let mut expected = vec![0x30, 0, 0, 0, 0, 2]; + expected.extend(model::ints(&[0])); + expected.push(0x80 | board.len() as u8); + expected.extend(board); + assert_eq!(request(&l1, 6, &model::ints(&[1, 2, 0, 116]), 4), encode(&expected)); + for (args, count, error) in [([2, 5, 0, 1], 4, 152), ([1, 5, 0, 1], 4, 151), ([1, 2, 0, 1], 3, 156)] { + assert_eq!(request(&l1, 6, &model::ints(&args), count), encode(&[0x30, 255, 255, 255, error, 0])); + } + l1.power_on(); + assert_eq!(request(&l1, 5, &[], 0), integer_reply(&[1, 48, 1])); +} + +/// A configured CPU configuration goes into component 0's board area, after +/// the identity fields, as format 2 with the one-byte selector 3, the Hub mode +/// bits, Config0 to Config3, the Hub and CPU MHz and three versions, the +/// unknown ones as empty fields; the I/O board's area and the default record +/// carry none. The layout is QEMU's (`l1_build_board_area`). +#[test] +fn the_node_board_record_carries_a_configured_cpu_configuration() { + let plain = L1::new(Default::default()).unwrap().record(0, 2).unwrap(); + let cpu = CpuContext { mode_word: 0x589b_ad85, cpu_mhz: 700, hub_mhz: 200 }; + let l1 = L1::new(L1Config { cpu_context: Some(cpu), ..Default::default() }).unwrap(); + let area = l1.record(0, 2).unwrap(); + assert_eq!(area.iter().fold(0u8, |s, x| s.wrapping_add(*x)), 0); + assert_eq!((area.len() % 8, usize::from(area[1]) * 8), (0, area.len())); + assert_eq!((plain[0], area[0]), (0, 2)); + // The identity fields end with the two-character text field "00". + let identity = plain.windows(3).position(|w| w == [0xc2, b'0', b'0']).unwrap() + 3; + assert_eq!(&area[2..identity], &plain[2..identity], "the identity fields are unchanged"); + let mut fields = vec![1, 3, 0, 4, 0x58, 0x9b, 0xad, 0x85, 0, 0, 0, 4, 0, 0, 0, 200, 4, 0, 0, 0x02, 0xbc, 0, 0, 0]; + for _ in 0..3 { + fields.extend_from_slice(&[4, 0, 0, 0, 0]); + } + fields.push(0xc1); + assert_eq!(&area[identity..identity + fields.len()], fields); + assert_eq!(l1.record(1, 2), L1::new(Default::default()).unwrap().record(1, 2), "the I/O board's area"); +} + +/// A board that names its node board gets that identity in component 0's +/// board area, empty fields included, and no I/O board record; with none +/// named the Fuel's placeholder answers in both, as before. +#[test] +fn a_named_node_board_answers_in_component_0_alone() { + let ip53 = BoardIdentity { name: "IP53_4CPU", part: "030-1868-001", revision: "-C" }; + let l1 = L1::new(L1Config { node_board: Some(ip53), ..Default::default() }).unwrap(); + let area = l1.record(0, 2).unwrap(); + assert_eq!(area.iter().fold(0u8, |s, x| s.wrapping_add(*x)), 0); + for text in [b"IP53_4CPU".as_slice(), b"030-1868-001", b"-C"] { + assert!(area.windows(text.len()).any(|x| x == text), "{}", String::from_utf8_lossy(text)); + } + assert!(!area.windows(4).any(|x| x == b"IP34")); + assert_eq!(l1.record(1, 2), None, "no I/O board record"); + assert_eq!(l1.record(1, 1), None); + let empty = BoardIdentity { name: "", part: "", revision: "" }; + let area = L1::new(L1Config { node_board: Some(empty), ..Default::default() }).unwrap().record(0, 2).unwrap(); + // SYNTHETIC, then an empty name, the serial, an empty part, the FRU file + // byte and an empty revision. + let at = area.windows(10).position(|x| x == b"\xc9SYNTHETIC").unwrap() + 10; + assert_eq!(&area[at..at + 11], b"\xc0\xc6EMUN00\xc0\x00\xc0"); + let fuel = L1::new(Default::default()).unwrap(); + assert_eq!(fuel.record(0, 2), L1::new(L1Config { node_board: None, ..Default::default() }).unwrap().record(0, 2)); + assert!(fuel.record(1, 2).is_some()); +} + +/// A successful reply carrying one text field. +fn text_reply(text: &str) -> Vec { + let mut p = vec![0x30, 0, 0, 0, 0, 1]; + p.push(1); + p.extend_from_slice(text.as_bytes()); + p.push(0); + encode(&p) +} + +/// With no configuration the controller reports the labelled model defaults: +/// firmware 1.48.1 and the synthetic brick serial. +#[test] +fn the_l1_reports_the_default_revision_and_brick_serial() { + let l1 = L1::new(Default::default()).unwrap(); + l1.power_on(); + assert_eq!(request(&l1, 5, &[], 0), integer_reply(&[1, 48, 1])); + assert_eq!(request(&l1, 4, &[], 0), text_reply("EMUN00")); +} + +/// Both identity values can be replaced. +#[test] +fn the_l1_revision_and_brick_serial_can_be_overridden() { + let l1 = L1::new(L1Config { + revision: Some(parse_revision("2.13.40").unwrap()), + brick_serial: Some("EMUN07".into()), + ..Default::default() + }) + .unwrap(); + l1.power_on(); + assert_eq!(request(&l1, 5, &[], 0), integer_reply(&[2, 13, 40])); + assert_eq!(request(&l1, 4, &[], 0), text_reply("EMUN07")); + assert_eq!( + L1::new(L1Config { brick_serial: Some("x".repeat(38)), ..Default::default() }).err().as_deref(), + Some("L1 brick serial must fit a 37-byte text field") + ); +} + +/// The revision override is exactly three decimal numbers. +#[test] +fn the_l1_revision_text_is_three_decimal_parts() { + assert_eq!(parse_revision("1.48.1"), Ok([1, 48, 1])); + for bad in ["", "1", "1.48", "1.48.1.0", "1..1", "a.b.c", "1.48.-1", "1.48.+1", " 1.48.1", "1.48.99999999999"] { + assert!(parse_revision(bad).is_err(), "{bad:?}"); + } +} + +/// Request 0x0008 is the configured station address as twelve uppercase +/// hexadecimal digits in address order, the text the PROM turns back into the +/// Ethernet address. +#[test] +fn the_l1_system_serial_is_the_station_address_in_hex() { + let l1 = L1::new(Default::default()).unwrap(); + l1.power_on(); + assert_eq!(request(&l1, 8, &[], 0), text_reply("080069123456")); + let l1 = L1::new(L1Config { mac: [0x00, 0xA0, 0xc9, 0x1f, 0x02, 0xbe], ..Default::default() }).unwrap(); + l1.power_on(); + assert_eq!(request(&l1, 8, &[], 0), text_reply("00A0C91F02BE")); +} + +/// The L1 parser recovers after malformed frames. +#[test] +fn the_l1_resynchronizes_after_bad_frames() { + let l1 = L1::new(Default::default()).unwrap(); + send(&l1, &[0x7e, 1, 2, 3, 0x7e]); + send(&l1, &vec![0x41; 259]); + send(&l1, &[0x7e]); + assert!(drain(&l1).is_empty()); + assert_eq!(request(&l1, 0x2000, &[], 0), encode(&[0x30, 255, 255, 255, 155, 0])); + send(&l1, &encode(&[0x10, 0x4f, 0xff, 0xf0, 6, 0, 3, 0])); + assert_eq!(drain(&l1), encode(&[0x30, 255, 255, 255, 255, 0, 0, 0, 8])); + // A good frame after them is answered. + assert_eq!(request(&l1, 9, &[], 0), integer_reply(&[0])); +} + +/// The PROM may reset the L1 during boot: the UART and parser restart, and +/// an event subchannel the operating system set (23) survives while the +/// PROM's own (3) does not. +#[test] +fn l1_reset_clears_uart_state_but_keeps_the_os_event_channel() { + for channel in [3, 23] { + let l1 = L1::new(Default::default()).unwrap(); + let (irq, line) = crate::irq::LevelProbe::line(); + l1.set_irq(line); + request(&l1, 0x1007, &model::ints(&[channel]), 1); + l1.write64(8, 1); + l1.send_event(0x8000, 7, "before reset"); + assert!(irq.level()); + let stale = encode(&[0x10, 0x0f, 0xff, 0xf0, 6, 0, 9, 0]); + send(&l1, &stale[..4]); + l1.write64(0x18, 0x80); + l1.write64(0, 42); + l1.write64(8, 7); + l1.power_on(); + assert!(!irq.level()); + assert_eq!(l1.read64(0x18).data, 0); + assert_eq!(l1.read64(8).data, 0); + assert!(drain(&l1).is_empty()); + l1.write64(0x18, 0x80); + // Ns16550 resets its divisor latch to 0x000c for 9600 baud. + assert_eq!(l1.read64(0).data, 0x0c); + assert_eq!(l1.read64(8).data, 0); + l1.write64(0x18, 0); + send(&l1, &stale[4..]); + assert!(drain(&l1).is_empty()); + l1.write64(8, 1); + l1.send_event(0x8000, 7, "after reset"); + assert_eq!(irq.level(), channel == 23); + let emitted = drain(&l1); + assert_eq!(emitted.is_empty(), channel == 3); + assert!(!irq.level()); + } +} + +/// Requests neither the stock PROM nor IRIX sends are refused as unknown +/// (minus 101), with no other effect. +#[test] +fn requests_no_guest_sends_are_refused() { + let l1 = L1::new(Default::default()).unwrap(); + for cmd in [2, 7, 0xf, 0x1001, 0x1004, 0x4002] { + assert_eq!(request(&l1, cmd, &[], 0), encode(&[0x30, 255, 255, 255, 155, 0]), "{cmd:#x}"); + } +} + +/// A local console power-down request requires a second confirmation command. +#[test] +fn interactive_power_down_waits_for_a_second_command() { + let l1 = Arc::new(L1::new(Default::default()).unwrap()); + let actions = Arc::new(Mutex::new(Vec::new())); + let observed = actions.clone(); + let weak = Arc::downgrade(&l1); + l1.set_event_handler(Arc::new(move |event| { + assert!(!weak.upgrade().unwrap().node_powered()); + observed.lock().push(event); + })); + request(&l1, 0x1007, &model::ints(&[23]), 1); + l1.receive_console(&[0x14]); + l1.receive_console(b"pwr d\r"); + assert!(l1.node_powered()); + assert!(actions.lock().is_empty()); + let mut warning = vec![0x57, 0, 4, 0x10, 4, 0, 0, 0x80, 0, 2]; + warning.extend(model::ints(&[0])); + warning.push(1); + warning.extend_from_slice(b"\r\n001c01 ATTN: Power Down: issue 'pwr d' again to immediately power down.\r\n\0"); + assert_eq!(drain(&l1), encode(&warning)); + l1.power_on(); + l1.receive_console(&[0x14]); + l1.receive_console(b"pwr d\r"); + assert!(l1.node_powered(), "reset clears the first confirmation"); + drain(&l1); + l1.receive_console(&[0x14]); + l1.receive_console(b"power down\r"); + assert!(!l1.node_powered()); + assert_eq!(*actions.lock(), [L1Event::Power(false), L1Event::Shutdown]); +} + +/// Protocol power-down actions must finish before the L1 request reply. +#[test] +fn protocol_and_task_three_power_down_commands_are_immediate() { + let protocol = vec![0x10, 0x0f, 0xff, 0xf0, 6, 0x20, 5, 0]; + let mut task_three = vec![0x10, 0x0f, 0xff, 0xf0, 3, 0, 0, 1, 1]; + task_three.extend_from_slice(b"pwr d\0"); + for (case, packet) in [ + ("protocol command", protocol), + ("task three console", task_three), + ] { + let l1 = L1::new(Default::default()).unwrap(); + let actions = Arc::new(Mutex::new(Vec::new())); + let observed = actions.clone(); + l1.set_event_handler(Arc::new(move |event| observed.lock().push(event))); + send(&l1, &encode(&packet)); + assert_eq!(drain(&l1), integer_reply(&[]), "{case}"); + assert!(!l1.node_powered(), "{case}"); + assert_eq!( + *actions.lock(), + [L1Event::Power(false), L1Event::Shutdown], + "{case}" + ); + } +} + +/// A CPU may read the UART in its interrupt callback while the worker updates the line. +#[test] +fn l1_worker_irq_delivery_stays_ordered_across_reentrant_cpu_reads() { + use std::sync::mpsc; + struct BlockingIrq { + entered: mpsc::SyncSender<()>, + resume: std::sync::Mutex>, + blocked: AtomicBool, + levels: Mutex>, + device: std::sync::Weak, + } + impl crate::irq::InterruptController for BlockingIrq { + fn set_line(&self, _: u32, level: bool) { + if level && !self.blocked.swap(true, Ordering::AcqRel) { + self.entered.send(()).unwrap(); + self.resume.lock().unwrap().recv().unwrap(); + // A machine interrupt handler may inspect the UART immediately. + assert_eq!(self.device.upgrade().unwrap().read64(0x28).data, 0x60); + } + self.levels.lock().push(level); + } + } + let l1 = Arc::new(L1::new(Default::default()).unwrap()); + let backend = Arc::new(crate::dev::serial_host::CiSerialBackend::new()); + let (entered_tx, entered_rx) = mpsc::sync_channel(1); + let (resume_tx, resume_rx) = mpsc::sync_channel(1); + let irq = Arc::new(BlockingIrq { + entered: entered_tx, + resume: std::sync::Mutex::new(resume_rx), + blocked: AtomicBool::new(false), + levels: Mutex::new(Vec::new()), + device: Arc::downgrade(&l1), + }); + l1.set_irq(IrqLine::new(irq.clone(), 0)); + l1.set_serial_backend(backend.clone()); + l1.write64(8, 1); + l1.start(); + backend.push_host(b"x"); + let entered = entered_rx.recv_timeout(std::time::Duration::from_secs(2)); + if entered.is_ok() { + l1.write64(0x10, 2); + } + resume_tx.send(()).unwrap(); + l1.stop(); + entered.unwrap(); + assert_eq!(*irq.levels.lock(), [false, true, false]); +} + +/// Concurrent worker output and platform actions must preserve their order. +#[test] +fn concurrent_l1_outputs_and_actions_are_delivered_in_order() { + use std::sync::mpsc; + struct BlockingBackend { + entered: mpsc::SyncSender<()>, + resume: std::sync::Mutex>, + output: Mutex>, + } + impl SerialBackend for BlockingBackend { + fn send_byte(&self, byte: u8) { + if byte == b'A' { + self.entered.send(()).unwrap(); + self.resume.lock().unwrap().recv().unwrap(); + } + self.output.lock().push(byte); + } + fn recv_byte(&self) -> std::io::Result { + Err(std::io::ErrorKind::WouldBlock.into()) + } + } + let l1 = Arc::new(L1::new(Default::default()).unwrap()); + let (entered_tx, entered_rx) = mpsc::sync_channel(1); + let (resume_tx, resume_rx) = mpsc::sync_channel(1); + let backend = Arc::new(BlockingBackend { + entered: entered_tx, + resume: std::sync::Mutex::new(resume_rx), + output: Mutex::new(Vec::new()), + }); + l1.set_serial_backend(backend.clone()); + let actions = Arc::new(Mutex::new(Vec::new())); + let captured = actions.clone(); + let weak = Arc::downgrade(&l1); + l1.set_event_handler(Arc::new(move |action| { + // Reentrant register reads must not recursively dispatch notifications. + assert_eq!(weak.upgrade().unwrap().read64(0x28).data, 0x60); + captured.lock().push(action); + })); + { + let mut s = l1.state.lock(); + s.output.extend(b"AB"); + s.actions.push(L1Event::Reset); + } + let worker_device = l1.clone(); + let worker = std::thread::spawn(move || worker_device.flush()); + let entered = entered_rx.recv_timeout(std::time::Duration::from_secs(2)); + if entered.is_ok() { + let mut s = l1.state.lock(); + s.output.extend(b"CD"); + s.actions.push(L1Event::Shutdown); + drop(s); + l1.flush(); + } + resume_tx.send(()).unwrap(); + worker.join().unwrap(); + entered.unwrap(); + assert_eq!(*backend.output.lock(), b"ABCD"); + assert_eq!(*actions.lock(), [L1Event::Reset, L1Event::Shutdown]); +} + +/// Request 0xa returns two integers; the first is a PROM display selector. +/// Ported from test_port_speed in tests/qtest/sn1-l1-test.c. +#[test] +fn the_port_speed_reply_carries_the_configured_first_integer() { + let l1 = L1::new(Default::default()).unwrap(); + assert_eq!(request(&l1, 0xa, &[], 0), integer_reply(&[0, 0])); + let l1 = L1::new(L1Config { port_speed_first: 1, ..Default::default() }).unwrap(); + assert_eq!(request(&l1, 0xa, &[], 0), integer_reply(&[1, 0])); + l1.power_on(); + drain(&l1); + assert_eq!(request(&l1, 0xa, &[], 0), integer_reply(&[1, 0])); + assert!(L1::new(L1Config { port_speed_first: 2, ..Default::default() }).is_err()); +} + +const BRICK: [u8; 53] = [ + 0x10, 0, 0, 0, 0, 0x30, 1, 1, 0, 0x2b, 0, 0x2b, 2, 0, 0, 0x81, 0, 0, 0x0f, 0xff, 0xf0, 3, 0x2f, + 0xff, 0xf0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0x0a, 1, 0, 7, 0, 0, 0, 0x34, b'b', b'r', + b'i', b'c', b'k', b' ', 0, +]; + +fn router_l1(type_code: u32, rack: u32, bay: u32, serial: Option<&str>) -> L1 { + L1::new(L1Config { + router: true, + type_code, + rack, + bay, + brick_serial: serial.map(str::to_string), + ..Default::default() + }) + .unwrap() +} + +fn chunk(channel: u8, command: u16, count: u8, length: u16, data: &[u8]) -> Vec { + let mut p = vec![channel, 0, 0, 0, 0, (command >> 8) as u8, command as u8, count]; + p.extend_from_slice(&length.to_be_bytes()); + p.extend_from_slice(data); + p +} + +fn send_packet(l1: &L1, packet: &[u8]) { + send(l1, &encode(packet)); +} + +/// Collect one reply message from its chunks: the channel, and the inner bytes. +fn router_receive(l1: &L1) -> (u8, Vec) { + let wire = drain(l1); + let mut frames = wire.split(|b| *b == 0x7e).filter(|f| !f.is_empty()); + let mut decode = || -> Vec { + let raw = frames.next().expect("a reply frame"); + let mut out = Vec::new(); + let mut it = raw.iter(); + while let Some(&b) = it.next() { + out.push(if b == 0x7d { it.next().unwrap() ^ 0x20 } else { b }); + } + out.truncate(out.len() - 2); + out + }; + let first = decode(); + assert_eq!(first[1..5], [0; 4]); + assert_eq!(first[5..8], [0x30, 1, 1]); + let channel = first[0]; + let mut inner = first[10..].to_vec(); + while inner.len() < usize::from(u16::from_be_bytes([inner[0], inner[1]])) { + let next = decode(); + assert_eq!(next[0], channel); + assert_eq!(next[5..10], [0x30, 3, 0, 0, 0]); + inner.extend_from_slice(&next[10..]); + } + assert!(frames.next().is_none()); + (channel, inner) +} + +fn expect_brick(l1: &L1, channel: u8, destination: u32, source: u32, text: &str) { + let (got, inner) = router_receive(l1); + assert_eq!(got, channel); + assert_eq!(usize::from(u16::from_be_bytes([inner[0], inner[1]])), inner.len()); + assert_eq!(u32::from_be_bytes(inner[8..12].try_into().unwrap()), destination); + assert_eq!(u32::from_be_bytes(inner[12..16].try_into().unwrap()), source); + assert_eq!(&inner[36..], [text.as_bytes(), &[0]].concat()); +} + +const DEFAULT_TEXT: &str = "rack: 001, slot: 04, partition: none, type: Chimera Server \ + [?MB flash], serial:(null), source: OTHER\n\n"; + +/// Ported from test_irouter_brick_exact in tests/qtest/sn1-l1-test.c. +#[test] +fn the_router_brick_reply_matches_the_captured_layout() { + let l1 = router_l1(0x37, 1, 4, None); + send_packet(&l1, &BRICK); + let mut inner = vec![ + 0, 0x8b, 2, 0, 0, 0x81, 1, 0, 0, 0x04, 0x42, 0x15, 0, 0x04, 0x40, 3, 2, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0x0a, 1, 0, 0x67, 0, 0, 0, 0x34, + ]; + inner.extend_from_slice(DEFAULT_TEXT.as_bytes()); + inner.push(0); + assert_eq!(inner.len(), 139); + let mut first = vec![0x30, 0, 0, 0, 0, 0x30, 1, 1, 0, 139]; + first.extend_from_slice(&inner[..113]); + let mut rest = vec![0x30, 0, 0, 0, 0, 0x30, 3, 0, 0, 0]; + rest.extend_from_slice(&inner[113..]); + let mut expected = encode(&first); + expected.extend(encode(&rest)); + assert_eq!(drain(&l1), expected); +} + +/// Subchannels are answered in place, and a request split across chunks +/// completes on the last continuation. Ported from +/// test_irouter_fragment_channel_transport (the single-transport part). +#[test] +fn the_router_reassembles_chunks_and_answers_on_the_request_subchannel() { + let l1 = router_l1(0x37, 1, 4, None); + let rest = &BRICK[10..]; + send_packet(&l1, &chunk(0x10, 0x3001, 1, 43, &rest[..12])); + assert!(drain(&l1).is_empty()); + let mut other = BRICK; + other[0] = 0x11; + send_packet(&l1, &other); + expect_brick(&l1, 0x31, 0x0004_4235, 0x0004_4003, DEFAULT_TEXT); + send_packet(&l1, &chunk(0x10, 0x3003, 0, 0, &rest[12..])); + expect_brick(&l1, 0x30, 0x0004_4215, 0x0004_4003, DEFAULT_TEXT); +} + +/// Ported from test_irouter_reset_and_negative. +#[test] +fn a_reset_drops_a_pending_router_message_and_unsupported_types_stay_silent() { + let l1 = router_l1(0x37, 1, 4, None); + let rest = &BRICK[10..]; + send_packet(&l1, &chunk(0x10, 0x3001, 1, 43, &rest[..12])); + l1.power_on(); + drain(&l1); + send_packet(&l1, &chunk(0x10, 0x3003, 0, 0, &rest[12..])); + assert!(drain(&l1).is_empty()); + send_packet(&l1, &BRICK); + expect_brick(&l1, 0x30, 0x0004_4215, 0x0004_4003, DEFAULT_TEXT); + + let off = L1::new(Default::default()).unwrap(); + send_packet(&off, &BRICK); + assert_eq!(drain(&off), encode(&[0x30, 255, 255, 255, 255, 0, 0, 0, 8])); + + let l1 = router_l1(0x37, 1, 4, None); + send_packet(&l1, &BRICK[..7]); + assert!(drain(&l1).is_empty()); + send_packet(&l1, &chunk(0x10, 0x3001, 1, 43, &rest[..12])); + send_packet(&l1, &BRICK); + expect_brick(&l1, 0x30, 0x0004_4215, 0x0004_4003, DEFAULT_TEXT); + + let l1 = router_l1(0x34, 1, 4, None); + send_packet(&l1, &BRICK); + send_packet(&l1, &chunk(0x10, 0x3003, 0, 0, rest)); + assert!(drain(&l1).is_empty()); + + let l1 = router_l1(0x37, 1, 4, None); + let mut reset = BRICK; + reset[46..52].copy_from_slice(b"reset\0"); + send_packet(&l1, &reset); + assert!(drain(&l1).is_empty()); +} + +/// A new start replaces a pending message. Ported from +/// test_irouter_replace_pending. +#[test] +fn a_new_router_start_replaces_the_pending_message() { + let l1 = router_l1(0x37, 1, 4, None); + let mut old = BRICK; + old[14] = 0x12; + old[27] = 0x34; + send_packet(&l1, &chunk(0x10, 0x3001, 1, 43, &old[10..22])); + send_packet(&l1, &BRICK); + expect_brick(&l1, 0x30, 0x0004_4215, 0x0004_4003, DEFAULT_TEXT); + send_packet(&l1, &chunk(0x10, 0x3003, 0, 0, &old[22..])); + assert!(drain(&l1).is_empty()); +} + +/// Messages of more than one fragment have no known descriptor layout. +/// Ported from test_irouter_multifragment_refusal. +#[test] +fn a_multi_fragment_router_message_is_refused_and_later_requests_work() { + let l1 = router_l1(0x37, 1, 4, None); + let mut request = BRICK; + request[15] = 0x82; + send_packet(&l1, &chunk(0x10, 0x3001, 2, 43, &request[10..])); + assert!(drain(&l1).is_empty()); + send_packet(&l1, &chunk(0x10, 0x3002, 0, 4, &[0xde, 0xad, 0xbe, 0xef])); + assert!(drain(&l1).is_empty()); + request[0] = 0x11; + request[15] = 0x81; + send_packet(&l1, &request); + expect_brick(&l1, 0x31, 0x0004_4235, 0x0004_4003, DEFAULT_TEXT); +} + +/// Ported from test_irouter_identity_overrides, less its partition cases: +/// this L1 is never partitioned. The type 0x39 case is the Tezro identity. +#[test] +fn the_router_reply_reports_rack_slot_type_and_serial() { + let l1 = router_l1(0x39, 2, 5, Some("SERIAL-X")); + send_packet(&l1, &BRICK); + expect_brick( + &l1, + 0x30, + 0x0008_5215, + 0x0008_5003, + "rack: 002, slot: 05, partition: none, type: Chimera WS [?MB flash], \ + serial:SERIAL-X, source: OTHER\n\n", + ); + let l1 = router_l1(0x51, 3, 2, Some("")); + send_packet(&l1, &BRICK); + expect_brick( + &l1, + 0x30, + 0x000c_2215, + 0x000c_2003, + "rack: 003, slot: 02, partition: none, type: Rackmount WS [?MB flash], \ + serial:, source: OTHER\n\n", + ); +} diff --git a/src/dev/mod.rs b/src/dev/mod.rs index 2e4ebf637..c2990d6f6 100644 --- a/src/dev/mod.rs +++ b/src/dev/mod.rs @@ -13,12 +13,17 @@ pub mod mace_audio; pub mod mace_isa; pub mod crime; pub mod battery_clock; +pub mod deadline_timers; pub mod ds1687; +pub mod ds1744; pub mod heart; pub mod heart_mem; pub mod ioc3; +pub mod ioc_serial_dma; pub mod ioc3_eth; +pub mod ioc3_kbc; pub mod ioc3_port; +pub mod ioc4; pub mod isp1040; pub mod mace_id; pub mod mace_pci; @@ -75,7 +80,16 @@ pub mod hub_md; pub mod hub_ni; pub mod hub_pi; pub mod dp83840; +pub mod bedrock; +pub mod bedrock_flash; +pub mod bcm5701; pub mod dp83843; pub mod ics1890; -pub mod timekeeper; +pub mod isp12160; +pub mod l1; +pub mod ohci; +pub mod pic; +pub mod pxbow; pub mod prom_image; +pub mod timekeeper; +pub mod xbridge; diff --git a/src/dev/ns16550.rs b/src/dev/ns16550.rs index 82a756d04..5be6990ed 100644 --- a/src/dev/ns16550.rs +++ b/src/dev/ns16550.rs @@ -117,6 +117,8 @@ pub mod com_reg { /// LSR: a received character is waiting. pub const LSR_DR: u8 = 0x01; pub const LSR_OE: u8 = 0x02; + pub const LSR_PE: u8 = 0x04; + pub const LSR_FE: u8 = 0x08; pub const LSR_BI: u8 = 0x10; /// LSR: transmit holding register empty. pub const LSR_THRE: u8 = 0x20; @@ -261,6 +263,19 @@ impl Ns16550 { self.lcr & LCR_DLAB != 0 } + /// How long a character takes on the line at the programmed rate and + /// frame. Nothing here waits for it; a DMA engine in front of the port + /// may. + pub fn char_time(&self) -> Duration { + self.char_transmit_time + } + + /// The holding register is empty, so a DMA engine may write a byte. + /// Transmit is instant, so it always is once the write returns. + pub fn tx_ready(&self) -> bool { + self.lsr & LSR_THRE != 0 + } + pub fn loopback(&self) -> bool { self.mcr & MCR_LOOP != 0 } diff --git a/src/dev/odyssey/board.rs b/src/dev/odyssey/board.rs index 13fc458c8..f91a42619 100644 --- a/src/dev/odyssey/board.rs +++ b/src/dev/odyssey/board.rs @@ -1,5 +1,5 @@ //! The Odyssey (VPro) board's constants and state. The interrupt packet path -//! is the `irq` field of the host interface (`HostInterface`, interface.rs). +//! is the `packets` field of the host interface (`HostInterface`, interface.rs). //! //! Ported from include/hw/misc/sgi-odyssey.h in jamesbraid/qemu-sgi-origami. diff --git a/src/dev/odyssey/debug.rs b/src/dev/odyssey/debug.rs index ce8b6f677..bb40c2d07 100644 --- a/src/dev/odyssey/debug.rs +++ b/src/dev/odyssey/debug.rs @@ -463,7 +463,7 @@ impl Odyssey { f.route_dest_upper, f.route_dest_lower, f.route_vectors, (f.route_vectors >> 24) & 0xff, (f.route_vectors >> 16) & 0xff, (f.route_vectors >> 8) & 0xff, f.route_vectors & 0xff, f.routed()); let _ = writeln!(out, " swap arm {:#x} pending {:#x} control {:#x} companion {:#x} set {:#x} clear {:#x} displayed {:#010x}{}", f.swap_arm, f.swap_pending, f.swap_control, f.swap_control_companion, f.swap_set, f.swap_clear, f.displayed_buffer, if f.displayed_written { " (written, repaint pending)" } else { "" }); - let _ = writeln!(out, " sw token {:#x} flow-control target {:#x} heart widget {}", f.sw_token, f.flctl_target, f.heart_widget); + let _ = writeln!(out, " sw token {:#x} flow-control target {:#x}", f.sw_token, f.flctl_target); } let g = self.quiesced()?; let wg = &g.widget; diff --git a/src/dev/odyssey/interface.rs b/src/dev/odyssey/interface.rs index 49727c3a8..26fe1557a 100644 --- a/src/dev/odyssey/interface.rs +++ b/src/dev/odyssey/interface.rs @@ -29,14 +29,13 @@ use super::display_fifo::display_fifo_access; use super::fifo::CommandFifo; use super::mmio::{access_waits, command_store, fifo_port}; use super::registers::reg; +use super::PacketSender; use parking_lot::Mutex; use std::sync::Arc; pub(super) struct HostInterface { - /// Where interrupt packets go (the iris glue's HEART). - pub irq: Option>, - /// The widget number of HEART, the packets' destination. - pub heart_widget: u32, + /// Where interrupt packets go: the machine's fabric. + pub packets: Option, /// ODSY_IDENTITY, composed from the revision at reset. pub identity: u32, @@ -88,8 +87,7 @@ pub(super) struct HostInterface { impl Default for HostInterface { fn default() -> Self { HostInterface { - irq: None, - heart_widget: 8, + packets: None, identity: 0, widget_control: 0, xt_req_timeout: 0, @@ -498,18 +496,28 @@ pub(super) fn sgi_xio_compose(who: &str, widget: u32, address: u64, packet: &mut /// HEART's interrupt-packet register in its widget space. const HEART_PACKET: u64 = 0x80; +/// The Octane's packet sender: HEART takes a packet at 0x80 of widget 8 and +/// drives the line the vector names. +pub(super) fn heart_sender(irq: Arc) -> PacketSender { + Arc::new(move |widget, address, vector, level| { + if widget != crate::ip30::WIDGET_HEART || address != HEART_PACKET { + return false; + } + irq.set_line(vector as u32, level); + true + }) +} + impl HostInterface { - /// The interrupt-modify write to the interrupt target. HEART takes it at - /// 0x80 of widget 8. + /// The interrupt-modify write to the interrupt target: the packet goes to + /// the machine's sender, which routes it by widget and address. pub fn send_interrupt(&mut self, packet: u64, number: u8, set: bool) { let widget = (packet >> XIO_PACKET_ADDRESS_BITS) as u32; let address = packet & ((1u64 << XIO_PACKET_ADDRESS_BITS) - 1); - if widget != self.heart_widget || address != HEART_PACKET { - crate::dlog!(LogModule::Odyssey, "odyssey: interrupt packet to widget {widget} address {address:#x} has no receiver"); - return; - } - if let Some(irq) = &self.irq { - irq.set_line(number as u32, set); + if let Some(send) = &self.packets { + if !send(widget as u8, address, number, set) { + crate::dlog!(LogModule::Odyssey, "odyssey: interrupt packet to widget {widget} address {address:#x} has no receiver"); + } } } } diff --git a/src/dev/odyssey/mod.rs b/src/dev/odyssey/mod.rs index e5cad6423..6d036f9c6 100644 --- a/src/dev/odyssey/mod.rs +++ b/src/dev/odyssey/mod.rs @@ -127,19 +127,34 @@ use scanout::monotonic_ns; /// The VPro board on widget `slot`: a `BusDevice` for the widget's /// windows. pub struct Odyssey { - small: u64, - large: u64, + aperture: Aperture, state: BoardLocks, display: Arc, running: Arc, thread: Mutex>>, } +/// Carries a board interrupt packet into a machine's crosstalk fabric: the +/// destination widget, the address within it, the vector, and whether the +/// source rose or fell. False when nothing at the destination takes it. +pub type PacketSender = Arc bool + Send + Sync>; + +/// How the processor's addresses reach the board, chosen when it is built. +#[derive(Clone, Copy)] +enum Aperture { + /// The Octane's: the widget's small and large IP30 windows. + Ip30Windows { small: u64, large: u64 }, + /// Every address is already an aperture offset, as a machine whose I/O + /// interface hands widgets offsets (Bedrock) presents it. + Offsets, +} + impl Odyssey { - /// A board of `product` (v12, v10, v8, v6) at `widget`: its interrupt - /// packets reach `irq` (HEART), its DMA reaches `mem`. `pool` is the - /// CPU raster unit's workers and tile size, `raster` which unit draws - /// and, for the GPU, the wgpu backend (`[graphics]`). + /// A board of `product` (v12, v10, v8, v6) at `widget` on the Octane: + /// the processor reaches it through the IP30 widget windows, its + /// interrupt packets reach `irq` (HEART), its DMA reaches `mem`. `pool` + /// is the CPU raster unit's workers and tile size, `raster` which unit + /// draws and, for the GPU, the wgpu backend (`[graphics]`). #[allow(clippy::too_many_arguments, reason = "the board's wiring")] pub fn new( widget: u8, @@ -149,6 +164,37 @@ impl Odyssey { status: Arc, pool: (usize, u32), raster: (crate::config::RasterUnit, &str), + ) -> Result, String> { + let aperture = Aperture::Ip30Windows { small: widget_small_base(widget), large: widget_large_base(widget) }; + Self::build(widget, product, interface::heart_sender(irq), aperture, mem, status, pool, raster) + } + + /// A board of `product` at `widget` on a machine that hands the board + /// widget-relative addresses, as Bedrock's I/O interface does, and + /// carries its interrupt packets through `packets`. + #[allow(clippy::too_many_arguments, reason = "the board's wiring")] + pub fn new_widget_relative( + widget: u8, + product: &str, + packets: PacketSender, + mem: Arc, + status: Arc, + pool: (usize, u32), + raster: (crate::config::RasterUnit, &str), + ) -> Result, String> { + Self::build(widget, product, packets, Aperture::Offsets, mem, status, pool, raster) + } + + #[allow(clippy::too_many_arguments, reason = "the board's wiring")] + fn build( + widget: u8, + product: &str, + packets: PacketSender, + aperture: Aperture, + mem: Arc, + status: Arc, + pool: (usize, u32), + raster: (crate::config::RasterUnit, &str), ) -> Result, String> { let mut s = Board::new(product, widget as u32)?; // The fragment pipeline's workers and tile size (`[graphics]`). @@ -168,15 +214,14 @@ impl Odyssey { }); let mem: Arc = Arc::new(record::RecordingMem { inner: mem, rec: rec.clone() }); s.guest = Some(mem); - s.front.lock().irq = Some(irq); + s.front.lock().packets = Some(packets); if let Ok(path) = std::env::var("IRIS_ODSY_TRACE") { s.trace.start(&path, trace::TRACE_ALL).map_err(|e| format!("IRIS_ODSY_TRACE {path}: {e}"))?; } let mut state = BoardLocks::new(s); state.rec = rec; Ok(Arc::new(Self { - small: widget_small_base(widget), - large: widget_large_base(widget), + aperture, state, display: Arc::new(Display::new(status)), running: Arc::new(AtomicBool::new(false)), @@ -192,7 +237,10 @@ impl Odyssey { /// The aperture offset of `addr`, in either window. fn offset(&self, addr: u64) -> Option { - let off = if addr >= self.large { addr - self.large } else { addr.wrapping_sub(self.small) }; + let off = match self.aperture { + Aperture::Ip30Windows { small, large } => if addr >= large { addr - large } else { addr.wrapping_sub(small) }, + Aperture::Offsets => addr, + }; (off < ODSY_APERTURE).then_some(off) } diff --git a/src/dev/odyssey/tests/interrupts.rs b/src/dev/odyssey/tests/interrupts.rs index 6bbb67ca9..8f89604cf 100644 --- a/src/dev/odyssey/tests/interrupts.rs +++ b/src/dev/odyssey/tests/interrupts.rs @@ -437,3 +437,26 @@ fn a_held_flag_is_sent_with_the_source_enables_clear() { reg_write(&mut s, route::DEST_UPPER, HEART_UPPER); assert_eq!(rec.events(FLAG_VECTOR), [true]); } + +/// A board built for a machine whose fabric carries packets to any widget +/// hands them to that machine's sender, to the widget the routing names, +/// and never to HEART's line: the Fuel's widget 13 reaches Bedrock this way. +#[test] +fn a_packet_sender_takes_the_interrupt_to_the_routed_widget() { + let (mut s, rec) = board("v12"); + let seen = Arc::new(Mutex::new(Vec::new())); + let log = seen.clone(); + s.front.lock().packets = Some(Arc::new(move |widget, address, vector, level| { + log.lock().push((widget, address, vector, level)); + true + })); + s.mmio_write(ROUTE_DEST_UPPER, 1 << 25, 4); + s.mmio_write(ROUTE_DEST_LOWER, 0x1230, 4); + s.mmio_write(ROUTE_VECTORS, TEST_VECTOR as u64, 4); + s.mmio_write(INTR_ENABLE_SET, INTR_FIFO_LOW_WATER, 4); + assert_eq!(*seen.lock(), vec![(1, 0x1230, TEST_VECTOR as u8, true)]); + assert!(rec.silent(), "the packet did not go to HEART"); + + s.mmio_write(INTR_CLEAR, INTR_FIFO_LOW_WATER, 4); + assert_eq!(seen.lock().last(), Some(&(1, 0x1230, TEST_VECTOR as u8, false))); +} diff --git a/src/dev/odyssey/tests/mod.rs b/src/dev/odyssey/tests/mod.rs index ff2f655b9..96ea31208 100644 --- a/src/dev/odyssey/tests/mod.rs +++ b/src/dev/odyssey/tests/mod.rs @@ -88,7 +88,7 @@ pub(super) fn board(product: &str) -> (Box, Arc) { let rec = Arc::new(Recorder::default()); let mut s = Board::new(product, VPRO_WIDGET).expect("board"); s.test_screen(); - s.front.lock().irq = Some(rec.clone()); + s.front.lock().packets = Some(super::interface::heart_sender(rec.clone())); (s, rec) } diff --git a/src/dev/odyssey/tests/monitor.rs b/src/dev/odyssey/tests/monitor.rs index 01c33271c..9fdbae56f 100644 --- a/src/dev/odyssey/tests/monitor.rs +++ b/src/dev/odyssey/tests/monitor.rs @@ -27,6 +27,12 @@ fn board() -> Arc { v } +/// The base of the board's large IP30 window, where a processor's stores +/// reach its aperture. +fn large() -> u64 { + crate::ip30::widget_large_base(WIDGET) +} + /// The monitor's writer: what a command wrote, read back after it. #[derive(Clone, Default)] struct Sink(Arc>>); @@ -289,9 +295,9 @@ fn rec_starts_marks_and_stops_a_recording_from_the_monitor() { // Two command stores and a front-register load through the bus, as a // processor makes them; then a pixel lands and a mark takes it. - assert_eq!(v.write32(v.large + cmd::FIFO, cmd::REGISTER_TAG | reg::STROBE), crate::traits::BUS_OK); - assert_eq!(v.write32(v.large + cmd::FIFO, cmd::REGISTER_TAG | reg::STROBE), crate::traits::BUS_OK); - let status0 = v.read32(v.large + ODSY_STATUS0); + assert_eq!(v.write32(large() + cmd::FIFO, cmd::REGISTER_TAG | reg::STROBE), crate::traits::BUS_OK); + assert_eq!(v.write32(large() + cmd::FIFO, cmd::REGISTER_TAG | reg::STROBE), crate::traits::BUS_OK); + let status0 = v.read32(large() + ODSY_STATUS0); assert!(status0.is_ok()); pixel_write(v.state.test_board(), 3, 4, 0x1234_5600); let h1 = memory_hash_bytes(v.state.test_board()); @@ -353,12 +359,12 @@ fn trace_needs_gputrace_and_then_writes_the_classes_asked_for() { // A GL begin and three integer vertices through the kernel FIFO port: the // words are FIFO lines, the third vertex completes a triangle (a DRAW // line); three identical status polls are one CPU line. - let fifo = v.large + cmd::FIFO; + let fifo = large() + cmd::FIFO; for word in [cmd::REGISTER_TAG | reg::PATTERN_MODE, 1 << 13, cmd::REGISTER_TAG | reg::GL_BEGIN, gl_cmd::TRIANGLES, gl_cmd::VERTEX_2I, 10, 20, gl_cmd::VERTEX_2I, 30, 20, gl_cmd::VERTEX_2I, 10, 40] { assert_eq!(v.write32(fifo, word), crate::traits::BUS_OK); } for _ in 0..3 { - assert!(v.read32(v.large + ODSY_STATUS0).is_ok()); + assert!(v.read32(large() + ODSY_STATUS0).is_ok()); } assert_eq!(ok(&v, "trace flush").trim(), "trace flushed"); let text = std::fs::read_to_string(&path).unwrap(); diff --git a/src/dev/ohci.rs b/src/dev/ohci.rs new file mode 100644 index 000000000..214b5d3d3 --- /dev/null +++ b/src/dev/ohci.rs @@ -0,0 +1,613 @@ +// Derived from MAME's xbox_usb.cpp and xbox_usb.h (the OHCI host controller), +// BSD-3-Clause, copyright Samuele Zannoli. + +//! The Lucent USS-312 USB controller: two root ports, OHCI registers and +//! descriptor processing through the PCI bridge's DMA interface. +//! +//! The Fuel's root ports are empty, as on a machine with nothing plugged in. +//! The tests plug a function into a port with `attach` and drive the +//! descriptor and packet scheduling engine through it. Isochronous +//! traffic, suspend/resume, external hubs, USB storage and legacy keyboard/SMI +//! routing are unsupported. Scheduling uses synthetic work units rather than +//! wire time. Register behavior follows OpenHCI 1.0a and the USS-312 revision 5 +//! data sheet. +//! +//! Ported from src/mame/shared/xbox_usb.cpp and xbox_usb.h in MAME, +//! revision 2ee9b1e30299b122b131cab0bd7912abc2b095ba. + +use crate::dev::heart::{host_clock, Clock}; +use crate::irq::IrqLine; +use crate::pci::{Bar, PciConfig, PciConfigPolicy, PciDma, PciFunction, PciFunctionState, PCI_INTA}; +use crate::traits::{Device, Resettable}; +use parking_lot::{Mutex, MutexGuard}; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; +use std::sync::{Arc, OnceLock}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +#[repr(u32)] +pub enum Pid { + Setup = 0, + Out = 1, + In = 2, +} +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct UsbPacket { + pub endpoint: u8, + pub pid: Pid, + pub toggle: bool, + /// IN requests name capacity; OUT and SETUP carry their bytes. + pub capacity: usize, + pub data: Vec, + pub time_ms: u64, +} +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum UsbOutcome { + Ack(Vec), + Nak, + Stall, + NoResponse, + CrcError, + BitStuffError, +} +pub trait UsbFunction: Send + Sync { + fn reset(&self); + fn address(&self) -> u8; + fn low_speed(&self) -> bool { + false + } + fn transact(&self, packet: UsbPacket) -> UsbOutcome; +} + +mod engine; +mod schedule; + +const EVENTS: u32 = 0x4000_007f; +const WDH: u32 = 2; +const UE: u32 = 16; +const RHSC: u32 = 64; +const PPS: u32 = 0x100; +const CHANGES: u32 = 0x1f0000; +/// The state lock as a frame holds it: it lets go around every DMA transfer +/// and USB transaction, and takes the lock back after. +type Guard<'a> = MutexGuard<'a, State>; +struct State { + r: [u32; 23], + ports: [Option>; 2], + low_speed: [bool; 2], + reset_ms: [u8; 2], + done_delay: u8, + now_ms: u64, + active_fi: u32, + bit_credit: u64, +} +impl State { + fn new() -> Self { + let mut r = [0; 23]; + r[0] = 0x110; + r[13] = 0x2edf; + r[17] = 0x628; + r[18] = 0x1000_0902; + r[19] = 0x60000; + Self { + r, + ports: [None, None], + low_speed: [false; 2], + reset_ms: [0; 2], + done_delay: 7, + now_ms: 0, + active_fi: 0x2edf, + bit_credit: 0, + } + } + fn soft_reset(&mut self) { + let control = (self.r[1] & 0x300) | 0xc0; + let keep = [self.r[18], self.r[19], self.r[20], self.r[21], self.r[22]]; + let ports = self.ports.clone(); + let low_speed = self.low_speed; + let resets = self.reset_ms; + let time = self.now_ms; + *self = Self::new(); + self.r[1] = control; + self.r[18..23].copy_from_slice(&keep); + self.ports = ports; + self.low_speed = low_speed; + self.reset_ms = resets; + self.now_ms = time; + } + fn irq(&self) -> bool { + self.r[1] & 0x100 == 0 && self.r[4] & 0x80000000 != 0 && self.r[3] & self.r[4] & EVENTS != 0 + } + fn powered(&self, p: usize) -> bool { + self.r[18] & 0x200 != 0 || self.r[21 + p] & PPS != 0 + } + fn individual(&self, p: usize) -> bool { + self.r[18] & 0x100 != 0 && self.r[19] & (1 << (17 + p)) != 0 + } + fn power(&mut self, p: usize, on: bool) { + if self.r[18] & 0x200 != 0 || on { + self.r[21 + p] |= PPS; + if self.ports[p].is_some() && self.r[21 + p] & 1 == 0 { + self.r[21 + p] |= 1 | 0x10000 | if self.low_speed[p] { 0x200 } else { 0 }; + self.r[3] |= RHSC; + } + } else { + let previous = self.r[21 + p]; + self.r[21 + p] &= !(PPS | 0x17); + if previous & 1 != 0 { + self.r[21 + p] |= 0x10000; + } + if previous & 2 != 0 { + self.r[21 + p] |= 0x20000; + } + if previous & 3 != 0 { + self.r[3] |= RHSC; + } + self.reset_ms[p] = 0; + } + } +} + +pub struct Ohci { + pci: PciFunctionState, + state: Mutex, + running: AtomicBool, + /// Source of the frame count: the machine's own clock, so the guest's + /// idea of a millisecond and the controller's agree. + time: OnceLock, + /// `time` reading at the last frame boundary counted; `None` while stopped. + clock: Mutex>, + /// `time` reading of the next frame boundary (`u64::MAX` while stopped), + /// so a caller with no boundary to deliver takes no lock. + due: AtomicU64, + /// Frames counted from the clock that no thread has run yet. + owed: AtomicU64, + /// A thread is running frames. Frames run one at a time, in order, on + /// whichever thread finds them owed and this clear; the others leave + /// theirs to it. + busy: AtomicBool, +} +impl Ohci { + pub fn new() -> Arc { + let config = PciConfig::new_with_policy( + 0x11c1, + 0x5802, + 0x0c0310, + 0x10, + [ + Bar::Mem(0x1000), + Bar::Unused, + Bar::Unused, + Bar::Unused, + Bar::Unused, + Bar::Unused, + ], + PCI_INTA, + PciConfigPolicy { + status: 0x0200, + status_w1c_mask: 0x8000, + command_mask: 0x0006, + latency_mask: 0, + cache_line_mask: 0, + registers: &[], + }, + ); + config.write(11, 0xffff_ffff); + Arc::new(Self { + pci: PciFunctionState::new(config), + state: Mutex::new(State::new()), + running: AtomicBool::new(false), + time: OnceLock::new(), + clock: Mutex::new(None), + due: AtomicU64::new(u64::MAX), + owed: AtomicU64::new(0), + busy: AtomicBool::new(false), + }) + } + /// Use `clock` for frame timing. Call before `start`. + pub fn set_clock(&self, clock: Clock) { + assert!( + !self.running.load(Ordering::Acquire) && self.time.set(clock).is_ok(), + "connect the OHCI clock once, before start" + ); + } + fn now(&self) -> u64 { + (self.time.get_or_init(host_clock))() + } + /// Deliver every frame boundary the clock has passed. Every register + /// access calls this, so a read sees the frame count the guest's own + /// clock implies, however late the machine's poll ran. The clock lock + /// covers the count alone; the frames run with no lock held. + pub fn sync_clock(&self) { + let now = self.now(); + if now < self.due.load(Ordering::Acquire) { + return; + } + let elapsed = { + let mut clock = self.clock.lock(); + let Some(at) = clock.as_mut() else { return }; + let elapsed = now.saturating_sub(*at) / 1_000_000; + *at += elapsed * 1_000_000; + self.due.store(*at + 1_000_000, Ordering::Release); + elapsed + }; + self.advance_frames(elapsed); + } + fn drive_irq(&self) { + loop { + let level = self.state.lock().irq(); + if let Some(line) = self.pci.irq() { + line.set(level); + } + // A callback may acknowledge the interrupt or another writer may + // change it while delivery is outside the state lock. + if self.state.lock().irq() == level { + break; + } + } + } + #[cfg(test)] + pub fn attach(&self, port: usize, function: Arc) -> Result<(), String> { + if port >= 2 { + return Err("OHCI has two ports".into()); + } + if self.state.lock().ports[port].is_some() { + return Err("OHCI port occupied".into()); + } + let low_speed = function.low_speed(); + function.reset(); + { + let mut s = self.state.lock(); + if s.ports[port].is_some() { + return Err("OHCI port occupied".into()); + } + if s.powered(port) { + s.r[21 + port] |= 1 | 0x10000; + } + s.low_speed[port] = low_speed; + if low_speed { + s.r[21 + port] |= 0x200; + } + s.ports[port] = Some(function); + s.r[3] |= RHSC; + } + self.drive_irq(); + Ok(()) + } + #[cfg(test)] + pub fn detach(&self, port: usize) -> Result<(), String> { + if port >= 2 { + return Err("OHCI has two ports".into()); + } + { + let mut s = self.state.lock(); + let enabled = s.r[21 + port] & 2; + s.ports[port] = None; + s.low_speed[port] = false; + s.reset_ms[port] = 0; + s.r[21 + port] = + (s.r[21 + port] & PPS) | 0x10000 | if enabled != 0 { 0x20000 } else { 0 }; + s.r[3] |= RHSC; + } + self.drive_irq(); + Ok(()) + } + /// Deterministic millisecond ticks; `sync_clock` delivers the same ticks from the clock. + pub fn advance_frames(&self, count: u64) { + if count != 0 { + self.owed.fetch_add(count, Ordering::AcqRel); + } + // Whoever wins `busy` runs every frame owed, including ones another + // thread adds meanwhile. Checking `owed` again after letting go of + // `busy` catches frames added just as the runner finished. + while self.owed.load(Ordering::Acquire) != 0 && !self.busy.swap(true, Ordering::Acquire) { + loop { + let n = self.owed.swap(0, Ordering::AcqRel); + if n == 0 { + break; + } + for _ in 0..n { + if !self.running.load(Ordering::Acquire) { + break; + } + self.frame(); + self.drive_irq(); + } + } + self.busy.store(false, Ordering::Release); + } + } + fn frame(&self) { + let s = &mut self.state.lock(); + s.now_ms = s.now_ms.wrapping_add(1); + for p in 0..2 { + if s.reset_ms[p] != 0 { + s.reset_ms[p] -= 1; + if s.reset_ms[p] == 0 { + s.r[21 + p] = (s.r[21 + p] & !16) | 2 | (1 << 20); + s.r[3] |= RHSC; + } + } + } + if s.r[1] & 0xc0 != 0x80 || s.r[3] & UE != 0 { + return; + } + s.bit_credit += 12000; + while s.bit_credit >= u64::from((s.active_fi & 0x3fff) + 1) { + s.bit_credit -= u64::from((s.active_fi & 0x3fff) + 1); + s.active_fi = s.r[13]; + if self.schedule(s).is_err() { + s.r[3] |= UE; + s.bit_credit = 0; + self.pci.config().set_status_bits(0x8000); + break; + } + } + } +} +impl PciFunction for Ohci { + fn state(&self) -> &PciFunctionState { + &self.pci + } + fn attached(&self) { + self.drive_irq(); + } + fn reset(&self) { + Resettable::power_on(self); + } + fn bar_read(&self, bar: usize, off: u32, width: usize) -> u32 { + if bar != 0 || width != 4 || off & 3 != 0 || off > 0x58 { + crate::dlog!( + crate::devlog::LogModule::Usb, + "ohci unsupported read at register offset {off:#x}, width {width}" + ); + return 0; + } + self.sync_clock(); + let s = self.state.lock(); + let i = (off / 4) as usize; + if i == 5 { + s.r[4] + } else if i == 21 || i == 22 { + s.r[i] | if s.r[18] & 0x200 != 0 { PPS } else { 0 } + } else { + s.r[i] + } + } + fn bar_write(&self, bar: usize, off: u32, val: u32, width: usize) { + if bar != 0 || width != 4 || off & 3 != 0 || off > 0x58 { + crate::dlog!( + crate::devlog::LogModule::Usb, + "ohci unsupported write at register offset {off:#x}, width {width}" + ); + return; + } + self.sync_clock(); + // Only a control write can restart the frame phase. The clock lock is + // taken before the state lock; nothing takes them the other way round. + let mut anchor = (off == 4).then(|| self.clock.lock()); + let mut reset = None; + { + let mut s = self.state.lock(); + match off / 4 { + 1 => { + let old = s.r[1]; + s.r[1] = val & 0x7ff; + if val & 0xc0 == 0x80 && old & 0xc0 != 0x80 { + // OpenHCI 1.0a 6.2.1, 7.3.2, 7.3.3: entering + // UsbOperational loads HcFmRemaining from the frame + // interval and increments HcFmNumber, with no SOF at + // the load; the first follows one frame later. The + // free-running clock's phase therefore restarts here. + s.bit_credit = 0; + s.active_fi = s.r[13]; + s.r[14] = (s.r[13] & 0x8000_0000) | (s.r[13] & 0x3fff); + let old_frame = s.r[15] as u16; + let frame = old_frame.wrapping_add(1); + s.r[15] = u32::from(frame); + if (old_frame ^ frame) & 0x8000 != 0 { + s.r[3] |= 32; + } + if let Some(Some(at)) = anchor.as_deref_mut() { + *at = self.now(); + self.due.store(*at + 1_000_000, Ordering::Release); + } + } + if val & 0xc0 == 0 && old & 0xc0 != 0 { + s.r[18] = 0x10000902; + for p in 0..2 { + s.power(p, false); + } + } + if val & 0xc0 == 0x40 || val & 0xc0 == 0xc0 { + crate::dlog!( + crate::devlog::LogModule::Usb, + "ohci suspend/resume timing unsupported at control register offset 0x04" + ); + } + } + 2 => { + if val & 1 != 0 { + s.soft_reset(); + } else { + s.r[2] |= val & 6; + if val & 8 != 0 { + s.r[3] |= 1 << 30; + } + } + } + 3 => s.r[3] &= !(val & EVENTS), + 4 => s.r[4] |= val & (EVENTS | 0x80000000), + 5 => s.r[4] &= !(val & (EVENTS | 0x80000000)), + 6 => s.r[6] = val & !255, + 7..=11 => s.r[(off / 4) as usize] = val & !15, + 13 => s.r[13] = val & 0xffff3fff, + 16 => s.r[16] = val & 0x3fff, + 17 => s.r[17] = val & 0xfff, + 18 => { + s.r[18] = (s.r[18] & !0xff001b00) | (val & 0xff001b00); + if val & 0x200 != 0 { + for p in 0..2 { + s.power(p, true); + } + } + } + 19 => s.r[19] = val & 0x60006, + 20 => { + if val & (1 << 15) != 0 { + s.r[20] |= 1 << 15; + } + if val & (1 << 31) != 0 { + s.r[20] &= !(1 << 15); + } + if val & (1 << 17) != 0 { + s.r[20] &= !(1 << 17); + } + if s.r[18] & 0x200 == 0 { + for p in 0..2 { + if !s.individual(p) { + if val & 1 != 0 { + s.power(p, false); + } + if val & (1 << 16) != 0 { + s.power(p, true); + } + } + } + } + } + 21 | 22 => { + let p = (off / 4 - 21) as usize; + let i = 21 + p; + s.r[i] &= !(val & CHANGES); + if val & 1 != 0 { + s.r[i] &= !2; + } + if val & 2 != 0 { + if s.r[i] & 1 != 0 && s.powered(p) { + s.r[i] |= 2; + } else { + s.r[i] |= 1 << 16; + } + } + if val & 12 != 0 { + crate::dlog!( + crate::devlog::LogModule::Usb, + "ohci port suspend/resume unsupported at port register offset {:#x}", + 0x54 + p * 4 + ); + } + if val & 16 != 0 { + if s.r[i] & 1 != 0 && s.powered(p) { + s.r[i] |= 16; + s.r[i] &= !2; + s.reset_ms[p] = 10; + reset = s.ports[p].clone(); + } else { + s.r[i] |= 1 << 16; + } + } + if s.individual(p) && s.r[18] & 0x200 == 0 { + if val & PPS != 0 { + s.power(p, true); + } + if val & 0x200 != 0 { + s.power(p, false); + } + } + if s.r[i] & CHANGES != 0 { + s.r[3] |= RHSC; + } + } + _ => { + crate::dlog!( + crate::devlog::LogModule::Usb, + "ohci write to read-only register offset {off:#x}" + ); + } + } + } + drop(anchor); + if let Some(f) = reset { + f.reset(); + } + self.drive_irq(); + } +} +impl Resettable for Ohci { + fn power_on(&self) { + let ports = { + let mut s = self.state.lock(); + let ports = s.ports.clone(); + let low_speed = s.low_speed; + *s = State::new(); + s.ports = ports.clone(); + s.low_speed = low_speed; + ports + }; + for f in ports.into_iter().flatten() { + f.reset(); + } + self.drive_irq(); + } +} +impl Device for Ohci { + fn start(&self) { + if self.running.swap(true, Ordering::AcqRel) { + return; + } + let now = self.now(); + *self.clock.lock() = Some(now); + self.due.store(now + 1_000_000, Ordering::Release); + } + fn stop(&self) { + self.sync_clock(); + self.running.store(false, Ordering::Release); + *self.clock.lock() = None; + self.due.store(u64::MAX, Ordering::Release); + // A frame another thread is running finishes before stop returns. + while self.busy.load(Ordering::Acquire) { + std::hint::spin_loop(); + } + } + fn is_running(&self) -> bool { + self.running.load(Ordering::Acquire) + } + fn step(&self, _: u64) {} + fn get_clock(&self) -> u64 { + 0 + } +} + +#[cfg(test)] +mod register_tests { + use super::*; + use crate::pci::PciFunction; + use crate::traits::Resettable; + + #[test] + fn lucent_identity_and_reset_contract() { + let hc = Ohci::new(); + assert_eq!(hc.state().config().read(0), 0x5802_11c1); + assert_eq!(hc.state().config().read(2), 0x0c03_1010); + assert_eq!(hc.state().config().read(11), 0xffff_ffff); + assert_eq!(hc.state().config().read(1), 0x0200_0000); + assert_eq!(hc.bar_read(0, 0, 4), 0x110); + assert_eq!(hc.bar_read(0, 0x10, 4), 0); + assert_eq!(hc.bar_read(0, 0x34, 4), 0x2edf); + assert_eq!(hc.bar_read(0, 0x48, 4), 0x1000_0902); + assert_eq!(hc.bar_read(0, 0x4c, 4), 0x0006_0000); + for p in [0x54, 0x58] { + assert_eq!(hc.bar_read(0, p, 4), 0); + } + hc.bar_write(0, 0x10, 0xffff_ffff, 4); + hc.bar_write(0, 0x08, 1, 4); + assert_eq!(hc.bar_read(0, 0x10, 4), 0); + assert_eq!(hc.bar_read(0, 0x04, 4) & 0xc0, 0xc0); + hc.power_on(); + assert_eq!(hc.bar_read(0, 0x04, 4), 0); + } +} + +#[cfg(test)] +#[path = "ohci/tests.rs"] +mod tests; diff --git a/src/dev/ohci/engine.rs b/src/dev/ohci/engine.rs new file mode 100644 index 000000000..0e25ef6c3 --- /dev/null +++ b/src/dev/ohci/engine.rs @@ -0,0 +1,227 @@ +//! General TD packet processing; guest DMA byte slices are the endianness edge. +use super::*; + +fn read_words(dma: &dyn PciDma, at: u32) -> Result<[u32; 4], ()> { + let mut bytes = [0; 16]; + if !dma.dma_read(at as u64, &mut bytes).is_ok() { + return Err(()); + } + Ok(std::array::from_fn(|i| { + u32::from_le_bytes(bytes[i * 4..i * 4 + 4].try_into().unwrap()) + })) +} +fn write_words(dma: &dyn PciDma, at: u32, words: &[u32]) -> Result<(), ()> { + let mut bytes = [0; 16]; + for (b, w) in bytes.chunks_exact_mut(4).zip(words) { + b.copy_from_slice(&w.to_le_bytes()); + } + if dma.dma_write(at as u64, &bytes[..words.len() * 4]).is_ok() { + Ok(()) + } else { + Err(()) + } +} +fn remaining(cbp: u32, be: u32) -> Result { + if cbp == 0 { + return Ok(0); + } + if cbp & !0xfff == be & !0xfff { + be.checked_sub(cbp).map(|n| n as usize + 1).ok_or(()) + } else { + Ok(0x1000 - (cbp as usize & 0xfff) + (be as usize & 0xfff) + 1) + } +} +fn advance(cbp: u32, be: u32, n: usize) -> u32 { + let first = 0x1000 - (cbp as usize & 0xfff); + if n < first { + cbp.wrapping_add(n as u32) + } else { + (be & !0xfff).wrapping_add((n - first) as u32) + } +} +fn payload_read(dma: &dyn PciDma, cbp: u32, be: u32, len: usize) -> Result, ()> { + let mut bytes = vec![0; len]; + let first = len.min(0x1000 - (cbp as usize & 0xfff)); + if first != 0 && !dma.dma_read(cbp as u64, &mut bytes[..first]).is_ok() { + return Err(()); + } + if len > first + && !dma + .dma_read((be & !0xfff) as u64, &mut bytes[first..]) + .is_ok() + { + return Err(()); + } + Ok(bytes) +} +fn payload_write(dma: &dyn PciDma, cbp: u32, be: u32, bytes: &[u8]) -> Result<(), ()> { + let first = bytes.len().min(0x1000 - (cbp as usize & 0xfff)); + if first != 0 && !dma.dma_write(cbp as u64, &bytes[..first]).is_ok() { + return Err(()); + } + if bytes.len() > first && !dma.dma_write((be & !0xfff) as u64, &bytes[first..]).is_ok() { + return Err(()); + } + Ok(()) +} +impl Ohci { + /// Returns the next ED and whether this ED still holds queued general work. + pub(super) fn endpoint( + &self, + s: &mut Guard<'_>, + dma: &dyn PciDma, + at: u32, + ) -> Result<(u32, bool, bool), ()> { + let mut ed = Guard::unlocked(s, || read_words(dma, at))?; + let next = ed[3] & !15; + if ed[0] & (1 << 15) != 0 { + crate::dlog!(crate::devlog::LogModule::Usb, "ohci isochronous ED unsupported at guest address {at:#x}"); + return Ok((next, false, false)); + } + if ed[0] & (1 << 14) != 0 || ed[2] & 1 != 0 || ed[2] & !15 == ed[1] & !15 { + return Ok((next, false, false)); + } + let td_at = ed[2] & !15; + let mut td = Guard::unlocked(s, || read_words(dma, td_at))?; + let mps = ((ed[0] >> 16) & 0x7ff) as usize; + if mps == 0 || mps > 1023 { + crate::dlog!(crate::devlog::LogModule::Usb, "ohci invalid maximum packet size {mps} at ED address {at:#x}"); + return Ok((next, false, false)); + } + let Ok(n) = remaining(td[1], td[3]) else { + crate::dlog!(crate::devlog::LogModule::Usb, "ohci malformed TD buffer range at ED address {at:#x}"); + return Ok((next, false, false)); + }; + let direction = match (ed[0] >> 11) & 3 { + 1 => 1, + 2 => 2, + _ => (td[0] >> 19) & 3, + }; + let pid = match direction { + 0 => Pid::Setup, + 1 => Pid::Out, + 2 => Pid::In, + _ => { + crate::dlog!(crate::devlog::LogModule::Usb, "ohci reserved PID at ED address {at:#x}"); + return Ok((next, false, false)); + } + }; + let length = n.min(mps); + let toggle = if td[0] & (1 << 25) != 0 { + td[0] & (1 << 24) != 0 + } else { + ed[2] & 2 != 0 + }; + let outgoing = if pid != Pid::In { + Guard::unlocked(s, || payload_read(dma, td[1], td[3], length))? + } else { + vec![] + }; + let address = (ed[0] & 127) as u8; + let mut functions = (0..2) + .filter(|&p| s.powered(p) && s.r[21 + p] & 0x16 == 2) + .filter_map(|p| s.ports[p].clone()) + .filter(|f| f.address() == address); + // Multiple matching functions are an unsupported topology. Do not choose + // a recipient by port order or claim to model electrical collisions. + let first = functions.next(); + let function = if functions.next().is_none() { + first + } else { + None + }; + let reply = if let Some(f) = function { + let packet = UsbPacket { + endpoint: ((ed[0] >> 7) & 15) as u8, + pid, + toggle, + capacity: if pid == Pid::In { mps } else { length }, + data: outgoing, + time_ms: s.now_ms, + }; + Guard::unlocked(s, || f.transact(packet)) + } else { + UsbOutcome::NoResponse + }; + let mut retire = false; + let mut cc = 0; + let mut halt = false; + match reply { + UsbOutcome::Nak => return Ok((next, true, true)), + UsbOutcome::Stall => { + cc = 4; + halt = true; + retire = true; + } + UsbOutcome::NoResponse | UsbOutcome::CrcError | UsbOutcome::BitStuffError => { + cc = match reply { + UsbOutcome::CrcError => 1, + UsbOutcome::BitStuffError => 2, + _ => 5, + }; + let ec = ((td[0] >> 26) & 3) + 1; + td[0] = (td[0] & !(3 << 26)) | ((ec.min(3)) << 26); + if ec >= 3 { + retire = true; + halt = true; + } + } + UsbOutcome::Ack(bytes) => { + let actual = if pid == Pid::In { bytes.len() } else { length }; + if pid == Pid::In { + Guard::unlocked(s, || payload_write(dma, td[1], td[3], &bytes[..actual.min(n).min(mps)]))?; + } + td[0] = (td[0] & !((3 << 24) | (3 << 26))) | ((2 | u32::from(!toggle)) << 24); + if actual > length { + cc = 8; + halt = true; + retire = true; + } else { + td[1] = if actual == n { + 0 + } else { + advance(td[1], td[3], actual) + }; + if actual < length { + retire = true; + if td[0] & (1 << 18) == 0 { + cc = 9; + halt = true; + } + } else if actual == n { + retire = true; + } + } + } + } + td[0] = (td[0] & 0x0fff_ffff) | (cc << 28); + if retire { + let next_td = td[2] & !15; + td[2] = s.r[12]; + // Failure stops immediately. Earlier successful writes and USB side + // effects remain visible; descriptor writeback is not a transaction. + ed[2] = next_td | ((td[0] >> 23) & 2) | u32::from(halt); + Guard::unlocked(s, || write_words(dma, at + 8, &[ed[2]]).and_then(|()| write_words(dma, td_at, &td)))?; + s.r[12] = td_at; + let di = ((td[0] >> 21) & 7) as u8; + s.done_delay = s.done_delay.min(if cc != 0 { 0 } else { di }); + Ok((next, next_td != ed[1] & !15 && !halt, true)) + } else { + Guard::unlocked(s, || write_words(dma, td_at, &td))?; + Ok((next, true, true)) + } + } + pub(super) fn publish(&self, s: &mut Guard<'_>, dma: &dyn PciDma) -> Result<(), ()> { + if s.done_delay == 0 && s.r[3] & WDH == 0 && s.r[12] != 0 { + let other = u32::from(s.r[3] & s.r[4] & EVENTS & !WDH != 0); + let (at, head) = (s.r[6] + 0x84, s.r[12] | other); + Guard::unlocked(s, || write_words(dma, at, &[head]))?; + s.r[12] = 0; + s.done_delay = 7; + s.r[3] |= WDH; + } else if s.done_delay > 0 && s.done_delay < 7 { + s.done_delay -= 1; + } + Ok(()) + } +} diff --git a/src/dev/ohci/schedule.rs b/src/dev/ohci/schedule.rs new file mode 100644 index 000000000..78a0111c3 --- /dev/null +++ b/src/dev/ohci/schedule.rs @@ -0,0 +1,97 @@ +//! Functional scheduling approximation: 256 synthetic units per ED visit. +//! Costs deliberately do not represent wire time. See the module's limitations. +use super::*; +const VISIT: u32 = 256; +impl Ohci { + fn list_ready(s: &mut Guard<'_>, head: usize, current: usize, filled: u32) -> bool { + if s.r[current] == 0 && s.r[2] & filled != 0 { + s.r[current] = s.r[head]; + s.r[2] &= !filled; + } + s.r[current] != 0 + } + fn nonperiodic( + &self, + s: &mut Guard<'_>, + dma: &dyn PciDma, + budget: &mut u32, + cutoff: u32, + credit: &mut u32, + ) -> Result<(), ()> { + let mut yielded: Option = None; + while *budget >= VISIT && *budget > cutoff { + let control = s.r[1] & 16 != 0 && Self::list_ready(s, 8, 9, 2); + let bulk = s.r[1] & 32 != 0 && Self::list_ready(s, 10, 11, 4); + if !control && !bulk { + break; + } + let choose_control = control + && (!bulk + || match yielded { + Some(was_control) => !was_control, + None => *credit > 0, + }); + let (current, filled) = if choose_control { (9, 2) } else { (11, 4) }; + let (next, queued, attempt) = self.endpoint(s, dma, s.r[current])?; + *budget -= VISIT; + s.r[current] = next; + if queued { + s.r[2] |= filled; + } + yielded = if attempt { None } else { Some(choose_control) }; + if attempt { + if choose_control { + *credit = credit.saturating_sub(1); + } else { + *credit = (s.r[1] & 3) + 1; + } + } + } + Ok(()) + } + pub(super) fn schedule(&self, s: &mut Guard<'_>) -> Result<(), ()> { + let old = s.r[15] as u16; + let frame = old.wrapping_add(1); + s.r[15] = u32::from(frame); + if (old ^ frame) & 0x8000 != 0 { + s.r[3] |= 32; + } + s.r[3] |= 4; + let mut budget = (s.active_fi & 0x3fff) + 1; + s.r[14] = (s.active_fi & 0x80000000) | budget.min(0x3fff); + if self.pci.config().command() & 4 == 0 { + return Ok(()); + } + let Some(dma) = self.pci.dma().cloned() else { + return Err(()); + }; + let frame_bytes = [frame as u8, (frame >> 8) as u8, 0, 0]; + let hcca = s.r[6]; + if !Guard::unlocked(s, || dma.dma_write((hcca + 0x80) as u64, &frame_bytes).is_ok()) { + return Err(()); + } + let mut credit = (s.r[1] & 3) + 1; + let cutoff = s.r[16].min(budget); + self.nonperiodic(s, &*dma, &mut budget, cutoff, &mut credit)?; + if s.r[1] & 4 != 0 { + let mut bytes = [0; 4]; + let at = (s.r[6] + u32::from(frame & 31) * 4) as u64; + if !Guard::unlocked(s, || dma.dma_read(at, &mut bytes).is_ok()) { + return Err(()); + } + s.r[7] = u32::from_le_bytes(bytes) & !15; + while s.r[7] != 0 && budget >= VISIT { + let (next, _, _) = self.endpoint(s, &*dma, s.r[7])?; + s.r[7] = next; + budget -= VISIT; + } + if s.r[7] != 0 { + s.r[3] |= 1; + s.r[2] = (s.r[2] & !0x30000) | ((s.r[2].wrapping_add(0x10000)) & 0x30000); + } + } + self.nonperiodic(s, &*dma, &mut budget, 0, &mut credit)?; + s.r[14] = (s.active_fi & 0x80000000) | budget.min(0x3fff); + self.publish(s, &*dma) + } +} diff --git a/src/dev/ohci/tests.rs b/src/dev/ohci/tests.rs new file mode 100644 index 000000000..152b7da01 --- /dev/null +++ b/src/dev/ohci/tests.rs @@ -0,0 +1,597 @@ +// The register and clock cases follow tests/qtest/usb-lucent-ohci-test.c in +// jamesbraid/qemu-sgi-origami: root-hub-descriptor-a-is-writable and +// frame-number-tracks-clock-while-timer-starved. The descriptor engine cases +// drive a test-only function and have no qtest counterpart. +use super::*; +use crate::pci::{PciDmaStatus, PciFunction}; +use crate::traits::{Device, Resettable}; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::Weak; + +struct Memory { + bytes: Mutex>, + refuse_read: Mutex>, + refuse_write: Mutex>, + writes: Mutex>, +} +impl Memory { + fn new() -> Arc { + Arc::new(Self { + bytes: Mutex::new(vec![0xa5; 0x10000]), + refuse_read: Mutex::new(None), + refuse_write: Mutex::new(None), + writes: Mutex::new(vec![]), + }) + } + fn words(&self, at: u32, words: &[u32]) { + for (i, w) in words.iter().enumerate() { + self.bytes.lock()[at as usize + i * 4..at as usize + i * 4 + 4] + .copy_from_slice(&w.to_le_bytes()); + } + } + fn word(&self, at: u32) -> u32 { + u32::from_le_bytes( + self.bytes.lock()[at as usize..at as usize + 4] + .try_into() + .unwrap(), + ) + } +} +impl PciDma for Memory { + fn dma_read(&self, addr: u64, dst: &mut [u8]) -> PciDmaStatus { + if self + .refuse_read + .lock() + .is_some_and(|x| (addr..addr + dst.len() as u64).contains(&(x as u64))) + { + return PciDmaStatus::MasterAbort; + } + let b = self.bytes.lock(); + let Some(s) = b.get(addr as usize..addr as usize + dst.len()) else { + return PciDmaStatus::MasterAbort; + }; + dst.copy_from_slice(s); + PciDmaStatus::Ok + } + fn dma_write(&self, addr: u64, src: &[u8]) -> PciDmaStatus { + if self + .refuse_write + .lock() + .is_some_and(|x| (addr..addr + src.len() as u64).contains(&(x as u64))) + { + return PciDmaStatus::MasterAbort; + } + let mut b = self.bytes.lock(); + let Some(s) = b.get_mut(addr as usize..addr as usize + src.len()) else { + return PciDmaStatus::MasterAbort; + }; + s.copy_from_slice(src); + self.writes.lock().push(addr); + PciDmaStatus::Ok + } +} +struct Function { + address: AtomicUsize, + calls: AtomicUsize, + reply: Mutex, + packets: Mutex>, +} +impl Function { + fn new(reply: UsbOutcome) -> Arc { + Arc::new(Self { + address: AtomicUsize::new(0), + calls: AtomicUsize::new(0), + reply: Mutex::new(reply), + packets: Mutex::new(vec![]), + }) + } +} +impl UsbFunction for Function { + fn reset(&self) {} + fn address(&self) -> u8 { + self.address.load(Ordering::SeqCst) as u8 + } + fn transact(&self, packet: UsbPacket) -> UsbOutcome { + self.calls.fetch_add(1, Ordering::SeqCst); + self.packets.lock().push(packet); + self.reply.lock().clone() + } +} +fn start_port_reset(h: &Ohci, port: u32) { + let status = 0x54 + port * 4; + h.bar_write(0, status, 0x100, 4); + h.bar_write(0, status, 0x10, 4); +} + +fn setup( + pid: Pid, + reply: UsbOutcome, + cbp: u32, + be: u32, + flags: u32, +) -> (Arc, Arc, Arc) { + let h = Ohci::new(); + let m = Memory::new(); + let f = Function::new(reply); + h.state().connect_dma(m.clone()); + h.attach(0, f.clone()).unwrap(); + h.start(); + start_port_reset(&h, 0); + h.advance_frames(10); + m.words(0x1000, &[0; 64]); + m.words(0x2000, &[(8 << 16), 0x3020, 0x3000, 0]); + m.words( + 0x3000, + &[0xf000_0000 | flags | ((pid as u32) << 19), cbp, 0x3020, be], + ); + h.state().config().write(1, 6); + h.bar_write(0, 0x18, 0x1000, 4); + h.bar_write(0, 0x20, 0x2000, 4); + h.bar_write(0, 0x08, 2, 4); + h.bar_write(0, 0x04, 0x90, 4); + (h, m, f) +} +/// The guest receives fatal status if the controller cannot access a descriptor. +#[test] +fn dma_refusal_preserves_descriptors_and_reports_a_fatal_error() { + for periodic in [false, true] { + for pid in [Pid::Out, Pid::In] { + let (h, m, f) = setup(pid, UsbOutcome::Ack(vec![5; 8]), 0x4000, 0x4007, 0); + if periodic { + m.words(0x1000, &[0x2000; 32]); + h.bar_write(0, 4, 0x84, 4); + } + if pid == Pid::In { + *m.refuse_write.lock() = Some(0x4000); + } else { + *m.refuse_read.lock() = Some(0x4000); + } + let before = m.bytes.lock()[0x3000..0x3010].to_vec(); + h.advance_frames(1); + let frame = h.bar_read(0, 0x3c, 4); + h.advance_frames(3); + assert_eq!(f.calls.load(Ordering::SeqCst), usize::from(pid == Pid::In)); + assert_eq!(&m.bytes.lock()[0x3000..0x3010], &before); + assert_eq!(m.word(0x2008), 0x3000); + assert_eq!(m.word(0x1084), 0); + assert_eq!(h.bar_read(0, 0x3c, 4), frame); + assert_ne!(h.bar_read(0, 0xc, 4) & UE, 0); + assert_eq!(h.state().config().read(1) >> 16, 0x8200); + h.state().config().write(1, 0x8000_0006); + assert_eq!(h.state().config().read(1) >> 16, 0x0200); + assert_ne!(h.bar_read(0, 0xc, 4) & UE, 0); + } + } +} + +/// Short and overlong USB replies leave bytes outside the packet untouched. +#[test] +fn short_or_overlong_in_packets_preserve_unwritten_bytes() { + for (name, payload, end, flags, bytes, current, condition) in [ + ("short", vec![1, 2, 3], 0x401f, (1 << 18) | (3 << 24), [1, 2, 3, 0xa5, 0xa5], 0x4003, 0), + ("overrun", vec![1; 8], 0x4003, 3 << 24, [1, 1, 1, 1, 0xa5], 0x4000, 8), + ] { + let (h, m, _) = setup(Pid::In, UsbOutcome::Ack(payload), 0x4000, end, flags); + h.advance_frames(1); + assert_eq!(&m.bytes.lock()[0x4000..0x4005], &bytes, "{name}"); + assert_eq!(m.word(0x3004), current, "{name}"); + assert_eq!(m.word(0x3000) >> 28, condition, "{name}"); + assert_eq!((m.word(0x3000) >> 24) & 3, 2, "{name}"); + if condition == 0 { + assert_eq!(m.word(0x2008) & !15, 0x3020); + assert_eq!(m.word(0x1084) & !15, 0x3000); + } + } +} + +/// The guest retries NAKs, stops on STALL, and retains consistent short-transfer state. +#[test] +fn nak_stall_and_short_in_completions_keep_the_endpoint_consistent() { + for (name, reply, flags, condition) in [ + ("NAK", UsbOutcome::Nak, 0, 15), + ("STALL", UsbOutcome::Stall, 3 << 24, 4), + ("no response", UsbOutcome::NoResponse, 0, 5), + ("short without rounding", UsbOutcome::Ack(vec![2]), 0, 9), + ] { + let (h, m, f) = setup(Pid::In, reply, 0x4000, 0x4007, flags); + h.bar_write(0, 0x34, 255, 4); + let before = m.bytes.lock()[0x3000..0x3010].to_vec(); + h.advance_frames(1); + if condition == 5 { + assert_eq!(f.calls.load(Ordering::SeqCst), 1); + assert_eq!((m.word(0x3000) >> 26) & 3, 1); + assert_eq!(m.word(0x2008), 0x3000); + h.advance_frames(2); + } + assert_eq!(m.word(0x3000) >> 28, condition, "{name}"); + if condition == 15 { + assert_eq!(&m.bytes.lock()[0x3000..0x3010], &before); + assert_eq!(m.word(0x2008), 0x3000); + } else { + assert_ne!(m.word(0x2008) & 1, 0, "{name}"); + } + if condition == 4 { + assert_eq!(m.word(0x3004), 0x4000); + assert_eq!((m.word(0x3000) >> 24) & 3, 3); + assert_eq!(m.word(0x2008) & 3, 3); + } + } +} + +/// Port and controller reset paths retain attached USB functions. +#[test] +fn attached_functions_survive_port_and_controller_resets() { + let h = Ohci::new(); + h.start(); + h.attach(1, Function::new(UsbOutcome::Nak)).unwrap(); + assert_eq!(h.bar_read(0, 0x58, 4) & 1, 0); + start_port_reset(&h, 1); + h.advance_frames(9); + assert_ne!(h.bar_read(0, 0x58, 4) & 16, 0); + h.advance_frames(1); + assert_eq!(h.bar_read(0, 0x58, 4) & 16, 0); + assert_ne!(h.bar_read(0, 0x58, 4) & (1 << 20), 0); + h.bar_write(0, 8, 1, 4); + assert_ne!(h.bar_read(0, 0x58, 4) & 1, 0); + h.bar_write(0, 0x58, 0x200, 4); + assert_eq!(h.bar_read(0, 0x58, 4) & 0x117, 0); + h.bar_write(0, 0x58, PPS, 4); + assert_ne!(h.bar_read(0, 0x58, 4) & 1, 0); + h.power_on(); + assert_eq!(h.bar_read(0, 0x58, 4), 0); + h.bar_write(0, 0x58, PPS, 4); + assert_ne!(h.bar_read(0, 0x58, 4) & 1, 0); +} + +/// The guest observes descriptor updates before the done bit and acknowledges them later. +#[test] +fn done_publication_orders_writes_and_waits_for_acknowledgement() { + let (h, m, _) = setup(Pid::In, UsbOutcome::Ack(vec![8; 8]), 0x4000, 0x4007, 0); + h.advance_frames(1); + { + let writes = m.writes.lock(); + assert!(writes.iter().position(|a| *a == 0x2008).unwrap() < writes.iter().position(|a| *a == 0x3000).unwrap()); + } + assert_eq!(m.word(0x1084) & !15, 0x3000); + m.words(0x3020, &[0xf0000000 | (2 << 19), 0x4100, 0x3040, 0x4107]); + m.words(0x2004, &[0x3040]); + h.bar_write(0, 8, 2, 4); + h.advance_frames(1); + assert_eq!(m.word(0x1084) & !15, 0x3000); + assert_eq!(h.bar_read(0, 0x30, 4), 0x3020); + h.bar_write(0, 0xc, 2, 4); + h.advance_frames(1); + assert_eq!(m.word(0x1084) & !15, 0x3020); + for delay in [0, 2, 7] { + let (h, m, _) = setup(Pid::In, UsbOutcome::Ack(vec![5; 8]), 0x4000, 0x4007, delay << 21); + h.advance_frames(1); + if delay != 0 { assert_eq!(m.word(0x1084), 0); } + h.advance_frames(u64::from(delay)); + assert_eq!(m.word(0x1084) & !15, if delay == 7 { 0 } else { 0x3000 }); + } + let (h, m, _) = setup(Pid::In, UsbOutcome::Ack(vec![1; 8]), 0x4000, 0x4007, 0); + *m.refuse_write.lock() = Some(0x1084); + h.advance_frames(1); + assert_ne!(h.bar_read(0, 0xc, 4) & UE, 0); + assert_eq!(h.bar_read(0, 0xc, 4) & WDH, 0); + assert_eq!(m.word(0x1084), 0); +} + +/// OHCI must yield to the other transfer list when a guest marks one list skipped. +#[test] +fn scheduler_shares_service_and_yields_past_skipped_lists() { + for (cbsr, skip_control, skip_bulk) in [(0, false, false), (3, false, false), (3, true, false), (0, false, true)] { + let (h, m, f) = setup(Pid::In, UsbOutcome::Nak, 0x4000, 0x4007, 0); + m.words(0x2000, &[(8 << 16) | (1 << 7) | (u32::from(skip_control) << 14), 0x3020, 0x3000, if skip_control { 0x2000 } else { 0 }]); + m.words(0x2100, &[(8 << 16) | (2 << 7) | (u32::from(skip_bulk) << 14), 0x3120, 0x3100, if skip_bulk { 0x2100 } else { 0 }]); + m.words(0x3100, &[0xf0100000, 0x4100, 0x3120, 0x4107]); + h.bar_write(0, 0x28, 0x2100, 4); + h.bar_write(0, 8, 6, 4); + h.bar_write(0, 4, 0xb0 | cbsr, 4); + h.advance_frames(1); + let packets = f.packets.lock(); + let control = packets.iter().filter(|p| p.endpoint == 1).count(); + let bulk = packets.iter().filter(|p| p.endpoint == 2).count(); + assert_eq!(control > 0, !skip_control); + assert_eq!(bulk > 0, !skip_bulk); + if skip_bulk { + assert!(control > 1, "control resumes after its credit is exhausted"); + } + if !skip_control && !skip_bulk { + assert!(control >= bulk * (cbsr as usize + 1)); + assert!(control <= bulk * (cbsr as usize + 1) + cbsr as usize + 1); + } + assert_eq!(h.bar_read(0, 0xc, 4) & UE, 0); + } +} + +/// The controller routes address zero only while one attached function can claim it. +#[test] +fn address_zero_is_routed_only_to_a_unique_eligible_function() { + for (name, address, peer_status, first_selected, peer_selected) in [ + ("ambiguous", 0, PPS | 1 | 2, false, false), + ("disabled peer", 0, PPS | 1, true, false), + ("assigned first", 5, PPS | 1 | 2, false, true), + ] { + let (h, m, f) = setup(Pid::In, UsbOutcome::Nak, 0x4000, 0x4007, 0); + let peer = Function::new(UsbOutcome::Nak); + h.attach(1, peer.clone()).unwrap(); + h.state.lock().r[22] = peer_status; + f.address.store(address, Ordering::SeqCst); + h.advance_frames(1); + assert_eq!(f.calls.load(Ordering::SeqCst) > 0, first_selected, "{name}"); + assert_eq!(peer.calls.load(Ordering::SeqCst) > 0, peer_selected, "{name}"); + if address != 0 { + let peer_calls = peer.calls.load(Ordering::SeqCst); + m.words(0x2000, &[(8 << 16) | 5]); + h.advance_frames(1); + assert!(f.calls.load(Ordering::SeqCst) > 0); + assert_eq!(peer.calls.load(Ordering::SeqCst), peer_calls); + } else if !first_selected { + assert_eq!(m.word(0x3000) >> 28, 5); + assert_eq!((m.word(0x3000) >> 26) & 3, 3); + } + } +} + + +/// A guest USB packet can cross physically nonadjacent DMA pages. +#[test] +fn packet_crosses_nonadjacent_pages() { + let (h, m, _) = setup( + Pid::In, + UsbOutcome::Ack(vec![1, 2, 3, 4, 5, 6, 7, 8]), + 0x4ffc, + 0x9003, + 0, + ); + h.advance_frames(1); + assert_eq!(&m.bytes.lock()[0x4ffc..0x5000], &[1, 2, 3, 4]); + assert_eq!(&m.bytes.lock()[0x9000..0x9004], &[5, 6, 7, 8]); + assert_eq!(m.bytes.lock()[0x5000], 0xa5); + assert_eq!(m.word(0x3004), 0); +} + +/// The frame counter catches up after its host timer was delayed. +#[test] +fn clock_read_catches_up_after_timer_starvation() { + let h = Ohci::new(); + let now = Arc::new(AtomicUsize::new(0)); + let t = now.clone(); + h.set_clock(Arc::new(move || t.load(Ordering::Relaxed) as u64)); + h.start(); + h.bar_write(0, 4, 0x80, 4); + now.store(5_000_000, Ordering::Relaxed); + assert!(h.bar_read(0, 0x3c, 4) >= 5); + h.stop(); + let frame = h.bar_read(0, 0x3c, 4); + h.advance_frames(10); + assert_eq!(h.bar_read(0, 0x3c, 4), frame); +} + +/// The frame count follows the guest's clock, not when the machine's poll or +/// a timer thread last ran: a guest that waits 1 ms after going operational +/// and reads HcFmNumber sees the frame, even if nothing else advanced the +/// controller in between (a slow or descheduled host thread). +#[test] +fn frame_number_follows_the_clock_with_no_poll_or_timer() { + let h = Ohci::new(); + let now = Arc::new(AtomicUsize::new(10_000_000)); + let t = now.clone(); + h.set_clock(Arc::new(move || t.load(Ordering::Relaxed) as u64)); + h.start(); + // Part-way into a frame period when the guest sets operational. + now.store(10_400_000, Ordering::Relaxed); + h.bar_write(0, 4, 0x80, 4); + // Entering Operational increments the frame number at once. + assert_eq!(h.bar_read(0, 0x3c, 4), 1); + // 1.1 ms on the guest's clock: the first boundary, 1 ms after the write, has passed. + now.store(11_500_000, Ordering::Relaxed); + assert_eq!(h.bar_read(0, 0x3c, 4), 2); + // A long guest-time gap is delivered in full at the next access. + now.store(31_500_000, Ordering::Relaxed); + assert_eq!(h.bar_read(0, 0x3c, 4), 22); + // Time that has not passed on the guest's clock delivers nothing. + assert_eq!(h.bar_read(0, 0x3c, 4), 22); + h.stop(); +} + +fn fake_clock_ohci(start_ns: usize) -> (Arc, Arc) { + let h = Ohci::new(); + let now = Arc::new(AtomicUsize::new(start_ns)); + let t = now.clone(); + h.set_clock(Arc::new(move || t.load(Ordering::Relaxed) as u64)); + h.start(); + (h, now) +} + +/// OpenHCI 1.0a sections 6.2.1 and 7.3.3: entering UsbOperational loads +/// HcFmRemaining from the frame interval, increments HcFmNumber, and sends the +/// first SOF one frame later, whatever the phase of the free-running clock. +fn operational_entry_restarts_the_frame(from_hcfs: Option) { + let (h, now) = fake_clock_ohci(10_000_000); + let m = Memory::new(); + h.state().connect_dma(m.clone()); + h.state().config().write(1, 6); + h.bar_write(0, 0x18, 0x1000, 4); + // Not operational: no frames count however much time passes. + if let Some(hcfs) = from_hcfs { + h.bar_write(0, 4, hcfs, 4); + } + now.store(15_000_000, Ordering::Relaxed); + assert_eq!(h.bar_read(0, 0x3c, 4), 0); + now.store(15_400_000, Ordering::Relaxed); + h.bar_write(0, 4, 0x80, 4); + assert_eq!(h.bar_read(0, 0x3c, 4), 1); + assert_eq!(h.bar_read(0, 0x38, 4), 0x2edf); + assert_eq!(h.bar_read(0, 0x0c, 4) & 4, 0, "no SOF at the load"); + assert_eq!(m.bytes.lock()[0x1080], 0xa5, "no HCCA write at the load"); + // Just short of one frame after the write: still nothing. + now.store(16_399_999, Ordering::Relaxed); + assert_eq!(h.bar_read(0, 0x3c, 4), 1); + assert_eq!(h.bar_read(0, 0x0c, 4) & 4, 0); + assert_eq!(m.bytes.lock()[0x1080], 0xa5); + // Exactly one frame after: the first SOF and its HCCA frame number. + now.store(16_400_000, Ordering::Relaxed); + assert_eq!(h.bar_read(0, 0x0c, 4) & 4, 4); + assert_eq!(h.bar_read(0, 0x3c, 4), 2); + assert_eq!(m.bytes.lock()[0x1080], 2); + h.stop(); +} + +#[test] +fn operational_entry_from_reset_restarts_the_frame() { + operational_entry_restarts_the_frame(None); +} + +#[test] +fn operational_entry_from_suspend_restarts_the_frame() { + // HCFS 11b. + operational_entry_restarts_the_frame(Some(0xc0)); +} + +#[test] +fn operational_entry_from_resume_restarts_the_frame() { + // HCFS 01b. + operational_entry_restarts_the_frame(Some(0x40)); +} + +/// The increment on entry carries into the frame-number overflow flag like a +/// frame's own, and HcFmRemaining takes the frame-interval toggle bit too. +#[test] +fn operational_entry_sets_overflow_and_loads_the_interval_toggle() { + for (before, after, overflow) in [(0x7fff, 0x8000, true), (0xffff, 0, true), (0x1234, 0x1235, false)] { + let (h, _now) = fake_clock_ohci(10_000_000); + h.state.lock().r[15] = before; + h.bar_write(0, 0x34, 0x8000_2edf, 4); + h.bar_write(0, 4, 0x80, 4); + assert_eq!(h.bar_read(0, 0x3c, 4), after, "from {before:#x}"); + assert_eq!(h.bar_read(0, 0x0c, 4) & 32 != 0, overflow, "from {before:#x}"); + assert_eq!(h.bar_read(0, 0x38, 4), 0x8000_2edf); + h.stop(); + } +} + +/// An interrupt handler can acknowledge a line while the controller raises it. +#[test] +fn reentrant_irq_acknowledgement_leaves_final_line_clear() { + struct Ack { + hc: Weak, + level: AtomicBool, + once: AtomicBool, + } + impl crate::irq::InterruptController for Ack { + fn set_line(&self, _: u32, level: bool) { + if level && !self.once.swap(true, Ordering::SeqCst) { + self.hc.upgrade().unwrap().bar_write(0, 0xc, RHSC, 4); + } + self.level.store(level, Ordering::SeqCst); + } + } + let h = Ohci::new(); + let ack = Arc::new(Ack { + hc: Arc::downgrade(&h), + level: AtomicBool::new(false), + once: AtomicBool::new(false), + }); + h.state().connect_irq(IrqLine::new(ack.clone(), 0)); + h.attached(); + h.bar_write(0, 0x10, 0x80000040, 4); + h.attach(0, Function::new(UsbOutcome::Nak)).unwrap(); + assert!(!ack.level.load(Ordering::SeqCst)); +} + +/// The guest sees the documented write masks, HCR exemptions, and ownership bit. +#[test] +fn register_masks_preserve_hcr_exempt_fields_and_ownership_status() { + let h = Ohci::new(); + h.bar_write(0, 0x18, 0x123456ff, 4); + assert_eq!(h.bar_read(0, 0x18, 4), 0x12345600); + h.bar_write(0, 0x20, 0x123456ff, 4); + assert_eq!(h.bar_read(0, 0x20, 4), 0x123456f0); + h.bar_write(0, 0x10, u32::MAX, 4); + assert_eq!(h.bar_read(0, 0x10, 4), 0xc000007f); + h.bar_write(0, 8, 8, 4); + assert_eq!(h.bar_read(0, 0xc, 4), 1 << 30); + h.bar_write(0, 0xc, 1 << 31, 4); + assert_eq!(h.bar_read(0, 0xc, 4), 1 << 30); + h.bar_write(0, 0xc, 1 << 30, 4); + assert_eq!(h.bar_read(0, 0xc, 4), 0); + h.bar_write(0, 0x48, u32::MAX, 4); + assert_eq!(h.bar_read(0, 0x48, 4), 0xff001b02); + h.bar_write(0, 4, 0x700, 4); + h.bar_write(0, 8, 1, 4); + assert_eq!(h.bar_read(0, 4, 4), 0x3c0); + assert_eq!(h.bar_read(0, 0x48, 4), 0xff001b02); + assert_eq!(h.bar_read(0, 0x10, 4), 0); + h.bar_write(0, 0x3c, 0xffff, 4); + assert_eq!(h.bar_read(0, 0x3c, 4), 0); + h.bar_write(0, 0x18, 1, 1); + assert_eq!(h.bar_read(0, 0x18, 4), 0); +} + +/// The periodic phase controls how the guest control and bulk lists share service. +#[test] +fn periodic_phase_preserves_control_and_bulk_credit() { + let (h, m, f) = setup(Pid::In, UsbOutcome::Nak, 0x4000, 0x4007, 0); + m.words(0x2000, &[(8 << 16) | (1 << 7), 0x3020, 0x3000, 0]); + m.words(0x2100, &[(8 << 16) | (2 << 7), 0x3120, 0x3100, 0]); + m.words(0x3100, &[0xf0100000, 0x4100, 0x3120, 0x4107]); + m.words(0x2200, &[(8 << 16) | (3 << 7), 0x3220, 0x3200, 0]); + m.words(0x3200, &[0xf0100000, 0x4200, 0x3220, 0x4207]); + for i in 0..32 { + m.words(0x1000 + i * 4, &[0x2200]); + } + let mut s = h.state.lock(); + s.active_fi = 5 * 256 - 1; + s.r[16] = 3 * 256; + s.r[1] = 0xb5; + s.r[10] = 0x2100; + s.r[2] |= 4; + h.schedule(&mut s).unwrap(); + let order: Vec<_> = f.packets.lock().iter().map(|p| p.endpoint).collect(); + assert_eq!(order, [1, 1, 3, 2, 1]); + assert_eq!(s.r[14], 0); +} + +/// A frame's DMA and USB transaction run with no controller lock held: a +/// parent that calls back into the controller from the transfer, here by +/// reading its registers (which first delivers the frames the clock owes), +/// neither deadlocks nor runs a second frame inside the first. +#[test] +fn a_frame_holds_no_lock_through_its_transfers() { + struct Reentrant { + memory: Arc, + hc: Mutex>, + frames: Mutex>, + } + impl PciDma for Reentrant { + fn dma_read(&self, addr: u64, dst: &mut [u8]) -> PciDmaStatus { + if let Some(hc) = self.hc.lock().upgrade() { + self.frames.lock().push(hc.bar_read(0, 0x3c, 4)); + } + self.memory.dma_read(addr, dst) + } + fn dma_write(&self, addr: u64, src: &[u8]) -> PciDmaStatus { + if let Some(hc) = self.hc.lock().upgrade() { + self.frames.lock().push(hc.bar_read(0, 0x3c, 4)); + } + self.memory.dma_write(addr, src) + } + } + let (h, now) = fake_clock_ohci(10_000_000); + let m = Memory::new(); + let dma = Arc::new(Reentrant { memory: m.clone(), hc: Mutex::new(Weak::new()), frames: Mutex::new(vec![]) }); + h.state().connect_dma(dma.clone()); + h.state().config().write(1, 6); + m.words(0x1000, &[0; 64]); + h.bar_write(0, 0x18, 0x1000, 4); + h.bar_write(0, 4, 0x80, 4); + *dma.hc.lock() = Arc::downgrade(&h); + // Three frames owed; each transfer's read sees the frame it belongs to. + now.store(13_000_000, Ordering::Relaxed); + assert_eq!(h.bar_read(0, 0x3c, 4), 4); + assert_eq!(*dma.frames.lock(), [2, 3, 4], "one HCCA write per frame, in order"); + assert_eq!(m.bytes.lock()[0x1080], 4); + h.stop(); +} diff --git a/src/dev/pic.rs b/src/dev/pic.rs new file mode 100644 index 000000000..4e141d14b --- /dev/null +++ b/src/dev/pic.rs @@ -0,0 +1,1439 @@ +//! The PIC, the PCI host bridge on the interface board of the Origin 350 and +//! Tezro (IP53 family). One chip sits in one XIO widget and carries two +//! independent bridge heads, at widget offset 0 and `0x800000`, each with its +//! own register block and PCI bus. Widget 15 holds the first PIC; the +//! two- and four-processor Tezro board adds a second at widget 11. +//! +//! A [`Pic`] is self-contained: the machine builds one per widget, hands the +//! processor's view of it to Bedrock IO ([`Pic::widget`]), gives it a +//! [`PicPeer`] for the packets it originates (interrupts and DMA), connects +//! the Bedrock PIs it raises interrupts on, and attaches PCI functions to the +//! bus slots. Each head owns a shared [`PciBus`] (`src/pci/`) for its +//! functions, BAR decode and interrupt pins, and is that bus's [`PciHost`]: +//! it supplies the DMA a slot gets and takes the level of each of its eight +//! interrupt inputs. The bus slots are hooks only: IOC4 at device 1 (PIC +//! input 0), the ISP12160 at device 3 (input 2) and the BCM5701 at device 4 +//! (input 3) are the board's choice, made by the machine; device `d`'s INTA +//! reaches input `d - 1`. +//! +//! Widget 0 of the board is the PXBOW of the PIC that carries IO9 +//! ([`Pic::enable_pxbow`]); the other PIC has none. +//! +//! The register offsets are the ones the Bridge family shares ([`reg`]); +//! the PIC leaves out the ATE, the alias windows and the second and third +//! write-request banks' meaning. +//! +//! # What is approximated here +//! +//! The model is the compatibility profile QEMU selects for Tezro: revision 3 +//! with identity bit 0 set, `DIR_MAP` bits 0 to 16 and 20 to 23, the control +//! and Device masks, and the reset values below are emulator choices that let +//! the PROM and IRIX run, not measured reset state. The CPU word view (a +//! 32-bit access to the low half of an eight-byte slot) and the processor's +//! byte-lane placement for PCI cycles are inferred from the public drivers. +//! The one-millisecond reassertion of a still-high PCI input after the +//! processor clears its interrupt bit is the Bedrock PI's +//! ([`Bedrock::set_level_source`]); this chip tells it which inputs are +//! active. Translated (ATE) DMA, PCI-X +//! errors and the bridge's own PCI function are unsupported and observable +//! ([`Pic::unsupported`]). The packet face a peer could read the heads +//! through has no user and is not modeled. All transfers are synchronous. +//! +//! A configuration read that no function claims is a PCI master timeout: +//! the processor gets a bus error back through Bedrock, the error view +//! records bit 11 whatever the enable, and interrupt status records it only +//! while enable bit 11 is set (the gating is inferred). Status clears one +//! bit at a time; any reset-register write clears the view. Device 0, which +//! the PIC keeps for itself, reads as unclaimed: an approximation, since +//! what the chip answers there is unknown. Which bit a bus in PCI-X mode +//! sets (11 or 34) is unknown; the model sets 11. An unclaimed +//! configuration write goes nowhere and records nothing. +//! +//! Ported from hw/pci-host/sgi-pic.c in jamesbraid/qemu-sgi-origami. + +use super::bedrock::Bedrock; +use super::heart::Clock; +use super::pxbow::{AttachError, Pxbow}; +use crate::devlog::LogModule; +use crate::irq::IrqLine; +use crate::pci::{ + BarDevice, BarMapping, BoardIrq, ByteOrder, CpuWindow, DevFn, IntxRouting, PciBus, PciDma, PciDmaStatus, PciFunction, + PciHost, Space, +}; +use crate::traits::{BusDevice, BusRead16, BusRead32, BusRead64, BusRead8, Resettable, BUS_ERR, BUS_OK}; +use parking_lot::{Mutex, RwLock}; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::{Arc, OnceLock, Weak}; + +/// Where the second head sits in the widget. +pub const HEAD_OFFSET: u64 = 0x80_0000; +/// The widget's address space. +pub const WIDGET_SIZE: u64 = 1 << 32; +/// Interrupt inputs per head. +pub const PINS: usize = 8; +/// PCI devices behind a head: configuration devices 1 to 4. Device 0 is the +/// bridge itself, and no PCI identity for it is established. +pub const DEVICES: u8 = 4; + +const REVISION: u32 = 3; +const MANUFACTURER: u32 = 0x24; +const PARTS: [u32; 2] = [0xd102, 0xd112]; + +const REGISTER_BYTES: usize = 0x640; +const REQUEST_BANKS: usize = 3; +const REQUEST_ENTRIES: usize = 256; +const REQUEST_BASE: usize = 0x18000; +const REQUEST_END: usize = 0x19800; +const REQUEST_BANK_SIZE: usize = 0x800; +const CONFIG_BASE: u64 = 0x20000; +const CONFIG_TYPE1: u64 = 0x28000; +const CONFIG_END: u64 = 0x29000; +const PCI_WINDOW: [u64; 2] = [0x4000_0000, 0xc000_0000]; +const PCI_WINDOW_SIZE: u64 = 1 << 30; +const SHORT_BASE: u64 = 0x20_0000; +/// The packet window of the 48-bit XIO address: bits 63:60 of a direct64 DMA +/// address name the destination widget, and a payload keeps 47 bits. +const PAYLOAD_BITS: u32 = 48; +const UNSUPPORTED_LIMIT: usize = 32; + +/// The widget flush register (read to order earlier writes). +const FLUSH: usize = 0x50; +/// The link-side error address slot, whose low-word view the PROM writes. +const ERROR_ADDRESS_WORD: u64 = 0x7c; + +const CONTROL_MASK: u64 = 0x0f00_f20f; +const CONTROL_RESET: u64 = 0x0f00_320f; +const REQUEST_TIMEOUT_RESET: u64 = 0xfffff; +const DIRECT_MAP_SUPPORTED: u64 = 0x1ffff | (0xf << 20); +const DEVICE_MASK: u64 = 0x1858_3fff; +const DEVICE_RESET: [u64; DEVICES as usize] = [0x1800_1000, 0x1800_1002, 0x1800_1004, 0x1800_1005]; +const DEVICE_SWAP: u64 = 1 << 19; +const DEVICE_STREAM: u64 = 1 << 13; +const DEVICE_WIDE: u64 = 1 << 22; +const DEVICE_MEMORY: u64 = 1 << 12; +const INVALID_ADDRESS: u64 = 1 << 24; +/// PCI master timeout: a cycle no device claimed. The same bit in the +/// interrupt status, enable and error view registers. +const MASTER_TIMEOUT: u64 = 1 << 11; +/// The error view: the error bits, whether or not they are enabled. +/// Read-only; any write to the reset register clears it. +const ERROR_VIEW: usize = reg::INTERRUPT_MULTIPLE; +const SET: u64 = 0x100; +const LEVELS: std::ops::RangeInclusive = 7..=127; + +/// The head registers the PIC shares with the Bridge family, as byte offsets +/// into a head's register block. +pub mod reg { + pub const IDENTIFICATION: usize = 0x0; + pub const CONTROL: usize = 0x20; + pub const REQUEST_TIMEOUT: usize = 0x28; + pub const INT_DEST_UPPER: usize = 0x30; + pub const INT_DEST_LOWER: usize = 0x38; + pub const DIRECT_MAP: usize = 0x80; + pub const PCI_TYPE1_CONFIG: usize = 0xc8; + pub const INTERRUPT_STATUS: usize = 0x100; + pub const INTERRUPT_ENABLE: usize = 0x108; + pub const INTERRUPT_RESET: usize = 0x110; + pub const INTERRUPT_MODE: usize = 0x118; + pub const INTERRUPT_DEVICE: usize = 0x120; + pub const INTERRUPT_ERROR_HOST: usize = 0x128; + pub const INTERRUPT_HOST_BASE: usize = 0x130; + pub const INTERRUPT_HOST_LAST: usize = 0x168; + pub const INTERRUPT_MULTIPLE: usize = 0x170; + pub const DEVICE_BASE: usize = 0x200; +} + +/// A run of a head's address space that one BAR covers whole, and the bus +/// device that answers it. +struct PioRun { + start: u64, + end: u64, + dev: Arc, +} + +/// Whole BAR-page runs are few and carry no RAM or native fault-handler state. +/// Readers clone one endpoint and release the table lock before calling a device. +#[derive(Default)] +struct PioDecode { + runs: RwLock>, +} +impl PioDecode { + fn publish(&self, mut runs: Vec) -> Vec { + runs.sort_unstable_by_key(|run| run.start); + debug_assert!(runs.windows(2).all(|pair| pair[0].end <= pair[1].start)); + std::mem::replace(&mut *self.runs.write(), runs) + } + fn target(&self, address: u64, size: usize) -> Option> { + let runs = self.runs.read(); + let i = runs.partition_point(|run| run.start <= address).checked_sub(1)?; + let run = &runs[i]; + (address.checked_add(size as u64)? <= run.end).then(|| run.dev.clone()) + } +} + +/// Widget 0's interrupt packets, which go to the peer as the PIC's do. +struct PxbowPackets(Arc); +impl crate::irq::InterruptController for PxbowPackets { + fn set_line(&self, line: u32, level: bool) { + self.packet(0, 0, line, level); + } + fn packet(&self, _source: u8, address: u64, line: u32, level: bool) -> bool { + let payload = u64::from(line) | if level { SET } else { 0 }; + let taken = self.0.peer.get().is_some_and(|peer| peer.interrupt(0, address, payload).is_ok()); + if !taken { + crate::dlog!(LogModule::Bedrock, "pic: PXBOW interrupt packet to {address:#x} refused"); + } + taken + } +} + +/// The widget identity word of a head: revision, part and manufacturer +/// packed as the common widget header lays them out, with bit 0 set. +pub const fn identity(head: usize) -> u32 { + (REVISION << 28) | (PARTS[head] << 12) | (MANUFACTURER << 1) | 1 +} + +/// A configuration cycle no device claimed: a PCI master timeout. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +struct Unclaimed; + +/// What the machine's side of the XIO link refused: no target, a permission +/// check, or an address with nothing behind it. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Refused; + +/// The far end of the link: where the packets this chip originates go. On +/// Tezro that is Bedrock IO. Bytes are in address order, as the XIO packet +/// carries them. +pub trait PicPeer: Send + Sync { + /// An interrupt packet: `payload` is the vector, with bit 8 for assert, + /// written to the 48-bit destination `address`. + fn interrupt(&self, source_widget: u8, address: u64, payload: u64) -> Result<(), Refused>; + + /// A DMA read of `data.len()` bytes at the 48-bit `address` of `widget`. + fn dma_read(&self, source_widget: u8, widget: u8, address: u64, data: &mut [u8]) -> Result<(), Refused>; + + fn dma_write(&self, source_widget: u8, widget: u8, address: u64, data: &[u8]) -> Result<(), Refused>; +} + +/// How the board uses a PCI function's slot. +#[derive(Clone, Copy, Debug, Default)] +pub struct Function { + /// The function's registers keep the processor's byte sequence (IOC4), + /// so the bridge leaves the address and value of a processor access + /// alone instead of placing the bytes for a little-endian adapter. + pub self_ordered_pio: bool, + /// The slot drives REQ64# and the function takes 64-bit data phases: + /// its swap-clear direct64 DMA reverses each doubleword, not each word. + pub req64: bool, +} + +/// The peer was already connected. +#[derive(Debug, PartialEq, Eq)] +pub struct AlreadyConnected; + +/// Wiring set once before the machine runs. +struct Wiring { + peer: OnceLock>, + /// The node whose PIs the chip's interrupts reach. Weak: the node holds + /// the widgets. + node: OnceLock>, + /// The first PI level source this chip owns; head h, pin p is + /// `source_base + h * PINS + p`. + source_base: AtomicUsize, +} + +struct State { + reg: [u64; REGISTER_BYTES / 8], + request: [[u64; REQUEST_ENTRIES]; REQUEST_BANKS], + pins: u8, + set_requests: [u32; PINS], + clear_requests: [u32; PINS], +} + +impl State { + fn new() -> Self { + let mut s = Self { + reg: [0; REGISTER_BYTES / 8], + request: [[0; REQUEST_ENTRIES]; REQUEST_BANKS], + pins: 0, + set_requests: [0; PINS], + clear_requests: [0; PINS], + }; + s.reset(); + s + } + + fn r(&self, off: usize) -> u64 { + self.reg[off / 8] + } + + fn set(&mut self, off: usize, value: u64) { + self.reg[off / 8] = value; + } + + fn reset(&mut self) { + *self = Self { + reg: [0; REGISTER_BYTES / 8], + request: [[0; REQUEST_ENTRIES]; REQUEST_BANKS], + pins: 0, + set_requests: [0; PINS], + clear_requests: [0; PINS], + }; + self.set(reg::REQUEST_TIMEOUT, REQUEST_TIMEOUT_RESET); + self.set(reg::CONTROL, CONTROL_RESET); + for (i, v) in DEVICE_RESET.iter().enumerate() { + self.set(reg::DEVICE_BASE + i * 8, *v); + } + } +} + +/// The head's PCI interrupt routing. Device `d`'s INTA reaches input `d - 1`; +/// INTB to INTD are not wired. The eight inputs are shared with the board's +/// own interrupts, which reach a head through [`Pic::pin_line`]. Device 0 is +/// the bridge itself and holds no function. +static PIC_ROUTING: IntxRouting = IntxRouting { + devices: DEVICES + 1, + irqs: PINS as u8, + intx: &[ + (1, [Some(0), None, None, None]), + (2, [Some(1), None, None, None]), + (3, [Some(2), None, None, None]), + (4, [Some(3), None, None, None]), + ], + board_irqs: &[0, 1, 2, 3, 4, 5, 6, 7], +}; + +/// The head's CPU windows, by [`CpuWindow`] index: the four short device-I/O +/// windows, then the large PCI memory window. Addresses are the widget's. +const SHORT_WINDOWS: usize = 4; +const BIG_WINDOW: usize = 4; + +/// The short windows' extents in the head's own space. +const SHORT: [(u64, u64); SHORT_WINDOWS] = [(0x20_0000, 0x20_0000), (0x40_0000, 0x20_0000), (0x60_0000, 0x10_0000), (0x70_0000, 0x10_0000)]; + +fn short_window(addr: u64) -> Option<(usize, u64, u64)> { + match addr { + 0x20_0000..=0x3f_ffff => Some((0, addr - 0x20_0000, 0x20_0000)), + 0x40_0000..=0x5f_ffff => Some((1, addr - 0x40_0000, 0x20_0000)), + 0x60_0000..=0x6f_ffff => Some((2, addr - 0x60_0000, 0x10_0000)), + 0x70_0000..=0x7f_ffff => Some((3, addr - 0x70_0000, 0x10_0000)), + _ => None, + } +} + +fn defined(off: usize) -> bool { + matches!( + off, + reg::IDENTIFICATION + | reg::CONTROL + | reg::REQUEST_TIMEOUT + | reg::INT_DEST_UPPER + | reg::INT_DEST_LOWER + | FLUSH + | reg::DIRECT_MAP + | reg::PCI_TYPE1_CONFIG + | reg::INTERRUPT_STATUS + | reg::INTERRUPT_ENABLE + | reg::INTERRUPT_RESET + | reg::INTERRUPT_MODE + | reg::INTERRUPT_DEVICE + | reg::INTERRUPT_ERROR_HOST + | reg::INTERRUPT_HOST_BASE..=reg::INTERRUPT_HOST_LAST + | ERROR_VIEW + | reg::DEVICE_BASE..=0x218 + ) +} + +fn request_slot(off: usize) -> (usize, usize) { + let rel = off - REQUEST_BASE; + (rel / REQUEST_BANK_SIZE, (rel % REQUEST_BANK_SIZE) / 8) +} + +/// The largest naturally aligned access of at most eight bytes that starts +/// at `address` and fits in `remaining`. +fn natural(address: u64, remaining: usize) -> usize { + let mut n = 8; + while n > remaining || !address.is_multiple_of(n as u64) { + n /= 2; + } + n +} + +fn le_bytes(size: usize, value: u64) -> [u8; 8] { + let mut b = [0; 8]; + b[..size].copy_from_slice(&value.to_le_bytes()[..size]); + b +} + +fn be_bytes(size: usize, value: u64) -> [u8; 8] { + let mut b = [0; 8]; + b[..size].copy_from_slice(&value.to_be_bytes()[8 - size..]); + b +} + +fn le_value(bytes: &[u8]) -> u64 { + bytes.iter().rev().fold(0, |v, &b| (v << 8) | b as u64) +} + +fn be_value(bytes: &[u8]) -> u64 { + bytes.iter().fold(0, |v, &b| (v << 8) | b as u64) +} + +/// One bridge head: registers, one PCI bus, and its interrupt inputs. +// Lock order: `order`, then `st`. Packets go to the peer and the PIs with +// `st` dropped; `order` keeps two threads' packets in the order decided. +struct Head { + index: usize, + wiring: Arc, + st: Mutex, + order: Mutex<()>, + /// The head's PCI bus: its functions, BAR windows and interrupt pins. + bus: PciBus, + /// How the board uses each device's slot, by device number. + slots: RwLock<[Option; DEVICES as usize + 1]>, + /// Whole-page BAR mappings of the windows, which the processor reaches + /// without a search of the bus. + pio: PioDecode, + unsupported: Mutex>, + /// This head, for the DMA a function is given: it must not keep the head alive. + me: Weak, +} + +struct SlotDma { + head: Weak, + device: u8, +} + +impl PciDma for SlotDma { + fn dma_read(&self, address: u64, buf: &mut [u8]) -> PciDmaStatus { + match self.head.upgrade() { + Some(head) => head.dma(self.device, address, buf, false), + None => { + buf.fill(0); + PciDmaStatus::MasterAbort + } + } + } + + fn dma_write(&self, address: u64, data: &[u8]) -> PciDmaStatus { + match self.head.upgrade() { + Some(head) => head.dma(self.device, address, &mut data.to_vec(), true), + None => PciDmaStatus::MasterAbort, + } + } +} + +/// What the head's bus asks of it: the DMA of a requester, and an input's +/// level, as the OR of every device line and board interrupt routed to it. +impl PciHost for Head { + fn dma_for(&self, requester: u8) -> Arc { + Arc::new(SlotDma { head: self.me.clone(), device: requester }) + } + + fn set_irq(&self, pin: u8, level: bool) { + self.pin(pin as usize, level); + } +} + +impl Head { + fn new(index: usize, wiring: Arc) -> Arc { + let this = Arc::new_cyclic(|me: &Weak| Self { + index, + wiring, + st: Mutex::new(State::new()), + order: Mutex::new(()), + // iris's bus numbers count per bus tree; the heads are separate trees. + bus: PciBus::new(index as u8 * 16, &PIC_ROUTING, me.clone()), + slots: RwLock::new([None; DEVICES as usize + 1]), + pio: PioDecode::default(), + unsupported: Mutex::new(Vec::new()), + me: me.clone(), + }); + this.bus.set_host_windows(&|| this.windows()); + let head = Arc::downgrade(&this); + this.bus.on_bar_mappings(Box::new(move |runs| { + if let Some(head) = head.upgrade() { + head.publish_mappings(runs); + } + })); + this + } + + fn note_unsupported(&self, addr: u64, size: usize, write: bool, value: u64) { + let mut seen = self.unsupported.lock(); + if seen.len() >= UNSUPPORTED_LIMIT { + return; + } + seen.push((self.index, write, addr, size)); + crate::dlog!( + LogModule::Bedrock, + "pic: head {} {} offset {addr:#x} size {size} value {value:#x} unsupported{}", + self.index, + if write { "write" } else { "read" }, + if seen.len() == UNSUPPORTED_LIMIT { "; further accesses suppressed" } else { "" } + ); + } + + fn source_widget(&self) -> u8 { + (self.st.lock().r(reg::CONTROL) & 0xf) as u8 + } + + // ---- interrupts ---- + + /// Send pin `pin`'s set or clear packet. True when the peer took it. + fn packet(&self, pin: usize, set: bool) -> bool { + let (destination, source) = { + let mut st = self.st.lock(); + if set { + st.set_requests[pin] += 1; + } else { + st.clear_requests[pin] += 1; + } + (st.r(reg::INTERRUPT_HOST_BASE + pin * 8), (st.r(reg::CONTROL) & 0xf) as u8) + }; + let level = destination >> 48; + if !LEVELS.contains(&level) { + crate::dlog!(LogModule::Bedrock, "pic: head {} pin {pin} rejected interrupt level {level:#x}", self.index); + return false; + } + let address = destination & ((1 << PAYLOAD_BITS) - 1); + let payload = level | if set { SET } else { 0 }; + let sent = self.send_interrupt(source, address, payload); + if !sent { + crate::dlog!(LogModule::Bedrock, "pic: head {} interrupt destination {destination:#x} refused", self.index); + } + sent + } + + fn send_interrupt(&self, source: u8, address: u64, payload: u64) -> bool { + match self.wiring.peer.get() { + Some(peer) => peer.interrupt(source, address, payload).is_ok(), + None => { + crate::dlog!(LogModule::Bedrock, "pic: head {} interrupt has no peer", self.index); + false + } + } + } + + /// Tell the Bedrock PI whether this pin's input is active at its + /// destination's level, so it can reassert a level the processor + /// cleared while the input stays high. + fn update_level_source(&self, pin: usize, delivered: bool) { + let (destination, active_pin) = { + let st = self.st.lock(); + let bit = 1u64 << pin; + (st.r(reg::INTERRUPT_HOST_BASE + pin * 8), st.pins as u64 & bit != 0 && st.r(reg::INTERRUPT_ENABLE) & bit != 0) + }; + let level = destination >> 48; + let target = usize::from(destination & (1 << 21) != 0); + let active = delivered && active_pin && LEVELS.contains(&level); + let source = self.wiring.source_base.load(Ordering::Acquire) + self.index * PINS + pin; + if let Some(node) = self.wiring.node.get().and_then(Weak::upgrade) { + for interface in 0..2 { + node.set_level_source(interface, source, level as u8, active && interface == target); + } + } + } + + /// Send pin `pin`'s set packet. The PI hears that the input is active + /// before the packet goes: a processor on another thread can take the + /// level and clear it as the packet lands, and a clear while the source + /// still reads inactive arms no reassertion, so with the input staying + /// high the interrupt would never come back. A refused packet withdraws + /// the source again. + fn assert_pin(&self, pin: usize) { + self.update_level_source(pin, true); + if !self.packet(pin, true) { + self.update_level_source(pin, false); + } + } + + fn pin(&self, pin: usize, level: bool) { + if pin >= PINS { + return; + } + let _order = self.order.lock(); + let bit = 1u8 << pin; + let (was_high, enabled, mode) = { + let mut st = self.st.lock(); + let was_high = st.pins & bit != 0; + st.pins = (st.pins & !bit) | if level { bit } else { 0 }; + if level && !was_high { + let status = st.r(reg::INTERRUPT_STATUS) | bit as u64; + st.set(reg::INTERRUPT_STATUS, status); + } + (was_high, st.r(reg::INTERRUPT_ENABLE) & bit as u64 != 0, st.r(reg::INTERRUPT_MODE) & bit as u64 != 0) + }; + if !level { + self.update_level_source(pin, false); + } + if level && !was_high && enabled { + self.assert_pin(pin); + } + if was_high && !level && mode { + self.packet(pin, false); + } + } + + /// The PROM's write to the link-side error address slot latches the + /// invalid-address condition and, when enabled, sends one packet. + fn invalid_address(&self) { + let _order = self.order.lock(); + let enabled = { + let mut st = self.st.lock(); + let status = st.r(reg::INTERRUPT_STATUS) | INVALID_ADDRESS; + st.set(reg::INTERRUPT_STATUS, status); + st.r(reg::INTERRUPT_ENABLE) & INVALID_ADDRESS != 0 + }; + if enabled { + self.error_interrupt(); + } + } + + /// An unclaimed configuration read: the error view always records the + /// master timeout, and interrupt status only while it is enabled, when + /// it also sends the error interrupt. + fn master_timeout(&self) { + let _order = self.order.lock(); + let enabled = { + let mut st = self.st.lock(); + let view = st.r(ERROR_VIEW) | MASTER_TIMEOUT; + st.set(ERROR_VIEW, view); + let enabled = st.r(reg::INTERRUPT_ENABLE) & MASTER_TIMEOUT != 0; + if enabled { + let status = st.r(reg::INTERRUPT_STATUS) | MASTER_TIMEOUT; + st.set(reg::INTERRUPT_STATUS, status); + } + enabled + }; + if enabled { + self.error_interrupt(); + } + } + + /// One set packet for the error interrupt, to the widget destination. + /// With `order` held. + fn error_interrupt(&self) { + let number = self.st.lock().r(reg::INTERRUPT_ERROR_HOST); + if number > 0x7f { + crate::dlog!(LogModule::Bedrock, "pic: head {} refused error interrupt number {number:#x}", self.index); + return; + } + let Some(address) = self.error_destination() else { return }; + let source = self.source_widget(); + if !self.send_interrupt(source, address, number | SET) { + crate::dlog!(LogModule::Bedrock, "pic: head {} error destination {address:#x} refused", self.index); + } + } + + /// The error interrupt's address from the widget destination pair: a + /// lower register that uses bits 47:32 is the address (the upper half + /// may repeat them), otherwise the pair joins into 48 bits. + fn error_destination(&self) -> Option { + let (upper, lower) = { + let st = self.st.lock(); + (st.r(reg::INT_DEST_UPPER), st.r(reg::INT_DEST_LOWER)) + }; + let upper_address = upper & 0xffff; + let lower_high = lower >> 32; + let refuse = |why: &str| { + crate::dlog!(LogModule::Bedrock, "pic: head {} error destination {why} (upper {upper:#x}, lower {lower:#x})", self.index); + None + }; + if upper >> 32 != 0 { + return refuse("has unsupported high bits"); + } + if lower >> 48 != 0 { + return refuse("exceeds 48 bits"); + } + if upper_address != 0 && lower_high != 0 && upper_address != lower_high { + return refuse("has conflicting address fields"); + } + let address = if lower_high != 0 { lower } else { (upper_address << 32) | (lower & 0xffff_ffff) }; + if address == 0 { + return refuse("is zero"); + } + Some(address) + } + + // ---- registers ---- + + fn identity(&self) -> u64 { + identity(self.index) as u64 + } + + /// The widget address of head address `addr`. + fn widget_address(&self, addr: u64) -> u64 { + self.index as u64 * HEAD_OFFSET + addr + } + + /// `None` is a read the processor gets back as a bus error: only an + /// unclaimed configuration read fails that way. + fn read(&self, addr: u64, size: usize) -> Option { + if short_window(addr).is_some() { + return Some(self.short(self.widget_address(addr), size, None).unwrap_or(0)); + } + if (CONFIG_BASE..CONFIG_END).contains(&addr) && size <= 4 { + return self.config(addr, size, None).ok(); + } + let off = addr as usize; + if size == 4 && off % 8 == 4 { + return Some(self.word_read(off)); + } + if size == 8 && off == 0 { + return Some(self.identity()); + } + if size == 8 && off % 8 == 0 && defined(off) { + return Some(self.st.lock().r(off)); + } + if size == 8 && off % 8 == 0 && (REQUEST_BASE..REQUEST_END).contains(&off) { + let (bank, index) = request_slot(off); + return Some(self.st.lock().request[bank][index]); + } + self.note_unsupported(addr, size, false, 0); + Some(0) + } + + fn write(&self, addr: u64, size: usize, value: u64) { + if short_window(addr).is_some() { + self.short(self.widget_address(addr), size, Some(value)); + return; + } + if addr == ERROR_ADDRESS_WORD && size == 4 { + self.invalid_address(); + return; + } + if (CONFIG_BASE..CONFIG_END).contains(&addr) && size <= 4 { + let _ = self.config(addr, size, Some(value)); + return; + } + let off = addr as usize; + if size == 4 && off % 8 == 4 { + self.word_write(off, value as u32); + return; + } + self.register_write(off, size, value); + } + + /// A 32-bit access to the low half of an eight-byte slot. + fn word_slot(off: usize) -> Option { + let slot = off.checked_sub(4).filter(|slot| slot % 8 == 0)?; + ((REQUEST_BASE..REQUEST_END).contains(&slot) || (defined(slot) && slot < REGISTER_BYTES)).then_some(slot) + } + + fn word_read(&self, off: usize) -> u64 { + match Self::word_slot(off) { + Some(slot) if slot != FLUSH => { + if slot == reg::IDENTIFICATION { + return self.identity(); + } + let st = self.st.lock(); + let full = if slot >= REQUEST_BASE { + let (bank, index) = request_slot(slot); + st.request[bank][index] + } else { + st.r(slot) + }; + full as u32 as u64 + } + _ => { + self.note_unsupported(off as u64, 4, false, 0); + 0 + } + } + } + + fn word_write(&self, off: usize, value: u32) { + let slot = match Self::word_slot(off) { + Some(slot) if !matches!(slot, reg::IDENTIFICATION | FLUSH | reg::INTERRUPT_STATUS | reg::INTERRUPT_DEVICE | ERROR_VIEW) => slot, + _ => { + self.note_unsupported(off as u64, 4, true, value as u64); + return; + } + }; + if slot >= REQUEST_BASE { + let (bank, index) = request_slot(slot); + let entry = &mut self.st.lock().request[bank][index]; + *entry = (*entry & 0xffff_ffff_0000_0000) | value as u64; + } else if slot == reg::INTERRUPT_RESET { + self.register_write(slot, 8, value as u64); + } else { + let merged = (self.st.lock().r(slot) & 0xffff_ffff_0000_0000) | value as u64; + self.register_write(slot, 8, merged); + } + } + + fn register_write(&self, off: usize, size: usize, value: u64) { + if size != 8 || off % 8 != 0 { + self.note_unsupported(off as u64, size, true, value); + return; + } + if (REQUEST_BASE..REQUEST_END).contains(&off) { + let (bank, index) = request_slot(off); + self.st.lock().request[bank][index] = value; + return; + } + if !defined(off) || off >= REGISTER_BYTES { + self.note_unsupported(off as u64, size, true, value); + return; + } + match off { + reg::IDENTIFICATION | FLUSH | reg::INTERRUPT_STATUS | ERROR_VIEW => { + self.note_unsupported(off as u64, size, true, value) + } + reg::CONTROL => { + if value & !CONTROL_MASK != 0 { + self.note_unsupported(off as u64, size, true, value); + } + self.st.lock().set(off, value & CONTROL_MASK); + } + reg::DIRECT_MAP => { + if value & !DIRECT_MAP_SUPPORTED != 0 { + self.note_unsupported(off as u64, size, true, value); + } + self.st.lock().set(off, value & DIRECT_MAP_SUPPORTED); + } + reg::INTERRUPT_ENABLE => { + let _order = self.order.lock(); + let (newly, pins, status) = { + let mut st = self.st.lock(); + let newly = value & !st.r(off); + st.set(off, value); + (newly, st.pins as u64, st.r(reg::INTERRUPT_STATUS)) + }; + for pin in 0..PINS { + if newly & status & pins & (1 << pin) != 0 { + self.assert_pin(pin); + } else if value & (1 << pin) == 0 { + self.update_level_source(pin, false); + } + } + } + reg::INTERRUPT_RESET => { + let _order = self.order.lock(); + let (pins, enable) = { + let mut st = self.st.lock(); + // Each error bit clears on its own one; any write + // clears the view. + let status = (st.r(reg::INTERRUPT_STATUS) & !value) | st.pins as u64; + st.set(reg::INTERRUPT_STATUS, status); + st.set(ERROR_VIEW, 0); + (st.pins as u64, st.r(reg::INTERRUPT_ENABLE)) + }; + for pin in 0..PINS { + if value & pins & enable & (1 << pin) != 0 { + self.assert_pin(pin); + } + } + } + reg::INTERRUPT_DEVICE => { + // The device-to-pin map has no effect here; the board wires + // each device to its pin. + self.note_unsupported(off as u64, size, true, value); + self.st.lock().set(off, value); + } + _ => { + let device = (reg::DEVICE_BASE..=0x218).contains(&off); + let stored = if device { + if value & !DEVICE_MASK != 0 { + self.note_unsupported(off as u64, size, true, value); + } + value & DEVICE_MASK + } else { + value + }; + self.st.lock().set(off, stored); + if device { + self.bus.set_host_windows(&|| self.windows()); + } + if (reg::INTERRUPT_HOST_BASE..=reg::INTERRUPT_HOST_LAST).contains(&off) { + let _order = self.order.lock(); + self.update_level_source((off - reg::INTERRUPT_HOST_BASE) / 8, false); + } + } + } + } + + // ---- PCI configuration ---- + + /// A processor configuration cycle. Bus 0 devices sit at `0x1000` + /// strides from `0x20000`; the type-1 window addresses the bus and + /// device `PCI_TYPE1_CONFIG` names. A read no function claims, device 0 + /// (the bridge's own) and every function but 0 included, is a master + /// timeout and fails; a write nobody claims goes nowhere. + fn config(&self, addr: u64, size: usize, value: Option) -> Result { + let mask = u32::MAX >> (32 - size * 8); + let function = ((addr >> 8) & 7) as u8; + let (bus, device) = if addr < CONFIG_TYPE1 { + (Some(self.bus.clone()), ((addr - CONFIG_BASE) >> 12) as u8) + } else { + let t1 = self.st.lock().r(reg::PCI_TYPE1_CONFIG); + (self.bus.find_bus(((t1 >> 16) & 0xff) as u8), ((t1 >> 11) & 0x1f) as u8) + }; + let register = ((addr & 0xff) as usize) ^ (4 - size); + let at = DevFn { device, function }; + let answered = match value { + Some(value) => bus.is_some_and(|bus| bus.config_write(at, register, size, value as u32 & mask)), + None => { + let read = bus.and_then(|bus| bus.config_read(at, register, size)); + if let Some(read) = read { + return Ok(read as u64); + } + false + } + }; + if !answered { + crate::dlog!( + LogModule::Bedrock, + "pic: head {} configuration {} miss at {addr:#x}", + self.index, + if value.is_some() { "write" } else { "read" } + ); + } + if value.is_some() { + return Ok(0); + } + self.master_timeout(); + Err(Unclaimed) + } + + // ---- PCI memory and I/O cycles ---- + + /// The CPU windows, from the Device registers; see [`SHORT_WINDOWS`]. A + /// short window whose Device register asks for a byte stream or a wide + /// device is one the model refuses, and has no run. + fn windows(&self) -> Vec { + let st = self.st.lock(); + let base = self.index as u64 * HEAD_OFFSET; + (0..=SHORT_WINDOWS) + .map(|kind| { + if kind < SHORT_WINDOWS { + let device = st.r(reg::DEVICE_BASE + kind * 8); + let (space, pci) = Self::device(device, kind); + let (start, len) = SHORT[kind]; + let len = if device & (DEVICE_STREAM | DEVICE_WIDE) != 0 { 0 } else { len }; + CpuWindow { space, cpu: base + start, pci, len, mask: u64::MAX } + } else { + CpuWindow { space: Space::Mem, cpu: PCI_WINDOW[self.index], pci: 0, len: PCI_WINDOW_SIZE, mask: u64::MAX } + } + }) + .collect() + } + + /// Device register `device` of short window `kind`: the space it opens + /// onto and the PCI address of the window's first byte. + fn device(device: u64, kind: usize) -> (Space, u32) { + let memory = device & DEVICE_MEMORY != 0; + let mut base = (device & if memory { 0x3ff } else { 0xfff }) as u32; + if kind < 2 { + base &= !1; + } + (if memory { Space::Mem } else { Space::Io }, base << 20) + } + + /// The byte order of a processor access reaching `f`. A conventional + /// adapter sees a numeric value at the address with its low two bits + /// corrected (MACE's rule); a self-ordered function (IOC4) keeps the + /// processor's byte sequence at the address as it is. The order of a + /// 64-bit access to a conventional adapter is unknown. + fn order(&self, f: &dyn PciFunction) -> ByteOrder { + let same = |g: &Arc| std::ptr::addr_eq(Arc::as_ptr(g), f as *const dyn PciFunction); + let device = (1..=DEVICES).find(|&device| self.bus.function(DevFn { device, function: 0 }).is_some_and(same)); + let native = device.is_some_and(|device| self.slots.read()[device as usize].is_some_and(|slot| slot.self_ordered_pio)); + if native { ByteOrder::Stream } else { ByteOrder::values(4) } + } + + /// The bus reports the BAR mappings the head's windows hold: serve them + /// from the head's own table. + fn publish_mappings(&self, runs: Vec) { + let runs = runs + .iter() + .map(|r| { + let base = r.cpu.wrapping_sub(r.off as u64); + PioRun { + start: r.cpu, + end: r.cpu + r.len, + dev: Arc::new(BarDevice::new(r.function.clone(), r.bar, base, u64::MAX, self.order(&*r.function))), + } + }) + .collect(); + // Replaced runs may run shutdown callbacks from their destructor. + drop(self.pio.publish(runs)); + } + + /// A processor access to the widget address `addr` of a short window or + /// the PCI memory window: a whole BAR page answers it from the head's + /// table, anything else through the bus's decode. Nothing claiming it + /// reads zero and drops a write; the caller says where that is a + /// refusal worth logging. + fn pci_access(&self, addr: u64, size: usize, value: Option) -> Option { + let crossing = addr & 0xfff > 0x1000 - size as u64; + if !crossing { + if let Some(endpoint) = self.pio.target(addr, size) { + return Some(match value { + Some(v) => { + match size { + 1 => endpoint.write8(addr, v as u8), + 2 => endpoint.write16(addr, v as u16), + 4 => endpoint.write32(addr, v as u32), + _ => endpoint.write64(addr, v), + }; + 0 + } + None => match size { + 1 => endpoint.read8(addr).data as u64, + 2 => endpoint.read16(addr).data as u64, + 4 => endpoint.read32(addr).data as u64, + _ => endpoint.read64(addr).data, + }, + }); + } + } + let (space, pci) = self.pci_address(addr)?; + let (function, bar, offset) = self.bus.decode(space, pci)?; + let order = self.order(function); + Some(match value { + None => order.read(function, bar, offset, size), + Some(v) => { + order.write(function, bar, offset, size, v); + 0 + } + }) + } + + /// The space and PCI address the widget address `addr` reaches. + fn pci_address(&self, addr: u64) -> Option<(Space, u32)> { + let rel = addr.checked_sub(self.index as u64 * HEAD_OFFSET); + if let Some((window, offset, _)) = rel.and_then(short_window) { + let (space, base) = Self::device(self.st.lock().r(reg::DEVICE_BASE + window * 8), window); + return Some((space, base.wrapping_add(offset as u32))); + } + let offset = addr.checked_sub(PCI_WINDOW[self.index])?; + (offset < PCI_WINDOW_SIZE).then_some((Space::Mem, offset as u32)) + } + + /// The short device-I/O windows: the Device register of the window + /// picks PCI memory or I/O space and the megabyte of it. + fn short(&self, addr: u64, size: usize, value: Option) -> Option { + let write = value.is_some(); + let (window, offset, window_size) = short_window(addr % HEAD_OFFSET)?; + if size as u64 > window_size - offset { + crate::dlog!(LogModule::Bedrock, "pic: head {} short window {addr:#x} rejects boundary-crossing {size}-byte access", self.index); + return None; + } + let device = self.st.lock().r(reg::DEVICE_BASE + window * 8); + let refused = device & (DEVICE_STREAM | DEVICE_WIDE); + if refused != 0 { + crate::dlog!( + LogModule::Bedrock, + "pic: head {} short window {addr:#x} refuses unsupported Device attributes {refused:#x}", + self.index + ); + return None; + } + let result = self.pci_access(addr, size, value); + if result.is_none() { + crate::dlog!( + LogModule::Bedrock, + "pic: head {} short window {addr:#x} PCI {} failed for {size} bytes", + self.index, + if write { "write" } else { "read" } + ); + } + result + } + + /// The large PCI memory window of this head. + fn pci_window(&self, addr: u64, size: usize, value: Option) -> u64 { + let result = self.pci_access(addr, size, value); + if result.is_none() { + self.note_unsupported(addr, size, value.is_some(), value.unwrap_or(0)); + } + result.unwrap_or(0) + } + + // ---- DMA ---- + + fn dma(&self, device: u8, address: u64, bytes: &mut [u8], write: bool) -> PciDmaStatus { + if !(1..=DEVICES).contains(&device) { + crate::dlog!(LogModule::Bedrock, "pic: head {} unsupported requester device {device}", self.index); + if !write { + bytes.fill(0); + } + return PciDmaStatus::MasterAbort; + } + let mut done = 0; + while done < bytes.len() { + let at = address.wrapping_add(done as u64); + let n = (0x1000 - (at as usize & 0xfff)).min(bytes.len() - done); + let ok = address.checked_add(done as u64).is_some() && self.dma_page(device, at, &mut bytes[done..done + n], write); + if !ok { + if !write { + bytes[done..].fill(0); + } + return PciDmaStatus::MasterAbort; + } + done += n; + } + PciDmaStatus::Ok + } + + fn dma_page(&self, device: u8, at: u64, chunk: &mut [u8], write: bool) -> bool { + if at > u32::MAX as u64 { + return self.direct64(device, at, chunk, write); + } + if at < 0x8000_0000 { + crate::dlog!(LogModule::Bedrock, "pic: head {} unsupported DMA {at:#x} from device {device}", self.index); + return false; + } + self.direct32(device, at, chunk, write) + } + + fn send_dma(&self, source: u8, widget: u8, address: u64, data: &mut [u8], write: bool) -> bool { + let Some(peer) = self.wiring.peer.get() else { + crate::dlog!(LogModule::Bedrock, "pic: head {} DMA has no peer", self.index); + return false; + }; + if write { + peer.dma_write(source, widget, address, data).is_ok() + } else { + peer.dma_read(source, widget, address, data).is_ok() + } + } + + /// The direct map: PCI `0x80000000` and up reach the widget `DIR_MAP` + /// names, at the address its high bits and the PCI offset make. + fn direct32(&self, device: u8, at: u64, chunk: &mut [u8], write: bool) -> bool { + let (direct, attributes, source) = { + let st = self.st.lock(); + (st.r(reg::DIRECT_MAP), st.r(reg::DEVICE_BASE + (device as usize - 1) * 8), (st.r(reg::CONTROL) & 0xf) as u8) + }; + let widget = ((direct >> 20) & 0xf) as u8; + let target = ((direct & 0x1ffff) << 31) | (at & 0x7fff_ffff); + if attributes & DEVICE_SWAP != 0 { + // A swapped slot puts each byte at its address with the low two + // bits flipped. + return self.send_flipped(source, widget, target, chunk, write, 3, |_, _| true); + } + let mut done = 0; + while done < chunk.len() { + let n = natural(target + done as u64, chunk.len() - done); + if !self.send_dma(source, widget, target + done as u64, &mut chunk[done..done + n], write) { + return false; + } + done += n; + } + true + } + + /// Why a direct64 request cannot be carried, if it cannot. It is + /// checked in full before any packet leaves. + fn direct64_refusal(address: u64, size: usize) -> Option<&'static str> { + let payload = address & ((1 << PAYLOAD_BITS) - 1); + if address >> 60 == 0 { + Some("target-zero") + } else if address & ((1 << 54) | (0x3f << 48)) != 0 { + Some("reserved-bits") + } else if payload & (1 << 47) != 0 { + Some("register-space") + } else if size as u64 > (1 << 47) - payload { + Some("range") + } else { + None + } + } + + /// A PCI address above 32 bits carries the target widget in bits 63:60 + /// and the XIO payload below. Bit 55 selects an ordered byte stream; + /// clear, the bytes keep 32-bit numeric values (the byte address low + /// bits flip by 3, or by 7 for a REQ64 slot's 64-bit requester). + fn direct64(&self, device: u8, at: u64, chunk: &mut [u8], write: bool) -> bool { + let wide = self.slots.read()[device as usize].is_some_and(|slot| slot.req64); + let source = self.source_widget(); + let check = |address: u64, size: usize| match Self::direct64_refusal(address, size) { + Some(why) => { + crate::dlog!(LogModule::Bedrock, "pic: head {} direct64 {why} address {address:#018x} from device {device}", self.index); + false + } + None => true, + }; + if !check(at, 1) { + return false; + } + let widget = (at >> 60) as u8; + let payload = at & ((1 << PAYLOAD_BITS) - 1); + if at & (1 << 55) == 0 { + let mask = if wide { 7 } else { 3 }; + return self.send_flipped(source, widget, payload, chunk, write, mask, |offset, n| check(at + offset, n)); + } + let mut done = 0; + while done < chunk.len() { + let n = 8.min(chunk.len() - done); + if !check(at + done as u64, n) || !self.send_dma(source, widget, payload + done as u64, &mut chunk[done..done + n], write) { + return false; + } + done += n; + } + true + } + + /// Carry a transfer whose bytes land at their addresses with the low + /// bits in `mask` flipped, which reverses the bytes of each aligned + /// group of `mask + 1`. The flip moves bytes within a group, not the + /// transfer apart: a run of whole groups goes as one packet, so a + /// processor reading a word the device is updating sees all of it old + /// or all of it new, as it would of the burst on the bus. Only a group + /// the transfer covers in part moves byte by byte. `fits` vets each + /// packet by its offset in `chunk` and its length before it leaves. + #[allow(clippy::too_many_arguments)] + fn send_flipped( + &self, + source: u8, + widget: u8, + address: u64, + chunk: &mut [u8], + write: bool, + mask: u64, + fits: impl Fn(u64, usize) -> bool, + ) -> bool { + let group = mask as usize + 1; + let mut done = 0; + while done < chunk.len() { + let here = address + done as u64; + let whole = if here & mask == 0 { (chunk.len() - done) / group * group } else { 0 }; + if whole == 0 { + if !fits(done as u64, 1) || !self.send_dma(source, widget, here ^ mask, &mut chunk[done..done + 1], write) { + return false; + } + done += 1; + continue; + } + let run = &mut chunk[done..done + whole]; + let mut flipped = run.to_vec(); + if write { + flipped.chunks_mut(group).for_each(<[u8]>::reverse); + } + if !fits(done as u64, whole) || !self.send_dma(source, widget, here, &mut flipped, write) { + return false; + } + if !write { + flipped.chunks_mut(group).for_each(<[u8]>::reverse); + run.copy_from_slice(&flipped); + } + done += whole; + } + true + } + + fn attach(&self, device: u8, function: Arc, options: Function) { + assert!((1..=DEVICES).contains(&device), "PIC devices are 1 to {DEVICES}"); + self.slots.write()[device as usize] = Some(options); + self.bus.attach(DevFn { device, function: 0 }, function); + } + + fn reset(&self) { + { + let _order = self.order.lock(); + self.st.lock().reset(); + } + for pin in 0..PINS { + self.update_level_source(pin, false); + } + // RST#: headers, then functions, which drop their own lines. The + // Device registers are back at reset, so the windows move too. + self.bus.reset(); + self.bus.set_host_windows(&|| self.windows()); + self.unsupported.lock().clear(); + } +} + +/// The PCI host bridge in one XIO widget. +pub struct Pic { + widget: u8, + wiring: Arc, + heads: [Arc; 2], + pxbow: OnceLock>, +} + +impl Pic { + /// A PIC that answers as XIO widget `widget`. + pub fn new(widget: u8) -> Arc { + let wiring = Arc::new(Wiring { + peer: OnceLock::new(), + node: OnceLock::new(), + source_base: AtomicUsize::new(0), + }); + Arc::new(Self { + widget, + heads: [Head::new(0, wiring.clone()), Head::new(1, wiring.clone())], + wiring, + pxbow: OnceLock::new(), + }) + } + + pub fn widget_number(&self) -> u8 { + self.widget + } + + /// The widget as the processor reaches it: both heads, the short + /// device-I/O windows and the PCI memory windows. Bedrock IO takes this. + pub fn widget(self: &Arc) -> Arc { + Arc::new(CpuWidget { heads: self.heads.clone() }) + } + + /// Give the chip its PXBOW, the widget-0 face of the board that owns + /// it, reached from `host_widget`. On Tezro the PIC that carries IO9 + /// supplies widget 0; a second PIC gets none. The PIC's own widget is + /// attached behind it, and widget 0's interrupt goes to the peer when it + /// targets the host. Once, before the machine runs. + pub fn enable_pxbow(self: &Arc, host_widget: u8, clock: Clock) -> Result, AttachError> { + let pxbow = Pxbow::new(host_widget, self.widget, clock)?; + pxbow.attach_endpoint(self.widget, self.widget())?; + pxbow.connect_host(Arc::new(PxbowPackets(self.wiring.clone()))); + self.pxbow.set(pxbow.clone()).map_err(|_| AttachError::Occupied(self.widget))?; + Ok(pxbow) + } + + /// The chip's PXBOW, if it has one. + pub fn pxbow(&self) -> Option<&Arc> { + self.pxbow.get() + } + + /// Where interrupts and DMA this chip originates go. Once, before the + /// machine runs. + pub fn connect_peer(&self, peer: Arc) -> Result<(), AlreadyConnected> { + self.wiring.peer.set(peer).map_err(|_| AlreadyConnected) + } + + /// Connect the Bedrock whose PIs this chip's interrupts reach, once. + /// `source_base` is the first of the 16 level sources the chip owns on + /// each PI; two PICs on one node need different bases. + pub fn connect_node(&self, node: &Arc, source_base: usize) { + self.wiring.source_base.store(source_base, Ordering::Release); + let _ = self.wiring.node.set(Arc::downgrade(node)); + } + + /// Put a PCI function at configuration `device` (1 to 4) of `head`'s + /// bus. It gets its DMA path and, if its header names an interrupt pin, + /// the line of input `device - 1`. + pub fn attach(&self, head: usize, device: u8, function: Arc, options: Function) { + self.heads[head].attach(device, function, options); + } + + /// The PCI bus of `head`. + pub fn bus(&self, head: usize) -> &PciBus { + &self.heads[head].bus + } + + /// Interrupt input `pin` (0 to 7) of `head`, for a device whose + /// interrupt the board wires to it directly. + pub fn pin_line(&self, head: usize, pin: usize) -> IrqLine { + assert!(pin < PINS); + self.heads[head].bus.board_irq(BoardIrq(pin as u8)) + } + + /// Set interrupt input `pin` of `head`. + pub fn set_pin(&self, head: usize, pin: usize, level: bool) { + self.pin_line(head, pin).set(level); + } + + /// The set and clear packets `head` has requested for `pin`, delivered + /// or not. + pub fn request_counts(&self, head: usize, pin: usize) -> (u32, u32) { + let st = self.heads[head].st.lock(); + (st.set_requests[pin], st.clear_requests[pin]) + } + + /// Accesses the model refused or could not carry out, as (head, write, + /// offset, size); the first 32 per head. + pub fn unsupported(&self) -> Vec<(usize, bool, u64, usize)> { + self.heads.iter().flat_map(|head| head.unsupported.lock().clone()).collect() + } + + pub fn register_locks(self: &Arc) { + use crate::locks::register_lock_fn; + for head in &self.heads { + for (what, probe) in [ + ("state", (|h: &Head| h.st.is_locked()) as fn(&Head) -> bool), + ("irq_order", |h| h.order.is_locked()), + ] { + let h = Arc::downgrade(head); + register_lock_fn(format!("pic{:x}::head{}::{what}", self.widget, head.index), move || { + h.upgrade().is_some_and(|h| probe(&h)) + }); + } + } + if let Some(pxbow) = self.pxbow.get() { + pxbow.xbow().register_locks(); + } + } +} + +impl Resettable for Pic { + fn power_on(&self) { + if let Some(pxbow) = self.pxbow.get() { + pxbow.xbow().power_on(); + } + for head in &self.heads { + head.reset(); + } + } +} + +/// The processor's view of the widget. +struct CpuWidget { + heads: [Arc; 2], +} + +impl CpuWidget { + /// `None` is a bus error: a malformed access or an unclaimed + /// configuration read. Accesses the PIC cannot make sense of that are + /// not malformed (unsupported registers, refused windows) read zero and + /// write nowhere without one. + fn access(&self, addr: u64, size: usize, value: Option) -> Option { + let write = value.is_some(); + let short = addr < 2 * HEAD_OFFSET && short_window(addr % HEAD_OFFSET).is_some(); + if !short && !addr.is_multiple_of(size as u64) { + return None; + } + if addr >= 2 * HEAD_OFFSET { + if addr < 2 * HEAD_OFFSET + 8 { + crate::dlog!(LogModule::Bedrock, "pic: short-window guard {} at {addr:#x} refused for {size} bytes", if write { "write" } else { "read" }); + return Some(0); + } + let window = PCI_WINDOW.iter().position(|&base| (base..base + PCI_WINDOW_SIZE).contains(&addr))?; + return Some(self.heads[window].pci_window(addr, size, value)); + } + let head = &self.heads[(addr / HEAD_OFFSET) as usize]; + let rel = addr % HEAD_OFFSET; + match value { + Some(v) => { + head.write(rel, size, v); + Some(0) + } + None => head.read(rel, size), + } + } +} + +macro_rules! cpu_access { + ($r:ident, $w:ident, $t:ty, $out:ident, $n:expr) => { + fn $r(&self, addr: u64) -> $out { + self.access(addr, $n, None).map_or_else($out::err, |v| $out::ok(v as $t)) + } + fn $w(&self, addr: u64, value: $t) -> u32 { + if self.access(addr, $n, Some(value as u64)).is_some() { BUS_OK } else { BUS_ERR } + } + }; +} + +impl BusDevice for CpuWidget { + cpu_access!(read8, write8, u8, BusRead8, 1); + cpu_access!(read16, write16, u16, BusRead16, 2); + cpu_access!(read32, write32, u32, BusRead32, 4); + cpu_access!(read64, write64, u64, BusRead64, 8); +} + +#[cfg(test)] +mod tests; diff --git a/src/dev/pic/tests.rs b/src/dev/pic/tests.rs new file mode 100644 index 000000000..e808b91f6 --- /dev/null +++ b/src/dev/pic/tests.rs @@ -0,0 +1,1104 @@ +// Cases follow tests/qtest/sgi-pic-test.c in jamesbraid/qemu-sgi-origami, for +// the Tezro runtime profile only: identity-fields, runtime-identity-bit, +// heads-and-widths, runtime-reset-compatibility-trial, widget-registers, +// cpu-word-view-registers, runtime-host-error-field, +// cpu-pio-overlapping-bars-use-active-function, +// runtime-dir-map-unsupported-fields, configuration, slot-bounds, +// runtime-profile-reserved-device-and-dma, type1-configuration, +// cpu-pio-conventional-lanes, cpu-pio-actual-ioc4, +// short-device-io-windows, short-device-io-attributes-boundaries, +// direct-dma, direct-dma-head-targets, direct64-card-write, +// direct64-requester-width, direct64-rejects-unsupported-addresses, +// direct64-range-boundary-is-split, direct64-target-error-preserves-data, +// pin-and-reset, remaining-pin-edges, bedrock-interrupt-level-bounds, +// runtime-profile-bedrock-level-mirror, runtime-profile-bedrock-rearm-one-level, +// runtime-profile-bedrock-rearm-staggered-levels, +// runtime-invalid-address-error-trial and unwired-pci-interrupt. The qtest +// fixtures' packet face, translated DMA, migration streams and profile +// selection have no counterpart here. +use super::*; +use crate::dev::bedrock::{reg::{pi as pi_reg, PI1_BASE}, BedrockConfig}; +use crate::irq::LevelProbe; +use crate::traits::Device; +use crate::pci::{Bar, PciConfig, PciFunctionState, PCI_INTA}; +use std::collections::HashMap; +use std::sync::atomic::{AtomicBool, AtomicU64}; + +const HEAD1: u64 = HEAD_OFFSET; +const BAR_BASE: u32 = 0x0100_0000; +const MEMORY_AND_MASTER: u32 = 6; + +/// A PCI function with one 4 KiB memory BAR that stores what it is given. +struct Card { + state: PciFunctionState, + bar: Mutex<[u8; 0x1000]>, +} + +impl Card { + fn new(device: u16) -> Arc { + let bars = [Bar::Mem(0x1000), Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused]; + Arc::new(Self { + state: PciFunctionState::new(PciConfig::new(0x1234, device, 0x028000, 1, bars, PCI_INTA)), + bar: Mutex::new([0; 0x1000]), + }) + } + + fn dma(&self) -> &Arc { + self.state.dma().expect("attached to a bridge") + } + + fn raise(&self, level: bool) { + self.state.irq().expect("attached to a bridge").set(level); + } +} + +impl PciFunction for Card { + fn state(&self) -> &PciFunctionState { + &self.state + } + + fn bar_read(&self, _bar: usize, off: u32, width: usize) -> u32 { + le_value(&self.bar.lock()[off as usize..off as usize + width]) as u32 + } + + fn bar_write(&self, _bar: usize, off: u32, val: u32, width: usize) { + self.bar.lock()[off as usize..off as usize + width].copy_from_slice(&val.to_le_bytes()[..width]); + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +struct Dma { + source: u8, + widget: u8, + address: u64, + len: usize, + write: bool, +} + +/// The link's far end: sparse memory per widget, and a log of what arrived. +#[derive(Default)] +struct Peer { + interrupts: Mutex>, + dma: Mutex>, + memory: Mutex>, + refuse: AtomicBool, +} + +impl Peer { + fn byte(&self, widget: u8, address: u64) -> u8 { + self.memory.lock().get(&(widget, address)).copied().unwrap_or(0) + } + + fn bytes(&self, widget: u8, address: u64, len: usize) -> Vec { + (0..len as u64).map(|i| self.byte(widget, address + i)).collect() + } + + fn put(&self, widget: u8, address: u64, data: &[u8]) { + for (i, b) in data.iter().enumerate() { + self.memory.lock().insert((widget, address + i as u64), *b); + } + } +} + +impl PicPeer for Peer { + fn interrupt(&self, source_widget: u8, address: u64, payload: u64) -> Result<(), Refused> { + if self.refuse.load(Ordering::SeqCst) { + return Err(Refused); + } + self.interrupts.lock().push((source_widget, address, payload)); + Ok(()) + } + + fn dma_read(&self, source: u8, widget: u8, address: u64, data: &mut [u8]) -> Result<(), Refused> { + self.dma.lock().push(Dma { source, widget, address, len: data.len(), write: false }); + if self.refuse.load(Ordering::SeqCst) { + return Err(Refused); + } + for (i, b) in data.iter_mut().enumerate() { + *b = self.byte(widget, address + i as u64); + } + Ok(()) + } + + fn dma_write(&self, source: u8, widget: u8, address: u64, data: &[u8]) -> Result<(), Refused> { + self.dma.lock().push(Dma { source, widget, address, len: data.len(), write: true }); + if self.refuse.load(Ordering::SeqCst) { + return Err(Refused); + } + self.put(widget, address, data); + Ok(()) + } +} + +struct Rig { + pic: Arc, + bus: Arc, + peer: Arc, +} + +fn rig() -> Rig { + let pic = Pic::new(15); + let peer = Arc::new(Peer::default()); + pic.connect_peer(peer.clone()).unwrap(); + Rig { bus: pic.widget(), pic, peer } +} + +impl Rig { + fn r64(&self, addr: u64) -> u64 { + let r = self.bus.read64(addr); + assert!(r.is_ok(), "read64 {addr:#x}"); + r.data + } + + fn r32(&self, addr: u64) -> u32 { + let r = self.bus.read32(addr); + assert!(r.is_ok(), "read32 {addr:#x}"); + r.data + } + + fn w64(&self, addr: u64, v: u64) { + assert_eq!(self.bus.write64(addr, v), BUS_OK, "write64 {addr:#x}"); + } + + fn w32(&self, addr: u64, v: u32) { + assert_eq!(self.bus.write32(addr, v), BUS_OK, "write32 {addr:#x}"); + } + + /// Fit `card` at `device` of `head` and map its BAR at `BAR_BASE`. + fn fit(&self, head: usize, device: u8, card: &Arc, options: Function) { + self.pic.attach(head, device, card.clone(), options); + let cfg = config(head, device as u64); + self.w32(cfg + 0x10, BAR_BASE); + self.w32(cfg + 4, MEMORY_AND_MASTER); + } +} + +fn head_base(head: usize) -> u64 { + head as u64 * HEAD_OFFSET +} + +fn config(head: usize, device: u64) -> u64 { + head_base(head) + CONFIG_BASE + device * 0x1000 +} + +/// The processor address of byte `off` of a card's BAR through the large +/// PCI memory window. +fn window(head: usize, off: u64) -> u64 { + PCI_WINDOW[head] + BAR_BASE as u64 + off +} + +fn decode(word: u64, head: usize) { + assert_eq!((word >> 28) & 0xf, 3, "revision"); + assert_eq!((word >> 12) & 0xffff, [0xd102, 0xd112][head], "part"); + assert_eq!((word & 0xffe) >> 1, 0x24, "manufacturer, packed one bit up"); + assert_eq!(word & 1, 1, "bit 0, which the PROM's identity check needs"); +} + +/// One of a Bedrock's PIs, through the chip's register window. +#[derive(Clone)] +struct Pi { + chip: Arc, + regs: Arc, + base: u64, +} +impl Pi { + fn read64(&self, at: u64) -> BusRead64 { + self.regs.read64(self.base + at) + } + fn write64(&self, at: u64, value: u64) -> u32 { + self.regs.write64(self.base + at, value) + } + /// The chip's other PI. + fn other(&self) -> Self { + Self { base: PI1_BASE - self.base, ..self.clone() } + } +} + +/// A Bedrock's PI0, which the PIC raises its levels on, and the chip's clock. +fn pi_with_clock() -> (Pi, Arc) { + let now = Arc::new(AtomicU64::new(0)); + let clock = now.clone(); + let chip = Bedrock::new(BedrockConfig { + name: "test", + clock: Arc::new(move || clock.load(Ordering::SeqCst)), + processors: Default::default(), + banks_mib: &[64], + events: None, + }) + .unwrap(); + chip.start(); + let regs = chip.registers(); + (Pi { chip, regs, base: 0 }, now) +} + +/// The destination a pin's packet goes to: `level` in bits 63:48 and an +/// address in the PI's window. +fn destination(level: u64) -> u64 { + (level << 48) | 0x1800090 +} + +/// The PIC's identity is the one thing the PROM checks before it trusts the +/// bridge, and each head has its own. +#[test] +fn each_head_has_its_own_identity_at_both_read_widths() { + let r = rig(); + for head in 0..2 { + decode(r.r64(head_base(head)), head); + assert_eq!(r.r32(head_base(head) + 4) as u64, r.r64(head_base(head))); + assert_eq!(identity(head) as u64, r.r64(head_base(head))); + } + assert_ne!(r.r64(0), r.r64(HEAD1)); + r.w64(0x20, 5); + r.pic.power_on(); + decode(r.r64(0), 0); + assert_eq!(r.r64(0x20), 0x0f00_320f, "reset restores control but not identity edits"); +} + +#[test] +fn reset_restores_the_selected_control_and_device_values() { + let r = rig(); + for head in 0..2 { + assert_eq!(r.r64(head_base(head) + 0x20), 0x0f00_320f); + for (i, v) in [0x18001000, 0x18001002, 0x18001004, 0x18001005].into_iter().enumerate() { + assert_eq!(r.r64(head_base(head) + 0x200 + i as u64 * 8), v); + } + assert_eq!(r.r64(head_base(head) + 0x28), 0xfffff); + } + // The low word of control merges into the register, per head. + r.w32(0x24, 0x0f00_420f); + assert_eq!(r.r32(0x24), 0x0f00_420f); + assert_eq!(r.r32(HEAD1 + 0x24), 0x0f00_320f); + r.w64(HEAD1 + 0x210, 0x1858_1007); + assert_eq!(r.r64(HEAD1 + 0x210), 0x1858_1007); + for head in 0..2 { + for i in 0..4 { + r.w64(head_base(head) + 0x200 + i * 8, 0xdead_0000 + head as u64 * 0x100 + i); + } + } + r.pic.power_on(); + for head in 0..2 { + assert_eq!(r.r64(head_base(head) + 0x20), 0x0f00_320f); + assert_eq!(r.r64(head_base(head) + 0x200), 0x18001000); + assert_eq!(r.r64(head_base(head) + 0x218), 0x18001005); + } +} + +/// The PROM writes the widget registers a word at a time through the low +/// half of each slot; a full doubleword write is the other form. +#[test] +fn widget_registers_take_full_and_low_word_writes_on_each_head() { + let r = rig(); + for head in 0..2 { + let base = head_base(head); + for (n, seed) in [0xfffff_u64, 0, 0].into_iter().enumerate() { + let slot = base + 0x28 + n as u64 * 8; + let initial = (0x1234_5678_0000_0000 | seed) ^ ((head as u64) << 40); + let low = if n == 1 && head == 1 { 0x5a5a_5a5a } else { 0xa5a5_a5a5 }; + assert_eq!(r.r64(slot), seed); + r.w64(slot, initial); + assert_eq!(r.r64(slot), initial); + r.w32(slot + 4, low); + assert_eq!(r.r32(slot + 4), low); + assert_eq!(r.r64(slot), (initial & 0xffff_ffff_0000_0000) | low as u64); + r.w64(slot, initial ^ 0x55aa_55aa); + // Other widths are refused and change nothing. + assert_eq!(r.bus.write8(slot, 0x7e), BUS_OK); + assert_eq!(r.bus.write16(slot, 0x7e5a), BUS_OK); + r.w32(slot, 0x7e5a_3c1d); + assert_eq!(r.bus.read8(slot).data, 0); + assert_eq!(r.bus.read16(slot).data, 0); + assert_eq!(r.r32(slot), 0); + assert_eq!(r.r64(slot), initial ^ 0x55aa_55aa); + } + } + assert!(r.pic.unsupported().iter().any(|&(_, write, off, size)| write && off == 0x28 && size == 1)); + r.pic.power_on(); + for head in 0..2 { + assert_eq!(r.r64(head_base(head) + 0x28), 0xfffff); + assert_eq!(r.r64(head_base(head) + 0x30), 0); + assert_eq!(r.r32(head_base(head) + 0x2c), 0xfffff); + } +} + +#[test] +fn the_host_error_field_has_a_low_word_view() { + let r = rig(); + for head in 0..2u64 { + let base = head * HEAD_OFFSET; + assert_eq!(r.r64(base + 0x128), 0); + r.w64(base + 0x128, 0x1234_5678_0000_0000 | (head + 1)); + assert_eq!(r.r32(base + 0x12c) as u64, head + 1); + r.w32(base + 0x12c, 0x55 + head as u32); + assert_eq!(r.r64(base + 0x128), 0x1234_5678_0000_0000 | (0x55 + head)); + assert_eq!(r.r64(base + 0x100), 0, "the write raised no status"); + } + r.pic.power_on(); + assert_eq!(r.r64(0x128), 0); +} + +#[test] +fn dir_map_keeps_only_the_implemented_bits() { + let r = rig(); + for head in 0..2 { + let at = head_base(head) + 0x80; + r.w64(at, 0x1ffff | (0xf << 20)); + assert_eq!(r.r64(at), 0x1ffff | (0xf << 20)); + let before = r.pic.unsupported().len(); + r.w64(at, (1 << 17) | (1 << 40) | 0x1234); + assert_eq!(r.r64(at), 0x1234, "unsupported bits clear and the rest stay"); + assert!(r.pic.unsupported().len() > before, "the refusal is observable"); + } +} + +/// Device 0 is the bridge and answers nothing; the function at device `n` +/// is `n` windows up, and every miss reads all ones. +#[test] +fn configuration_cycles_reach_the_function_one_window_above_its_device() { + let r = rig(); + let a = Card::new(0x5001); + let b = Card::new(0x5002); + r.pic.attach(0, 1, a.clone(), Function::default()); + r.pic.attach(0, 3, b.clone(), Function::default()); + assert_eq!(r.r32(config(0, 1)), 0x5001_1234); + assert_eq!(r.r32(config(0, 3)), 0x5002_1234); + assert!(!r.bus.read32(config(1, 1)).is_ok(), "head 1 has its own bus"); + for device in [0, 2, 4, 5, 7] { + assert!(!r.bus.read32(config(0, device)).is_ok(), "device {device}"); + r.w32(config(0, device) + 4, 7); + } + // Bytes and halfwords of the big-endian processor land on the right + // little-endian bytes of the register. + assert_eq!(r.bus.read16(config(0, 1)).data, 0x5001, "offset 0 is the register's upper half"); + assert_eq!(r.bus.read16(config(0, 1) + 2).data, 0x1234); + assert_eq!(r.bus.read8(config(0, 1)).data, 0x50); + assert_eq!(r.bus.read8(config(0, 1) + 3).data, 0x34); + // Only function 0 answers. + assert!(!r.bus.read32(config(0, 1) + 0x100).is_ok()); + r.w32(config(0, 1) + 4, MEMORY_AND_MASTER); + assert_eq!(r.r32(config(0, 1) + 4) & 0xffff, MEMORY_AND_MASTER); + r.w32(config(0, 1) + 0x10, BAR_BASE); + assert_eq!(r.r32(config(0, 1) + 0x10), BAR_BASE); + assert_eq!(a.state.config().bar_base(0), BAR_BASE); +} + +#[test] +fn type_one_configuration_selects_the_bus_and_device_through_the_register() { + let r = rig(); + let a = Card::new(0x5001); + r.pic.attach(0, 2, a, Function::default()); + r.w64(0xc8, 2 << 11); + assert_eq!(r.r32(CONFIG_TYPE1), 0x5001_1234); + r.w64(0xc8, (1 << 16) | (2 << 11)); + assert!(!r.bus.read32(CONFIG_TYPE1).is_ok(), "a subordinate bus has nothing"); + r.w64(0xc8, 3 << 11); + assert!(!r.bus.read32(CONFIG_TYPE1).is_ok()); + r.w64(0xc8, 2 << 11); + assert_eq!(r.r32(CONFIG_TYPE1 + 8), 0x0280_0001, "class and revision"); +} + +/// The processor's byte lanes: an adapter wired little-endian sees the +/// numeric value of a word, and a halfword at offset 2 reaches offset 0. +#[test] +fn a_conventional_function_sees_cpu_pio_with_its_address_corrected() { + let r = rig(); + let card = Card::new(0x6001); + r.fit(0, 1, &card, Function::default()); + r.w32(window(0, 0x10), 0x1234_5678); + assert_eq!(&card.bar.lock()[0x10..0x14], [0x78, 0x56, 0x34, 0x12]); + assert_eq!(r.r32(window(0, 0x10)), 0x1234_5678); + assert_eq!(r.bus.write16(window(0, 0x22), 0x9abc), BUS_OK); + assert_eq!(&card.bar.lock()[0x20..0x22], [0xbc, 0x9a], "offset 2 reaches offset 0"); + assert_eq!(r.bus.read16(window(0, 0x22)).data, 0x9abc); + assert_eq!(r.bus.write8(window(0, 0x33), 0x5a), BUS_OK); + assert_eq!(card.bar.lock()[0x30], 0x5a, "byte offset 3 reaches offset 0"); + assert_eq!(r.bus.read8(window(0, 0x33)).data, 0x5a); + assert_eq!(r.bus.read8(window(0, 0x30)).data, 0, "offset 0 is a different byte"); + // The same card through the short window of head 0, memory space. + r.w64(0x200, (1 << 12) | (BAR_BASE >> 20) as u64); + r.w32(SHORT_BASE + 0x40, 0xcafe_f00d); + assert_eq!(&card.bar.lock()[0x40..0x44], [0x0d, 0xf0, 0xfe, 0xca]); + assert_eq!(r.r32(SHORT_BASE + 0x40), 0xcafe_f00d); +} + +/// IOC4 keeps the processor's byte sequence and the address as written. +#[test] +fn a_self_ordered_function_keeps_the_processors_byte_sequence() { + let r = rig(); + let card = Card::new(0x6002); + r.fit(1, 1, &card, Function { self_ordered_pio: true, req64: false }); + r.w32(window(1, 0x10), 0x1234_5678); + assert_eq!(&card.bar.lock()[0x10..0x14], [0x12, 0x34, 0x56, 0x78]); + assert_eq!(r.r32(window(1, 0x10)), 0x1234_5678); + assert_eq!(r.bus.write8(window(1, 0x21), 0xa5), BUS_OK); + assert_eq!(card.bar.lock()[0x21], 0xa5, "no address correction"); + assert_eq!(r.bus.read8(window(1, 0x21)).data, 0xa5); + assert_eq!(r.bus.write16(window(1, 0x22), 0x9abc), BUS_OK); + assert_eq!(&card.bar.lock()[0x22..0x24], [0x9a, 0xbc]); +} + +/// A short window follows its Device register: space, megabyte and size +/// limit, and it refuses the attributes it does not model. +#[test] +fn short_windows_follow_their_device_registers() { + let r = rig(); + let card = Card::new(0x6003); + r.fit(0, 1, &card, Function::default()); + // Window 0 and 1 are 2 MiB and ignore the low bit of the base; windows + // 2 and 3 are 1 MiB. + for (window_index, cpu) in [(0u64, 0x20_0000u64), (1, 0x40_0000), (2, 0x60_0000), (3, 0x70_0000)] { + // The two big windows ignore the base's low bit; the small ones keep it. + let base = (BAR_BASE >> 20) as u64 | if window_index < 2 { 1 } else { 0 }; + r.w64(0x200 + window_index * 8, (1 << 12) | base); + card.bar.lock().fill(0); + r.w32(cpu + 0x80, 0x1111_1111 * (window_index as u32 + 1)); + assert_eq!(le_value(&card.bar.lock()[0x80..0x84]), 0x1111_1111 * (window_index + 1), "window {window_index}"); + } + // The last word of a window fits; a doubleword across its end does not. + r.w32(0x70_0000 + 0xf_fffc, 7); + let before = r.pic.unsupported().len(); + assert_eq!(r.bus.write64(0x70_0000 + 0xf_fffc, 7), BUS_OK); + assert_eq!(r.r64(0x70_0000 + 0xf_fffc), 0, "a boundary-crossing access moves nothing"); + assert_eq!(r.pic.unsupported().len(), before, "it is logged, not listed as unsupported"); + // Swapped and wide programmed I/O are refused rather than guessed. + r.w64(0x200, (1 << 12) | (BAR_BASE >> 20) as u64 | (1 << 13)); + card.bar.lock().fill(0); + r.w32(SHORT_BASE + 0x40, 0xffff_ffff); + assert_eq!(card.bar.lock()[0x40], 0); + r.w64(0x200, (1 << 12) | (BAR_BASE >> 20) as u64 | (1 << 22)); + assert_eq!(r.r32(SHORT_BASE + 0x40), 0); + assert_eq!(card.bar.lock()[0x40], 0); +} + +#[test] +fn the_guard_after_the_second_head_reads_zero_and_the_gaps_fault() { + let r = rig(); + assert_eq!(r.r64(2 * HEAD_OFFSET), 0); + assert_eq!(r.bus.write64(2 * HEAD_OFFSET, 1), BUS_OK); + assert!(!r.bus.read64(2 * HEAD_OFFSET + 8).is_ok()); + assert!(!r.bus.read64(0x8000_0000).is_ok(), "the 64-bit alias window is not this chip's"); + assert!(!r.bus.read32(0x1_0000_0000).is_ok()); + assert!(r.bus.read32(0x20_0001).is_ok(), "short windows take unaligned accesses"); + assert!(!r.bus.read32(0x1002).is_ok(), "registers need natural alignment"); +} + +/// Both heads send direct-map DMA to the widget and address their own +/// `DIR_MAP` names. +#[test] +fn direct_map_dma_goes_where_each_heads_direct_map_says() { + let r = rig(); + let a = Card::new(0x7001); + let b = Card::new(0x7002); + r.fit(0, 1, &a, Function::default()); + r.fit(1, 1, &b, Function::default()); + r.w64(0x80, 10 << 20); + r.w64(HEAD1 + 0x80, (11 << 20) | 1); + let data: Vec = (0x11..0x19).collect(); + assert!(a.dma().dma_write(0x8000_1000, &data).is_ok()); + let other: Vec = (0x21..0x29).collect(); + assert!(b.dma().dma_write(0x8000_2000, &other).is_ok()); + assert_eq!(r.peer.bytes(10, 0x1000, 8), data); + assert_eq!(r.peer.bytes(11, (1 << 31) | 0x2000, 8), other); + assert_eq!(r.peer.bytes(10, 0x2000, 8), vec![0; 8], "the other head's target is untouched"); + let log = r.peer.dma.lock().clone(); + assert_eq!(log[0], Dma { source: 15, widget: 10, address: 0x1000, len: 8, write: true }); + // Reads come back through the same path. + let mut back = [0u8; 8]; + assert!(a.dma().dma_read(0x8000_1000, &mut back).is_ok()); + assert_eq!(back.to_vec(), data); + // A transfer at an odd address is cut into naturally aligned pieces, and + // one that crosses a page is translated again at the page. + r.peer.dma.lock().clear(); + assert!(a.dma().dma_write(0x8000_1ffd, &[1, 2, 3, 4, 5, 6]).is_ok()); + let sizes: Vec<_> = r.peer.dma.lock().iter().map(|p| (p.address, p.len)).collect(); + assert_eq!(sizes, [(0x1ffd, 1), (0x1ffe, 2), (0x2000, 2), (0x2002, 1)].map(|(a, l)| (a as u64, l))); +} + +/// A slot whose Device register sets the swap bit puts each byte at its +/// address with the low two bits flipped, and still moves whole words as +/// one packet. +#[test] +fn a_swapping_slot_flips_the_byte_address_within_each_word() { + let r = rig(); + let card = Card::new(0x7003); + r.fit(0, 1, &card, Function::default()); + r.w64(0x80, 10 << 20); + r.w64(0x200, 0x18001000 | (1 << 19)); + assert!(card.dma().dma_write(0x8000_0100, &[1, 2, 3, 4, 5, 6, 7, 8]).is_ok()); + assert_eq!(r.peer.bytes(10, 0x100, 8), [4, 3, 2, 1, 8, 7, 6, 5]); + let mut back = [0u8; 8]; + assert!(card.dma().dma_read(0x8000_0100, &mut back).is_ok()); + assert_eq!(back, [1, 2, 3, 4, 5, 6, 7, 8]); + let packets: Vec<_> = r.peer.dma.lock().iter().map(|p| (p.address, p.len)).collect(); + assert_eq!(packets, [(0x100, 8), (0x100, 8)]); +} + +#[test] +fn dma_outside_the_direct_window_is_refused_and_a_read_comes_back_zero() { + let r = rig(); + let card = Card::new(0x7004); + r.fit(0, 1, &card, Function::default()); + r.peer.put(0, 0, &[0xaa; 16]); + assert_eq!(card.dma().dma_write(0x7fff_f000, &[1; 8]), PciDmaStatus::MasterAbort, "below the window"); + let mut buf = [0xffu8; 8]; + assert_eq!(card.dma().dma_read(0x4000_0000, &mut buf), PciDmaStatus::MasterAbort, "translated DMA is unsupported"); + assert_eq!(buf, [0; 8]); + assert!(r.peer.dma.lock().is_empty(), "nothing reached the peer"); + // A peer that refuses ends the transfer and zeroes the unread rest. + r.peer.refuse.store(true, Ordering::SeqCst); + let mut buf = [0xffu8; 8]; + assert_eq!(card.dma().dma_read(0x8000_0000, &mut buf), PciDmaStatus::MasterAbort); + assert_eq!(buf, [0; 8]); + // Device 0 and devices past 4 have no DMA path. + r.peer.refuse.store(false, Ordering::SeqCst); + assert_eq!(r.pic.heads[0].dma(0, 0x8000_0000, &mut [0; 4], false), PciDmaStatus::MasterAbort); + assert_eq!(r.pic.heads[0].dma(5, 0x8000_0000, &mut [0; 4], false), PciDmaStatus::MasterAbort); +} + +fn direct64(widget: u64, flags: u64, payload: u64) -> u64 { + (widget << 60) | flags | payload +} + +/// Bit 55 set is an ordered byte stream; clear, each 32-bit word's bytes +/// swap, or each doubleword's for a 64-bit requester. +#[test] +fn direct64_dma_keeps_a_stream_and_swaps_the_words_of_a_swap_clear_request() { + let r = rig(); + let narrow = Card::new(0x7101); + let wide = Card::new(0x7102); + r.fit(0, 1, &narrow, Function::default()); + r.fit(0, 4, &wide, Function { self_ordered_pio: false, req64: true }); + let data = [0x12u8, 0x34, 0x56, 0x78, 0x9a, 0xbc, 0xde, 0xf0]; + assert!(narrow.dma().dma_write(direct64(0xa, 1 << 55, 0x3000), &data).is_ok()); + assert_eq!(r.peer.bytes(0xa, 0x3000, 8), data, "ordered bytes unchanged"); + assert!(narrow.dma().dma_write(direct64(0xa, 0, 0x4000), &data).is_ok()); + assert_eq!(r.peer.bytes(0xa, 0x4000, 8), [0x78, 0x56, 0x34, 0x12, 0xf0, 0xde, 0xbc, 0x9a]); + assert!(wide.dma().dma_write(direct64(0xa, 0, 0x5000), &data).is_ok()); + assert_eq!(r.peer.bytes(0xa, 0x5000, 8), [0xf0, 0xde, 0xbc, 0x9a, 0x78, 0x56, 0x34, 0x12]); + assert!(wide.dma().dma_write(direct64(0xa, 1 << 55, 0x6000), &data).is_ok()); + assert_eq!(r.peer.bytes(0xa, 0x6000, 8), data, "a stream is a stream at either width"); + let mut back = [0u8; 8]; + assert!(narrow.dma().dma_read(direct64(0xa, 0, 0x4000), &mut back).is_ok()); + assert_eq!(back, data); + // Prefetch, precise, virtual channel 1 and barrier are recognized and + // ignored; DIR_MAP does not apply. + r.w64(0x80, 3 << 20); + for bits in [1 << 59, 1 << 58, 1 << 57, 1 << 56] { + assert!(narrow.dma().dma_write(direct64(0xa, bits | (1 << 55), 0x7000), &data).is_ok()); + } + assert!(r.peer.dma.lock().iter().all(|p| p.widget == 0xa)); +} + +/// The swap moves bytes within a word, not the transfer apart: whole words +/// leave as one packet, so a processor reading a word the card is updating +/// (a ring index in a status block) never sees half of it new. Only the +/// ragged ends go byte by byte. +#[test] +fn a_swap_clear_direct64_transfer_keeps_its_words_whole() { + let r = rig(); + let narrow = Card::new(0x7101); + let wide = Card::new(0x7102); + r.fit(0, 1, &narrow, Function::default()); + r.fit(0, 4, &wide, Function { self_ordered_pio: false, req64: true }); + let data: Vec = (0x10..0x30).collect(); + for (card, mask, at) in [(&narrow, 3u64, 0x4000u64), (&wide, 7, 0x5000)] { + r.peer.dma.lock().clear(); + assert!(card.dma().dma_write(direct64(0xa, 0, at), &data).is_ok()); + let packets: Vec<(u64, usize)> = r.peer.dma.lock().iter().map(|p| (p.address, p.len)).collect(); + assert_eq!(packets, [(at, data.len())], "mask {mask}: one packet"); + let want: Vec = (0..data.len() as u64).map(|i| data[(i ^ mask) as usize]).collect(); + assert_eq!(r.peer.bytes(0xa, at, data.len()), want, "mask {mask}: each word reversed"); + let mut back = vec![0u8; data.len()]; + assert!(card.dma().dma_read(direct64(0xa, 0, at), &mut back).is_ok()); + assert_eq!(back, data, "mask {mask}: read back"); + } + // Two bytes before a word and one after it: the word still goes whole. + r.peer.dma.lock().clear(); + assert!(narrow.dma().dma_write(direct64(0xa, 0, 0x6002), &[0xa1, 0xa2, 1, 2, 3, 4, 0xb1]).is_ok()); + let packets: Vec<(u64, usize)> = r.peer.dma.lock().iter().map(|p| (p.address, p.len)).collect(); + assert_eq!(packets, [(0x6001, 1), (0x6000, 1), (0x6004, 4), (0x600b, 1)]); + assert_eq!(r.peer.bytes(0xa, 0x6000, 4)[..2], [0xa2, 0xa1]); + assert_eq!(r.peer.bytes(0xa, 0x6004, 4), [4, 3, 2, 1]); + assert_eq!(r.peer.byte(0xa, 0x600b), 0xb1); +} + +#[test] +fn direct64_refuses_what_it_cannot_carry_before_any_packet_leaves() { + let r = rig(); + let card = Card::new(0x7103); + r.fit(1, 2, &card, Function::default()); + let sentinel = [0xa5u8; 8]; + r.peer.put(0xa, 0x1000, &sentinel); + for bit in 48..=54 { + assert_eq!(card.dma().dma_write(direct64(0xa, 1 << bit, 0x1000), &[1; 4]), PciDmaStatus::MasterAbort, "bit {bit}"); + } + assert_eq!(card.dma().dma_write(direct64(0xa, 0, (1 << 47) | 0x1000), &[1; 4]), PciDmaStatus::MasterAbort, "register space"); + assert_eq!(card.dma().dma_write(1 << 32, &[1; 4]), PciDmaStatus::MasterAbort, "target zero"); + assert!(r.peer.dma.lock().is_empty()); + assert_eq!(r.peer.bytes(0xa, 0x1000, 8), sentinel); +} + +/// A transfer that runs into the register-space boundary delivers the part +/// below it and fails at the page that crosses. +#[test] +fn direct64_splits_at_the_register_space_boundary() { + let r = rig(); + let card = Card::new(0x7104); + r.fit(0, 1, &card, Function::default()); + let boundary = 1u64 << 47; + let status = card.dma().dma_write(direct64(0xa, 1 << 55, boundary - 2), &[0x90, 0x91, 0x92, 0x93]); + assert_eq!(status, PciDmaStatus::MasterAbort); + assert_eq!(r.peer.bytes(0xa, boundary - 2, 2), [0x90, 0x91]); + assert_eq!(r.peer.dma.lock().len(), 1); +} + +#[test] +fn a_refused_direct64_write_leaves_the_target_as_it_was() { + let r = rig(); + let card = Card::new(0x7105); + r.fit(0, 1, &card, Function::default()); + r.peer.put(0xa, 0x3000, &[0xa5; 8]); + r.peer.refuse.store(true, Ordering::SeqCst); + for flags in [0, 1 << 55, 1 << 58, (1 << 56) | (1 << 55)] { + assert_eq!(card.dma().dma_write(direct64(0xa, flags, 0x3000), &[1; 8]), PciDmaStatus::MasterAbort); + } + assert_eq!(r.peer.bytes(0xa, 0x3000, 8), [0xa5; 8]); + assert!(r.peer.dma.lock().len() >= 4, "each attempt reached the peer"); +} + +/// An input that rises sets its status and, once enabled, sends the set +/// packet to its destination; a falling edge sends a clear if its mode bit +/// asks for one. +#[test] +fn a_rising_enabled_pin_sends_a_set_packet_and_a_falling_one_a_clear() { + let r = rig(); + for pin in 0..PINS as u64 { + r.w64(0x130 + pin * 8, destination(0x20 + pin)); + } + r.pic.set_pin(0, 3, true); + assert_eq!(r.r64(0x100), 1 << 3); + assert!(r.peer.interrupts.lock().is_empty(), "disabled"); + r.w64(0x108, 1 << 3); + assert_eq!(r.peer.interrupts.lock().as_slice(), [(15, 0x1800090, 0x123)], "enabling a pending input sends it"); + assert_eq!(r.pic.request_counts(0, 3), (1, 0)); + r.pic.set_pin(0, 3, false); + assert_eq!(r.pic.request_counts(0, 3), (1, 0), "no clear without the mode bit"); + r.w64(0x118, 1 << 3); + r.pic.set_pin(0, 3, true); + r.pic.set_pin(0, 3, false); + assert_eq!(r.pic.request_counts(0, 3), (2, 1)); + assert_eq!(r.peer.interrupts.lock().last().copied(), Some((15, 0x1800090, 0x23))); + // The status bit stays until software clears it, and a pin that is + // still high comes back. + assert_eq!(r.r64(0x100), 1 << 3); + r.w64(0x110, 1 << 3); + assert_eq!(r.r64(0x100), 0); + r.pic.set_pin(1, 0, true); + assert_eq!(r.r64(HEAD1 + 0x100), 1, "the heads' pins are separate"); + assert_eq!(r.r64(0x100), 0); + // Reset drops pins and status. + r.pic.power_on(); + assert_eq!(r.r64(0x100), 0); + r.pic.set_pin(0, 3, true); + assert_eq!(r.pic.request_counts(0, 3), (0, 0), "counts restart and the enable is gone"); +} + +#[test] +fn a_clear_write_resends_a_pin_that_is_still_high_and_enabled() { + let r = rig(); + r.w64(0x130, destination(9)); + r.w64(0x108, 1); + r.pic.set_pin(0, 0, true); + assert_eq!(r.pic.request_counts(0, 0), (1, 0)); + r.w64(0x110, 1); + assert_eq!(r.pic.request_counts(0, 0), (2, 0), "the status bit comes back with the level, and so does the packet"); + assert_eq!(r.r64(0x100), 1); + // A masked pin stays quiet on a clear write. + r.w64(0x108, 0); + r.w64(0x110, 1); + assert_eq!(r.pic.request_counts(0, 0), (2, 0)); +} + +#[test] +fn only_levels_a_pi_can_take_leave_the_chip() { + let r = rig(); + r.pic.set_pin(0, 0, true); + for level in [0, 6, 128, 0x7fff] { + r.w64(0x130, destination(level)); + r.w64(0x108, 0); + r.w64(0x108, 1); + assert!(r.peer.interrupts.lock().is_empty(), "level {level}"); + } + assert_eq!(r.pic.request_counts(0, 0).0, 4, "each attempt was counted"); + for level in [7, 127] { + r.w64(0x130, destination(level)); + r.w64(0x108, 0); + r.w64(0x108, 1); + } + let sent: Vec<_> = r.peer.interrupts.lock().iter().map(|p| p.2).collect(); + assert_eq!(sent, [0x107, 0x17f]); + // A peer that refuses is not an error the guest sees, but it is counted. + r.peer.refuse.store(true, Ordering::SeqCst); + r.w64(0x108, 0); + r.w64(0x108, 1); + assert_eq!(r.peer.interrupts.lock().len(), 2); +} + +/// A PCI function's INTA reaches the input numbered one less than its +/// device, whichever functions are fitted, and nothing else raises it. +#[test] +fn pci_interrupts_reach_the_input_of_their_device() { + let r = rig(); + let card = Card::new(0x7201); + let other = Card::new(0x7202); + r.pic.attach(0, 3, card.clone(), Function::default()); + r.pic.attach(0, 4, other.clone(), Function::default()); + r.w64(0x130 + 2 * 8, destination(0x31)); + r.w64(0x108, 1 << 2); + card.raise(true); + assert_eq!(r.r64(0x100), 1 << 2, "device 3 is input 2"); + assert_eq!(r.peer.interrupts.lock().as_slice(), [(15, 0x1800090, 0x131)]); + card.raise(false); + other.raise(true); + assert_eq!(r.r64(0x100), (1 << 2) | (1 << 3), "device 4 is input 3, which the enable leaves unsent"); + assert_eq!(r.peer.interrupts.lock().len(), 1); + // A board routes its own interrupt to an input through the bus. + r.pic.pin_line(0, 6).set(true); + assert_eq!(r.r64(0x100) & (1 << 6), 1 << 6); +} + +/// Both PIs, told apart the way the node does: by address bit 21. +struct ToPi([Pi; 2]); +impl PicPeer for ToPi { + fn interrupt(&self, _: u8, address: u64, payload: u64) -> Result<(), Refused> { + self.0[(address >> 21) as usize & 1].write64(pi_reg::PENDING_MODIFY, payload); + Ok(()) + } + fn dma_read(&self, _: u8, _: u8, _: u64, _: &mut [u8]) -> Result<(), Refused> { + Err(Refused) + } + fn dma_write(&self, _: u8, _: u8, _: u64, _: &[u8]) -> Result<(), Refused> { + Err(Refused) + } +} + +/// The PI is told which inputs are active, so it can raise a level again +/// after the processor clears it while the line stays high. +#[test] +fn the_pi_reasserts_a_level_whose_input_is_still_active() { + let (pi, now) = pi_with_clock(); + let pic = Pic::new(15); + pic.connect_peer(Arc::new(ToPi([pi.clone(), pi.other()]))).unwrap(); + pic.connect_node(&pi.chip, 0); + let bus = pic.widget(); + bus.write64(0x130, destination(12)); + bus.write64(0x108, 1); + pic.set_pin(0, 0, true); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data, 1 << 12); + // The processor clears its pending bit; the input is still high. + pi.write64(pi_reg::PENDING_MODIFY, 12); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data, 0); + now.store(999_999, Ordering::SeqCst); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data, 0, "not before a millisecond"); + now.store(1_000_000, Ordering::SeqCst); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data, 1 << 12); + // Dropping the input ends the reassertion. + pi.write64(pi_reg::PENDING_MODIFY, 12); + pic.set_pin(0, 0, false); + now.store(5_000_000, Ordering::SeqCst); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data, 0); +} + +/// A processor on another thread that takes and clears the level the moment +/// the set packet lands, before the sending thread has gone on. +struct TakenAtOnce(Pi); +impl PicPeer for TakenAtOnce { + fn interrupt(&self, _: u8, _: u64, payload: u64) -> Result<(), Refused> { + self.0.write64(pi_reg::PENDING_MODIFY, payload); + self.0.write64(pi_reg::PENDING_MODIFY, payload & !SET); + Ok(()) + } + fn dma_read(&self, _: u8, _: u8, _: u64, _: &mut [u8]) -> Result<(), Refused> { + Err(Refused) + } + fn dma_write(&self, _: u8, _: u8, _: u64, _: &[u8]) -> Result<(), Refused> { + Err(Refused) + } +} + +/// The PI learns the input is active before the set packet can be cleared, +/// so a clear that comes as the packet lands still brings the level back: +/// the input stays high and nothing else would send it again. Both ways a +/// packet leaves while the PI holds the input inactive are covered: a rising +/// input and an enable. +#[test] +fn a_level_cleared_as_its_packet_lands_comes_back() { + for enable_last in [false, true] { + let what = if enable_last { "enable" } else { "rising input" }; + let (pi, now) = pi_with_clock(); + let pic = Pic::new(15); + pic.connect_peer(Arc::new(TakenAtOnce(pi.clone()))).unwrap(); + pic.connect_node(&pi.chip, 0); + let bus = pic.widget(); + bus.write64(0x130, destination(12)); + if enable_last { + pic.set_pin(0, 0, true); + bus.write64(0x108, 1); + } else { + bus.write64(0x108, 1); + pic.set_pin(0, 0, true); + } + assert_eq!(pic.request_counts(0, 0), (1, 0), "{what}: one packet"); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data, 0, "{what}: the processor took it"); + now.store(1_000_000, Ordering::SeqCst); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data, 1 << 12, "{what}: the level comes back"); + } +} + +/// Each level keeps its own deadline, a masked input is not reasserted and +/// a destination in the other PI's window leaves this one alone. +#[test] +fn levels_reassert_independently_and_only_while_enabled() { + let (pi, now) = pi_with_clock(); + let other = pi.other(); + let pic = Pic::new(15); + pic.connect_peer(Arc::new(ToPi([pi.clone(), other.clone()]))).unwrap(); + pic.connect_node(&pi.chip, 0); + let bus = pic.widget(); + for (pin, level) in [(0u64, 12u64), (1, 13)] { + bus.write64(0x130 + pin * 8, destination(level)); + } + // Pin 2 targets the second PI (destination bit 21). + bus.write64(0x140, destination(14) | (1 << 21)); + bus.write64(0x108, 0b111); + for pin in 0..3 { + pic.set_pin(0, pin, true); + } + pi.write64(pi_reg::PENDING_MODIFY, 12); + now.store(400_000, Ordering::SeqCst); + pi.write64(pi_reg::PENDING_MODIFY, 13); + now.store(1_000_000, Ordering::SeqCst); + let pending = pi.read64(pi_reg::PENDING_STATUS_A).data; + assert_eq!(pending & (1 << 12), 1 << 12, "level 12 is due"); + assert_eq!(pending & (1 << 13), 0, "level 13 is not"); + now.store(1_400_000, Ordering::SeqCst); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data & (1 << 13), 1 << 13); + // Masking an input stops its level from coming back. + pi.write64(pi_reg::PENDING_MODIFY, 12); + bus.write64(0x108, 0b110); + now.store(3_000_000, Ordering::SeqCst); + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data & (1 << 12), 0); + // The destination of pin 2 is the other PI's, so this one never held it. + assert_eq!(pi.read64(pi_reg::PENDING_STATUS_A).data & (1 << 14), 0); + assert_eq!(other.read64(pi_reg::PENDING_STATUS_A).data & (1 << 14), 1 << 14); + other.write64(pi_reg::PENDING_MODIFY, 14); + now.store(3_999_999, Ordering::SeqCst); + assert_eq!(other.read64(pi_reg::PENDING_STATUS_A).data & (1 << 14), 0, "not before a millisecond"); + now.store(4_000_000, Ordering::SeqCst); + assert_eq!(other.read64(pi_reg::PENDING_STATUS_A).data & (1 << 14), 1 << 14, "the other PI reasserts it"); +} + +/// The PROM writes the link-side error address slot's low word; the chip +/// latches the invalid-address status and, if enabled, sends one packet to +/// the destination the widget registers name. +#[test] +fn an_invalid_address_write_latches_status_and_sends_one_packet() { + let r = rig(); + let invalid = 1u64 << 24; + // Other widths and the doubleword form of the slot do nothing. + r.bus.write8(0x7c, 0x11); + r.bus.write16(0x7c, 0x2233); + r.bus.write32(0x78, 0x4455_6677); + r.w64(0x78, 0x8899_aabb_ccdd_eeff); + assert_eq!(r.r64(0x100), 0); + r.w32(0x7c, 0xdead_beef); + assert_eq!(r.r32(0x104) as u64, invalid); + assert_eq!(r.r32(HEAD1 + 0x104), 0, "per head"); + assert!(r.peer.interrupts.lock().is_empty(), "not enabled"); + r.w32(0x114, invalid as u32); + assert_eq!(r.r32(0x104), 0, "written one to clear"); + let target = 0x1_0000_3000u64; + r.w64(0x30, 0x3400_0000); + r.w64(0x38, target); + r.w32(0x10c, invalid as u32); + for bad in [0x80u64, 0x100] { + r.w64(0x128, bad); + r.w32(0x7c, 0); + assert!(r.peer.interrupts.lock().is_empty(), "interrupt number {bad:#x}"); + } + r.w64(0x128, 0x65); + r.w32(0x7c, 0); + assert_eq!(r.peer.interrupts.lock().as_slice(), [(15, target, 0x165)]); + r.w32(0x7c, 0); + assert_eq!(r.peer.interrupts.lock().len(), 2, "every request sends while the status stays latched"); + // A repeat of the lower register's bits 47:32 in the upper register is + // accepted; a different one, or a lone upper word, is not. + r.w64(0x30, 1); + r.w32(0x7c, 0); + assert_eq!(r.peer.interrupts.lock().len(), 3); + r.w64(0x30, 2); + r.w32(0x7c, 0); + r.w64(0x30, 0x3400_0000); + r.w64(0x38, 0); + r.w32(0x7c, 0); + r.w64(0x30, 0x1_0000_0001); + r.w64(0x38, target); + r.w32(0x7c, 0); + r.w64(0x30, 0x3400_0000); + r.w64(0x38, 0x000f_0001_0000_3000); + r.w32(0x7c, 0); + assert_eq!(r.peer.interrupts.lock().len(), 3, "conflicts, zero and overflow are refused"); + // The PROM's two word writes join into the address. + r.w64(0x30, 0); + r.w64(0x38, 0); + r.w32(0x34, 1); + r.w32(0x3c, 0x3000); + r.w32(0x7c, 0); + assert_eq!(r.peer.interrupts.lock().last().map(|p| p.1), Some(target)); +} + +#[test] +fn two_pics_are_independent_devices() { + let a = Pic::new(11); + let b = Pic::new(15); + let peer_a = Arc::new(Peer::default()); + let peer_b = Arc::new(Peer::default()); + a.connect_peer(peer_a.clone()).unwrap(); + b.connect_peer(peer_b.clone()).unwrap(); + assert_eq!(a.connect_peer(peer_b.clone()), Err(AlreadyConnected)); + assert_eq!((a.widget_number(), b.widget_number()), (11, 15)); + let (bus_a, bus_b) = (a.widget(), b.widget()); + bus_a.write64(0x20, 0x0f00_3205); + assert_eq!(bus_b.read64(0x20).data, 0x0f00_320f, "state is per chip"); + bus_a.write64(0x130, destination(9)); + bus_a.write64(0x108, 1); + a.set_pin(0, 0, true); + assert_eq!(peer_a.interrupts.lock().as_slice(), [(5, 0x1800090, 0x109)], "the packet carries the control widget field"); + assert!(peer_b.interrupts.lock().is_empty()); + let card = Card::new(1); + a.attach(0, 1, card.clone(), Function::default()); + assert!(!bus_b.read32(config(0, 1)).is_ok(), "the other chip's slot stays empty"); +} + +// An empty slot's configuration read fails as a master timeout. QEMU's +// sgi-pic.c returns all ones with no fault and has no counterpart case. + +const MASTER_TIMEOUT_BIT: u64 = 1 << 11; +const STATUS: u64 = 0x100; +const ENABLE: u64 = 0x108; +const RESET: u64 = 0x110; +const VIEW: u64 = 0x170; + +/// With every enable clear, as the PROM probes: an empty device fails on +/// either head, function 0 or 1, for every width the processor uses. +/// Error view bit 11 records it and status bit 11 stays clear. +#[test] +fn an_empty_slot_read_faults_and_only_the_error_view_records_it() { + let r = rig(); + for head in [0, 1] { + let base = head_base(head); + for (device, function) in [(2u64, 0u64), (2, 1), (0, 0), (5, 1)] { + let at = config(head, device) + function * 0x100; + assert!(!r.bus.read32(at).is_ok(), "head {head} device {device}.{function}"); + assert!(!r.bus.read16(at + 2).is_ok()); + assert!(!r.bus.read8(at + 3).is_ok()); + assert_eq!(r.r64(base + VIEW) & MASTER_TIMEOUT_BIT, MASTER_TIMEOUT_BIT, "head {head}"); + assert_eq!(r.r32(base + VIEW + 4) as u64 & MASTER_TIMEOUT_BIT, MASTER_TIMEOUT_BIT, "low-word view"); + assert_eq!(r.r64(base + STATUS) & MASTER_TIMEOUT_BIT, 0, "status needs the enable"); + assert_eq!(r.r32(base + STATUS + 4) as u64 & MASTER_TIMEOUT_BIT, 0); + // The PROM's low-word write of 0x41 clears the view. + r.w32(base + RESET + 4, 0x41); + assert_eq!(r.r64(base + VIEW), 0); + } + // The other head's view is its own. + assert!(!r.bus.read32(config(head, 2)).is_ok()); + assert_eq!(r.r64(head_base(1 - head) + VIEW), 0); + r.w64(base + RESET, 0x41); + assert_eq!(r.r64(base + VIEW), 0, "an aligned write clears it too"); + } + // The view is read-only and an unclaimed write records nothing. + r.w64(VIEW, MASTER_TIMEOUT_BIT); + r.w32(config(0, 2) + 4, 7); + assert_eq!(r.r64(VIEW), 0); + assert!(r.pic.unsupported().iter().any(|&(_, write, off, _)| write && off == VIEW)); + assert!(r.peer.interrupts.lock().is_empty()); +} + +/// With enable bit 11 set, status bit 11 sets too and the error interrupt +/// goes out. Status clears one bit at a time: the PROM's 0x41 leaves it set, +/// a one at bit 11 clears it. +#[test] +fn an_enabled_master_timeout_sets_status_until_bit_11_is_written() { + let r = rig(); + r.w64(0x38, 0x1800090); + r.w64(0x128, 9); + r.w64(ENABLE, MASTER_TIMEOUT_BIT); + assert!(!r.bus.read32(config(0, 2)).is_ok()); + assert_eq!(r.r64(STATUS) & MASTER_TIMEOUT_BIT, MASTER_TIMEOUT_BIT); + assert_eq!(r.r64(VIEW) & MASTER_TIMEOUT_BIT, MASTER_TIMEOUT_BIT); + assert_eq!(r.peer.interrupts.lock().len(), 1, "one error interrupt"); + r.w32(RESET + 4, 0x41); + assert_eq!(r.r64(STATUS) & MASTER_TIMEOUT_BIT, MASTER_TIMEOUT_BIT, "0x41 leaves status bit 11"); + assert_eq!(r.r64(VIEW), 0, "but clears the view"); + r.w64(RESET, MASTER_TIMEOUT_BIT); + assert_eq!(r.r64(STATUS) & MASTER_TIMEOUT_BIT, 0); + assert_eq!(r.r64(HEAD1 + STATUS) & MASTER_TIMEOUT_BIT, 0, "head 1 untouched"); +} + +/// A populated device after an empty one still reads present once the +/// PROM's 0x41 has been written, as widget 15's second head lists device 3 +/// after an empty device 2 on a real Tezro. A present single-function card +/// has header-type bit 7 clear, so the PROM never asks for its function 1, +/// which would fault. +#[test] +fn a_device_after_an_empty_one_is_still_found() { + let r = rig(); + let card = Card::new(0x5003); + r.pic.attach(1, 3, card, Function::default()); + let probe = |device: u64| { + r.w32(HEAD1 + RESET + 4, 0x41); + let read = r.bus.read32(config(1, device)); + let absent = !read.is_ok() || r.r32(HEAD1 + STATUS + 4) as u64 & MASTER_TIMEOUT_BIT != 0; + (!absent).then_some(read.data) + }; + assert_eq!(probe(1), None); + assert_eq!(probe(2), None); + assert_eq!(probe(3), Some(0x5003_1234)); + assert_eq!(probe(4), None); + assert_eq!(r.r32(config(1, 3) + 0xc) >> 16 & 0x80, 0, "not multifunction"); + assert!(!r.bus.read32(config(1, 3) + 0x100).is_ok(), "its function 1 is unclaimed"); +} + +/// A reset clears the view with the rest of the head. +#[test] +fn reset_clears_the_error_view() { + let r = rig(); + assert!(!r.bus.read32(config(0, 2)).is_ok()); + assert_ne!(r.r64(VIEW), 0); + r.pic.power_on(); + assert_eq!(r.r64(VIEW), 0); +} + +/// Two functions that decode the same range, as a probing PROM leaves them: +/// the shared bus answers from the lower device, and the other answers when +/// that one stops decoding. What real silicon does with two comparators +/// matching is not defined. +/// (cpu-pio-overlapping-bars-use-active-function) +#[test] +fn overlapping_bars_go_to_the_lower_device() { + let r = rig(); + let first = Card::new(0x8001); + let second = Card::new(0x8002); + r.fit(0, 1, &first, Function::default()); + r.fit(0, 2, &second, Function::default()); + let at = window(0, 0x10); + r.w32(at, 0x40); + assert_eq!(le_value(&first.bar.lock()[0x10..0x14]), 0x40, "device 1 answers"); + assert_eq!(le_value(&second.bar.lock()[0x10..0x14]), 0); + // Switching the winner off leaves the other one. + r.w32(config(0, 1) + 4, 0); + r.w32(at, 0x20); + assert_eq!(le_value(&second.bar.lock()[0x10..0x14]), 0x20); + assert_eq!(r.r32(at), 0x20); + r.w32(config(0, 1) + 4, MEMORY_AND_MASTER); + assert_eq!(r.r32(at), 0x40, "device 1 answers again"); +} diff --git a/src/dev/pxbow.rs b/src/dev/pxbow.rs new file mode 100644 index 000000000..2476c1561 --- /dev/null +++ b/src/dev/pxbow.rs @@ -0,0 +1,183 @@ +//! PXBOW, the crossbar-like widget 0 that sits behind the PIC on Tezro and +//! the Origin 350, and the forwarding of processor accesses through it. +//! +//! The model reuses the Crossbow ([`Xbow`]) at revision 5 behaviour with +//! PXBOW's identity word `0x3d100001` (revision 3, part `0xd100`, +//! manufacturer 0), and a source field in its +//! status reads. That is a software choice, not a claim that the part is a +//! Crossbow 2.0 or that the processor's route to widget 0 passes a real +//! switch: its physical route, fitted revision and register set beyond the +//! reused ones are unknown and stay unsupported and observable +//! ([`Xbow::unsupported`]). +//! +//! The chip knows two links when it is built: the host's widget (Bedrock +//! IO, 10) and the PIC's own. A further widget, such as the optional V12 at +//! widget 12 on the two- and four-processor Tezro, becomes reachable when +//! the machine attaches it with [`Pxbow::attach_endpoint`]; until then its +//! link is absent and an access to it is a bus error. Each processor access +//! to a widget through [`Pxbow::route`] needs both the host's link and the +//! target's to be alive, so a link reset takes the PIC out of reach for the +//! 20 ns of its pulse. +//! +//! Ported from hw/pci-host/sgi-pic.c and hw/misc/sgi-xbow.c in +//! jamesbraid/qemu-sgi-origami. + +use super::heart::Clock; +use super::xbow::Xbow; +use crate::irq::InterruptController; +use crate::xtalk::XtalkHost; +use crate::traits::{BusDevice, BusRead16, BusRead32, BusRead64, BusRead8, BUS_ERR}; +use parking_lot::RwLock; +use std::sync::Arc; + +/// The Crossbow revision the PIC's PXBOW selects for behaviour. +pub const REVISION: u32 = 5; +/// The revision its identification word reports. Established from real +/// Tezro, Origin 350 and Onyx 4 PROM inventories (revision 3). +pub const REPORTED_REVISION: u32 = 3; +// Part and manufacturer are INFERRED; a 2009 OpenBSD dmesg names the part +// Bedrock, a conflict that stays open. +pub const PART: u32 = 0xd100; +pub const MANUFACTURER: u32 = 0; + +/// Why a widget could not be attached. +#[derive(Debug, PartialEq, Eq)] +pub enum AttachError { + /// Links carry widgets 8 to 15 only. + NotALink(u8), + /// The host's own widget has no endpoint behind the switch. + Host(u8), + /// The widget already has an endpoint. + Occupied(u8), +} + +pub struct Pxbow { + xbow: Arc, + host: u8, + pic: u8, + endpoints: RwLock<[Option>; 16]>, +} + +impl Pxbow { + /// A PXBOW reached from `host` (the widget Bedrock IO answers as) with + /// its PIC at `pic`. Both links are present and alive. `clock` is the + /// machine's virtual time in nanoseconds, which times link resets. + pub fn new(host: u8, pic: u8, clock: Clock) -> Result, AttachError> { + for widget in [host, pic] { + if !(8..16).contains(&widget) { + return Err(AttachError::NotALink(widget)); + } + } + if host == pic { + return Err(AttachError::Occupied(pic)); + } + let xbow = Xbow::new(REVISION, (1 << host) | (1 << pic)) + .with_identity(PART, MANUFACTURER) + .with_reported_revision(REPORTED_REVISION) + .with_status_source(host) + // PROM 6.210's crossbar check writes the read-only lower error + // address on purpose. + .with_posted_error_address_writes() + .with_clock(clock); + Ok(Arc::new(Self { + xbow: Arc::new(xbow), + host, + pic, + endpoints: RwLock::new(std::array::from_fn(|_| None)), + })) + } + + pub fn host_widget(&self) -> u8 { + self.host + } + + pub fn pic_widget(&self) -> u8 { + self.pic + } + + /// The register block the processor reaches as widget 0. + pub fn xbow(&self) -> &Arc { + &self.xbow + } + + /// Widget 0 as a bus device, for Bedrock IO's widget 0. + pub fn widget_zero(&self) -> Arc { + self.xbow.clone() + } + + /// Where widget 0's own interrupt goes: packets to the host widget, + /// whose address names the destination. Once, before the machine runs. + pub fn connect_host(&self, host: Arc) { + let xtalk = XtalkHost { widget: self.host, int_packet: 0, int_packet_mask: 0, vectors: 256 }; + self.xbow.connect_host(xtalk, host); + } + + /// Put `device` behind the switch as `widget` and bring its link up. + /// The PIC's own endpoint is set this way too. + pub fn attach_endpoint(&self, widget: u8, device: Arc) -> Result<(), AttachError> { + if !(8..16).contains(&widget) { + return Err(AttachError::NotALink(widget)); + } + if widget == self.host { + return Err(AttachError::Host(widget)); + } + let mut endpoints = self.endpoints.write(); + if endpoints[widget as usize].is_some() { + return Err(AttachError::Occupied(widget)); + } + endpoints[widget as usize] = Some(device); + drop(endpoints); + self.xbow.attach(widget); + Ok(()) + } + + /// Widgets with an endpoint, as a mask with bit n for widget n. + pub fn endpoint_widgets(&self) -> u16 { + self.endpoints.read().iter().enumerate().fold(0, |m, (w, e)| m | (u16::from(e.is_some()) << w)) + } + + /// `widget` as the processor reaches it through the switch: forwarded + /// to the endpoint while the host's link and the widget's are alive, a + /// bus error otherwise. + pub fn route(self: &Arc, widget: u8) -> Arc { + assert!((8..16).contains(&widget)); + Arc::new(Routed { pxbow: self.clone(), widget }) + } +} + +struct Routed { + pxbow: Arc, + widget: u8, +} + +impl Routed { + /// The endpoint, if the route is open. + fn open(&self) -> Option> { + let p = &self.pxbow; + if !p.xbow.carries_timed(p.host) || !p.xbow.carries_timed(self.widget) { + return None; + } + p.endpoints.read()[self.widget as usize].clone() + } +} + +macro_rules! forward { + ($r:ident, $w:ident, $t:ty, $out:ident) => { + fn $r(&self, addr: u64) -> $out { + self.open().map_or_else($out::err, |d| d.$r(addr)) + } + fn $w(&self, addr: u64, value: $t) -> u32 { + self.open().map_or(BUS_ERR, |d| d.$w(addr, value)) + } + }; +} + +impl BusDevice for Routed { + forward!(read8, write8, u8, BusRead8); + forward!(read16, write16, u16, BusRead16); + forward!(read32, write32, u32, BusRead32); + forward!(read64, write64, u64, BusRead64); +} + +#[cfg(test)] +mod tests; diff --git a/src/dev/pxbow/tests.rs b/src/dev/pxbow/tests.rs new file mode 100644 index 000000000..d7e026387 --- /dev/null +++ b/src/dev/pxbow/tests.rs @@ -0,0 +1,239 @@ +// Cases follow tests/qtest/sn-builder-test.c in jamesbraid/qemu-sgi-origami: +// ip53-widget-zero-pxbow-identity, ip53-widget-zero-status-source and +// ip53-pxbow-route-fences-and-interrupt, and the widget-12 absence check of +// tests/qtest/sgi-graphics-pipes-test.c (ip53-v12). The qtest's machine +// properties, link-pulse timing and migration have no counterpart here. +use super::*; +use crate::dev::pic::{PicPeer, Refused, Pic}; +use crate::traits::{BusRead32, BUS_OK}; +use parking_lot::Mutex; +use std::sync::atomic::{AtomicU64, Ordering}; + +const STATUS: u64 = 0x0c; +const STATUS_CLEAR: u64 = 0x54; +const LINK_STATUS_ALIVE: u32 = 1 << 31; +const AUX_PRESENT: u32 = 1 << 5; + +/// A clock the test advances, in nanoseconds. +fn clock() -> (Clock, Arc) { + let now = Arc::new(AtomicU64::new(0)); + let c = now.clone(); + (Arc::new(move || c.load(Ordering::SeqCst)), now) +} + +fn link(widget: u64, register: u64) -> u64 { + 0x100 + (widget - 8) * 0x40 + register + 4 +} + +/// A widget that answers one word and remembers a write. +struct Stub(Mutex); +impl BusDevice for Stub { + fn read32(&self, addr: u64) -> BusRead32 { + if addr == 0x40 { BusRead32::ok(*self.0.lock()) } else { BusRead32::err() } + } + fn write32(&self, addr: u64, value: u32) -> u32 { + if addr != 0x40 { + return BUS_ERR; + } + *self.0.lock() = value; + BUS_OK + } +} + +#[derive(Default)] +struct Peer(Mutex>); +impl PicPeer for Peer { + fn interrupt(&self, source: u8, address: u64, payload: u64) -> Result<(), Refused> { + self.0.lock().push((source, address, payload)); + Ok(()) + } + fn dma_read(&self, _: u8, _: u8, _: u64, _: &mut [u8]) -> Result<(), Refused> { + Err(Refused) + } + fn dma_write(&self, _: u8, _: u8, _: u64, _: &[u8]) -> Result<(), Refused> { + Err(Refused) + } +} + +/// Widget zero names itself PXBOW, not Crossbow and not the PIC, and shows +/// the host's and the PIC's links up. +#[test] +fn widget_zero_identifies_as_pxbow_with_the_host_and_pic_links_up() { + let p = Pxbow::new(10, 15, clock().0).unwrap(); + let w = p.widget_zero(); + let id = w.read32(4).data; + assert_eq!(id, 0x3d10_0001); + assert_eq!((id >> 28) & 0xf, 3); + assert_eq!((id >> 12) & 0xffff, 0xd100); + assert_eq!((id >> 1) & 0x7ff, 0); + assert_eq!(id & 1, 1); + for widget in [10, 15] { + assert_ne!(w.read32(link(widget, 0x10)).data & LINK_STATUS_ALIVE, 0, "widget {widget} alive"); + assert_ne!(w.read32(link(widget, 0x38)).data & AUX_PRESENT, 0); + } + for widget in [8, 9, 11, 12, 13, 14] { + assert_eq!(w.read32(link(widget, 0x10)).data & LINK_STATUS_ALIVE, 0, "widget {widget} absent"); + assert_eq!(w.read32(link(widget, 0x38)).data & AUX_PRESENT, 0); + } + assert_eq!(p.endpoint_widgets(), 0); +} + +#[test] +fn status_reads_carry_the_hosts_widget_as_their_source() { + let p = Pxbow::new(10, 15, clock().0).unwrap(); + let w = p.widget_zero(); + // Seed the status with an access error first. + let _ = w.read32(0x1004); + let status = w.read32(STATUS).data; + assert_eq!((status >> 6) & 0xf, 10); + assert_ne!(status & (1 << 5), 0, "the seeded access error shows"); + assert_eq!((w.read32(STATUS_CLEAR).data >> 6) & 0xf, 10); + assert_eq!(w.read32(STATUS).data & (1 << 5), 0, "the alias acknowledged it"); + let other = Pxbow::new(11, 14, clock().0).unwrap(); + assert_eq!((other.widget_zero().read32(STATUS).data >> 6) & 0xf, 11); +} + +/// PROM 6.210's crossbar check on the Tezro writes 0xdeadbeef to the +/// read-only lower error address and 0 to the upper one. The store +/// completes, the registers keep their value, and widget 0's status reports +/// the access error (NEC U10278EJ4V0UM p.104). +#[test] +fn the_proms_error_address_writes_complete_and_are_reported() { + let p = Pxbow::new(10, 15, clock().0).unwrap(); + let w = p.widget_zero(); + assert_eq!(w.write32(0x1c, 0xdead_beef), BUS_OK); + assert_eq!(w.write32(0x14, 0), BUS_OK); + assert_eq!(w.read32(0x1c).data, 0); + assert_ne!(w.read32(STATUS).data & (1 << 5), 0, "the access error shows"); + assert!(p.xbow().unsupported().contains(&(0x1c, 4, true)), "the write stays observable"); + assert_ne!(w.write32(0x04, 0), BUS_OK, "the identification register still refuses"); +} + +/// Revision 5 decodes the fences and the lock; the lock takes a write only +/// while it reads zero and any write to its clear alias releases it. +#[test] +fn fences_and_the_lock_are_stored_and_read_back() { + let p = Pxbow::new(10, 15, clock().0).unwrap(); + let w = p.widget_zero(); + for (off, v) in [(0x78u64, 0xff_u32), (0x80, 0xff), (0xb8, 0xff), (0xc0, 0), (0xc8, 0)] { + assert_eq!(w.read32(off + 4).data, v, "reset value at {off:#x}"); + } + for (off, v) in [(0x78u64, 0x5a), (0x90, 0xa5), (0xb8, 0x3c)] { + assert_eq!(w.write32(off + 4, v), BUS_OK); + assert_eq!(w.read32(off + 4).data, v); + } + assert_eq!(w.read32(0x78 + 4).data, 0x5a); + assert_eq!(w.write32(0xc0 + 4, 0x11), BUS_OK); + assert_eq!(w.write32(0xc0 + 4, 0x22), BUS_OK); + assert_eq!(w.read32(0xc0 + 4).data, 0x11, "a held lock ignores a second write"); + assert_eq!(w.read32(0xc8 + 4).data, 0x11, "both offsets read the one value"); + assert_eq!(w.write32(0xc8 + 4, 0), BUS_OK); + assert_eq!(w.read32(0xc0 + 4).data, 0); + assert_eq!(w.write32(0xc0 + 4, 0x22), BUS_OK); + assert_eq!(w.read32(0xc0 + 4).data, 0x22); + crate::traits::Resettable::power_on(&*p.xbow().clone()); + assert_eq!(w.read32(0x78 + 4).data, 0xff, "reset restores the fences"); + assert_eq!(w.read32(0xc0 + 4).data, 0, "and releases the lock"); + // The upper word of a slot and an eight-byte access are still refused. + assert!(!w.read32(0x78).is_ok()); + assert!(!w.read64(0x78).is_ok()); +} + +/// Widget zero's own error interrupt is a packet to the host, with the +/// vector and address its destination registers hold. +#[test] +fn a_widget_zero_error_sends_an_interrupt_packet_to_the_host() { + let pic = Pic::new(15); + let peer = Arc::new(Peer::default()); + pic.connect_peer(peer.clone()).unwrap(); + let p = pic.enable_pxbow(10, clock().0).unwrap(); + let w = p.widget_zero(); + let address = 0x1_0180_0090u64; + assert_eq!(w.write32(0x34, (0x34 << 24) | (10 << 16) | ((address >> 32) as u32 & 0xffff)), BUS_OK); + assert_eq!(w.write32(0x3c, address as u32), BUS_OK); + assert_eq!(w.write32(0x24, 1 << 5), BUS_OK, "enable the access-error condition"); + assert!(peer.0.lock().is_empty()); + let _ = w.read32(0x1004); + assert_ne!(w.read32(STATUS).data & (1 << 23), 0, "the request bit is up"); + assert_eq!(peer.0.lock().as_slice(), [(0, address, 0x134)]); + let _ = w.read32(STATUS_CLEAR); + assert_eq!(peer.0.lock().last().copied(), Some((0, address, 0x34)), "acknowledging lowers it"); + // A target that is not the host goes nowhere. + assert_eq!(w.write32(0x34, (0x34 << 24) | (11 << 16)), BUS_OK); + let _ = w.read32(0x1004); + assert_eq!(peer.0.lock().len(), 2); +} + +/// The V12's widget is absent until the machine fits it, then forwards to +/// the endpoint with the widget-relative address unchanged. +#[test] +fn a_widget_is_unreachable_until_an_endpoint_is_attached() { + let p = Pxbow::new(10, 15, clock().0).unwrap(); + let w12 = p.route(12); + assert!(!w12.read32(0x40).is_ok()); + assert_eq!(w12.write32(0x40, 1), BUS_ERR); + let stub = Arc::new(Stub(Mutex::new(0))); + p.attach_endpoint(12, stub.clone()).unwrap(); + assert_eq!(p.endpoint_widgets(), 1 << 12); + let wz = p.widget_zero(); + assert_ne!(wz.read32(link(12, 0x10)).data & LINK_STATUS_ALIVE, 0, "its link comes up"); + assert_ne!(wz.read32(link(12, 0x38)).data & AUX_PRESENT, 0); + assert_eq!(w12.write32(0x40, 0xcafe), BUS_OK); + assert_eq!(*stub.0.lock(), 0xcafe); + assert_eq!(w12.read32(0x40).data, 0xcafe); + assert!(!w12.read32(0x44).is_ok(), "the endpoint's own refusal passes through"); + assert!(!p.route(11).read32(0x40).is_ok(), "other widgets stay absent"); +} + +#[test] +fn attachment_refuses_the_host_a_taken_widget_and_a_non_link() { + let p = Pxbow::new(10, 15, clock().0).unwrap(); + let stub: Arc = Arc::new(Stub(Mutex::new(0))); + assert_eq!(p.attach_endpoint(10, stub.clone()), Err(AttachError::Host(10))); + assert_eq!(p.attach_endpoint(3, stub.clone()), Err(AttachError::NotALink(3))); + assert_eq!(p.attach_endpoint(12, stub.clone()), Ok(())); + assert_eq!(p.attach_endpoint(12, stub.clone()), Err(AttachError::Occupied(12))); + assert!(Pxbow::new(10, 10, clock().0).is_err()); + assert!(Pxbow::new(10, 2, clock().0).is_err()); +} + +/// The PIC that owns IO9 supplies widget 0; the other PIC does not, and +/// the PIC's own widget is reached through the switch. +#[test] +fn only_the_pic_that_enables_it_has_a_pxbow_and_it_forwards_to_the_pic() { + let with = Pic::new(15); + let without = Pic::new(11); + assert!(with.pxbow().is_none()); + let p = with.enable_pxbow(10, clock().0).unwrap(); + assert!(without.pxbow().is_none()); + assert_eq!(p.pic_widget(), 15); + assert_eq!(p.endpoint_widgets(), 1 << 15); + assert!(Arc::ptr_eq(with.pxbow().unwrap(), &p)); + assert!(with.enable_pxbow(10, clock().0).is_err(), "once"); + let via_switch = p.route(15); + let id = via_switch.read64(0).data; + assert_eq!((id >> 28) & 0xf, 3, "the PIC's head 0 answers through widget 15"); + assert_eq!(via_switch.read64(crate::dev::pic::HEAD_OFFSET).data >> 12 & 0xffff, 0xd112); + assert!(Pic::new(15).enable_pxbow(15, clock().0).is_err(), "host and PIC differ"); +} + +/// A link reset on the host's link or the PIC's takes the PIC out of reach +/// for the pulse and brings it back, as the qtest's 20 ns clock step does. +#[test] +fn a_link_reset_makes_the_pic_unreachable_for_the_pulse() { + let (c, now) = clock(); + let pic = Pic::new(15); + let p = pic.enable_pxbow(10, c).unwrap(); + let via = p.route(15); + let wz = p.widget_zero(); + assert!(via.read64(0).is_ok()); + for (widget, reset) in [(10u64, 0x1b4u64), (15, 0x2f4)] { + now.fetch_add(100, Ordering::SeqCst); + assert_eq!(wz.write32(reset, 0), BUS_OK, "widget {widget}'s link reset"); + assert!(!via.read64(0).is_ok(), "down during the pulse"); + assert_eq!(via.write64(0x20, 0), BUS_ERR); + now.fetch_add(20, Ordering::SeqCst); + assert!(via.read64(0).is_ok(), "back after 20 ns"); + assert_eq!(via.read64(0x20).data, 0x0f00_320f, "the PIC kept its registers"); + } +} diff --git a/src/dev/xbow.rs b/src/dev/xbow.rs index 3f3352254..68cfce5ee 100644 --- a/src/dev/xbow.rs +++ b/src/dev/xbow.rs @@ -23,13 +23,44 @@ //! stock PROM resets each link it finds and expects it back), and register //! accesses are 32-bit at the data half of each eight-byte slot only. //! +//! # The XBridge's crossbar +//! +//! The IP35 boards' XBridge carries widget 0's Crossbow register map for its +//! own crossbar, built as [`Part::XBridge`] with [`Xbow::xbridge`]. It +//! differs in identification (part 0xd000, revision 2, with the fences and +//! the resource lock), Control's reset value and writable bits, the +//! performance counters' reset value and select field, the requesting +//! port in status bits 9:6, and the MicroLAN register's busy bit. It +//! interrupts for access errors alone, not for link events, and records +//! no multiple error; a malformed access answers zero rather than a bus +//! error. These follow hw/misc/sgi-xbridge-xbow.c in +//! jamesbraid/qemu-sgi-origami, not a published specification. +//! +//! # Behind a PIC +//! +//! The chip can also carry a part number and manufacturer, report a +//! revision other than the one whose behaviour it has, and name a source +//! widget in its status reads: what [`Pxbow`](super::pxbow::Pxbow) needs +//! behind a PIC. +//! +//! With a clock ([`Xbow::with_clock`]) a link reset is the two-core-cycle +//! pulse the QEMU model has: the link is down for 20 ns of the clock, and +//! the chip ends the pulse the next time anything reads its registers or +//! asks [`Xbow::carries_timed`], so what the guest sees follows the clock and not +//! host scheduling. Without one the pulse is over at once. [`Xbow::warm_reset`] +//! takes a warm reset from a link through the revision-5 fences. No board in +//! this tree sends one yet: the Hub's link control is the only sender the +//! QEMU model names. +//! //! Ported from hw/misc/sgi-xbow.c in jamesbraid/qemu-sgi-origami. -use std::sync::atomic::{AtomicBool, AtomicU8, Ordering}; +use std::sync::atomic::{AtomicBool, AtomicU64, AtomicU8, Ordering}; use std::sync::{Arc, OnceLock}; use parking_lot::Mutex; +use super::heart::Clock; + use crate::irq::InterruptController; use crate::snapshot::{field_bool, field_u32, field_u32s, field_u8, StateTable}; use crate::traits::{BusDevice, BusRead16, BusRead32, BusRead64, BusRead8, BUS_ERR, BUS_OK, Saveable}; @@ -72,6 +103,15 @@ const FENCE_MASK: u32 = 0xff; /// The first revision code with fences and the lock (Crossbow 2.0 in the /// software tables; an Octane2's code-5 part shows as 1.4 in its inventory). const REVISION_FENCES: u32 = 5; +const CORE_CYCLE_NS: u64 = 10; +/// How long a link reset holds the link down: two core cycles. +const LINK_RESET_NS: u64 = 2 * CORE_CYCLE_NS; +/// A warm reset arriving this many cycles after another is ignored. +const WARM_RESET_QUIET_CYCLES: u64 = 1024; +/// Where a status read carries the widget that asked, when the chip is told +/// to. +const STATUS_SOURCE_SHIFT: u32 = 6; +const STATUS_SOURCE_MASK: u32 = 0xf << STATUS_SOURCE_SHIFT; const LINK_BASE: u32 = 0x100; const LINK_STRIDE: u32 = 0x40; const LINK_END: u32 = LINK_BASE + LINK_STRIDE * LINKS as u32; @@ -153,24 +193,50 @@ struct Regs { fences: [u32; FENCES], /// The resource lock's byte. lock: u32, + /// When each link's reset pulse ends, in clock ns; 0 when none runs. + reset_due: [u64; LINKS], + /// Links that were alive when their pulse began and return alive. + reset_restore: u8, + /// A warm reset before this time is ignored. + warm_quiet_until: u64, +} + +/// Which crossbar a [`Xbow`] is: see the module documentation. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Part { + /// The Octane's and the Origin 2000's Crossbow. + Crossbow, + /// The crossbar inside the IP35 boards' XBridge. + XBridge, } +/// The XBridge crossbar's identification: part 0xd000, revision 2. +const XBRIDGE_ID: u32 = widget_id(2, 0xd000, 0); +const XBRIDGE_CONTROL_RESET: u32 = 0x80; +const XBRIDGE_CONTROL_MASK: u32 = 0xfe; +const XBRIDGE_PERF_SELECT_MASK: u32 = 0x0070_0000; +/// Status bits 9:6 name the port whose request this is. +const XBRIDGE_REQUESTER_SHIFT: u32 = 6; +/// The XBridge clears its widget 0 errors and request together. +const XBRIDGE_STATUS_CLEARED: u32 = STATUS_REQUEST | 0x3f; + impl Regs { /// `startup`: the links present, which come up alive. - fn reset(startup: u8) -> Self { + fn reset(part: Part, startup: u8) -> Self { + let xbridge = part == Part::XBridge; Self { status: 0, err_upper: 0, err_lower: 0, - control: 0, + control: if xbridge { XBRIDGE_CONTROL_RESET } else { 0 }, packet_timeout: PACKET_TIMEOUT_MASK, int_upper: INT_UPPER_TARGET_HIGH, int_lower: 0, err_cmd: 0, llp_control: LLP_CONTROL_RESET, arb_reload: ARB_RELOAD_RESET, - perf_a: PERF_RESET, - perf_b: PERF_RESET, + perf_a: if xbridge { 0 } else { PERF_RESET }, + perf_b: if xbridge { 0 } else { PERF_RESET }, nic: NIC_RESET, link_control: [LINK_CREDIT_RESET; LINKS], link_status: [0; LINKS], @@ -184,6 +250,9 @@ impl Regs { initializing: 0, eight_bit: 0, interrupt_asserted: false, + reset_due: [0; LINKS], + reset_restore: 0, + warm_quiet_until: 0, } } } @@ -191,15 +260,29 @@ impl Regs { /// What an access leaves to do once the register lock is dropped. #[derive(Default)] struct Out { - /// Widget 0's interrupt packets to HEART: (vector, set). - packets: Vec<(u8, bool)>, - /// A link reset's far-end widget. - link_reset: Option, + /// Widget 0's interrupt packets to the host: (address, vector, set). + packets: Vec<(u64, u8, bool)>, + /// Link resets' far-end widgets. + link_resets: Vec, } pub struct Xbow { regs: Mutex, revision: u32, + part: Part, + /// Revision field of the identification word when it differs from the + /// revision that selects behaviour. + reported_revision: Option, + /// Part number and manufacturer in a Crossbow's identification word. + identity: (u32, u32), + /// The widget a status read reports as its source, if the chip does. + status_source: Option, + clock: Option, + /// The earliest running pulse's end, or `u64::MAX`. Written under `regs`. + reset_due: AtomicU64, + /// A write to an error address register completes; see + /// [`Xbow::with_posted_error_address_writes`]. + posted_error_address_writes: bool, /// Links with something attached, bit n for widget 8 + n. present: AtomicU8, /// Links passing traffic, as `present`. Written under `regs`. @@ -230,10 +313,27 @@ impl Xbow { /// A Crossbow of `revision` with the links in `widgets` (a widget mask, /// bit n for widget n) attached and alive at power-on. pub fn new(revision: u32, widgets: u16) -> Self { + Self::with_part(Part::Crossbow, revision, widgets) + } + + /// The XBridge's crossbar with the links in `widgets` attached and + /// alive at power-on. + pub fn xbridge(widgets: u16) -> Self { + Self::with_part(Part::XBridge, 2, widgets) + } + + fn with_part(part: Part, revision: u32, widgets: u16) -> Self { let present = (widgets >> FIRST_WIDGET) as u8; Self { - regs: Mutex::new(Regs::reset(present)), + regs: Mutex::new(Regs::reset(part, present)), revision, + part, + reported_revision: None, + identity: (0, 0), + status_source: None, + clock: None, + reset_due: AtomicU64::new(u64::MAX), + posted_error_address_writes: false, present: AtomicU8::new(present), alive: AtomicU8::new(present), irq_order: Mutex::new(()), @@ -245,6 +345,47 @@ impl Xbow { } } + /// Complete a write to either error address register instead of + /// refusing it. The registers are read-only and the write still changes + /// nothing, sets the register-access error and is listed in + /// [`Xbow::unsupported`]: the specification makes a write to a read-only + /// register an error condition that widget 0's status reports. What the + /// processor sees is the other half: an uncached store is a write + /// request with no completion response (NEC U10278EJ4V0UM p.104, table + /// 6-1), so no error can come back to it. A stock IP35 PROM writes the + /// lower register on purpose in its crossbar check. QEMU's + /// `complete-posted-error-address-writes` property. + pub fn with_posted_error_address_writes(mut self) -> Self { + self.posted_error_address_writes = true; + self + } + + /// Answer to `part` and `manufacturer` in the identification word + /// instead of the Crossbow's zeros. + pub fn with_identity(mut self, part: u32, manufacturer: u32) -> Self { + self.identity = (part, manufacturer); + self + } + + /// Report `revision` in the identification word while behaviour stays + /// that of the revision given to [`Xbow::new`]. + pub fn with_reported_revision(mut self, revision: u32) -> Self { + self.reported_revision = Some(revision); + self + } + + /// Time link resets and warm resets by `clock`, in nanoseconds. + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = Some(clock); + self + } + + /// Report `widget` in bits 9:6 of every status read. + pub fn with_status_source(mut self, widget: u8) -> Self { + self.status_source = Some(widget); + self + } + /// Where widget 0's interrupt packets go: the host chip addressed as /// `host`, whose line is the vector the packet names. Once, before the /// machine runs. @@ -266,11 +407,9 @@ impl Xbow { } pub fn register_locks(self: &Arc) { - use crate::locks::register_lock_fn; - let me = self.clone(); - register_lock_fn("xbow::regs", move || me.regs.is_locked()); - let me = self.clone(); - register_lock_fn("xbow::irq_order", move || me.irq_order.is_locked()); + use crate::locks::register_mutex_fn; + register_mutex_fn("xbow::regs", self, |x| x.regs.is_locked()); + register_mutex_fn("xbow::irq_order", self, |x| x.irq_order.is_locked()); } /// Run `f` on the registers, then do what it left outside the lock: @@ -279,10 +418,17 @@ impl Xbow { let order = self.irq_order.lock(); let before = self.alive.load(Ordering::Acquire); let mut out = Out::default(); - let ret = f(&mut self.regs.lock(), &mut out); + let ret = { + let mut r = self.regs.lock(); + self.end_pulses(&mut r, &mut out); + let ret = f(&mut r, &mut out); + let due = r.reset_due.iter().copied().filter(|&d| d != 0).min().unwrap_or(u64::MAX); + self.reset_due.store(due, Ordering::Release); + ret + }; if let Some((_, h)) = self.host.get() { - for (vector, set) in out.packets { - h.set_line(vector as u32, set); + for (address, vector, set) in out.packets { + h.packet(0, address, vector as u32, set); } } drop(order); @@ -292,8 +438,10 @@ impl Xbow { cb(FIRST_WIDGET + link, self.alive.load(Ordering::Acquire) >> link & 1 != 0); } } - if let (Some(w), Some(cb)) = (out.link_reset, self.on_link_reset.get()) { - cb(w); + if let Some(cb) = self.on_link_reset.get() { + for w in out.link_resets { + cb(w); + } } ret } @@ -309,18 +457,48 @@ impl Xbow { } } - /// Whether the switch passes traffic to and from `widget`. Lock-free. + /// Whether the switch passes traffic to and from `widget`. A link reset + /// pulse ends here only on a register access; a chip with a clock asks + /// [`Self::carries_timed`]. pub fn carries(&self, widget: u8) -> bool { link_bit(widget).is_some_and(|b| self.alive.load(Ordering::Acquire) & b != 0) } + /// [`Self::carries`] on a chip with a clock: a link reset pulse that is + /// over by the clock ends first. Lock-free unless one is. + pub fn carries_timed(&self, widget: u8) -> bool { + if let Some(clock) = &self.clock { + if clock() >= self.reset_due.load(Ordering::Acquire) { + self.locked(|_, _| {}); + } + } + self.carries(widget) + } + pub fn unsupported(&self) -> Vec<(u32, u8, bool)> { self.unsupported.lock().clone() } fn identification(&self) -> u32 { - // Part number and manufacturer are zero on Crossbow. - (self.revision << 28) | 1 + match self.part { + // Part number and manufacturer are zero on Crossbow unless the + // board names others. + Part::Crossbow => { + let (part, manufacturer) = self.identity; + (self.reported_revision.unwrap_or(self.revision) << 28) | (part << 12) | (manufacturer << 1) | 1 + } + Part::XBridge => XBRIDGE_ID, + } + } + + /// The status register as the host reads it: on the XBridge, with the + /// host's port as the requester; on a chip told its source, with that. + fn status(&self, r: &Regs) -> u32 { + match (self.part, self.host.get(), self.status_source) { + (Part::XBridge, Some((host, _)), _) => r.status | u32::from(host.widget) << XBRIDGE_REQUESTER_SHIFT, + (Part::Crossbow, _, Some(widget)) => (r.status & !STATUS_SOURCE_MASK) | (u32::from(widget) << STATUS_SOURCE_SHIFT), + _ => r.status, + } } fn link_state(&self, r: &Regs, link: usize) -> u32 { @@ -365,7 +543,8 @@ impl Xbow { /// deassertion once every request is acknowledged. A packet to anything /// but the host chip goes nowhere on this machine. fn update_interrupt(&self, r: &mut Regs, out: &mut Out) { - for link in 0..LINKS { + // The XBridge's crossbar interrupts for its own errors alone. + for link in (0..LINKS).filter(|_| self.part == Part::Crossbow) { if Self::link_requesting(r, link) { r.status |= 1 << (24 + link); } else { @@ -385,12 +564,13 @@ impl Xbow { Some((host, _)) if target == host.widget && self.carries(target) => {} _ => return, } - out.packets.push((vector, pending)); + let address = (((r.int_upper & 0xffff) as u64) << 32) | r.int_lower as u64; + out.packets.push((address, vector, pending)); r.interrupt_asserted = pending; } fn record_access_error(&self, r: &mut Regs, out: &mut Out) { - if r.status & STATUS_ACCESS_ERROR != 0 { + if r.status & STATUS_ACCESS_ERROR != 0 && self.part == Part::Crossbow { r.status |= STATUS_MULTIPLE_ERROR; } r.status |= STATUS_ACCESS_ERROR; @@ -411,7 +591,8 @@ impl Xbow { } fn accessible(&self, slot: u32, write: bool) -> bool { - if self.revision >= REVISION_FENCES && ((FENCE_BASE..FENCE_END).contains(&slot) || slot == LOCK || slot == LOCK_CLEAR) { + let fences = self.revision >= REVISION_FENCES || self.part == Part::XBridge; + if fences && ((FENCE_BASE..FENCE_END).contains(&slot) || slot == LOCK || slot == LOCK_CLEAR) { return slot % 8 == 0; } match slot { @@ -431,9 +612,10 @@ impl Xbow { /// Decode an access to widget 0's window. Registers are 32 bits in the /// low half of an eight-byte slot; an eight-byte access has no /// established packet form and is refused without a status flag, any - /// other malformed access is an access error. + /// other malformed access is an access error. The XBridge's crossbar + /// takes an eight-byte access as an access error too. fn decode(&self, r: &mut Regs, out: &mut Out, off: u32, size: usize, write: bool) -> Option { - if size == 8 { + if size == 8 && self.part == Part::Crossbow { self.note(off, size, write); return None; } @@ -447,12 +629,15 @@ impl Xbow { } /// A processor read of widget 0's window at `off`; `None` is a bus error. + /// The XBridge's crossbar answers a malformed read with zero. pub fn read(&self, off: u32, size: usize) -> Option { self.locked(|r, out| { - let slot = self.decode(r, out, off, size, false)?; + let Some(slot) = self.decode(r, out, off, size, false) else { + return (self.part == Part::XBridge).then_some(0); + }; let v = match slot { IDENTIFICATION => self.identification(), - STATUS => r.status, + STATUS => self.status(r), ERR_UPPER => r.err_upper, ERR_LOWER => r.err_lower, CONTROL => r.control, @@ -464,8 +649,11 @@ impl Xbow { STATUS_CLEAR => { // Acknowledges widget 0's own errors and request; each // link's request is acknowledged at its own alias. - let v = r.status; - r.status &= !(STATUS_WIDGET0_ERRORS | STATUS_REQUEST); + let v = self.status(r); + r.status &= !match self.part { + Part::Crossbow => STATUS_WIDGET0_ERRORS | STATUS_REQUEST, + Part::XBridge => XBRIDGE_STATUS_CLEARED, + }; self.update_interrupt(r, out); v } @@ -501,14 +689,23 @@ impl Xbow { }) } - /// A processor write; false is a bus error. + /// A processor write; false is a bus error. The XBridge's crossbar + /// takes a malformed write and drops it. pub fn write(&self, off: u32, size: usize, val: u64) -> bool { + let xbridge = self.part == Part::XBridge; self.locked(|r, out| { - let Some(slot) = self.decode(r, out, off, size, true) else { return false }; + let Some(slot) = self.decode(r, out, off, size, true) else { + // Refused and recorded either way; a posted error address + // write completes all the same. + return xbridge + || (self.posted_error_address_writes + && size == 4 + && matches!(off.wrapping_sub(4), ERR_UPPER | ERR_LOWER)); + }; let val = val as u32; match slot { CONTROL => { - r.control = val & CONTROL_MASK; + r.control = val & if xbridge { XBRIDGE_CONTROL_MASK } else { CONTROL_MASK }; self.update_interrupt(r, out); } PACKET_TIMEOUT => r.packet_timeout = val & PACKET_TIMEOUT_MASK, @@ -523,8 +720,13 @@ impl Xbow { LLP_CONTROL => r.llp_control = val & LLP_CONTROL_MASK, ARB_RELOAD => r.arb_reload = val & ARB_RELOAD_MASK, // The selector is kept; nothing counts. - PERF_A => r.perf_a = val & PERF_SELECT_MASK, - PERF_B => r.perf_b = val & PERF_SELECT_MASK, + PERF_A | PERF_B => { + let v = val & if xbridge { XBRIDGE_PERF_SELECT_MASK } else { PERF_SELECT_MASK }; + *if slot == PERF_A { &mut r.perf_a } else { &mut r.perf_b } = v; + } + // The XBridge's one-wire master is not modelled: it reports + // itself done (bit 1) at once. + NIC if xbridge => r.nic = (val & NIC_MASK & !3) | NIC_RESET, NIC => r.nic = val & NIC_MASK, s if (FENCE_BASE..FENCE_END).contains(&s) => r.fences[((s - FENCE_BASE) / 8) as usize] = val & FENCE_MASK, // Test-and-set: stored only while the lock is free. @@ -548,7 +750,7 @@ impl Xbow { self.link_reset(r, link, out); // After the lock: the widget's reset may ask // the switch about its links. - out.link_reset = Some(FIRST_WIDGET + link as u8); + out.link_resets.push(FIRST_WIDGET + link as u8); } _ => unreachable!(), } @@ -558,13 +760,17 @@ impl Xbow { }) } - /// A link reset. The pulse is two core cycles, over before the + /// A link reset. Without a clock the pulse is over before the /// processor can look again, so it is not kept. A fitted link that was /// alive comes back alive with an alive event, as the Octane's /// restore-alive policy has it; one that was down stays down, still /// initializing; an empty port fails initialization. fn link_reset(&self, r: &mut Regs, link: usize, out: &mut Out) { let bit = 1u8 << link; + if let Some(clock) = &self.clock { + self.begin_pulse(r, link, clock(), out); + return; + } let was_alive = self.alive.load(Ordering::Acquire) & bit != 0; r.eight_bit &= !bit; if self.present.load(Ordering::Acquire) & bit != 0 { @@ -585,13 +791,102 @@ impl Xbow { } self.update_interrupt(r, out); } + + /// A link reset with a clock: the link drops now and the pulse ends + /// two core cycles on. A fitted link that was alive comes back alive + /// then, with an alive event; one that was down comes out of + /// initialization still down; an empty port fails initialization and + /// stays failed. + fn begin_pulse(&self, r: &mut Regs, link: usize, now: u64, out: &mut Out) { + let bit = 1u8 << link; + let present = self.present.load(Ordering::Acquire) & bit != 0; + if present && self.alive.load(Ordering::Acquire) & bit != 0 { + r.reset_restore |= bit; + } else { + r.reset_restore &= !bit; + } + self.alive.fetch_and(!bit, Ordering::AcqRel); + r.eight_bit &= !bit; + if present { + r.initializing |= bit; + } else { + r.initializing &= !bit; + r.came_up &= !bit; + } + r.reset_due[link] = now + LINK_RESET_NS; + self.update_interrupt(r, out); + } + + /// End every pulse the clock says is over. + fn end_pulses(&self, r: &mut Regs, out: &mut Out) { + let Some(clock) = &self.clock else { return }; + if r.reset_due.iter().all(|&d| d == 0) { + return; + } + let now = clock(); + for link in 0..LINKS { + let bit = 1u8 << link; + if r.reset_due[link] == 0 || r.reset_due[link] > now { + continue; + } + r.reset_due[link] = 0; + if self.present.load(Ordering::Acquire) & bit == 0 { + continue; + } + r.initializing &= !bit; + if r.reset_restore & bit != 0 { + self.alive.fetch_or(bit, Ordering::AcqRel); + r.came_up |= bit; + r.link_event[link] |= LINK_ALIVE; + if r.link_control[link] & LINK_EIGHT_BIT_FORCE != 0 { + r.eight_bit |= bit; + } + } + r.reset_restore &= !bit; + self.update_interrupt(r, out); + } + } + + /// A warm reset arriving on the link of `source` widget, at `now` ns. + /// The first in 1024 core cycles is taken and later ones are ignored. + /// On revision 5 and later each destination has a fence naming the + /// sources it admits: the core resets only if its fence admits the + /// source, and each link whose fence does gets a reset pulse out to + /// its far end ([`Self::on_link_reset`]). Fences keep their programmed + /// value; the lock clears with the core. Earlier revisions admit every + /// source. + pub fn warm_reset(&self, source: u8, now: u64) { + let Some(port) = link_bit(source).map(|b| b.trailing_zeros() as usize) else { return }; + let fenced = self.revision >= REVISION_FENCES || self.part == Part::XBridge; + self.locked(|r, out| { + if now < r.warm_quiet_until { + crate::dlog!(crate::devlog::LogModule::Bedrock, "xbow: warm reset from widget {source} inside the quiet window; ignored"); + return; + } + let fences = r.fences; + let admits = |fence: u32| !fenced || fence >> port & 1 != 0; + if admits(fences[0]) { + let present = self.present.load(Ordering::Acquire); + *r = Regs::reset(self.part, present); + self.alive.store(present, Ordering::Release); + r.fences = fences; + self.update_interrupt(r, out); + } + r.warm_quiet_until = now + WARM_RESET_QUIET_CYCLES * CORE_CYCLE_NS; + for (link, &fence) in fences[1..].iter().enumerate() { + if admits(fence) { + out.link_resets.push(FIRST_WIDGET + link as u8); + } + } + }); + } } impl crate::traits::Resettable for Xbow { fn power_on(&self) { self.locked(|r, _| { let present = self.present.load(Ordering::Acquire); - *r = Regs::reset(present); + *r = Regs::reset(self.part, present); self.alive.store(present, Ordering::Release); }); } @@ -769,8 +1064,15 @@ impl Saveable for Xbow { interrupt_asserted: field_bool(v, "interrupt_asserted")?, fences: field_u32s(v, "fences")?, lock: field_u32(v, "lock")?, + // A link reset lasts two core cycles and the warm-reset quiet + // time is counted on the board clock; a snapshot carries + // neither, so a restored crossbow has none in progress. + reset_due: [0; LINKS], + reset_restore: 0, + warm_quiet_until: 0, }; *self.regs.lock() = n; + self.reset_due.store(u64::MAX, Ordering::Release); self.alive.store(field_u8(v, "alive")?, Ordering::Release); Ok(()) } @@ -880,6 +1182,133 @@ mod tests { assert_eq!(isr.0.load(Ordering::SeqCst), 1 << 0x2b); } + /// A host that decodes the packet itself gets the destination address + /// the interrupt registers name, from widget 0. + #[test] + fn a_host_that_decodes_packets_sees_their_address() { + #[derive(Default)] + struct Decoder(Mutex>); + impl InterruptController for Decoder { + fn set_line(&self, _: u32, _: bool) { + unreachable!("a packet host takes packets"); + } + fn packet(&self, source: u8, address: u64, line: u32, level: bool) -> bool { + self.0.lock().push((source, address, line, level)); + true + } + } + let x = octane(); + let decoder = Arc::new(Decoder::default()); + x.connect_host(HEART_HOST, decoder.clone()); + assert!(x.write(INT_UPPER + 4, 4, (0x2a << 24) | (8 << 16) | 0x12)); + assert!(x.write(INT_LOWER + 4, 4, 0x0180_0090)); + assert!(x.write(CONTROL + 4, 4, STATUS_ACCESS_ERROR as u64)); + assert_eq!(x.read(0x00, 4), None); + x.read(STATUS_CLEAR + 4, 4).unwrap(); + assert_eq!(*decoder.0.lock(), [(0, 0x12_0180_0090, 0x2a, true), (0, 0x12_0180_0090, 0x2a, false)]); + } + + /// The lock registry outlives a machine's tests; it must not keep the + /// Crossbow alive. + #[test] + fn registering_the_locks_does_not_keep_the_crossbow_alive() { + let x = Arc::new(octane()); + x.register_locks(); + let weak = Arc::downgrade(&x); + drop(x); + assert!(weak.upgrade().is_none()); + } + + // The XBridge's crossbar. Cases follow tests/qtest/sn1-xbridge-xbow-test.c + // (reset-values, lock, fences) and sn1-xbridge-test.c (access-errors) in + // jamesbraid/qemu-sgi-origami. + + /// The Fuel's: Bedrock at 10 and the two PCI heads at 14 and 15. + fn xbridge() -> Xbow { + Xbow::xbridge(1 << 10 | 1 << 14 | 1 << 15) + } + + fn get(x: &Xbow, slot: u32) -> u32 { + x.read(slot + 4, 4).unwrap() as u32 + } + + /// Software probing the crossbar after reset sees these values. Each + /// link's control word resets to 0x80000; that value belongs in the + /// control word, not in the link reset register, which no read can see. + #[test] + fn the_xbridge_crossbar_resets_to_its_own_values() { + let x = xbridge(); + for (slot, value) in [ + (IDENTIFICATION, 0x2d00_0001), + (ERR_UPPER, 0), + (ERR_LOWER, 0), + (CONTROL, 0x80), + (PACKET_TIMEOUT, 0xf_ffff), + (INT_UPPER, 0x8_0000), + (INT_LOWER, 0), + (ERR_CMD, 0), + (LLP_CONTROL, 0x03ff_1810), + (ARB_RELOAD, 2), + (PERF_A, 0), + (PERF_B, 0), + (NIC, 2), + (FENCE_BASE, 0xff), + (LOCK, 0), + ] { + assert_eq!(get(&x, slot), value, "{slot:#x}"); + } + for link in 0..LINKS as u32 { + let base = LINK_BASE + link * LINK_STRIDE; + assert_eq!(get(&x, FENCE_BASE + 8 + link * 8), 0xff); + assert_eq!(get(&x, base + LINK_CONTROL), 0x8_0000, "link {}", link + 8); + assert_eq!(get(&x, base + LINK_ARB_UPPER), 0); + assert_eq!(get(&x, base + LINK_ARB_LOWER), 0); + } + assert!(x.carries(10) && x.carries(14) && x.carries(15) && !x.carries(13)); + } + + /// A write takes the resource lock only while it is free, and every + /// reset fence bit is read and write in the low eight bits. + #[test] + fn the_xbridge_crossbar_has_the_lock_and_the_fences() { + let x = xbridge(); + assert!(x.write(LOCK + 4, 4, 0x5a)); + assert!(x.write(LOCK + 4, 4, 0x3c)); + assert_eq!((get(&x, LOCK), get(&x, LOCK_CLEAR)), (0x5a, 0x5a)); + assert!(x.write(LOCK_CLEAR + 4, 4, 0)); + assert!(x.write(LOCK + 4, 4, 0x3c)); + assert_eq!(get(&x, LOCK), 0x3c); + for slot in (FENCE_BASE..FENCE_END).step_by(8) { + assert!(x.write(slot + 4, 4, 0)); + assert_eq!(get(&x, slot), 0); + assert!(x.write(slot + 4, 4, u64::from(u32::MAX))); + assert_eq!(get(&x, slot), 0xff); + } + } + + /// A malformed access answers zero and latches the access error, which + /// interrupts once while enabled; status carries the host's port in + /// bits 9:6, and reading the clear alias drops the interrupt. Link + /// events do not interrupt. + #[test] + fn the_xbridge_crossbar_reports_its_first_error_once() { + let x = xbridge(); + let isr = Arc::new(Isr::default()); + x.connect_host(XtalkHost { widget: 10, int_packet: 0, int_packet_mask: 0, vectors: 256 }, isr.clone()); + assert!(x.write(INT_UPPER + 4, 4, 0x150a_0000)); + assert!(x.write(LINK_BASE + LINK_STRIDE * 7 + LINK_CONTROL + 4, 4, LINK_ALIVE_ENABLE as u64)); + assert!(x.write(LINK_BASE + LINK_STRIDE * 7 + LINK_RESET + 4, 4, 0)); + assert_eq!(isr.0.load(Ordering::SeqCst), 0, "a link event does not interrupt"); + assert!(x.write(CONTROL + 4, 4, u64::from(STATUS_ACCESS_ERROR))); + assert_eq!(x.read(IDENTIFICATION, 8), Some(0)); + assert!(x.write(IDENTIFICATION + 4, 4, 1)); + assert_eq!(isr.0.load(Ordering::SeqCst), 1 << 0x15); + assert_eq!(get(&x, STATUS), STATUS_REQUEST | STATUS_ACCESS_ERROR | 10 << 6); + assert_eq!(get(&x, STATUS_CLEAR) & (STATUS_REQUEST | STATUS_ACCESS_ERROR), STATUS_REQUEST | STATUS_ACCESS_ERROR); + assert_eq!(isr.0.load(Ordering::SeqCst), 0); + assert_eq!(get(&x, STATUS), 10 << 6); + } + #[test] fn reset_restores_startup_links() { let x = octane(); @@ -906,6 +1335,166 @@ mod tests { assert_eq!(x.write32(0x1000_0000 + IDENTIFICATION as u64 + 4, 0), BUS_ERR); } + /// A write to a read-only error address register is an access error and + /// changes nothing. With posted error + /// address writes the store still completes, as QEMU's crossbar does with + /// `complete-posted-error-address-writes` (hw/misc/sgi-xbow.c, asserted + /// set on the Origin 2000 by sn0-origin2000-test's xio-owner case); other + /// read-only registers are refused as before. + #[test] + fn a_posted_error_address_write_completes_and_still_records_the_access_error() { + let refusing = octane(); + assert_eq!(refusing.write32(ERR_LOWER as u64 + 4, 0xdead_beef), BUS_ERR); + let x = octane().with_posted_error_address_writes(); + assert_eq!(x.write32(ERR_LOWER as u64 + 4, 0xdead_beef), BUS_OK); + assert_eq!(x.read(ERR_LOWER + 4, 4), Some(0), "the register keeps its value"); + assert_eq!(x.read(STATUS + 4, 4).unwrap() as u32 & STATUS_ACCESS_ERROR, STATUS_ACCESS_ERROR); + assert_eq!(x.write32(ERR_UPPER as u64 + 4, 0), BUS_OK); + assert_eq!(x.read(STATUS + 4, 4).unwrap() as u32 & STATUS_MULTIPLE_ERROR, STATUS_MULTIPLE_ERROR); + assert_eq!(x.unsupported(), vec![(ERR_LOWER + 4, 4, true), (ERR_UPPER + 4, 4, true)]); + assert_eq!(x.write32(IDENTIFICATION as u64 + 4, 0), BUS_ERR, "only the error address registers"); + assert_eq!(x.write64(ERR_LOWER as u64, 0), BUS_ERR, "nor an eight-byte write"); + } + + /// A Crossbow built without the posted-write option, as the Octane and + /// any other standalone one is, refuses both error address registers and + /// still records the access error; only the PXBOW completes them. + #[test] + fn a_standalone_crossbow_refuses_both_error_address_writes() { + for revision in [4, 5] { + let x = Xbow::new(revision, 0x8400); + assert_eq!(x.write32(ERR_LOWER as u64 + 4, 0xdead_beef), BUS_ERR, "revision {revision}"); + assert_eq!(x.write32(ERR_UPPER as u64 + 4, 0), BUS_ERR, "revision {revision}"); + assert_eq!(x.read(STATUS + 4, 4).unwrap() as u32 & STATUS_ACCESS_ERROR, STATUS_ACCESS_ERROR); + } + } + + /// The fences, the lock and a non-zero part number belong to revision 5 + /// and later; the Octane's revision 4 leaves them reserved. + #[test] + fn fences_the_lock_and_the_identity_fields_need_the_revision_that_has_them() { + let x = octane(); + for slot in [FENCE_BASE, (FENCE_BASE + 8), LOCK, LOCK_CLEAR] { + assert_eq!(x.read(slot + 4, 4), None, "slot {slot:#x} is reserved on revision 4"); + assert!(!x.write(slot + 4, 4, 1)); + } + assert_eq!(x.read(IDENTIFICATION + 4, 4), Some(0x4000_0001)); + let y = Xbow::new(5, 0x8400).with_identity(0xd100, 0); + assert_eq!(y.read(IDENTIFICATION + 4, 4), Some(0x5d10_0001)); + assert_eq!(y.read(FENCE_BASE + 4, 4), Some(0xff)); + assert!(y.write(FENCE_BASE + 4, 4, 0x1ff)); + assert_eq!(y.read(FENCE_BASE + 4, 4), Some(0xff), "a fence is eight bits"); + } + + fn timed(revision: u32) -> (Xbow, Arc) { + let now = Arc::new(AtomicU64::new(0)); + let clock = now.clone(); + let x = Xbow::new(revision, 0x8100).with_clock(Arc::new(move || clock.load(Ordering::SeqCst))); + (x, now) + } + + /// Link reset is a pulse of two core cycles: the link is down and not + /// initializing-failed while it lasts, then a fitted live link returns + /// with an alive event. (sn0-xbow-test link-reset-pulse) + #[test] + fn a_link_reset_holds_the_link_down_for_twenty_nanoseconds() { + let (x, now) = timed(4); + let status = |link| LINK_BASE + LINK_STRIDE * link + LINK_STATUS + 4; + now.store(1000, Ordering::SeqCst); + assert!(x.write(LINK_BASE + LINK_STRIDE * 3 + LINK_RESET + 4, 4, 0), "widget 11 has nothing fitted"); + assert!(x.write(LINK_BASE + LINK_STRIDE * 7 + LINK_RESET + 4, 4, 0)); + assert!(!x.carries_timed(15)); + assert_eq!(x.read(status(7), 4), Some(0), "initializing shows none of the three bits"); + assert_eq!(x.read(LINK_BASE + LINK_STRIDE * 7 + LINK_AUX_STATUS + 4, 4), Some(AUX_PRESENT as u64)); + now.store(1019, Ordering::SeqCst); + assert!(!x.carries_timed(15)); + now.store(1020, Ordering::SeqCst); + assert!(x.carries_timed(15), "the pulse is over by the clock without any register access"); + assert_eq!(x.read(status(7), 4).unwrap() as u32 & LINK_ALIVE, LINK_ALIVE); + assert!(!x.carries_timed(11), "an empty port stays down"); + assert_eq!(x.read(status(3), 4), Some(LINK_MAX_RETRY as u64)); + assert!(x.carries_timed(8), "other links were not touched"); + } + + /// A register read ends a finished pulse before it answers. + #[test] + fn a_register_read_after_the_pulse_sees_the_link_back() { + let (x, now) = timed(4); + assert!(x.write(LINK_BASE + LINK_STRIDE * 7 + LINK_RESET + 4, 4, 0)); + now.store(20, Ordering::SeqCst); + let aux = x.read(LINK_BASE + LINK_STRIDE * 7 + LINK_AUX_STATUS + 4, 4).unwrap() as u32; + assert_eq!(aux & (AUX_PRESENT | AUX_FAILURE_MODE), AUX_PRESENT); + } + + fn resets(x: &Xbow) -> Arc>> { + let seen = Arc::new(Mutex::new(Vec::new())); + let s = seen.clone(); + x.on_link_reset(Box::new(move |w| s.lock().push(w))); + seen + } + + /// Fences gate a warm reset per destination, an accepted one leaves + /// them as programmed, and a machine reset restores them. + /// (sn0-xbow-test fence-persists-across-warm-reset, + /// fence-independence-link-outlives-core-rejection) + #[test] + fn a_warm_reset_obeys_each_destinations_fence_and_keeps_the_fences() { + let (x, now) = timed(5); + let seen = resets(&x); + assert!(x.write(CONTROL + 4, 4, 0x26)); + assert!(x.write(FENCE_BASE + 4, 4, 0xbf)); + assert!(x.write((FENCE_BASE + 8) + 8 * 7 + 4, 4, 0x3c)); + assert!(x.write((FENCE_BASE + 8) + 4, 4, 0xfd)); + x.warm_reset(15, 0); + assert_eq!(x.read(CONTROL + 4, 4), Some(0), "the core admitted widget 15 and reset"); + assert_eq!(x.read(FENCE_BASE + 4, 4), Some(0xbf)); + assert_eq!(x.read((FENCE_BASE + 8) + 8 * 7 + 4, 4), Some(0x3c)); + // Link 7's fence refuses port 15 (bit 7 clear); the rest admit it. + assert_eq!(*seen.lock(), (8..15).collect::>()); + // A core that refuses the source is left alone while links still admit. + assert!(x.write(CONTROL + 4, 4, 0x26)); + assert!(x.write(FENCE_BASE + 4, 4, 0xbf & !(1 << 7))); + now.store(20_000, Ordering::SeqCst); + x.warm_reset(15, 20_000); + assert_eq!(x.read(CONTROL + 4, 4), Some(0x26)); + crate::traits::Resettable::power_on(&x); + assert_eq!(x.read(FENCE_BASE + 4, 4), Some(0xff)); + assert_eq!(x.read((FENCE_BASE + 8) + 8 * 7 + 4, 4), Some(0xff)); + } + + /// A second warm reset inside 1024 core cycles is the first one's echo. + /// The lock clears with the core only. (sn0-xbow-test + /// lock-clears-with-core-reset-only) + #[test] + fn a_warm_reset_inside_the_quiet_window_is_ignored_and_the_lock_follows_the_core() { + let (x, _) = timed(5); + let seen = resets(&x); + assert!(x.write(FENCE_BASE + 4, 4, 0xff & !(1 << 7))); + assert!(x.write(LOCK + 4, 4, 0x66)); + x.warm_reset(15, 100); + assert_eq!(x.read(LOCK + 4, 4), Some(0x66), "fenced out: the core is untouched"); + assert_eq!(seen.lock().len(), 8, "every link admitted the source"); + assert!(x.write(FENCE_BASE + 4, 4, 0xff)); + x.warm_reset(15, 100 + 10_239); + assert_eq!(x.read(LOCK + 4, 4), Some(0x66), "still inside the window"); + assert_eq!(seen.lock().len(), 8); + x.warm_reset(15, 100 + 10_240); + assert_eq!(x.read(LOCK + 4, 4), Some(0)); + assert_eq!(seen.lock().len(), 16); + } + + /// Revision 4 has no fences, so a warm reset always reaches the core and + /// every link. (sn0-xbow-test warm-reset-revision-4-unaffected-by-fence) + #[test] + fn revision_four_admits_every_warm_reset() { + let (x, _) = timed(4); + let seen = resets(&x); + assert!(x.write(CONTROL + 4, 4, 0x26)); + assert!(!x.write(FENCE_BASE + 4, 4, 0), "reserved on this revision"); + x.warm_reset(15, 0); + assert_eq!(x.read(CONTROL + 4, 4), Some(0)); + assert_eq!(seen.lock().len(), 8); + } #[test] fn link_changes_reach_the_callback_outside_the_lock() { let x = Arc::new(octane()); diff --git a/src/dev/xbridge.rs b/src/dev/xbridge.rs new file mode 100644 index 000000000..3514caead --- /dev/null +++ b/src/dev/xbridge.rs @@ -0,0 +1,174 @@ +//! The Fuel's XBridge: two PCI heads at crossbar ports 14 and 15, and the +//! crossbar that joins them, and any XIO board, to the host at port 10. +//! +//! Each head is a [`Bridge`] built as [`Part::XBridge`], and the crossbar a +//! Crossbow built as [`Xbow::xbridge`]; `bridge.rs` and `xbow.rs` say where +//! they differ from the Octane's chips. This module wires them together: +//! every head sends its DMA and interrupt packets to the host the board +//! names, and a link reset of a head's port resets the head and runs RST# +//! on its bus before the board resets what it attached. +//! +//! # What is approximated here +//! +//! The crossbar's reset fences are kept but gate nothing: no warm reset +//! travels through it. Link training is not modelled; a link reset brings a +//! fitted link back alive at once, as on the Crossbow. +//! +//! Ported from hw/pci-host/sgi-xbridge.c and hw/misc/sgi-xbridge-xbow.c +//! in jamesbraid/qemu-sgi-origami, revision +//! fe128635b6dfe2511f1bd451684cb30007686df1. + +use crate::dev::bridge::{Bridge, Part}; +use crate::dev::xbow::Xbow; +use crate::irq::InterruptController; +use crate::pci::{IntxRouting, PciBus, PciDma, PciHost}; +use crate::traits::{BusDevice, Resettable}; +use crate::xtalk::{Upstream, XtalkHost}; +use parking_lot::RwLock; +use std::sync::{Arc, Weak}; + +pub const HOST_WIDGET: u8 = 10; +pub const HEAD_E: u8 = 14; +pub const HEAD_F: u8 = 15; + +/// The head's PCI interrupt routing. Device `d`'s INTA reaches interrupt pin `d`; the +/// pins are shared with the board's own IRQs (the IOC3's SuperIO interrupt +/// is `BoardIrq(0)`). The Fuel uses devices 1, 4 and 5; the others follow the +/// same rule, and INTB to INTD are not wired. +pub static PCI_ROUTING: IntxRouting = IntxRouting { + devices: 8, + irqs: 8, + intx: &[ + (0, [Some(0), None, None, None]), + (1, [Some(1), None, None, None]), + (2, [Some(2), None, None, None]), + (3, [Some(3), None, None, None]), + (4, [Some(4), None, None, None]), + (5, [Some(5), None, None, None]), + (6, [Some(6), None, None, None]), + (7, [Some(7), None, None, None]), + ], + board_irqs: &[0, 1, 2, 3, 4, 5, 6, 7], +}; + +/// What the board resets when a port's link reset reaches it. +type ResetOutput = Arc; + +/// One chip: both heads and the crossbar. +pub struct XBridge { + heads: [Arc; 2], + /// Each head's XIO space, as the host reaches it at the head's port. + windows: [Arc; 2], + xbow: Arc, + reset_output: Arc>>, +} + +impl XBridge { + /// An XBridge whose crossbar has `host` at its port. `port(widget)` is + /// the host as a widget sending through the crossbar reaches it: the + /// memory its DMA lands in and the input its interrupt packets set. + /// The crossbar's own packets come from widget 0. + pub fn new(host: XtalkHost, port: &dyn Fn(u8) -> (Arc, Arc)) -> Arc { + let xbow = Arc::new(Xbow::xbridge(1 << host.widget | 1 << HEAD_E | 1 << HEAD_F)); + xbow.connect_host(host, port(0).1); + let heads = [HEAD_E, HEAD_F].map(|widget| { + let (mem, irq) = port(widget); + let head = Bridge::with_part(Part::XBridge, Upstream::single(host, mem, irq), &PCI_ROUTING); + head.set_revision(2); + head.set_widget(widget); + // Head E's bus runs at 66 MHz (speed code 1), head F's at 33. + head.set_control_strap(u32::from(widget == HEAD_E) << 4); + let links = xbow.clone(); + head.set_link_alive(Box::new(move |w| links.carries(w))); + head + }); + let reset_output: Arc>> = Default::default(); + let resets: [Weak; 2] = heads.each_ref().map(Arc::downgrade); + let output = reset_output.clone(); + xbow.on_link_reset(Box::new(move |port| { + if let Some(head) = port.checked_sub(HEAD_E).and_then(|i| resets.get(usize::from(i))).and_then(Weak::upgrade) { + head.reset(); + head.reset_pci(); + } + // Cloned out, so the board may replace it or reach the chip. + let out = output.read().clone(); + if let Some(out) = out { + out(port); + } + })); + let windows = heads.each_ref().map(|head| head.head_window()); + Arc::new(Self { heads, windows, xbow, reset_output }) + } + + pub fn register_locks(&self) { + for head in &self.heads { + head.register_locks(); + } + self.xbow.register_locks(); + } + + fn index(widget: u8) -> usize { + assert!((HEAD_E..=HEAD_F).contains(&widget), "widget {widget} is not an XBridge head"); + usize::from(widget - HEAD_E) + } + + /// What the host connects at crossbar port `widget`: the crossbar's + /// registers at 0 and at the host's own port, a head's XIO space at its + /// port. A board's port is the one [`XBridge::attach_widget`] returns. + pub fn widget(&self, widget: u8) -> Option> { + match widget { + 0 | HOST_WIDGET => Some(self.xbow.clone()), + HEAD_E | HEAD_F => Some(self.windows[Self::index(widget)].clone()), + _ => None, + } + } + + /// The crossbar's widget 0 registers. + pub fn crossbar(&self) -> Arc { + self.xbow.clone() + } + + /// The PCI bus behind head `widget`: functions are fitted with + /// [`PciBus::attach`] and the SuperIO interrupt is `bus.board_irq(BoardIrq(0))`. + pub fn bus(&self, widget: u8) -> &PciBus { + self.heads[Self::index(widget)].bus() + } + + /// The DMA head `widget` gives the function in `slot`. + pub fn dma(&self, widget: u8, slot: u8) -> Arc { + assert!(slot < 8); + self.heads[Self::index(widget)].dma_for(slot) + } + + /// Fit a board at crossbar port `widget`, its link up. What the host + /// connects at that port is returned: the board while its link is up. + pub fn attach_widget(&self, widget: u8, board: Arc) -> Arc { + assert!((8..HEAD_E).contains(&widget) && widget != HOST_WIDGET); + self.xbow.attach(widget); + self.xbow.port(widget, board) + } + + /// Told of each port's link reset, after a head's own reset and RST# on + /// its bus and with no chip lock held: the board resets the engines it + /// attached behind that port. + pub fn set_reset_output(&self, output: ResetOutput) { + *self.reset_output.write() = Some(output); + } + + pub fn reset(&self) { + for head in &self.heads { + head.reset(); + head.reset_pci(); + } + self.xbow.power_on(); + } +} + +impl Resettable for XBridge { + fn power_on(&self) { + self.reset(); + } +} + +#[cfg(test)] +mod tests; diff --git a/src/dev/xbridge/tests.rs b/src/dev/xbridge/tests.rs new file mode 100644 index 000000000..677adb311 --- /dev/null +++ b/src/dev/xbridge/tests.rs @@ -0,0 +1,290 @@ +// Cases follow tests/qtest/sn1-xbridge-chip-test.c (head-dma-reaches-the-host) +// and sn1-xbridge-test.c (interrupt-packets, link-reset) in +// jamesbraid/qemu-sgi-origami. The heads' and the crossbar's register +// contracts are tested with their parts in bridge_tests.rs and xbow.rs. +use super::*; +use crate::pci::{Bar, BoardIrq, DevFn, PciConfig, PciConfigPolicy, PciDmaStatus, PciFunction, PciFunctionState}; +use crate::traits::{BusRead32, BusRead8, BUS_ERR, BUS_OK}; +use parking_lot::Mutex; + +const BEDROCK: XtalkHost = XtalkHost { widget: HOST_WIDGET, int_packet: 0, int_packet_mask: 0, vectors: 256 }; +const INT_DEST_UPPER: u64 = 0x30 + 4; +const INT_DEST_LOWER: u64 = 0x38 + 4; +const INT_ENABLE: u64 = 0x108 + 4; +const INT_MODE: u64 = 0x118 + 4; +const INT_STATUS: u64 = 0x100 + 4; +const INT_HOST_0: u64 = 0x130 + 4; +const CONTROL: u64 = 0x20 + 4; +const DIRECT_MAP: u64 = 0x80 + 4; +const PCI_TYPE1_CONFIG: u64 = 0xc8 + 4; +const XBOW_INT_UPPER: u64 = 0x30 + 4; +const XBOW_INT_LOWER: u64 = 0x38 + 4; +const XBOW_CONTROL: u64 = 0x20 + 4; +/// Port 15's (link 7's) reset register in the crossbar. +const LINK_RESET_15: u64 = 0x100 + 7 * 0x40 + 0x30 + 4; + +fn at(device: u8) -> DevFn { + DevFn { device, function: 0 } +} + +/// Bytes in address order, as the host's memory holds them. +struct Memory(Mutex>); +impl BusDevice for Memory { + fn read8(&self, a: u64) -> BusRead8 { + self.0.lock().get(a as usize).map_or(BusRead8::err(), |&b| BusRead8::ok(b)) + } + fn write8(&self, a: u64, v: u8) -> u32 { + self.0.lock().get_mut(a as usize).map_or(BUS_ERR, |b| { + *b = v; + BUS_OK + }) + } + fn read32(&self, a: u64) -> BusRead32 { + let m = self.0.lock(); + m.get(a as usize..a as usize + 4).map_or(BusRead32::err(), |b| BusRead32::ok(u32::from_be_bytes(b.try_into().unwrap()))) + } + fn write32(&self, a: u64, v: u32) -> u32 { + self.0.lock().get_mut(a as usize..a as usize + 4).map_or(BUS_ERR, |b| { + b.copy_from_slice(&v.to_be_bytes()); + BUS_OK + }) + } +} + +/// The host as a widget reaches it: one memory and the packets taken, with +/// the widget that asked for this port. +#[derive(Default)] +struct Packets(Mutex>); +struct Port { + widget: u8, + packets: Arc, +} +impl InterruptController for Port { + fn set_line(&self, _: u32, _: bool) { + unreachable!("the host takes packets"); + } + fn packet(&self, source: u8, address: u64, line: u32, level: bool) -> bool { + self.packets.0.lock().push((self.widget, source, address, line, level)); + true + } +} + +fn chip() -> (Arc, Arc, Arc) { + let mem = Arc::new(Memory(Mutex::new(vec![0; 0x8000]))); + let packets = Arc::new(Packets::default()); + let (m, p) = (mem.clone(), packets.clone()); + let chip = XBridge::new(BEDROCK, &|widget| { + (m.clone() as Arc, Arc::new(Port { widget, packets: p.clone() }) as Arc) + }); + (chip, mem, packets) +} + +struct Card { + state: PciFunctionState, + bytes: Mutex>, +} +impl Card { + fn new() -> Arc { + Self::with_status_events(0) + } + fn with_status_events(mask: u16) -> Arc { + Arc::new(Self { + state: PciFunctionState::new(PciConfig::new_with_policy( + 0x1234, + 0x5678, + 0, + 0, + [Bar::Mem(0x1000), Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused, Bar::Unused], + 1, + PciConfigPolicy { status_w1c_mask: mask, ..Default::default() }, + )), + bytes: Mutex::new((0..0x1000).map(|i| i as u8).collect()), + }) + } +} +impl PciFunction for Card { + fn state(&self) -> &PciFunctionState { + &self.state + } + fn bar_read(&self, _: usize, off: u32, width: usize) -> u32 { + let b = self.bytes.lock(); + (0..width).fold(0, |v, i| v | ((b[off as usize + i] as u32) << (i * 8))) + } + fn bar_write(&self, _: usize, off: u32, value: u32, width: usize) { + let mut b = self.bytes.lock(); + for i in 0..width { + b[off as usize + i] = (value >> (i * 8)) as u8; + } + } + fn reset(&self) { + if let Some(line) = self.state.irq() { + line.set(false); + } + } +} + +/// The machine lock registry must not extend the lifetime of a chip or attached function. +#[test] +fn lock_registry_does_not_keep_the_chip_or_its_devices_alive() { + let (chip, mem, _) = chip(); + let memory = Arc::downgrade(&mem); + drop(mem); + let chip_ref = Arc::downgrade(&chip); + let heads = chip.heads.each_ref().map(Arc::downgrade); + let crossbar = Arc::downgrade(&chip.xbow); + let card = Card::new(); + let card_ref = Arc::downgrade(&card); + chip.bus(HEAD_F).attach(at(1), card.clone()); + chip.set_reset_output(Arc::new(move |_| card.state.config().reset())); + chip.register_locks(); + assert!(heads.iter().all(|head| head.upgrade().is_some())); + assert!(crossbar.upgrade().is_some() && card_ref.upgrade().is_some() && memory.upgrade().is_some()); + drop(chip); + assert!(chip_ref.upgrade().is_none()); + assert!(heads.iter().all(|head| head.upgrade().is_none())); + assert!(crossbar.upgrade().is_none()); + assert!(card_ref.upgrade().is_none()); + assert!(memory.upgrade().is_none()); +} + +/// Each head has a bus of its own: the same device number on the two heads +/// is two functions, configuration reaches only the head addressed, and a +/// function's INTA raises its own head's pin of that number alone. +#[test] +fn each_head_has_its_own_configuration_space_and_interrupt_pins() { + let (c, _, _) = chip(); + let (first, second) = (Card::new(), Card::new()); + c.bus(HEAD_E).attach(at(3), first.clone()); + c.bus(HEAD_F).attach(at(3), second.clone()); + let (e, f) = (c.widget(HEAD_E).unwrap(), c.widget(HEAD_F).unwrap()); + for head in [&e, &f] { + assert_eq!(head.read32(0x23000).data, 0x5678_1234); + } + assert_eq!(e.write32(0x2303c, 0x5a), BUS_OK); + assert_eq!(first.state.config().read(15) & 0xff, 0x5a); + assert_eq!(second.state.config().read(15) & 0xff, 0, "the other head's function is untouched"); + let pins = |head: &Arc| head.read32(INT_STATUS).data & 255; + second.state.irq().unwrap().set(true); + assert_eq!((pins(&e), pins(&f)), (0, 1 << 3)); + first.state.irq().unwrap().set(true); + assert_eq!((pins(&e), pins(&f)), (1 << 3, 1 << 3)); + second.state.irq().unwrap().set(false); + assert_eq!((pins(&e), pins(&f)), (1 << 3, 0)); + // The board's own wire shares pin 0. + c.bus(HEAD_F).board_irq(BoardIrq(0)).set(true); + assert_eq!(pins(&f), 1); +} + +/// Each head sends to the host through the port of its own widget: DMA +/// lands in that port's memory, and a pin's packet carries the widget ID +/// the guest gave the head and the address it programmed. The crossbar's +/// packets come from widget 0. +#[test] +fn the_heads_and_the_crossbar_reach_the_host_through_their_own_ports() { + let (c, mem, packets) = chip(); + let f = c.widget(HEAD_F).unwrap(); + f.write32(DIRECT_MAP, u32::from(HOST_WIDGET) << 20); + assert_eq!(c.dma(HEAD_F, 0).dma_write(0x8000_2000, &[0, 1, 2, 3]), PciDmaStatus::Ok); + assert_eq!(&mem.0.lock()[0x2000..0x2004], &[3, 2, 1, 0]); + f.write32(CONTROL, (f.read32(CONTROL).data & !0xf) | u32::from(HEAD_F)); + f.write32(INT_DEST_UPPER, 0xa_0001); + f.write32(INT_DEST_LOWER, 0x2345_0080); + f.write32(INT_HOST_0, (0x155 << 8) | 0x42); + f.write32(INT_ENABLE, 1); + f.write32(INT_MODE, 1); + let wire = c.bus(HEAD_F).board_irq(BoardIrq(0)); + wire.set(true); + wire.set(false); + let address = (0x1_2345_0080u64 & !(0x3ff << 38)) | (0x155 << 38); + assert_eq!(*packets.0.lock(), [(HEAD_F, HEAD_F, address, 0x42, true), (HEAD_F, HEAD_F, address, 0x42, false)]); + packets.0.lock().clear(); + let x = c.crossbar(); + x.write32(XBOW_INT_UPPER, 0x150a_0000); + x.write32(XBOW_INT_LOWER, 0x80); + x.write32(XBOW_CONTROL, 0x20); + x.write32(4, 1); + assert_eq!(*packets.0.lock(), [(0, 0, 0x80, 0x15, true)]); +} + +/// A link reset of a head's port resets the head and runs RST# on its bus +/// before the board's output runs, so a function's BAR stops answering; +/// the output may reach the chip and replace itself. +#[test] +fn a_link_reset_resets_the_head_and_its_bus_before_the_board() { + use crate::dev::isp1040::{mbox, reg as isp_reg, CTRL_INT_ENABLE, CTRL_RISC_INT_ENABLE, HOST_CMD_SHIFT}; + use crate::dev::isp12160::Isp12160; + + let (c, _, _) = chip(); + let isp = Arc::new(Isp12160::new(Isp12160::new_bus(), Isp12160::new_bus())); + c.bus(HEAD_F).attach(at(1), isp.clone()); + let f = c.widget(HEAD_F).unwrap(); + let pin1 = |f: &Arc| f.read32(INT_STATUS).data & 2 != 0; + // BAR 1 at PCI 0x0200_0000, memory and mastering on, through the head. + assert_eq!(f.write32(0x21014, 0x0200_0000), BUS_OK); + assert_eq!(f.write32(0x21004, 6), BUS_OK); + isp.bar_write(1, isp_reg::BUS_CTRL, u32::from(CTRL_INT_ENABLE | CTRL_RISC_INT_ENABLE), 2); + isp.bar_write(1, isp_reg::MAILBOX_BASE, u32::from(mbox::NOP), 2); + isp.bar_write(1, isp_reg::HOST_CMD, 0x5 << HOST_CMD_SHIFT, 2); + assert!(pin1(&f)); + assert_eq!(f.read16(0x4200_0000 + u64::from(isp_reg::MAILBOX_BASE) ^ 2).data, mbox::COMPLETE); + + let weak = Arc::downgrade(&c); + let reset_isp = isp.clone(); + c.set_reset_output(Arc::new(move |port| { + assert_eq!(port, HEAD_F); + let chip = weak.upgrade().unwrap(); + // The head is back at reset, its function's header too. + assert_eq!(chip.widget(port).unwrap().read32(DIRECT_MAP).data, 0x20000); + assert_eq!(reset_isp.state().config().command(), 0); + reset_isp.power_on(); + chip.set_reset_output(Arc::new(|_| {})); + })); + f.write32(DIRECT_MAP, 0xa0_0000); + c.crossbar().write32(LINK_RESET_15, 1); + assert!(!pin1(&f)); + assert_eq!(isp.state().config().bar_base(1), 0); + assert_eq!(f.read32(0x4200_0000).data, u32::MAX, "the BAR no longer answers"); + assert_eq!(isp.bar_read(1, isp_reg::BUS_CTRL, 2), 0); + assert_eq!(isp.bar_read(1, isp_reg::MAILBOX_BASE + 2, 2), 0x4953); + assert!(c.xbow.carries(HEAD_F), "the link comes back"); +} + +/// A board fitted at a port answers there while its link is up. +#[test] +fn an_attached_board_answers_at_its_port() { + let (c, _, _) = chip(); + let board = Arc::new(Memory(Mutex::new(vec![0x5a; 8]))); + let port = c.attach_widget(13, board); + assert_eq!(port.read8(3).data, 0x5a); + assert_ne!(c.crossbar().read32(0x100 + 5 * 0x40 + 0x10 + 4).data & (1 << 31), 0, "link 13 is alive"); +} + +/// Narrow guest configuration writes preserve status bits outside the +/// written bytes, through either aperture. +#[test] +fn partial_configuration_writes_preserve_unwritten_status_events() { + let (c, _, _) = chip(); + let f = c.widget(HEAD_F).unwrap(); + let card = Card::with_status_events(0xc000); + c.bus(HEAD_F).attach(at(0), card.clone()); + for (bus, expected) in [(0, 0x5678_1234), (1, u32::MAX)] { + f.write32(PCI_TYPE1_CONFIG, bus << 16); + assert_eq!(f.read32(0x28000).data, expected); + } + card.state.config().set_status_bits(0xc000); + assert_eq!(f.write16(0x20006, 0x0102), BUS_OK); + assert_eq!(f.read32(0x20004).data, 0xc200_0102); + assert_eq!(f.write8(0x20007, 3), BUS_OK); + assert_eq!(f.read32(0x20004).data, 0xc200_0103); + assert_eq!(f.write8(0x20006, 2), BUS_OK); + assert_eq!(f.read32(0x20004).data, 0xc200_0203); + assert_eq!(f.write8(0x20005, 0xff), BUS_OK); + assert_eq!(f.read32(0x20004).data, 0xc200_0203); + assert_eq!(f.write8(0x20004, 0x80), BUS_OK); + assert_eq!(f.read32(0x20004).data, 0x4200_0203); + assert_eq!(f.write16(0x20004, 0x4000), BUS_OK); + assert_eq!(f.read32(0x20004).data, 0x0200_0203); + card.state.config().set_status_bits(0xc000); + assert_eq!(f.write32(0x20004, 0x8000_0002), BUS_OK); + assert_eq!(f.read32(0x20004).data, 0x4200_0002); +} diff --git a/src/devlog.rs b/src/devlog.rs index 6cd909a16..6dd8e2d9f 100644 --- a/src/devlog.rs +++ b/src/devlog.rs @@ -70,6 +70,8 @@ pub enum LogModule { /// The NVM Express controller: register writes, controller state /// changes, and the commands it runs (categories `NVME_LOG_*`). Nvme = 34, + L1 = 35, + Usb = 36, } // Mask bits shared by L1i / L1d / L2c @@ -84,7 +86,7 @@ pub const NVME_LOG_REGS: u32 = 0x4; // every register read and write below the pub const NVME_LOG_IO: u32 = 0x8; // I/O doorbells, commands and completions impl LogModule { - pub const COUNT: usize = 35; + pub const COUNT: usize = 37; pub fn from_str(s: &str) -> Option { match s { @@ -123,6 +125,8 @@ impl LogModule { "bridge" => Some(Self::Bridge), "heart" => Some(Self::Heart), "nvme" => Some(Self::Nvme), + "l1" => Some(Self::L1), + "usb" => Some(Self::Usb), _ => None, } } @@ -164,6 +168,8 @@ impl LogModule { Self::Bridge => "bridge", Self::Heart => "heart", Self::Nvme => "nvme", + Self::L1 => "l1", + Self::Usb => "usb", } } @@ -176,6 +182,7 @@ impl LogModule { Self::L1i, Self::L1d, Self::L2c, Self::Ultra, Self::Scsi1, Self::Nveeprom, Self::Gr2, Self::Rad1, Self::Odyssey, Self::Bedrock, Self::Hub, Self::Pci, Self::Bridge, Self::Heart, Self::Nvme, + Self::L1, Self::Usb, ] } @@ -340,6 +347,8 @@ impl DevLog { ModuleLog::new(), // Bridge ModuleLog::new(), // Heart ModuleLog::new(), // Nvme + ModuleLog::new(), // L1 + ModuleLog::new(), // Usb ], } } diff --git a/src/ip30_machine.rs b/src/ip30_machine.rs index 8f6d75c63..d257c062a 100644 --- a/src/ip30_machine.rs +++ b/src/ip30_machine.rs @@ -175,7 +175,7 @@ impl OctaneMachine { if prom.len() > FLASH_SIZE { return Err(format!("Octane PROM {prom_path} is {} bytes; the flash holds {FLASH_SIZE}", prom.len())); } - let cpu_count = cfg.machine.cpus; + let cpu_count = cfg.machine.cpus(); MachineProfile::OctaneIp30.accepts_cpus(cpu_count)?; let banks = heart_banks(cfg); let kind = cfg.graphics_board(); diff --git a/src/ip34_machine.rs b/src/ip34_machine.rs new file mode 100644 index 000000000..128811fa1 --- /dev/null +++ b/src/ip34_machine.rs @@ -0,0 +1,856 @@ +//! The Fuel (IP34 board, IP35 software platform): an R14000 on Bedrock, +//! XBridge and the base I/O devices, with Iris's CPU thread, monitor and serial console. +//! +//! Selected with `[machine] profile = "fuel_ip34"`. Shared keys select the +//! PROM, NVRAM, memory and SCSI devices; `[fuel]` selects the serial console +//! and writable flash. `ip35::Ip35Bus` maps the processor views over the +//! same Physical memory banks. The IOC3 carries PS/2 keyboard and mouse ports +//! (default) and the OHCI controller can take a USB pair; without graphics the +//! machine runs on a serial console. With `[graphics] board = "v12"` (or +//! `"v10"`) a VPro board answers at crossbar widget 13, the Fuel's graphics +//! slot. +//! +//! The Bedrock node itself, and its approximations, are in `ip35::node`. +//! +//! # What is approximated here +//! +//! The processor interfaces' graphics flow-control registers store what +//! IRIX writes, which is what its VPro driver checks, but count no stores: +//! no uncached store into the graphics page takes a credit and the board +//! returns none, so the credit-starvation interrupt never fires. The board's +//! DMA reaches memory through Bedrock's inbound path as widget 13. +//! +//! Board composition ported from hw/mips/sgi/sn1-node.c +//! in jamesbraid/qemu-sgi-origami. + +use crate::config::{scsi_label, FuelConsole, GraphicsBoard, MachineConfig}; +use crate::cpu::mips_exec::{CpuDevice, MipsCpu, MipsCpuConfig, MipsExecutor}; +use crate::cpu::mips_tlb::MipsTlb; +use crate::dev::bedrock::{Bedrock, Population}; +use crate::dev::bedrock_flash::{FirmwareRecord, Flash}; +use crate::dev::heart::{host_clock, Clock}; +use crate::dev::ioc3::{Ioc3, Ioc3Config}; +use crate::dev::odyssey::{Odyssey, PacketSender}; +use crate::dev::ioc3_eth::Ioc3Eth; +use crate::dev::isp12160::Isp12160; +use crate::dev::l1::{L1, L1Config}; +use crate::dev::ohci::Ohci; +use crate::dev::prom_image; +use crate::dev::ps2::Ps2Controller; +use crate::dev::serial_host::{null_serial_backend, open_log, tcp_serial_backend, CiSerialBackend, LogMode, SerialBackend, TeeBackend}; +use crate::dev::timekeeper::Timekeeper; +use crate::dev::xbridge::XBridge; +use crate::pci::{BoardIrq, DevFn}; +use crate::devlog::{init_devlog, LogModule}; +use crate::gdb_stub::CpuDebug; +use crate::gfx_display::GfxDisplay; +use crate::hptimer::TimerManager; +use crate::cpu::mips_cache_shadow::R12000ShadowCache; +use crate::cpu::parts::{self, Part}; +use crate::cpu::mips_cache_v2::MipsCache; +use crate::ip35::node::{connect_cpus, Node, NodeConfig}; +use crate::ip35::{Ip35Bus, CPU_PHYS_MASK}; +use crate::machine_state::GuestState; +use crate::monitor::{Monitor, DEFAULT_PORT as DEFAULT_MONITOR_PORT}; +use crate::net::{GatewayConfig, Link, NatControl}; +use crate::perf_monitor::PerfMonitor; +use crate::scsi::{attach_from_config, ScsiBus}; +use crate::traits::{BusDevice, Device, MachineEvent, Resettable, BUS_OK}; +use crate::workstation::{register_system_controller, DiscChanger, MachineRef, MachineSlot, Workstation}; +use parking_lot::Mutex; +use std::net::{Ipv4Addr, SocketAddr}; +use std::path::{Path, PathBuf}; +use std::sync::mpsc::{Receiver, Sender}; +use std::sync::{atomic::{AtomicBool, AtomicU64, Ordering}, Arc}; + +pub struct FuelMachine { + board: Arc, + cpu: Arc, + timers: Arc, + monitor: Arc, + monitor_port: u16, + ci_serial: Option>, + console: usize, + ci: bool, + scratch: Option, + timekeeper: Arc, + nvram: String, + frames: Option>, + nat: Option<( + Arc, + (Ipv4Addr, Ipv4Addr, Ipv4Addr), + )>, + events: Option>, + slot: Option, + lifecycle: GuestState, +} + +impl FuelMachine { + pub fn new(cfg: &MachineConfig) -> Result { + cfg.validate()?; + let prom = prom_image::load(Path::new(&cfg.prom), 8 * 1024 * 1024) + .map_err(|e| format!("Fuel {e}"))?; + let timekeeper = Arc::new(Timekeeper::fuel_with_nvram_file( + Path::new(&cfg.nvram), + )?); + let board = Arc::new(FuelBoard::with_config(FuelBoardConfig { + payload: &prom, + clock: host_clock(), + flash_path: cfg.fuel.flash.as_deref().map(Path::new), + station_mac: cfg.network().mac, + timekeeper: timekeeper.clone(), + ps2_input: cfg.fuel.ps2_input, + l1: L1Identity { + revision: cfg.fuel.l1_revision.as_deref().map(crate::dev::l1::parse_revision).transpose()?, + brick_serial: cfg.fuel.brick_serial.clone(), + }, + graphics: cfg.graphics_board(), + graphics_pool: (cfg.graphics.threads, cfg.graphics.tile), + graphics_raster: (cfg.graphics.raster, cfg.graphics.gpu_backend.as_str()), + })?); + let timers = Arc::new(TimerManager::new()); + let part = cfg.cpu(); + let mut exec = board.build_cpu(part); + // The CPU's Compare matches blink the graphics status bar's LED. + exec.core.fasttick_count = board.fasttick.clone(); + exec.core.set_timer_manager(timers.clone()); + exec.core.set_count_hz( + cfg.clock + .fixed_mhz + .map_or(part.count_hz(), |mhz| (mhz * 1_000_000.0) as u64) + .max(1), + ); + exec.core.count_write_samples = true; + let poll_board = board.clone(); + exec.batch_hook = Some(Box::new(move |_| poll_board.poll())); + let cpu: Arc = Arc::new(MipsCpu::<_, R12000ShadowCache>::new(*exec)); + board.status.set_cpu_cycles(cpu.cycles_ptr()); + board.status.set_count_hz(cpu.count_hz_atomic()); + connect_cpus("Fuel", &board.bedrock, &board.l1, &board.events, std::slice::from_ref(&cpu))?; + board.bedrock.set_timer_manager(&timers); + // After the windows above: each compile worker takes the cache's + // geometry and the CPU model's policies once, as it starts, and + // without the windows the geometry has no inline load or store path. + // Workers read plain physical code addresses through the CPU's code + // bus: the cached bus, plus the reset PROM's uncached-fetch window. + // The uncached views would decode plain addresses as Hub registers. + #[cfg(feature = "jitv2")] + { + let cpu_device: Arc = cpu.clone().as_device(); + let jit = cpu.jitv2(); + let mut g = jit.lock(); + g.compile_queue.set_thread_count(cfg.jitv2.threads.max(1)); + g.compile_queue.set_cpu(Arc::downgrade(&cpu_device)); + g.compile_queue.set_owner(Arc::downgrade(&jit)); + // Not under lockstep, as on the other machines: lockstep compiles + // inline on the CPU thread, and a running pool both owns the + // Codegen inline needs and deadlocks the CPU thread's flush. + #[cfg(not(feature = "jitv2_lockstep"))] + { + let stats = g.stats.clone(); + g.compile_queue.start(board.code_bus(), stats); + } + // The compile pool keeps only a weak handle on the CPU. + std::mem::forget(cpu_device); + } + let scratch = Self::attach_scsi(cfg, &board)?; + let console = match cfg.fuel.console { + FuelConsole::L1 => 0, + FuelConsole::Ioc3A => 1, + FuelConsole::Ioc3B => 2, + }; + let ci_serial = cfg.ci.then(|| board.serial[console].clone()); + Self::connect_serial(cfg, &board, console)?; + let monitor = Monitor::new(); + monitor.register_device(init_devlog()); + monitor.register_device(Arc::new(crate::cpu::cpu_monitor::CpuMonitor::new(vec![cpu.clone()]))); + monitor.register_device(PerfMonitor::new( + cpu.running_flag(), cpu.cycles_ptr(), cpu.fasttick_count(), None, None, + )); + for device in board.monitor_devices() { + monitor.register_device(device); + } + cpu.register_locks(); + board.scsi.register_locks(); + board.ioc3.register_locks(); + board.xbridge.register_locks(); + board.flash.register_locks(); + board.bedrock.register_locks(); + let (mut nat, mut frames) = (None, None); + let net = GatewayConfig::from_network(&cfg.network()); + if let Some(link) = Link::with_gateway(net.clone())? { + frames = Some(link.frames()); + nat = Some((link.nat(), (net.client_ip, net.gateway_ip, net.netmask))); + board.eth.attach(link); + } + let events = board.receiver.lock().take(); + eprintln!( + "iris: Fuel IP34, {}, 512 MiB, {}, serial console {:?}", + cfg.cpu().label(), + if board.vpro.is_some() { cfg.graphics_board().label() } else { "no graphics" }, + cfg.fuel.console + ); + Ok(Self { + board, + cpu, + timers, + monitor: Arc::new(monitor), + monitor_port: cfg.monitor_port.unwrap_or(DEFAULT_MONITOR_PORT), + ci_serial, + console, + ci: cfg.ci, + scratch, + timekeeper, + nvram: cfg.nvram.clone(), + nat, + frames, + events, + slot: None, + lifecycle: GuestState::Idle, + }) + } + + fn attach_scsi(cfg: &MachineConfig, board: &FuelBoard) -> Result, String> { + let attached = attach_from_config(cfg, Some("fuel"), &|channel| { + board.scsi.bus_for_channel(channel) + .ok_or_else(|| format!("controller {channel} (a Fuel has channels 0 and 1)")) + })?; + if let Some(device) = attached.daynaports.first() { + return Err(format!("{}: kind = \"daynaport\" is not available on the Fuel", + scsi_label(device.controller, device.id))); + } + Ok(attached.scratch) + } + + fn connect_serial( + cfg: &MachineConfig, + board: &FuelBoard, + console: usize, + ) -> Result<(), String> { + let [a, b] = cfg.serial_ports(); + let ports = [cfg.fuel.l1_port.unwrap_or(8882), a, b]; + if ports[0] == ports[1] || ports[0] == ports[2] || ports[1] == ports[2] { + return Err("Fuel L1 and IOC3 serial ports must be distinct".into()); + } + let labels = ["L1", "IOC3 A", "IOC3 B"]; + let settings = ["fuel.l1_port", "serial_port_a", "serial_port_b"]; + let mut log_config = cfg.clone(); + for (index, port) in ports.into_iter().enumerate() { + log_config.serial_log = if index == console { + cfg.serial_log.clone() + } else if index == 0 { + cfg.fuel.l1_log.clone() + } else { + None + }; + let tag = format!("Fuel {}", labels[index]); + let log = open_log(&log_config, &tag, LogMode::Fresh); + let backend: Arc = if cfg.ci && index == console { + if let Some(log) = log { + board.serial[index].set_log(log); + } + board.serial[index].clone() + } else { + let mut backend = null_serial_backend(); + match tcp_serial_backend(port) { + Some((socket, actual_port)) => { + eprintln!("iris: {tag} on 127.0.0.1:{actual_port}"); + backend = socket; + } + None => eprintln!( + "iris: {tag}: port {port} unavailable; is another iris using it? ({})", + settings[index] + ), + } + match log { + Some(log) => Arc::new(TeeBackend::new(backend, log)), + None => backend, + } + }; + if index == 0 { + board.l1.set_serial_backend(backend); + } else { + board.ioc3.set_serial_backend(index - 1, backend); + } + } + Ok(()) + } + + #[cfg(test)] + pub(crate) fn connect_serial_for_test(cfg: &MachineConfig, board: &FuelBoard, console: usize) -> Result<(), String> { + Self::connect_serial(cfg, board, console) + } + +} + +impl Workstation for FuelMachine { + fn start(&mut self) { + self.lifecycle = GuestState::Starting; + self.monitor + .clone() + .start_server(format!("127.0.0.1:{}", self.monitor_port)); + self.restart_peripherals(); + if !self.ci { + self.cpu_start(); + } + } + fn stop(&mut self) { + self.cpu.clone().as_device().stop(); + self.board.stop(); + if let Err(e) = self.timekeeper.flush_nvram() { + dlog!(LogModule::Bedrock, "Fuel NVRAM: {e}"); + } + if self.lifecycle != GuestState::Idle { + self.lifecycle = GuestState::Halted; + } + } + fn reset(&mut self) { + self.stop(); + self.cpu.power_on(); + self.board.bedrock.power_on(); + self.board.l1.power_on(); + self.board.flash.power_on(); + self.board.xbridge.power_on(); + self.board.ioc3.power_on(); + self.board.eth.power_on(); + self.board.scsi.power_on(); + self.board.usb.power_on(); + if let Some(vpro) = &self.board.vpro { + vpro.power_on(); + } + self.timekeeper.power_on(); + self.restart_peripherals(); + } + fn restart_peripherals(&mut self) { + self.board.start(); + } + fn cpu_start(&self) { + self.cpu.clone().as_device().start(); + } + fn cpu_is_running(&self) -> bool { + self.cpu.clone().as_device().is_running() + } + fn machine_slot(&mut self) -> MachineSlot { + let ptr = self as *mut Self as *mut dyn Workstation; + self.slot + .get_or_insert_with(|| Arc::new(Mutex::new(MachineRef(ptr)))) + .clone() + } + fn register_system_controller(&mut self) { + let slot = self.machine_slot(); + let events = self + .events + .take() + .expect("Fuel system controller registered twice"); + register_system_controller(&self.monitor, slot, Some(events)); + } + fn monitor_addr(&self) -> Option { + self.monitor.bound_addr() + } + fn lifecycle(&self) -> GuestState { + self.lifecycle + } + fn set_lifecycle(&mut self, state: GuestState) { + self.lifecycle = state; + } + fn get_display(&self) -> Option> { + self.board.vpro.as_ref().map(|vpro| vpro.display()) + } + fn register_monitor_device(&self, device: Arc) { + self.monitor.register_device(device); + } + fn get_ps2(&self) -> Arc { + self.board.ioc3.ps2() + } + fn get_timer_manager(&self) -> Arc { + self.timers.clone() + } + fn get_cpu_debug(&self) -> Arc { + self.cpu.clone().debug_adapter() + } + fn fasttick_count(&self) -> Arc { + self.cpu.fasttick_count() + } + fn get_ci_serial(&self) -> Option> { + self.ci_serial.clone() + } + fn inject_serial_console(&self, bytes: &[u8]) { + self.board.serial[self.console].push_host(bytes); + } + fn as_any_mut(&mut self) -> &mut dyn std::any::Any { + self + } + fn save_nvram(&self, path: Option<&str>) -> Result { + if path.is_some_and(|p| p != self.nvram) { + return Err("Fuel NVRAM save uses the configured path".into()); + } + self.timekeeper.flush_nvram()?; + Ok(self.nvram.clone()) + } + fn scratch_path(&self) -> Option<&Path> { + self.scratch.as_deref() + } + fn disc_changer(&self) -> Option> { + Some(self.board.scsi.clone() as Arc) + } + fn pending_chd_sync_count(&self) -> usize { + (0..2).map(|channel| self.board.scsi.bus_for_channel(channel).unwrap().pending_chd_sync_count()).sum() + } + fn flush_disks(&self) -> std::io::Result<()> { + self.board.flush_disks() + } + fn sync_chd_disks( + &self, + only: Option, + progress: &mut dyn FnMut(usize, usize, f32), + cancel: &dyn Fn() -> bool, + ) -> std::io::Result { + let buses: Vec<_> = (0..2).map(|channel| self.board.scsi.bus_for_channel(channel).unwrap()).collect(); + ScsiBus::sync_chd_disks(&buses, only, progress, cancel) + } + fn net_frames(&self) -> Option> { self.frames.clone() } + fn nat_control(&self) -> Option> { + self.nat.as_ref().map(|(n, _)| n.clone()) + } + fn nat_expected(&self) -> Option<(Ipv4Addr, Ipv4Addr, Ipv4Addr)> { + self.nat.as_ref().map(|(_, a)| *a) + } +} + +impl Drop for FuelMachine { + fn drop(&mut self) { + if let Some(slot) = self.slot.take() { + slot.lock().0 = MachineRef::null(); + } + self.stop(); + self.monitor.shutdown(); + } +} + +/// The processors a Fuel can be built with, the default first: the R14000A +/// the Fuel ships with. +pub const CPUS: &[&Part] = &[&parts::R14000_500]; + +/// The cache type `with_cpu!` builds for every entry of [`CPUS`]; a test pins +/// it. The board's devices are not generic over the processor, so it names the +/// type once here. +pub(crate) type FuelCpu = MipsExecutor; + +/// The board-id word. Bits 15:12 are the Fuel's product code, 4, as OpenBSD +/// reads them (IP35_FUEL, sys/arch/sgi/sgi/ip27.h). The low bits are chosen +/// to match what PROM 6.210 expects, not read from a board: with them and the +/// flash record's machine type 2 the PROM's CPU configuration check passes, +/// as on a real Fuel. A real board's bits 11:0 are unknown. +const FUEL_BOARD_ID: u64 = 0x4004; +/// Two 256 MiB banks, locational banks 0 and 1. +const FUEL_BANKS_MIB: [u32; 2] = [256, 256]; + + +/// Build the Fuel's processor, `part` being the one the config resolved, and +/// set the PROM boot-mode inputs. +pub(crate) fn build_cpu(bus: Arc, part: &Part) -> Box { + let extent = crate::dev::bedrock::RamExtent::new(&FUEL_BANKS_MIB); + let cfg = MipsCpuConfig::for_model::(Arc::new(crate::ip35::memory_map(&extent))) + .with_prid(part.prid()); + let mut cpu = Box::new(MipsExecutor::new( + bus as Arc, + MipsTlb::new(cfg.tlb_entries), + &cfg, + )); + cpu.phys_mask = CPU_PHYS_MASK; + // The flash record's mode word selects these boot-mode inputs. A + // mismatch enters the PROM's error routine before GP is initialized. + cpu.core.cp0_config = FirmwareRecord::FUEL.reset_config0(); + cpu +} + +/// Shared device composition for a single-processor Fuel. +/// Graphics is absent; the serial workstation shares these base I/O devices. +pub(crate) struct FuelBoard { + pub bus: Arc, + /// Cacheable memory at plain physical addresses: what the cache fills + /// from, and the one bus the JIT and its compile pool read code through. + cached: Arc, + /// What the JIT compiles code through: `cached`, and the reset PROM's + /// uncached-fetch window ([`Ip35Bus::code_view`]). + #[cfg(feature = "jitv2")] + code: Arc, + pub bedrock: Arc, + pub l1: Arc, + pub flash: Arc, + /// The node's resets and power events, and their receiver until the + /// system controller takes it. + events: Sender, + receiver: Mutex>>, + pub xbridge: Arc, + pub ioc3: Arc, + pub eth: Arc, + pub scsi: Arc, + pub usb: Arc, + /// The VPro board in the graphics slot, widget 13. + pub vpro: Option>, + /// The status bar's activity bits and the CPU's Compare matches, which + /// the graphics board shows. + pub heartbeat: Arc, + pub fasttick: Arc, + /// What the graphics board's status bar shows: the activity bits, the + /// Compare matches and the CPU's cycle count and Count rate. + pub status: Arc, + pub serial: [Arc; 3], + #[cfg(test)] + pub irq: Arc, + clock: Clock, + last_clock: AtomicU64, + running: AtomicBool, +} + +pub(crate) struct FuelBoardConfig<'a> { + pub payload: &'a [u8], + pub clock: Clock, + pub flash_path: Option<&'a Path>, + pub station_mac: [u8; 6], + pub timekeeper: Arc, + /// The IOC3's PS/2 keyboard and mouse controller. + pub ps2_input: bool, + /// Replacements for the L1's reported identity; unset keeps its defaults. + pub l1: L1Identity, + /// `None`, `V12` or `V10`; validation refuses the others. + pub graphics: GraphicsBoard, + /// The VPro's fragment pipeline: worker count and tile size (`[graphics]`). + pub graphics_pool: (usize, u32), + /// Which unit draws and, for the GPU, the wgpu backend (`[graphics]`). + pub graphics_raster: (crate::config::RasterUnit, &'a str), +} + +/// What a Fuel config may replace in the L1's identity replies. +#[derive(Clone, Debug, Default)] +pub(crate) struct L1Identity { + pub revision: Option<[u32; 3]>, + pub brick_serial: Option, +} + +impl FuelBoard { + /// Flush the disks on both channels of the storage controller. + pub fn flush_disks(&self) -> std::io::Result<()> { + (0..2).try_for_each(|channel| self.scsi.bus_for_channel(channel).unwrap().flush()) + } + + #[cfg(test)] + pub fn new(payload: &[u8], clock: Clock) -> Result { + Self::with_config(FuelBoardConfig { + payload, clock, flash_path: None, + station_mac: L1Config::default().mac, + timekeeper: Arc::new(Timekeeper::fuel()?), + ps2_input: true, + l1: L1Identity::default(), + graphics: GraphicsBoard::None, + graphics_pool: (2, 32), + graphics_raster: (crate::config::RasterUnit::Cpu, "auto"), + }) + } + + pub fn with_config(config: FuelBoardConfig<'_>) -> Result { + let FuelBoardConfig { payload, clock, flash_path, station_mac, timekeeper, ps2_input, l1: l1_identity, graphics, graphics_pool, graphics_raster } = config; + let mut l1_config = L1Config { mac: station_mac, ..Default::default() }; + if let Some(revision) = l1_identity.revision { + l1_config.revision = Some(revision); + } + if let Some(serial) = l1_identity.brick_serial { + l1_config.brick_serial = Some(serial); + } + let node = Node::build(NodeConfig { + name: "Fuel", + payload, + clock: clock.clone(), + flash_path, + banks_mib: &FUEL_BANKS_MIB, + processors: Population::SINGLE, + firmware: FirmwareRecord::FUEL, + board_id: FUEL_BOARD_ID, + l1: l1_config, + })?; + let Node { + bus, + bedrock, + l1, + flash, + cached, + #[cfg(feature = "jitv2")] + code, + events, + receiver, + } = node; + #[cfg(test)] + let irq = Arc::new(PendingIrq::default()); + #[cfg(test)] + bedrock.set_cpu_interrupts(0, &irq.0); + let serial = std::array::from_fn(|_| Arc::new(CiSerialBackend::new())); + l1.set_serial_backend(serial[0].clone()); + let weak = Arc::downgrade(&bedrock); + let xbridge = XBridge::new(BEDROCK_PORT, &|widget| { + let port = Arc::new(Inbound { bedrock: weak.clone(), widget }); + (port.clone() as Arc, port as Arc) + }); + for widget in [0, crate::dev::xbridge::HOST_WIDGET, 14, 15] { + bedrock.connect_widget(widget, xbridge.widget(widget).unwrap()); + } + let mac = l1.station_mac(); + let mut ioc3_config = Ioc3Config::fuel(mac); + ioc3_config.keyboard_mouse = ps2_input; + let ioc3 = Arc::new(Ioc3::new(ioc3_config)); + let eth = Ioc3Eth::new_fuel(mac); + ioc3.attach_eth(eth.clone()); + ioc3.attach_timekeeper(timekeeper); + ioc3.set_serial_backend(0, serial[1].clone()); + ioc3.set_serial_backend(1, serial[2].clone()); + let scsi = Arc::new(Isp12160::new( + Isp12160::new_bus(), + Isp12160::new_bus(), + )); + let heartbeat = Arc::new(AtomicU64::new(0)); + let fasttick = Arc::new(AtomicU64::new(0)); + scsi.set_heartbeat(heartbeat.clone()); + ioc3.set_heartbeat(heartbeat.clone()); + eth.set_heartbeat(heartbeat.clone()); + let status = crate::disp::StatusSource::new(heartbeat.clone(), fasttick.clone()); + let vpro = if graphics == GraphicsBoard::None { + None + } else { + // Interrupt packets reach Bedrock as the heads' do; DMA enters + // it as this widget's inbound traffic. + let port = Inbound { bedrock: Arc::downgrade(&bedrock), widget: WIDGET_VPRO }; + let packets: PacketSender = Arc::new(move |widget, address, vector, level| { + use crate::irq::InterruptController; + widget == BEDROCK_PORT.widget && port.packet(WIDGET_VPRO, address, vector.into(), level) + }); + let dma = bedrock.widget_dma(WIDGET_VPRO); + let board = Odyssey::new_widget_relative( + WIDGET_VPRO, graphics.as_str(), packets, dma, status.clone(), graphics_pool, graphics_raster, + )?; + bedrock.connect_widget(WIDGET_VPRO, xbridge.attach_widget(WIDGET_VPRO, board.clone())); + Some(board) + }; + let pci = xbridge.bus(15); + let slot = |device| DevFn { device, function: 0 }; + pci.attach(slot(1), scsi.clone()); + pci.attach(slot(4), ioc3.clone()); + let usb = Ohci::new(); + usb.set_clock(clock.clone()); + pci.attach(slot(5), usb.clone()); + ioc3.connect_superio_interrupt(pci.board_irq(BoardIrq(0))); + let reset_ioc3 = ioc3.clone(); + let reset_eth = eth.clone(); + let reset_scsi = scsi.clone(); + let reset_usb = usb.clone(); + let reset_vpro = vpro.as_ref().map(Arc::downgrade); + xbridge.set_reset_output(Arc::new(move |port| { + if port == 15 { + reset_ioc3.power_on(); + reset_eth.power_on(); + reset_scsi.power_on(); + reset_usb.power_on(); + } + // A link reset of the graphics port resets the board behind it. + if port == WIDGET_VPRO { + if let Some(vpro) = reset_vpro.as_ref().and_then(|v| v.upgrade()) { + vpro.power_on(); + } + } + })); + Ok(Self { + bus, + cached, + #[cfg(feature = "jitv2")] + code, + bedrock, + l1, + flash, + events, + receiver: Mutex::new(Some(receiver)), + xbridge, + ioc3, + eth, + scsi, + usb, + vpro, + heartbeat, + fasttick, + status, + serial, + #[cfg(test)] + irq, + last_clock: AtomicU64::new(clock()), + clock, + running: AtomicBool::new(false), + }) + } + + pub fn cached_bus(&self) -> Arc { + self.cached.clone() + } + + /// The bus the JIT compiles code through; see [`Ip35Bus::code_view`]. + #[cfg(feature = "jitv2")] + pub fn code_bus(&self) -> Arc { + self.code.clone() + } + + pub fn build_cpu(&self, part: &Part) -> Box { + let mut cpu = build_cpu(self.bus.clone(), part); + cpu.cache = R12000ShadowCache::new(self.cached_bus()); + #[cfg(feature = "jitv2")] + cpu.set_cached_bus(self.code_bus()); + #[cfg(not(feature = "jitv2"))] + cpu.set_cached_bus(self.cached_bus()); + cpu + } + + pub fn start(&self) { + self.last_clock + .store((self.clock)(), Ordering::Release); + self.running + .store(true, Ordering::Release); + self.bedrock.start(); + self.l1.start(); + self.ioc3.start(); + self.eth.start(); + self.scsi.start(); + self.usb.start(); + if let Some(vpro) = &self.vpro { + vpro.start(); + } + } + /// The board's devices that take monitor commands: the physical map, the + /// L1, the SCSI controller, the keyboard and mouse (`ps2 type` and + /// friends) and, when one is fitted, the VPro (`vpro …`). + pub fn monitor_devices(&self) -> Vec> { + let mut devices: Vec> = vec![ + self.bedrock.physical().clone(), + self.l1.clone(), + self.scsi.clone(), + self.ioc3.ps2(), + ]; + if let Some(vpro) = &self.vpro { + devices.push(vpro.clone()); + } + devices + } + + pub fn poll(&self) { + if self.running.load(Ordering::Acquire) { + let now = (self.clock)(); + let previous = self + .last_clock + .swap(now, Ordering::AcqRel); + self.flash.advance_ns(now.saturating_sub(previous)); + self.usb.sync_clock(); + } + } + pub fn stop(&self) { + self.poll(); + self.running + .store(false, Ordering::Release); + if let Some(vpro) = &self.vpro { + vpro.stop(); + } + self.usb.stop(); + self.scsi.stop(); + self.eth.stop(); + self.ioc3.stop(); + self.l1.stop(); + self.bedrock.stop(); + } +} + +/// The crossbar widget of the Fuel's graphics slot: a Fuel IRIX inventory +/// lists its VPro under xtalk/13, and OpenBSD finds Odyssey at widget 13. +pub const WIDGET_VPRO: u8 = 13; + +impl Drop for FuelBoard { + fn drop(&mut self) { + self.stop(); + } +} + +/// Bedrock's crossbar port: widget 10, taking interrupt packets at any +/// address (the address picks the processor interface). +const BEDROCK_PORT: crate::xtalk::XtalkHost = + crate::xtalk::XtalkHost { widget: crate::dev::xbridge::HOST_WIDGET, int_packet: 0, int_packet_mask: 0, vectors: 256 }; + +/// Bedrock as a widget sending through the XBridge reaches it: the +/// widget's DMA as inbound reads and writes, a doubleword at a time where +/// the transfer allows, and its interrupt packets as inbound doubleword +/// writes to the address they name. Bedrock checks the widget's +/// permissions on each, for every master on the widget. +struct Inbound { + bedrock: std::sync::Weak, + widget: u8, +} + +impl Inbound { + fn access(&self, source: u8, a: u64, size: u8, value: Option) -> crate::traits::BusRead64 { + let Some(bedrock) = self.bedrock.upgrade() else { return crate::traits::BusRead64::err() }; + match value { + Some(v) => crate::traits::BusRead64 { status: bedrock.inbound_write(source, None, a, size, v), data: 0 }, + None => bedrock.inbound_read(source, None, a, size), + } + } +} + +macro_rules! inbound_access { + ($r:ident, $w:ident, $t:ty, $out:ident, $n:expr) => { + fn $r(&self, a: u64) -> crate::traits::$out { + let r = self.access(self.widget, a, $n, None); + crate::traits::$out { data: r.data as $t, status: r.status } + } + fn $w(&self, a: u64, v: $t) -> u32 { + self.access(self.widget, a, $n, Some(v as u64)).status + } + }; +} + +impl BusDevice for Inbound { + inbound_access!(read8, write8, u8, BusRead8, 1); + inbound_access!(read16, write16, u16, BusRead16, 2); + inbound_access!(read32, write32, u32, BusRead32, 4); + inbound_access!(read64, write64, u64, BusRead64, 8); + fn read_block(&self, a: u64, buf: &mut [u64]) -> u32 { + for (i, word) in buf.iter_mut().enumerate() { + let r = self.read64(a + 8 * i as u64); + if r.status != BUS_OK { + return r.status; + } + *word = r.data; + } + BUS_OK + } + fn write_block(&self, a: u64, buf: &[u64]) -> u32 { + for (i, &word) in buf.iter().enumerate() { + let status = self.write64(a + 8 * i as u64, word); + if status != BUS_OK { + return status; + } + } + BUS_OK + } +} + +impl crate::irq::InterruptController for Inbound { + fn set_line(&self, line: u32, _: bool) { + dlog!(LogModule::Bedrock, "Fuel: widget {} interrupt {line} with no packet address dropped", self.widget); + } + fn packet(&self, source: u8, address: u64, line: u32, level: bool) -> bool { + self.access(source, address, 8, Some(u64::from(line) | if level { 1 << 8 } else { 0 })).status == BUS_OK + } +} + +/// CPU 0's interrupt word, for tests that build the board without a CPU. +#[cfg(test)] +#[derive(Default)] +pub(crate) struct PendingIrq(pub(crate) AtomicU64); +#[cfg(test)] +impl PendingIrq { + pub fn bits(&self) -> u64 { + self.0.load(Ordering::Acquire) + } +} diff --git a/src/ip35.rs b/src/ip35.rs index 432d8f497..ce187590d 100644 --- a/src/ip35.rs +++ b/src/ip35.rs @@ -1,18 +1,79 @@ -//! The IP35 processor bus: windows over small MMIO apertures. +//! The IP35 processor bus: Bedrock's four XKPHYS access-space views. //! -//! Maps replace whole 4 KiB pages as Physical does, while each endpoint still -//! rejects padding and enforces its cell permission. +//! Raw address bits 58:57 select Hub-special, I/O, fetch operations or uncached +//! RAM. Each view has a small fixed set of MMIO apertures, decoded by sorted +//! windows rather than a full physical-address table. Maps replace whole 4 KiB +//! pages as Physical does, while each endpoint still rejects padding and +//! enforces its cell permission. +//! +//! Cached RAM alone uses Physical, including its direct bank pointers, ppmem +//! and generation mappings. The MMIO views and register sub-bus need none of +//! those RAM facilities. Giving each its own full table would consume several +//! entries in the process-wide fault-handler registry per machine. //! //! Ported from hw/mips/sgi/sn1-node.c in jamesbraid/qemu-sgi-origami. +use crate::dev::bedrock::RamExtent; use crate::physical::Physical; use crate::traits::*; #[cfg(feature = "jitv2")] -use std::sync::atomic::AtomicU64; +use std::sync::{atomic::AtomicU64, OnceLock}; use std::sync::Arc; +pub(crate) const CPU_PHYS_MASK: u64 = ((1u64 << 40) - 1) | (3u64 << 57); const PA_MASK: u64 = (1u64 << 40) - 1; +/// The reset PROM's compatibility window: the processor's reset vector and +/// the PROM's own code are at 0x1fc00000, fetched uncached. +pub(crate) const RESET_WINDOW: u64 = 0x1fc0_0000; +pub(crate) const RESET_WINDOW_SIZE: u64 = 0x40_0000; +/// Where an uncached instruction fetch of the reset window is presented to +/// the bus (`MipsExecutor::uncached_fetch_alias`): the same PROM at an +/// address of its own, above every RAM slot and below the 16 GiB the JIT's +/// frame map covers. +/// +/// The split is inferred. The hardware selects a space by the processor's +/// uncached attribute (established: ip35-address-map.md, "Processor-side +/// address spaces"), so an uncached attribute-0 access and a cached one are +/// different requests to Bedrock. What attribute 0 returns at 0x1fc00000 is +/// not established: no public source says how the reset request reaches flash +/// (ip35-address-map.md, reset vector), and the cached side is likewise open. +/// The reset PROM answering uncached fetches while cached accesses reach RAM +/// is what PROM 6.210's memory test needs, and that is all this decode rests on. +pub(crate) const RESET_WINDOW_FETCH: u64 = 0x2_0000_0000; + +/// Where a Bedrock node has memory: each 1 GiB software slot `extent` says +/// can hold RAM, as far as its largest bank reaches, and the reset PROM. +/// Which bank lands in which slot of a group of four depends on how the PROM +/// programs the memory controller; the Fuel's two 256 MiB banks can be in any +/// of the first four, and a Tezro's banks 4 and 5 in any of the next four. Uncached fetches of the PROM's own code land in its +/// range. A slot 0 larger than 508 MiB reaches the PROM's range, which the +/// node maps over it, so its RAM is listed on either side. +pub fn memory_map(extent: &RamExtent) -> crate::memory_map::MemoryMap { + use crate::dev::bedrock::{BANK_SPAN, SLOTS}; + use crate::memory_map::{MemKind, MemRange, MemoryMap}; + const NAMES: [&str; SLOTS] = + ["ram slot 0", "ram slot 1", "ram slot 2", "ram slot 3", "ram slot 4", "ram slot 5", "ram slot 6", "ram slot 7"]; + const PROM: (u64, u64) = (RESET_WINDOW, RESET_WINDOW + RESET_WINDOW_SIZE); + let mut ranges = vec![MemRange::new("prom", MemKind::Rom, PROM.0, PROM.1)]; + for (slot, &name) in NAMES.iter().enumerate() { + let (base, end) = (slot as u64 * BANK_SPAN, slot as u64 * BANK_SPAN + extent.prefix(slot)); + if base == end { + continue; + } + if base < PROM.1 && end > PROM.0 { + ranges.push(MemRange::new(name, MemKind::Ram, base, PROM.0)); + if end > PROM.1 { + ranges.push(MemRange::new("ram slot 0 above the prom", MemKind::Ram, PROM.1, end)); + } + } else { + ranges.push(MemRange::new(name, MemKind::Ram, base, end)); + } + } + ranges.push(MemRange::new("prom (uncached fetch)", MemKind::Rom, RESET_WINDOW_FETCH, RESET_WINDOW_FETCH + RESET_WINDOW_SIZE)); + MemoryMap::new(ranges) +} + struct Unclaimed; impl BusDevice for Unclaimed {} @@ -177,3 +238,199 @@ impl BusDevice for SparseWindowMap { sparse_access!(read64, write64, u64, BusRead64); } +/// The four uncached processor views decode MMIO separately from cached RAM. +pub struct Ip35Bus { + pub phys: Arc, + views: [SparseWindowMap; 4], +} +impl Ip35Bus { + pub fn new(phys: Arc) -> Self { + phys.set_default(Arc::new(Unclaimed), "IP35 unclaimed cached address"); + Self { + phys, + views: std::array::from_fn(|_| SparseWindowMap::default()), + } + } + /// Raw XKPHYS bits 58:57 select Hub-special, I/O, fetchop, or uncached RAM. + pub fn map(&mut self, raw_uc: u8, base: u64, size: u64, device: Arc) { + assert!(raw_uc < 4); + self.views[raw_uc as usize].map(base, size, device); + } + pub fn set_fallback(&mut self, raw_uc: u8, end: u64, device: Arc) { + self.map(raw_uc, 0, end, device); + } + fn target(&self, a: u64, n: u64) -> Option<(&SparseWindowMap, u64)> { + let pa = plain_address(a, n)?; + Some((&self.views[(a >> 57) as usize], pa)) + } + /// The bus the JIT compiles code through: the cached view, except the + /// reset PROM's uncached-fetch window ([`RESET_WINDOW_FETCH`]), which only + /// the uncached view holds. + #[cfg(feature = "jitv2")] + pub fn code_view(self: &Arc, cached: Arc) -> Arc { + Arc::new(CodeViews { cached, uncached: self.clone() }) + } + /// Cached RAM, with `plain_memory` answering wherever `extent` says + /// fitted RAM can be. + pub fn cached_view(&self, plain_memory: Arc, extent: RamExtent) -> Arc { + Arc::new(CachedPhysical { + phys: self.phys.clone(), + plain_memory, + extent, + }) + } +} +fn plain_address(a: u64, n: u64) -> Option { + if a & !CPU_PHYS_MASK != 0 { + return None; + } + let pa = a & PA_MASK; + (pa.checked_add(n)? <= PA_MASK + 1).then_some(pa) +} +/// Whether `a` is in the reset PROM's uncached-fetch window. +fn in_fetch_window(a: u64) -> bool { + a.wrapping_sub(RESET_WINDOW_FETCH) < RESET_WINDOW_SIZE +} +/// See [`Ip35Bus::code_view`]. Compiles read instruction words only. +#[cfg(feature = "jitv2")] +struct CodeViews { + cached: Arc, + uncached: Arc, +} +#[cfg(feature = "jitv2")] +impl BusDevice for CodeViews { + fn read32(&self, a: u64) -> BusRead32 { + if in_fetch_window(a) { + self.uncached.read32(a) + } else { + self.cached.read32(a) + } + } + fn gen_ptr(&self, a: u64) -> *const AtomicU64 { + if in_fetch_window(a) { self.uncached.gen_ptr(a) } else { self.cached.gen_ptr(a) } + } +} +struct CachedPhysical { + phys: Arc, + plain_memory: Arc, + extent: RamExtent, +} +impl CachedPhysical { + fn memory_target(&self, a: u64, n: u64) -> Option<(&dyn BusDevice, u64)> { + self.target(a, n).map(|(physical, pa)| { + // MD owns every guest write to a possible fitted-RAM prefix, so + // all paths update its stable logical generation counters. PROM + // and other devices retain the cached Physical connection. + let memory: &dyn BusDevice = if self.extent.contains(pa) { + self.plain_memory.as_ref() + } else { physical }; + (memory, pa) + }) + } + fn target(&self, a: u64, n: u64) -> Option<(&Physical, u64)> { + // UC selectors belong to the uncached SysAd connection. Reject them + // here rather than turning a malformed cached request into an alias. + if a & !PA_MASK != 0 || a & (n - 1) != 0 { + return None; + } + Some((self.phys.as_ref(), plain_address(a, n)?)) + } +} +macro_rules! processor_access { + ($r:ident,$w:ident,$t:ty,$out:ident,$n:expr,$writes:ident) => { + fn $r(&self, a: u64) -> $out { + self + .target(a, $n) + .map(|(v, pa)| v.$r(pa)) + .unwrap_or_else($out::err) + } + fn $w(&self, a: u64, v: $t) -> u32 { + self + .$writes(a, $n) + .map(|(view, pa)| view.$w(pa, v)) + .unwrap_or(BUS_ERR) + } + }; +} +impl BusDevice for Ip35Bus { + #[cfg(feature = "jitv2")] + fn gen_ptr(&self, address: u64) -> *const AtomicU64 { + // Only the reset PROM's page asks this bus: its generation is the PROM + // device's, so a flash command invalidates it and a store to RAM never + // does. The counter covers the whole flash, so any flash write + // invalidates every PROM page (over-invalidation, safe). + if in_fetch_window(address) { + return self.views[0].gen_ptr(address); + } + std::ptr::null() + } + fn write64_masked(&self, a: u64, val: u64, mask: u64) -> u32 { + if a & 7 != 0 { + return BUS_ERR; + } + self.target(a, 8) + .map(|(view, pa)| view.write64_masked(pa, val, mask)) + .unwrap_or(BUS_ERR) + } + fn write_narrow_from(&self, a: u64, size: usize, gpr: u64) -> u32 { + self.target(a, size as u64) + .map(|(view, pa)| view.write_narrow_from(pa, size, gpr)) + .unwrap_or(BUS_ERR) + } + processor_access!(read8, write8, u8, BusRead8, 1, target); + processor_access!(read16, write16, u16, BusRead16, 2, target); + processor_access!(read32, write32, u32, BusRead32, 4, target); + processor_access!(read64, write64, u64, BusRead64, 8, target); +} +impl BusDevice for CachedPhysical { + fn write64_masked(&self, address: u64, value: u64, mask: u64) -> u32 { + self.memory_target(address, 8).map(|(view, pa)| view.write64_masked(pa, value, mask)) + .unwrap_or(BUS_ERR) + } + fn mem_ptr(&self, a: u64) -> Option<*const u64> { + self.target(a, 8).and_then(|(view, pa)| view.mem_ptr(pa)) + } + #[cfg(feature = "jitv2")] + fn gen_ptr(&self, a: u64) -> *const AtomicU64 { + self.target(a, 1).map(|(view, pa)| { + let generation = self.plain_memory.gen_ptr(pa); + if generation.is_null() { view.gen_ptr(pa) } else { generation } + }).unwrap_or(std::ptr::null()) + } + fn read_block(&self, a: u64, out: &mut [u64]) -> u32 { + self.target(a, 8) + .map(|(view, pa)| view.read_block(pa, out)).unwrap_or(BUS_ERR) + } + fn write_block(&self, a: u64, data: &[u64]) -> u32 { + self.memory_target(a, 8) + .map(|(view, pa)| view.write_block(pa, data)).unwrap_or(BUS_ERR) + } + // A processor's SC to RAM the shadow cache cannot reach directly, such + // as a bank smaller than a direct-window region: the swap the store makes atomic, + // which matters once a node runs more than one processor. + fn cas32(&self, a: u64, old: u32, new: u32) -> Option { + self.memory_target(a, 4).and_then(|(view, pa)| view.cas32(pa, old, new)) + } + fn cas64(&self, a: u64, old: u64, new: u64) -> Option { + self.memory_target(a, 8).and_then(|(view, pa)| view.cas64(pa, old, new)) + } + processor_access!(read8, write8, u8, BusRead8, 1, memory_target); + processor_access!(read16, write16, u16, BusRead16, 2, memory_target); + processor_access!(read32, write32, u32, BusRead32, 4, memory_target); + processor_access!(read64, write64, u64, BusRead64, 8, memory_target); + fn dma_read64(&self, a: u64) -> BusRead64 { + plain_address(a, 8) + .map(|pa| self.plain_memory.dma_read64(pa)) + .unwrap_or_else(BusRead64::err) + } + fn dma_write64(&self, a: u64, v: u64) -> u32 { + plain_address(a, 8) + .map(|pa| self.plain_memory.dma_write64(pa, v)) + .unwrap_or(BUS_ERR) + } +} + +pub(crate) mod node; +pub(crate) mod presence; +#[cfg(test)] +mod tests; diff --git a/src/ip35/node.rs b/src/ip35/node.rs new file mode 100644 index 000000000..d6079f84c --- /dev/null +++ b/src/ip35/node.rs @@ -0,0 +1,301 @@ +//! The Bedrock node every IP35 board shares: the processor bus with its four +//! views, the Bedrock chip with its memory, the L1, the boot flash and the +//! Hub-special page, wired together and to the board's processors. A board +//! names its inputs in [`NodeConfig`] and adds its own crossbar widgets. +//! +//! # What is approximated here +//! +//! The two Hub-special head windows return zero on reads and acknowledge +//! writes without side effects. PROM 6.210 writes the 64-bit value 0x2000 +//! at relative offsets 0 and 0x40 before DRAM discovery. Their hardware +//! meaning is unknown; Bedrock logging records accesses to these windows. +//! +//! Node composition ported from hw/mips/sgi/sn1-node.c in +//! jamesbraid/qemu-sgi-origami. + +use super::{map_device, Ip35Bus, Window}; +use crate::cpu::mips_exec::CpuDevice; +use crate::dev::bedrock::{Bedrock, BedrockConfig, Population, BANK_SPAN, SLOTS}; +use crate::dev::bedrock_flash::{FirmwareRecord, Flash, PART_SIZE}; +use crate::dev::heart::Clock; +use crate::dev::l1::{L1Config, L1Event, L1}; +use crate::devlog::LogModule; +use crate::ppmem::devmap::SLOT_SHIFT; +use crate::traits::{BusDevice, BusRead16, BusRead32, BusRead64, BusRead8, MachineEvent, Signal, BUS_ERR, BUS_OK}; +use parking_lot::Mutex; +use std::path::Path; +use std::sync::mpsc::{channel, Receiver, Sender}; +use std::sync::Arc; + +/// What a board gives its Bedrock node. +pub(crate) struct NodeConfig<'a> { + /// The board's name in logs and map names. + pub name: &'static str, + pub payload: &'a [u8], + pub clock: Clock, + pub flash_path: Option<&'a Path>, + /// The memory banks in MiB, locational bank 0 upward. + pub banks_mib: &'a [u32], + /// The processor slots the board fits. + pub processors: Population, + pub firmware: FirmwareRecord, + /// The word the board reads back at the Hub-special board-id window. + pub board_id: u64, + pub l1: L1Config, +} + +/// One Bedrock node as the IP35 boards share it. +pub(crate) struct Node { + pub bus: Arc, + pub bedrock: Arc, + pub l1: Arc, + pub flash: Arc, + /// Cacheable memory at plain physical addresses: what a cache fills + /// from, and the one bus the JIT and its compile pool read code through. + pub cached: Arc, + /// What the JIT compiles code through: `cached`, plus the reset PROM's + /// uncached-fetch window ([`Ip35Bus::code_view`]). + #[cfg(feature = "jitv2")] + pub code: Arc, + /// The node's resets and power events, for the system controller. + pub events: Sender, + pub receiver: Receiver, +} + +impl Node { + pub fn build(config: NodeConfig<'_>) -> Result { + let NodeConfig { name, payload, clock, flash_path, banks_mib, processors, firmware, board_id, l1 } = config; + let (events, receiver) = channel(); + let bedrock = Bedrock::new(BedrockConfig { + name, + clock: clock.clone(), + processors, + banks_mib, + events: Some(events.clone()), + })?; + let phys = bedrock.physical().clone(); + let l1 = Arc::new(L1::new(l1)?); + l1.set_irq(bedrock.uart_irq()); + let flash = Arc::new(match flash_path { + Some(path) => Flash::with_file(payload, path, firmware), + None => Flash::new(payload, firmware), + }.map_err(|e| e.to_string())?); + let mut bus = Ip35Bus::new(phys.clone()); + bedrock.rebuild_memory(); + bus.set_fallback(1, 1 << 32, bedrock.pio()); + // The local and node XIO register forms: with processors on PI0 + // alone, the same window. + bus.map(1, 0x0100_0000, 0x80_0000, bedrock.local_registers()); + bus.map(1, 0x0180_0000, 0x80_0000, bedrock.registers()); + // Uncached RAM reaches the banks directly, as cached RAM does; the + // diagnostic views stay separate. + let uncached = bedrock.uncached_view(); + bus.map(3, 0, 1 << 33, uncached.clone()); + // Hub-special low memory: its first 256 MiB is the node memory view. + bus.map(0, 0, 0x1000_0000, uncached); + let prom = Arc::new(flash.array_view(0, PART_SIZE).map_err(|e| e.to_string())?); + let boot = Arc::new(flash.boot_view()); + for base in [0x1800_0000, 0x3800_0000] { + bus.map(0, base, 0x800_0000, boot.clone()); + } + // Reset execution always sees stored bytes, independently of commands. + bus.map(0, 0x1fc0_0000, 0x40_0000, prom.clone()); + // The same PROM where an uncached instruction fetch of that window is + // presented (`MipsExecutor::uncached_fetch_alias`). + bus.map(0, super::RESET_WINDOW_FETCH, super::RESET_WINDOW_SIZE, prom.clone()); + if !payload.is_empty() { + map_device(&phys, 0x1fc0_0000, payload.len() as u64, prom.clone(), &format!("{name} cached reset PROM")); + } + let cell = bedrock.clone(); + let permission: Arc bool + Send + Sync> = Arc::new(move || cell.cell_permits()); + let leds = Arc::new(Leds { bedrock: bedrock.clone(), values: Mutex::new([0; 4]) }); + // The three windows occupy the same processor register page. + let special = Arc::new(HubSpecialPage { + windows: [ + Window::new(0, 0x80, Arc::new(HubHead(name))).permitted(permission.clone()), + Window::new(0x80, 0x40, l1.clone()).permitted(permission.clone()), + Window::new(0xc0, 0x20, leds).permitted(permission), + ], + }); + for base in [0x1000_0000, 0x3000_0000] { + bus.map(0, base, 1 << SLOT_SHIFT, special.clone()); + bus.map(0, base + 0x400_0000, 8, Arc::new(BoardId { bedrock: bedrock.clone(), word: board_id })); + } + bus.map(0, 0x1_c000_0000, 1 << 30, bedrock.directory_bus()); + bus.map(0, 0x1_4000_0000, 1 << 30, bedrock.ecc_bus()); + bus.map(0, 0x1_0000_0000, 1 << 30, bedrock.ecc_scrub_bus()); + bus.map(2, 0, 1 << 33, bedrock.fetchop_bus()); + let bus = Arc::new(bus); + let cached = bus.cached_view(bedrock.dma_view(), bedrock.ram_extent()); + #[cfg(feature = "jitv2")] + let code = bus.code_view(cached.clone()); + Ok(Self { + bus, + bedrock, + l1, + flash, + cached, + #[cfg(feature = "jitv2")] + code, + events, + receiver, + }) + } + +} + +/// Give a node's memory and interrupts to its processors, CPU 0 first, each +/// in the slot the board's population gives it: a cached view of memory for +/// every one and its IP2..IP6 from its PI slot. The L1's power events reach +/// the system controller on `events`, and its soft reset every processor. +pub(crate) fn connect_cpus( + name: &'static str, + bedrock: &Bedrock, + l1: &L1, + events: &Sender, + cpus: &[Arc], +) -> Result<(), String> { + if let Some(space) = bedrock.physical().ppmem_space() { + // Each CPU reads memory through a cached view of its own. The PROM + // keeps its stack in primary data lines before it has set up memory, + // so lines may stand in every slot that can hold a bank without it. + let extent = bedrock.ram_extent(); + for slot in 0..SLOTS { + if extent.prefix(slot) > 0 { + let at = slot as u64 * BANK_SPAN; + space.allow_unbacked_lines(at, at + extent.prefix(slot)); + } + } + for cpu in cpus { + let v = space + .add_cached_view(&[cpu.ppmem_bitmap_ptr(), cpu.tcache_bitmap_ptr()]) + .map_err(|e| format!("ppmem: a CPU's cached view: {e}"))?; + unsafe { + cpu.set_tcache_window(space.view_base(v)); + cpu.set_cached_view(space, v); + cpu.set_tcache_shift(crate::ppmem::window_shift(space.window_size())); + #[cfg(feature = "jitv2")] + cpu.set_tcache_gen_window(bedrock.generation_base()); + } + } + bedrock.rebuild_memory(); + } + for (n, cpu) in cpus.iter().enumerate() { + bedrock.set_cpu_interrupts(n, cpu.interrupts_ptr()); + } + let tx = events.clone(); + l1.set_event_handler(Arc::new(move |event| { + dlog!(LogModule::L1, "{name}: L1 event {event:?}"); + if let Some(action) = machine_action(event) { + let _ = tx.send(action); + } + })); + Ok(()) +} + +/// What the machine does for an L1 power or reset event. A power change +/// alone does nothing: the shutdown or restart the L1 reports with it +/// carries the action. The L1 serves a power cycle (request 0x2006, which +/// the PROM sends from its failure paths) as power down then power up, and +/// a node that powers up starts from reset, so the cycle restarts the +/// machine; only a power down (request 0x2005 or the console's `pwr d`) +/// turns it off. +fn machine_action(event: L1Event) -> Option { + match event { + L1Event::Shutdown => Some(MachineEvent::PowerOff), + L1Event::Reset => Some(MachineEvent::HardReset), + L1Event::Power(_) => None, + } +} + +struct HubSpecialPage { + windows: [Window; 3], +} +impl HubSpecialPage { + fn target(&self, a: u64, n: u64) -> Option<&Window> { + self.windows.iter().find(|window| window.contains(a, n)) + } +} +macro_rules! hub_page_access { + ($r:ident,$w:ident,$t:ty,$out:ident,$n:expr) => { + fn $r(&self, a: u64) -> $out { + self.target(a, $n).map(|window| window.$r(a)).unwrap_or_else($out::err) + } + fn $w(&self, a: u64, value: $t) -> u32 { + self.target(a, $n).map(|window| window.$w(a, value)).unwrap_or(BUS_ERR) + } + }; +} +impl BusDevice for HubSpecialPage { + hub_page_access!(read8, write8, u8, BusRead8, 1); + hub_page_access!(read16, write16, u16, BusRead16, 2); + hub_page_access!(read32, write32, u32, BusRead32, 4); + hub_page_access!(read64, write64, u64, BusRead64, 8); +} + +struct BoardId { + bedrock: Arc, + word: u64, +} +impl BusDevice for BoardId { + fn read64(&self, a: u64) -> BusRead64 { + if a == 0 && self.bedrock.cell_permits() { + BusRead64::ok(self.word) + } else { + BusRead64::err() + } + } +} +// Compatibility head windows: their register contract is not established. +// PROM 6.210 writes 0x2000 at offsets 0 and 0x40 before DRAM discovery. +// Retain the existing zero-read/acknowledged-write model, with accesses +// observable through the Bedrock log, until the register behavior is known. +/// The board name for the log. +struct HubHead(&'static str); +macro_rules! head_access { + ($r:ident,$w:ident,$ty:ty,$result:ident) => { + fn $r(&self, a: u64) -> $result { + dlog!(LogModule::Bedrock, + "{}: unidentified Hub-special read {} bytes at {a:#x}", + self.0, + std::mem::size_of::<$ty>() + ); + $result::ok(0) + } + fn $w(&self, a: u64, v: $ty) -> u32 { + dlog!(LogModule::Bedrock, + "{}: unidentified Hub-special write {} bytes at {a:#x} value {v:#x}", + self.0, + std::mem::size_of::<$ty>() + ); + BUS_OK + } + }; +} +impl BusDevice for HubHead { + head_access!(read8, write8, u8, BusRead8); + head_access!(read16, write16, u16, BusRead16); + head_access!(read32, write32, u32, BusRead32); + head_access!(read64, write64, u64, BusRead64); +} +struct Leds { + bedrock: Arc, + values: Mutex<[u8; 4]>, +} +impl BusDevice for Leds { + fn read64(&self, a: u64) -> BusRead64 { + if a < 32 && a & 7 == 0 && self.bedrock.cell_permits() { + BusRead64::ok(self.values.lock()[a as usize / 8] as u64) + } else { + BusRead64::err() + } + } + fn write64(&self, a: u64, v: u64) -> u32 { + if a < 32 && a & 7 == 0 && self.bedrock.cell_permits() { + self.values.lock()[a as usize / 8] = v as u8; + BUS_OK + } else { + BUS_ERR + } + } +} diff --git a/src/ip35/presence.rs b/src/ip35/presence.rs new file mode 100644 index 000000000..fa9c67bcc --- /dev/null +++ b/src/ip35/presence.rs @@ -0,0 +1,320 @@ +//! The holds on the IP35 PROM's pair liveness check and its presence +//! publisher, outside the chip. +//! +//! The PROM brings a node's processors up on the assumption that they left +//! reset on one clock edge and have run in step since. Two handshakes +//! depend on it beyond what Bedrock can order by itself. +//! +//! - Pair liveness. After its presence join on LB scratch 1, each processor +//! of an interface with both slots fitted stores a status value to its own +//! PI real-time compare register and reads its partner's until the +//! partner's value appears, at most 200,000 reads (PROM 6.210 at +//! `0x1fc69234`). If the reads run out, it clears the partner's CPU enable +//! and prints `peer_pled failed: Other CPU on this node died.` The bound +//! is the poller's own progress. On hardware the pair reach their posts +//! microseconds apart; here each processor is a host thread whose +//! progress over the seconds since the reset barrier drifts from its +//! partner's, by more than the reads last once the poll runs compiled. +//! So the first of a pair to post is held until its partner posts. +//! - Presence publisher. The lowest processor of each interface takes the +//! hub lock (LB scratch 3, through its test-and-set twin), sets its +//! interface's slot bits in II scratch 0 (bit 4 + n for slot n: A and B on +//! PI0, C and D on PI1), releases the lock and polls II scratch 0 for the +//! other interface's bits. Its poll is inferred to end on the PI's +//! real-time counter, which runs on host time, so a publisher whose +//! partner interface is descheduled can run its window out and the PROM +//! loses that interface's processors. A publisher that has published, +//! with the hub lock free, while another populated interface has not, is +//! held until that interface publishes. +//! +//! A held processor ([`HOLD_RENDEZVOUS`]) stops at its batch boundary. The +//! check runs on every processor's batch boundary and reads atomics Bedrock +//! exposes ([`Bedrock::rt_posted`], [`Bedrock::io_scratch0`], +//! [`Bedrock::hub_lock_free`]); nothing calls back from the chip. A hold is +//! bounded by the cycles the processors it waits for have retired +//! ([`HOLD_CYCLES`]), not host time, so a host thread the scheduler leaves +//! off a core cannot run it out; host time backs it only as a deadlock +//! safety net ([`SAFETY_NS`]). A liveness hold that runs out ends only +//! itself, and the PROM then finds the partner dead as it would a dead one; +//! a publisher's hold that runs out ends the join until the next reset. +//! While nothing is held and the processor has no hold to start, the check +//! takes no lock; once every populated interface has published, it is one +//! atomic load. +//! +//! These are approximations with no hardware counterpart: they change when +//! a processor next runs, never a value it reads. A held processor stops at +//! its next batch boundary, not at once. Nothing holds with fewer than two +//! processors, and the liveness hold needs both slots of an interface +//! fitted. QEMU holds the same processors, the first poster at its post and +//! the publisher at its release of the hub lock, and bounds both at 2 s of +//! its virtual clock. +//! +//! Ported from hw/misc/sgi-bedrock.c in jamesbraid/qemu-sgi-origami. + +use crate::cpu::mips_core::CyclesPtr; +use crate::cpu::mips_exec::CpuDevice; +use crate::cpu::smp::{ParkSlot, HOLD_RENDEZVOUS}; +use crate::dev::bedrock::{Bedrock, Population}; +use crate::dev::heart::Clock; +use crate::devlog::LogModule; +use parking_lot::Mutex; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; +use std::sync::{Arc, OnceLock}; + +/// How far the processors a hold waits for may run, in their retired +/// cycles, before the hold gives up: 2 s of a 1 GHz processor, the order of +/// QEMU's 2 s bound. +pub(crate) const HOLD_CYCLES: u64 = 2_000_000_000; +/// The longest a hold lasts in board nanoseconds whatever the processors +/// do: a deadlock safety net, far past any host stall. +pub(crate) const SAFETY_NS: u64 = 120_000_000_000; + +/// One processor the hold may stop: its park slot and its retired cycles. +pub(crate) struct Cpu { + pub slot: Arc, + pub cycles: Box u64 + Send + Sync>, +} + +/// What a held processor waits for. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Kind { + /// Its pair partner's liveness post. + Partner, + /// The other interface's publication. + Publication, +} + +/// One held processor: what for, when the hold started, in board +/// nanoseconds, and how many cycles the processors it waits for had retired +/// by then. +#[derive(Clone, Copy)] +struct Held { + kind: Kind, + since: u64, + progress: u64, +} + +#[derive(Default)] +struct State { + held: [Option; 4], + /// A hold ran out: no more holds until the next reset. + abandoned: bool, +} + +pub(crate) struct PresenceHold { + processors: Population, + clock: Clock, + bedrock: Arc, + cpus: OnceLock>, + state: Mutex, + /// Every populated interface has published, or the join was abandoned, + /// and nothing is held: the check is over until the next reset. + done: AtomicBool, + /// Whether any processor is held, so a boundary with nothing to release + /// and no hold to start takes no lock. + holding: AtomicBool, + /// Each processor's pair partner: the other slot of its interface, if + /// fitted. + partners: [Option; 4], + /// Processors, one bit per CPU number, whose liveness hold ran out: not + /// held for their partner again until the next reset. + gave_up: AtomicU64, +} + +impl PresenceHold { + pub fn new(processors: Population, clock: Clock, bedrock: &Arc) -> Arc { + Arc::new(Self { + processors, + clock, + bedrock: bedrock.clone(), + cpus: OnceLock::new(), + state: Mutex::new(State::default()), + done: AtomicBool::new(processors.count() < 2), + holding: AtomicBool::new(false), + partners: std::array::from_fn(|cpu| { + let (interface, slot) = processors.slot_of(cpu)?; + (0..processors.count()).find(|&other| processors.slot_of(other) == Some((interface, 1 - slot))) + }), + gave_up: AtomicU64::new(0), + }) + } + + /// The processors, CPU 0 first, in the order the population numbers + /// them. Once. + pub fn set_cpus(&self, cpus: &[Arc]) { + self.set_processors(cpus.iter().map(|c| { + let cycles: CyclesPtr = c.cycles_ptr(); + Cpu { slot: c.park_slot(), cycles: Box::new(move || cycles.retired()) } + }).collect()); + } + + pub(crate) fn set_processors(&self, cpus: Vec) { + assert!(self.cpus.set(cpus).is_ok(), "presence hold processors set twice"); + } + + fn interface_of(&self, cpu: usize) -> Option { + self.processors.slot_of(cpu).map(|(interface, _)| interface) + } + + fn populated(&self, interface: u8) -> bool { + (0..2).any(|slot| self.processors.fitted(interface, slot)) + } + + /// Whether `interface` has published its processors in `presence`. + fn published(presence: u64, interface: u8) -> bool { + presence & (0x30 << (2 * interface)) != 0 + } + + /// The interface's publisher: its lowest fitted processor. + fn publisher(&self, interface: u8) -> Option { + (0..self.processors.count()).find(|&cpu| self.interface_of(cpu) == Some(interface)) + } + + /// The retired cycles of the processors on interfaces other than `own`. + fn progress(&self, own: u8) -> u64 { + let Some(cpus) = self.cpus.get() else { return 0 }; + cpus.iter().enumerate() + .filter(|&(cpu, _)| self.interface_of(cpu) != Some(own)) + .fold(0u64, |sum, (_, c)| sum.saturating_add((c.cycles)())) + } + + /// The retired cycles of processor `cpu`. + fn cycles(&self, cpu: usize) -> u64 { + self.cpus.get().and_then(|cpus| cpus.get(cpu)).map_or(0, |c| (c.cycles)()) + } + + /// What a hold of `cpu` started now waits for has retired so far. + fn waited_progress(&self, cpu: usize, kind: Kind) -> u64 { + match kind { + Kind::Partner => self.partners[cpu].map_or(0, |partner| self.cycles(partner)), + Kind::Publication => self.interface_of(cpu).map_or(0, |own| self.progress(own)), + } + } + + /// Whether `cpu` has made its liveness post and waits for its partner's: + /// the partner has not posted and `cpu`'s hold has not run out since the + /// last reset. + fn awaits_partner(&self, posted: u64, cpu: usize) -> bool { + let bit = |cpu: usize| 1u64 << cpu; + self.partners[cpu].is_some_and(|partner| posted & bit(partner) == 0) + && posted & bit(cpu) != 0 + && self.gave_up.load(Ordering::Relaxed) & bit(cpu) == 0 + } + + fn release(&self, s: &mut State, cpu: usize, outcome: &str) { + if s.held[cpu].take().is_some() { + dlog!(LogModule::Bedrock, "presence: CPU {cpu} hold {outcome}"); + if let Some(c) = self.cpus.get().and_then(|cpus| cpus.get(cpu)) { + c.slot.release(HOLD_RENDEZVOUS); + } + } + } + + /// CPU `cpu`'s batch boundary: end the holds that are satisfied or past + /// their bound, and hold `cpu` if it has posted and waits for its + /// partner's post, or is a publisher waiting for the other interface. + pub fn at_batch(&self, cpu: usize) { + if self.done.load(Ordering::Relaxed) { + return; + } + let presence = self.bedrock.io_scratch0(); + let posted = self.bedrock.rt_posted(); + // Nothing held and nothing for this processor to start: no lock. + if !self.holding.load(Ordering::Acquire) && !self.may_hold(presence, cpu) && !self.awaits_partner(posted, cpu) { + let all = (0..2).filter(|&i| self.populated(i)).all(|i| Self::published(presence, i)); + if all && !self.done.swap(true, Ordering::Relaxed) { + dlog!(LogModule::Bedrock, "presence: the join is over"); + } + return; + } + let now = (self.clock)(); + let mut s = self.state.lock(); + for held in 0..s.held.len() { + let Some(hold) = s.held[held] else { continue }; + let satisfied = match hold.kind { + Kind::Partner => !self.awaits_partner(posted, held), + Kind::Publication => self.interface_of(held).is_none_or(|own| Self::published(presence, 1 - own)), + }; + let ran_out = self.waited_progress(held, hold.kind).saturating_sub(hold.progress) >= HOLD_CYCLES + || now.saturating_sub(hold.since) >= SAFETY_NS; + if satisfied { + self.release(&mut s, held, "satisfied"); + } else if ran_out && hold.kind == Kind::Partner { + self.gave_up.fetch_or(1 << held, Ordering::Relaxed); + self.release(&mut s, held, "ran out; its partner is left to the PROM's check"); + } else if ran_out { + s.abandoned = true; + self.release(&mut s, held, "ran out; no more holds until a reset"); + } + } + let all = (0..2).filter(|&i| self.populated(i)).all(|i| Self::published(presence, i)); + if (all || s.abandoned) && s.held.iter().all(Option::is_none) { + if !self.done.swap(true, Ordering::Relaxed) { + dlog!(LogModule::Bedrock, "presence: the join is over"); + } + return; + } + let kind = if s.held[cpu].is_some() { + None + } else if self.awaits_partner(posted, cpu) { + Some(Kind::Partner) + } else if !s.abandoned && self.may_hold(presence, cpu) { + Some(Kind::Publication) + } else { + None + }; + let Some(kind) = kind else { + self.holding.store(s.held.iter().any(Option::is_some), Ordering::Release); + return; + }; + s.held[cpu] = Some(Held { kind, since: now, progress: self.waited_progress(cpu, kind) }); + self.holding.store(true, Ordering::Release); + match kind { + Kind::Partner => dlog!(LogModule::Bedrock, "presence: holding CPU {cpu} until its partner posts"), + Kind::Publication => dlog!(LogModule::Bedrock, "presence: holding CPU {cpu} until the other interface publishes"), + } + if let Some(c) = self.cpus.get().and_then(|cpus| cpus.get(cpu)) { + c.slot.hold(HOLD_RENDEZVOUS); + } + } + + /// Whether `cpu` is a publisher that should wait now: its interface has + /// published and another populated one has not, with the hub lock free. + fn may_hold(&self, presence: u64, cpu: usize) -> bool { + let Some(own) = self.interface_of(cpu) else { return false }; + let other = 1 - own; + self.publisher(own) == Some(cpu) + && Self::published(presence, own) + && self.populated(other) + && !Self::published(presence, other) + && self.bedrock.hub_lock_free() + } + + /// Every hold released and the join started over. + pub fn reset(&self) { + let mut s = self.state.lock(); + for cpu in 0..s.held.len() { + self.release(&mut s, cpu, "ended by a reset"); + } + *s = State::default(); + self.holding.store(false, Ordering::Release); + self.gave_up.store(0, Ordering::Relaxed); + self.done.store(self.processors.count() < 2, Ordering::Relaxed); + } + + /// The processors held now, for tests. + #[cfg(test)] + pub(crate) fn held(&self) -> Vec { + let s = self.state.lock(); + (0..s.held.len()).filter(|&cpu| s.held[cpu].is_some()).collect() + } + + /// Whether the check is over, for tests. + #[cfg(test)] + pub(crate) fn is_done(&self) -> bool { + self.done.load(Ordering::Relaxed) + } +} + +#[cfg(test)] +mod tests; diff --git a/src/ip35/presence/tests.rs b/src/ip35/presence/tests.rs new file mode 100644 index 000000000..947bc2839 --- /dev/null +++ b/src/ip35/presence/tests.rs @@ -0,0 +1,261 @@ +use super::*; +use crate::cpu::smp::HOST_PARK_BIT; +use crate::dev::bedrock::{reg, BedrockConfig}; +use crate::traits::{BusDevice, Device, BUS_OK}; +use std::sync::atomic::AtomicU64; + +/// A node of `count` processors with its register window, each processor a +/// park slot and a retired-cycle counter the test drives. +struct Rig { + hold: Arc, + regs: Arc, + words: Vec>, + cycles: Vec>, + slots: Vec>, + now: Arc, +} + +impl Rig { + fn new(count: usize) -> Self { + let now = Arc::new(AtomicU64::new(0)); + let clock_now = now.clone(); + let clock: Clock = Arc::new(move || clock_now.load(Ordering::SeqCst)); + let processors = Population::fitted_cpus(count).unwrap(); + let bedrock = Bedrock::new(BedrockConfig { name: "test", clock: clock.clone(), processors, banks_mib: &[64], events: None }).unwrap(); + bedrock.start(); + let hold = PresenceHold::new(processors, clock, &bedrock); + let words: Vec<_> = (0..count).map(|_| Arc::new(AtomicU64::new(0))).collect(); + let cycles: Vec<_> = (0..count).map(|_| Arc::new(AtomicU64::new(0))).collect(); + let slots: Vec<_> = words.iter().map(|w| Arc::new(ParkSlot::new(Arc::as_ptr(w)))).collect(); + hold.set_processors(slots.iter().zip(&cycles).map(|(slot, c)| { + let c = c.clone(); + Cpu { slot: slot.clone(), cycles: Box::new(move || c.load(Ordering::SeqCst)) } + }).collect()); + Self { hold, regs: bedrock.registers(), words, cycles, slots, now } + } + /// II scratch 0, as a publisher's store leaves it. + fn publish(&self, bits: u64) { + assert_eq!(self.regs.write64(reg::IO_BASE + 0x150, bits), BUS_OK); + } + /// Take the hub lock through scratch 3's test-and-set twin, or release it. + fn hub_lock(&self, take: bool) { + if take { + assert_eq!(self.regs.read64(reg::LB_BASE + 0x8058).data, 0, "the lock was free"); + } else { + assert_eq!(self.regs.write64(reg::LB_BASE + 0x8018, 0), BUS_OK); + } + } + /// A store of `value` at `offset` in the register window by processor + /// `cpu`'s thread. + fn store_as(&self, cpu: usize, offset: u64, value: u64) { + let regs = self.regs.clone(); + let status = std::thread::spawn(move || { + crate::cpu::smp::set_current_cpu(cpu as u8); + regs.write64(offset, value) + }) + .join() + .unwrap(); + assert_eq!(status, BUS_OK); + } + /// The PROM's reset barrier on LB scratch 1: every processor clears it, + /// CPU 0 releases, and then `joins` store again, their presence join. + fn barrier(&self, joins: &[usize]) { + for cpu in 0..self.slots.len() { + self.store_as(cpu, reg::LB_BASE + 0x8008, 0); + } + self.store_as(0, reg::LB_BASE + 0x8008, 0x07f8_0000); + for &cpu in joins { + self.store_as(cpu, reg::LB_BASE + 0x8008, 0); + } + } + /// Processor `cpu` stores `value` to the real-time compare register of + /// `slot` (0 = A, 1 = B) on PI `interface`. + fn compare(&self, cpu: usize, interface: u64, slot: u64, value: u64) { + self.store_as(cpu, interface * reg::PI1_BASE + 0x108 + 8 * slot, value); + } + /// Processor `cpu` posts its liveness value in its own compare register. + fn post(&self, cpu: usize) { + let (interface, slot) = Population::fitted_cpus(self.slots.len()).unwrap().slot_of(cpu).unwrap(); + self.compare(cpu, u64::from(interface), slot as u64, 0x24); + } + fn parked(&self, cpu: usize) -> bool { + self.words[cpu].load(Ordering::SeqCst) & HOST_PARK_BIT != 0 && self.slots[cpu].holds() & HOLD_RENDEZVOUS != 0 + } +} + +/// PI0's publisher, having published with the lock free while PI1 has not, +/// is held at its batch boundary; its partner on PI0 and PI1's processors +/// are not. PI1's publication releases it at the next boundary any +/// processor reaches, and the join is over. +#[test] +fn a_publisher_waits_for_the_other_interface() { + let rig = Rig::new(4); + rig.hub_lock(true); + rig.publish(0x30); + rig.hold.at_batch(0); + assert!(rig.hold.held().is_empty(), "not while the hub lock is held"); + rig.hub_lock(false); + for cpu in [2, 1, 3] { + rig.hold.at_batch(cpu); + } + assert!(rig.hold.held().is_empty(), "only a publisher whose interface has published"); + rig.hold.at_batch(0); + assert_eq!(rig.hold.held(), [0]); + assert!(rig.parked(0)); + rig.publish(0xf0); + rig.hold.at_batch(1); + assert!(rig.hold.held().is_empty() && !rig.parked(0)); + assert!(rig.hold.is_done()); +} + +/// The bound counts the waited-for processors' cycles: a PI1 that runs +/// HOLD_CYCLES without publishing ends the hold, and no later hold starts +/// until a reset, which starts the join over. +#[test] +fn a_hold_ends_once_the_other_interface_has_run_its_bound() { + let rig = Rig::new(2); + rig.publish(0x10); + rig.hold.at_batch(0); + assert_eq!(rig.hold.held(), [0]); + rig.cycles[1].store(HOLD_CYCLES - 1, Ordering::SeqCst); + rig.hold.at_batch(1); + assert_eq!(rig.hold.held(), [0], "not yet"); + rig.cycles[0].store(u64::MAX, Ordering::SeqCst); + rig.hold.at_batch(1); + assert_eq!(rig.hold.held(), [0], "the held processor's own cycles do not count"); + rig.cycles[1].store(HOLD_CYCLES, Ordering::SeqCst); + rig.hold.at_batch(1); + assert!(rig.hold.held().is_empty() && !rig.parked(0)); + assert!(rig.hold.is_done(), "abandoned"); + rig.hold.reset(); + assert!(!rig.hold.is_done()); + rig.hold.at_batch(0); + assert_eq!(rig.hold.held(), [0], "a reset starts the join over"); + rig.hold.reset(); + assert!(rig.hold.held().is_empty() && !rig.parked(0)); +} + +/// Host time backs the bound only as a safety net. +#[test] +fn the_safety_net_ends_a_hold_whatever_the_cycles() { + let rig = Rig::new(2); + rig.publish(0x10); + rig.hold.at_batch(0); + rig.now.store(SAFETY_NS - 1, Ordering::SeqCst); + rig.hold.at_batch(1); + assert_eq!(rig.hold.held(), [0]); + rig.now.store(SAFETY_NS, Ordering::SeqCst); + rig.hold.at_batch(1); + assert!(rig.hold.held().is_empty()); +} + +/// One processor has nobody to wait for. +#[test] +fn one_processor_is_never_held() { + let rig = Rig::new(1); + assert!(rig.hold.is_done()); + rig.publish(0x10); + rig.hold.at_batch(0); + assert!(rig.hold.held().is_empty()); +} + +/// A boundary with nothing held and no hold to start takes no lock: CPU 2, +/// not a publisher, passes its boundary while the state is locked, and so +/// does a publisher whose partner interface has published. +#[test] +fn a_boundary_with_nothing_to_do_takes_no_lock() { + let rig = Rig::new(4); + rig.publish(0x30); + let held = rig.hold.state.lock(); + rig.hold.at_batch(2); + rig.hold.at_batch(1); + drop(held); + rig.publish(0xf0); + let held = rig.hold.state.lock(); + rig.hold.at_batch(0); + drop(held); + assert!(rig.hold.is_done()); +} + +/// The first of a pair to post its liveness value after its join is held +/// until its partner posts: CPU 0 (PI0/A) waits for CPU 2 (PI0/B), and CPU 3 +/// (PI1/B) for CPU 1 (PI1/A). Nobody else is held, and the partner's post +/// releases the hold at the next boundary any processor reaches. +#[test] +fn the_first_of_a_pair_to_post_waits_for_its_partner() { + let rig = Rig::new(4); + rig.barrier(&[0, 1, 2, 3]); + rig.post(0); + for cpu in [1, 2, 3, 0] { + rig.hold.at_batch(cpu); + } + assert_eq!(rig.hold.held(), [0]); + assert!(rig.parked(0)); + rig.post(3); + rig.hold.at_batch(3); + assert_eq!(rig.hold.held(), [0, 3]); + rig.post(2); + rig.hold.at_batch(1); + assert_eq!(rig.hold.held(), [3], "CPU 2's post released CPU 0"); + assert!(!rig.parked(0)); + rig.post(1); + rig.hold.at_batch(2); + assert!(rig.hold.held().is_empty() && !rig.parked(3)); + for cpu in 0..4 { + rig.hold.at_batch(cpu); + } + assert!(rig.hold.held().is_empty(), "both pairs posted"); +} + +/// Only a processor's first store to its own compare register after its +/// join is a post: earlier stores are progress codes, and a store to the +/// partner's register is not the processor's. +#[test] +fn only_a_store_to_its_own_register_after_its_join_posts() { + let rig = Rig::new(4); + rig.post(0); + rig.hold.at_batch(0); + assert!(rig.hold.held().is_empty(), "a progress code before the join"); + rig.barrier(&[0, 2]); + rig.compare(0, 0, 1, 0x24); + rig.hold.at_batch(0); + assert!(rig.hold.held().is_empty(), "CPU 0 wrote CPU 2's register"); + rig.post(0); + rig.hold.at_batch(0); + assert_eq!(rig.hold.held(), [0]); +} + +/// A partner that runs HOLD_CYCLES without posting ends the hold, and the +/// processor is not held for it again: the PROM's own check then finds the +/// partner dead. The presence join goes on. +#[test] +fn a_liveness_hold_runs_out_on_the_partners_cycles_and_ends_only_itself() { + let rig = Rig::new(4); + rig.barrier(&[0, 1, 2, 3]); + rig.post(0); + rig.hold.at_batch(0); + assert_eq!(rig.hold.held(), [0]); + rig.cycles[1].store(u64::MAX, Ordering::SeqCst); + rig.hold.at_batch(1); + assert_eq!(rig.hold.held(), [0], "only the partner's cycles count"); + rig.cycles[2].store(HOLD_CYCLES, Ordering::SeqCst); + rig.hold.at_batch(2); + assert!(rig.hold.held().is_empty() && !rig.parked(0)); + rig.hold.at_batch(0); + assert!(rig.hold.held().is_empty(), "not held for that partner again"); + rig.publish(0x10); + rig.hold.at_batch(0); + assert_eq!(rig.hold.held(), [0], "the publisher's hold still applies"); +} + +/// Two processors sit one per interface: neither has a partner to wait for. +#[test] +fn two_processors_have_no_pair() { + let rig = Rig::new(2); + rig.barrier(&[0, 1]); + rig.post(0); + rig.post(1); + rig.hold.at_batch(0); + rig.hold.at_batch(1); + assert!(rig.hold.held().is_empty()); +} diff --git a/src/ip35/tests.rs b/src/ip35/tests.rs new file mode 100644 index 000000000..930652564 --- /dev/null +++ b/src/ip35/tests.rs @@ -0,0 +1,971 @@ +use super::*; +use crate::dev::bedrock::reg as bedrock_reg; +use crate::traits::{BusDevice, BUS_OK}; + +/// A Bedrock unit's registers, at its base in the node's register window. +struct Unit(Arc, u64); +impl Unit { + fn read64(&self, a: u64) -> BusRead64 { + self.0.read64(self.1 + a) + } + fn write64(&self, a: u64, v: u64) -> u32 { + self.0.write64(self.1 + a, v) + } +} +fn md_regs(board: &crate::ip34_machine::FuelBoard) -> Unit { + Unit(board.bedrock.registers(), bedrock_reg::MD_BASE) +} +fn pi_regs(board: &crate::ip34_machine::FuelBoard, interface: u64) -> Unit { + Unit(board.bedrock.registers(), interface * bedrock_reg::PI1_BASE) +} + +struct Word(u64); +impl BusDevice for Word { + fn read64(&self, _: u64) -> BusRead64 { + BusRead64::ok(self.0) + } +} + +#[derive(Clone, Debug, PartialEq, Eq)] +enum NarrowWriteCall { + Byte(u64, u8), + Half(u64, u16), + From(usize, u64, u64), +} +/// Records every narrow write it sees. With `sees_register` it overrides +/// `write_narrow_from` and keeps the whole source register, as Bedrock's ECC +/// view does; otherwise it takes the trait's default. +#[derive(Default)] +struct NarrowWriteSpy { + sees_register: bool, + calls: std::sync::Mutex>, +} +impl NarrowWriteSpy { + fn new(sees_register: bool) -> Arc { + Arc::new(Self { sees_register, ..Default::default() }) + } + fn calls(&self) -> Vec { + self.calls.lock().unwrap().clone() + } +} +impl BusDevice for NarrowWriteSpy { + fn write8(&self, address: u64, value: u8) -> u32 { + self.calls.lock().unwrap().push(NarrowWriteCall::Byte(address, value)); + BUS_OK + } + fn write16(&self, address: u64, value: u16) -> u32 { + self.calls.lock().unwrap().push(NarrowWriteCall::Half(address, value)); + BUS_OK + } + fn write_narrow_from(&self, address: u64, size: usize, gpr: u64) -> u32 { + if !self.sees_register { + return if size == 1 { self.write8(address, gpr as u8) } else { self.write16(address, gpr as u16) }; + } + self.calls.lock().unwrap().push(NarrowWriteCall::From(size, address, gpr)); + BUS_OK + } +} + +/// An uncached SB or SH reaches a device with the whole source register, at the +/// device-local address; only a device that overrides the call sees more than +/// the low bits, and a window outside every map is a bus error. +/// The Fuel's processor is the R14000 at revision 2.3 and 500 MHz, built as +/// the R12000 type with its PRId set over it; the Octane's own part is +/// revision 2.4. +#[test] +fn the_fuel_runs_the_r14000_at_revision_2_3() { + use crate::cpu::mips_cache_shadow::R12000ShadowCache; + use crate::cpu::mips_cache_v2::CpuModel as Model; + use crate::cpu::parts; + use crate::ip34_machine::CPUS; + assert_eq!(CPUS, &[&parts::R14000_500]); + let part = CPUS[0]; + assert_eq!((part.mhz, part.rev, part.l2_kb), (500, (2, 3), 2048)); + assert_eq!(part.prid(), 0x0f23); + assert_eq!(part.count_hz(), 250_000_000); + // `with_cpu!` builds the type `FuelCpu` names for this part. + let built = crate::with_cpu!(part, true, false, C => std::any::type_name::()); + assert_eq!(built, std::any::type_name::()); + // Typed R12000, it takes the R10000 family's behaviour: reserved + // secondary-tag bits read as zero, and a refused uncached store posts. + assert!(::R10K_FAMILY); + assert_ne!(::SECONDARY_TAG_MASK, u64::MAX); + assert!(::POSTS_UNCACHED_STORES); + assert!(::PARTIAL_LWR_KEEPS_UPPER); + // The executor carries the chip's PRId. + // The executor is large; the default test stack is not. + std::thread::Builder::new() + .stack_size(64 << 20) + .spawn(|| { + let cpu = crate::ip34_machine::build_cpu(Arc::new(Ip35Bus::new(Physical::new(Vec::new()))), crate::ip34_machine::CPUS[0]); + assert_eq!(cpu.core.cp0_prid, 0x0f23); + }) + .unwrap() + .join() + .expect("thread panicked"); +} + +#[test] +fn narrow_stores_carry_the_register_to_devices_that_ask_for_it() { + let mut bus = Ip35Bus::new(Physical::new(Vec::new())); + let plain = NarrowWriteSpy::new(false); + let ecc = NarrowWriteSpy::new(true); + bus.map(3, 0x1000, 0x1000, plain.clone()); + bus.map(3, 0x2000, 0x1000, ecc.clone()); + + let source = 0x1122_3344_5566_77a5u64; + let uc = 3u64 << 57; + assert_eq!(bus.write_narrow_from(uc | 0x1003, 1, source), BUS_OK); + assert_eq!(bus.write_narrow_from(uc | 0x1004, 2, source), BUS_OK); + assert_eq!( + plain.calls(), + vec![NarrowWriteCall::Byte(3, source as u8), NarrowWriteCall::Half(4, source as u16)], + "an ordinary window gets the architectural low bits", + ); + + assert_eq!(bus.write_narrow_from(uc | 0x2004, 2, source), BUS_OK); + assert_eq!(ecc.calls(), vec![NarrowWriteCall::From(2, 4, source)]); + + assert_eq!(bus.write_narrow_from(uc | 0x8000, 2, source), BUS_ERR, "no window there"); +} + +/// Equal physical offsets in different XKPHYS access spaces reach different devices. +#[test] +fn uc_spaces_keep_equal_offsets_distinct() { + let mut bus = Ip35Bus::new(Physical::new(Vec::new())); + for space in 0..4 { + bus.map( + space, + 0x0100_0000, + 0x1000, + Arc::new(Word(0x100 + space as u64)), + ); + } + for space in 0..4 { + let value = bus.read64(((space as u64) << 57) | 0x0100_0000); + assert_eq!(value.status, BUS_OK); + assert_eq!(value.data, 0x100 + space as u64); + } +} + +/// The processor must retain both XKPHYS access-space bits through translation. +#[test] +fn xkphys_uc_bits_survive_the_board_cpu_mask() { + // MipsExecutor and the TLB's inline arrays exceed the test worker's stack. + std::thread::Builder::new() + .stack_size(4 * 1024 * 1024) + .spawn(|| { + let bus = Arc::new(Ip35Bus::new(Physical::new(Vec::new()))); + let mut cpu = crate::ip34_machine::build_cpu(bus, crate::ip34_machine::CPUS[0]); + cpu.core.write_cp0( + 12, + (cpu.core.cp0_status | crate::cpu::mips_core::STATUS_KX) as u64, + ); + for raw_uc in 0..4u64 { + let va = 0x9000_0000_0100_0020 | (raw_uc << 57); + for _ in 0..2 { + let translated = cpu.debug_translate(va); + assert_eq!(translated.status, crate::cpu::mips_exec::TR_UNCACHED); + assert_eq!(translated.phys, (raw_uc << 57) | 0x0100_0020); + } + } + }) + .expect("spawn") + .join() + .expect("thread panicked"); +} + +/// RAM placed above the first GiB must remain shared by cached, uncached and DMA paths. +#[test] +fn cached_and_uncached_memory_keep_high_addresses() { + let board = crate::ip34_machine::FuelBoard::new(&[], Arc::new(|| 0)).unwrap(); + assert_eq!(md_regs(&board).write64(8, 0x33), BUS_OK); + let at = (1 << 30) + 0x1000; + assert_eq!(board.bedrock.dma_view().write64(at, 0x12345678), BUS_OK); + assert_eq!(board.cached_bus().read64(at).data, 0x12345678); + assert_eq!(board.bus.read64((3 << 57) | at).data, 0x12345678); + assert_eq!(board.bus.read64(at).status, BUS_ERR); + assert_eq!(board.bedrock.dma_view().read64(at).data, 0x12345678); +} + +/// A processor store through the cached view or an uncached one lands in +/// the bank the slot holds and advances that slot's logical counter, +/// whichever bank sits there after a swap, and no other slot's. +#[test] +fn processor_stores_advance_the_counter_of_the_slot_they_reach() { + let board = crate::ip34_machine::FuelBoard::new(&[], Arc::new(|| 0)).unwrap(); + let memory = board.bedrock.dma_view(); + let counter = |_at: u64| -> u64 { + #[cfg(feature = "jitv2")] + return unsafe { (*memory.gen_ptr(_at)).load(std::sync::atomic::Ordering::Relaxed) }; + #[cfg(not(feature = "jitv2"))] + 0 + }; + let advanced = |what: &str, before: u64, after: u64| { + assert!(!cfg!(feature = "jitv2") || after > before, "{what}: {before} -> {after}"); + }; + let (zero, one) = (0x2000, (1 << 30) + 0x2000); + // Bit 32 swaps the banks of a pair: slot 0 now reaches bank 1. + assert_eq!(md_regs(&board).write64(8, 0x33 | (1 << 32)), BUS_OK); + let banks = board.bedrock.physical().banks(); + let (before_zero, before_one) = (counter(zero), counter(one)); + assert_eq!(board.cached_bus().write64(zero, 0x44), BUS_OK); + assert_eq!(banks[1].read64(0x2000).data, 0x44, "slot 0 reaches bank 1"); + assert_eq!(banks[0].read64(0x2000).data, 0); + let after_zero = counter(zero); + advanced("cached store to slot 0", before_zero, after_zero); + assert_eq!(counter(one), before_one); + assert_eq!(board.bus.write64((3 << 57) | one, 0x55), BUS_OK); + assert_eq!(banks[0].read64(0x2000).data, 0x55, "slot 1 reaches bank 0"); + let after_one = counter(one); + advanced("uncached store to slot 1", before_one, after_one); + assert_eq!(board.bus.write8((3 << 57) | (one + 8), 0x77), BUS_OK); + advanced("uncached byte store", after_one, counter(one)); + assert_eq!(counter(zero), after_zero); +} + +/// Firmware resets the PCI link before reinitializing the attached storage controller. +#[test] +fn fuel_link_reset_resets_the_attached_scsi_engine() { + use crate::dev::isp1040::{mbox, reg, CTRL_INT_ENABLE, CTRL_RISC_INT_ENABLE, HOST_CMD_SHIFT}; + use crate::pci::PciFunction; + let board = crate::ip34_machine::FuelBoard::new(&[], Arc::new(|| 0)).unwrap(); + let isp = &board.scsi; + isp.state().config().write(1, 6); + isp.state().config().write(5, 0x02000000); + isp.bar_write(1, reg::BUS_CTRL, u32::from(CTRL_INT_ENABLE | CTRL_RISC_INT_ENABLE), 2); + isp.bar_write(1, reg::MAILBOX_BASE, u32::from(mbox::NOP), 2); + isp.bar_write(1, reg::HOST_CMD, 0x5 << HOST_CMD_SHIFT, 2); + assert_eq!(isp.bar_read(1, reg::MAILBOX_BASE, 2), u32::from(mbox::COMPLETE)); + assert_ne!(isp.bar_read(1, reg::BUS_CTRL, 2), 0); + assert_eq!(board.xbridge.widget(0).unwrap().write32(0x2f4, 1), BUS_OK); + assert_eq!(isp.state().config().command(), 0); + assert_eq!(isp.state().config().bar_base(1), 0); + assert_eq!(isp.bar_read(1, reg::BUS_CTRL, 2), 0); + assert_eq!(isp.bar_read(1, reg::MAILBOX_BASE, 2), 0); + assert_eq!(isp.bar_read(1, reg::MAILBOX_BASE + 2, 2), 0x4953); +} + +/// The PROM reads the station address from the L1 and IRIX from the +/// Ethernet; both must report the configured one. +#[test] +fn the_configured_station_address_reaches_the_l1_and_the_ethernet() { + let mac = [0x08, 0x00, 0x69, 0x21, 0x43, 0x65]; + let board = crate::ip34_machine::FuelBoard::with_config(crate::ip34_machine::FuelBoardConfig { + payload: &[], clock: Arc::new(|| 0), flash_path: None, + station_mac: mac, + timekeeper: Arc::new(crate::dev::timekeeper::Timekeeper::fuel().unwrap()), + ps2_input: true, + l1: Default::default(), + graphics: crate::config::GraphicsBoard::None, + graphics_pool: (2, 32), + graphics_raster: (crate::config::RasterUnit::Cpu, "auto"), + }).unwrap(); + assert_eq!(board.l1.station_mac(), mac); + assert_eq!(board.eth.mac(), mac); +} + +/// The L1's reported revision and brick serial keep their defaults unless +/// the board is built with replacements. +#[test] +fn the_l1_identity_overrides_reach_the_controller() { + use crate::ip34_machine::{FuelBoard, FuelBoardConfig, L1Identity}; + let build = |l1| FuelBoard::with_config(FuelBoardConfig { + payload: &[], clock: Arc::new(|| 0), flash_path: None, + station_mac: [8, 0, 0x69, 0x12, 0x34, 0x56], + timekeeper: Arc::new(crate::dev::timekeeper::Timekeeper::fuel().unwrap()), + ps2_input: true, l1, + graphics: crate::config::GraphicsBoard::None, + graphics_pool: (2, 32), + graphics_raster: (crate::config::RasterUnit::Cpu, "auto"), + }).unwrap().l1.config(); + let stock = build(L1Identity::default()); + assert_eq!(stock.revision, Some([1, 48, 1])); + assert_eq!(stock.brick_serial.as_deref(), Some("EMUN00")); + let set = build(L1Identity { revision: Some([2, 3, 4]), brick_serial: Some("EMUN09".into()) }); + assert_eq!(set.revision, Some([2, 3, 4])); + assert_eq!(set.brick_serial.as_deref(), Some("EMUN09")); +} + +fn fuel_dma_board() -> crate::ip34_machine::FuelBoard { + let board = crate::ip34_machine::FuelBoard::new(&[], Arc::new(|| 0)).unwrap(); + assert_eq!(md_regs(&board).write64(8, 0x33), BUS_OK); + board.xbridge.widget(15).unwrap().write32(0x24, 15); + board +} + +fn fuel_dma_address(address: u64) -> u64 { + (10 << 60) | (1 << 55) | address +} + +/// Storage DMA buffers can begin unaligned and cross a physical page boundary. +#[test] +fn dma_preserves_byte_order_and_crosses_ragged_pages() { + use crate::pci::PciDmaStatus; + let board = fuel_dma_board(); + let dma = board.xbridge.dma(15, 0); + let memory = board.bedrock.dma_view(); + for (address, length) in [(0x1030, 8), (0x1103, 27), (0x1ffb, 19)] { + let data: Vec = (0..length).map(|i| 0x31 + i as u8).collect(); + assert_eq!(dma.dma_write(fuel_dma_address(address), &data), PciDmaStatus::Ok); + for (i, &byte) in data.iter().enumerate() { + assert_eq!(memory.read8(address + i as u64).data, byte); + } + let mut back = vec![0xff; length]; + assert_eq!(dma.dma_read(fuel_dma_address(address), &mut back), PciDmaStatus::Ok); + assert_eq!(back, data); + } + // Without byte swapping enabled, the bridge reverses each 32-bit word. + assert_eq!(dma.dma_write((10 << 60) | 0x3000, &[0, 1, 2, 3, 4, 5, 6, 7]), PciDmaStatus::Ok); + assert_eq!(memory.read64(0x3000).data, 0x0302010007060504); +} + +/// A failed DMA read must clear only refused chunks and preserve completed data. +#[test] +fn dma_refusals_clear_failed_reads_without_losing_completed_chunks() { + use crate::pci::PciDmaStatus; + let board = fuel_dma_board(); + let dma = board.xbridge.dma(15, 0); + let memory = board.bedrock.dma_view(); + // Bank zero has 256 MiB fitted. The second qword is outside its backing. + let last = 0x0fff_fff8; + assert_eq!(memory.write64(last, 0x0102030405060708), BUS_OK); + let mut bytes = [0xff; 16]; + assert_eq!(dma.dma_read(fuel_dma_address(last), &mut bytes), PciDmaStatus::MasterAbort); + assert_eq!(bytes, [1, 2, 3, 4, 5, 6, 7, 8, 0, 0, 0, 0, 0, 0, 0, 0]); + assert_eq!(dma.dma_write(fuel_dma_address(last), &[0x42; 16]), PciDmaStatus::MasterAbort); + assert_eq!(memory.read64(last).data, 0x4242424242424242); +} + +/// Inbound Hub writes must preserve access width and the widget's permissions. +#[test] +fn dma_hub_access_preserves_width_and_source_permissions() { + use crate::pci::PciDmaStatus; + let board = fuel_dma_board(); + let dma = board.xbridge.dma(15, 0); + let hub = (1 << 47) | 0x1800000; + for (interface, offset) in [(0, 0), (1, 0x200000)] { + let address = fuel_dma_address(hub + offset + 0x90); + assert_eq!(dma.dma_write(address, &0x10cu64.to_be_bytes()), PciDmaStatus::Ok); + assert_ne!(pi_regs(&board, interface as u64).read64(0x98).data & (1 << 12), 0); + for width in [1, 2, 4] { + assert_eq!(dma.dma_write(address, &[0; 8][..width]), PciDmaStatus::MasterAbort); + assert_ne!(pi_regs(&board, interface as u64).read64(0x98).data & (1 << 12), 0); + } + assert_eq!(dma.dma_write(address, &12u64.to_be_bytes()), PciDmaStatus::Ok); + assert_eq!(pi_regs(&board, interface as u64).read64(0x98).data & (1 << 12), 0); + } + let address = fuel_dma_address(hub + 0x90); + // I/O modify must retain PI region protection and the widget's permissions. + assert_eq!(pi_regs(&board, 0 as u64).write64(0x10, 0), BUS_OK); + assert_eq!(dma.dma_write(address, &0x10cu64.to_be_bytes()), PciDmaStatus::MasterAbort); + assert_eq!(pi_regs(&board, 0 as u64).read64(0x98).data & (1 << 12), 0); + assert_eq!(pi_regs(&board, 0 as u64).write64(0x10, 1), BUS_OK); + assert_eq!(board.xbridge.dma(15, 1).dma_write(address, &0x10cu64.to_be_bytes()), PciDmaStatus::Ok); + assert_ne!(pi_regs(&board, 0 as u64).read64(0x98).data & (1 << 12), 0); + let mut bytes = [0xff; 8]; + assert_eq!(board.xbridge.dma(15, 1).dma_read( + fuel_dma_address(hub + 0x40), &mut bytes, + ), PciDmaStatus::MasterAbort); + assert_eq!(bytes, [0; 8]); + assert_eq!(board.bus.read64((1 << 57) | 0x1800040).status, BUS_OK); +} + +/// Rounded MMIO pages must not expose padding or bypass cell permissions. +#[test] +fn exact_windows_reject_padding_and_revoked_permissions() { + use std::sync::atomic::{AtomicBool, Ordering}; + struct Offset; + impl BusDevice for Offset { + fn read64(&self, offset: u64) -> BusRead64 { BusRead64::ok(offset) } + fn write64(&self, _offset: u64, _value: u64) -> u32 { BUS_OK } + } + let permitted = Arc::new(AtomicBool::new(true)); + let permission = permitted.clone(); + let window = Window::new(0x1000, 0x88, Arc::new(Offset)) + .permitted(Arc::new(move || permission.load(Ordering::Acquire))); + assert_eq!(window.read64(0x1080).data, 0x80); + assert_eq!(window.read64(0x1088).status, BUS_ERR); + assert_eq!(window.read64(0x1084).status, BUS_ERR); + assert_eq!(window.read64(0x0ff8).status, BUS_ERR); + assert_eq!(window.read64(u64::MAX).status, BUS_ERR); + permitted.store(false, Ordering::Release); + assert_eq!(window.read64(0x1000).status, BUS_ERR); + assert_eq!(window.write64(0x1000, 1), BUS_ERR); + permitted.store(true, Ordering::Release); + assert_eq!(window.write64(0x1000, 1), BUS_OK); +} + +/// Board default setup must preserve the direct RAM path and its shared DMA contents. +#[test] +fn direct_cached_banks_remain_visible_after_board_defaults_are_installed() { + let board = crate::ip34_machine::FuelBoard::new(&[], Arc::new(|| 0)).unwrap(); + assert_eq!(md_regs(&board).write64(8, 0x33), BUS_OK); + let cached = board.cached_bus(); + assert!(cached.mem_ptr(0x2000).is_some()); + assert_eq!(cached.write64(0x2000, 0x123456789abcdef0), BUS_OK); + assert_eq!(board.bedrock.dma_view().read64(0x2000).data, 0x123456789abcdef0); + assert!(cached.mem_ptr(3 << 57).is_none(), "UC selectors cannot alias cached RAM"); +} + +/// A device overlay replaces its pages, including padding, while its neighbors +/// keep their original address translation. +#[test] +fn sparse_overlays_keep_neighbors_and_refuse_the_new_windows_padding() { + let mut map = SparseWindowMap::default(); + map.map(0x1000, 0x4000, Arc::new(Word(1))); + map.map(0x2040, 0x80, Arc::new(Word(2))); + assert_eq!(map.read64(0x1000).data, 1); + assert_eq!(map.read64(0x2040).data, 2); + assert_eq!(map.read64(0x20b8).data, 2); + assert_eq!(map.read64(0x20bc).status, BUS_ERR); + assert_eq!(map.read64(0x2000).status, BUS_ERR); + assert_eq!(map.read64(0x20c0).status, BUS_ERR); + assert_eq!(map.read64(0x3000).data, 1); + assert_eq!(map.read64(0x5000).status, BUS_ERR); + map.map(0x1000, 0x4000, Arc::new(Word(3))); + assert_eq!(map.ranges.len(), 1); + assert_eq!(map.read64(0x2040).data, 3); +} + +/// Sparse MMIO holes remain bus errors and a revoked cell permission does not +/// fall through to an older mapping beneath the protected window. +#[test] +fn sparse_holes_and_revoked_permissions_never_expose_older_devices() { + use std::sync::atomic::{AtomicBool, Ordering}; + let permission = Arc::new(AtomicBool::new(true)); + let allowed = permission.clone(); + let mut map = SparseWindowMap::default(); + map.map(0, 0x3000, Arc::new(Word(1))); + map.map(0x1000, 8, Arc::new(Window::new(0, 8, Arc::new(Word(2))) + .permitted(Arc::new(move || allowed.load(Ordering::Acquire))))); + assert_eq!(map.read64(0x1000).data, 2); + permission.store(false, Ordering::Release); + assert_eq!(map.read64(0x1000).status, BUS_ERR); + assert_eq!(map.read64(0x1008).status, BUS_ERR); + assert_eq!(map.read64(0x2000).data, 1); + assert_eq!(map.read64(0x3000).status, BUS_ERR); +} + +/// Code-page lookup receives plain physical addresses and reads them through +/// the cached plane. Cached stores must also update the MD view. +#[cfg(feature = "jitv2")] +#[test] +fn code_generations_use_the_stable_ram_endpoint() { + struct Generations { + ram_only: bool, + generation: AtomicU64, + writes: std::sync::Mutex>, + } + impl Generations { + fn new(ram_only: bool) -> Arc { + Arc::new(Self { ram_only, generation: AtomicU64::new(0), writes: Default::default() }) + } + } + impl BusDevice for Generations { + fn gen_ptr(&self, address: u64) -> *const AtomicU64 { + if !self.ram_only || address < 0x1000 { &self.generation } else { std::ptr::null() } + } + fn write64(&self, address: u64, value: u64) -> u32 { + self.writes.lock().unwrap().push((address, value)); + BUS_OK + } + fn write64_masked(&self, address: u64, value: u64, mask: u64) -> u32 { + self.write64(address, value & mask) + } + fn write_block(&self, address: u64, values: &[u64]) -> u32 { + for (index, value) in values.iter().enumerate() { + self.write64(address + index as u64 * 8, *value); + } + BUS_OK + } + } + let physical = Physical::new(Vec::new()); + let bus = Ip35Bus::new(physical.clone()); + let raw_ram = Generations::new(false); + let stable_ram = Generations::new(true); + let rom = Generations::new(false); + map_device(&physical, 0, 0x1000, raw_ram.clone(), "raw RAM generation"); + map_device(&physical, 0x1fc00000, 0x1000, rom.clone(), "ROM generation"); + let cached = bus.cached_view(stable_ram.clone(), crate::dev::bedrock::RamExtent::new(&[256, 256])); + assert_eq!(cached.gen_ptr(0x40), &stable_ram.generation as *const AtomicU64); + assert_eq!(cached.gen_ptr(0x1fc00000), &rom.generation as *const AtomicU64); + assert_eq!(cached.write64(0x40, 0x1234), BUS_OK); + assert_eq!(cached.write_block(0x48, &[0x5678]), BUS_OK); + assert_eq!(cached.write64_masked(0x50, 0xaabb, 0xff), BUS_OK); + assert_eq!(*stable_ram.writes.lock().unwrap(), [(0x40, 0x1234), (0x48, 0x5678), (0x50, 0xbb)]); + assert_eq!(cached.write64(0x1fc00000, 0x4321), BUS_OK); + assert_eq!(*rom.writes.lock().unwrap(), [(0, 0x4321)], "the ROM window translates to a device-local offset"); + assert!(raw_ram.writes.lock().unwrap().is_empty()); +} + +/// The JIT's page lookup and inline compile read code through the CPU's cached +/// bus, which a board whose uncached connection decodes plain addresses as +/// something else sets to the view the cache fills from. The compile pool is +/// started on that same object. +#[test] +fn the_cpu_reads_code_through_the_cached_view_not_the_uncached_connection() { + // An executor is large; a test thread's stack is not. + std::thread::Builder::new().stack_size(64 << 20).spawn(cpu_cached_bus).unwrap().join().unwrap(); +} + +fn cpu_cached_bus() { + let board = crate::ip34_machine::FuelBoard::new(&[], Arc::new(|| 0)).unwrap(); + let cpu = board.build_cpu(crate::ip34_machine::CPUS[0]); + #[cfg(feature = "jitv2")] + let cached = board.code_bus(); + #[cfg(not(feature = "jitv2"))] + let cached = board.cached_bus(); + assert!(Arc::ptr_eq(cpu.cached_bus(), &cached)); + let sysad: Arc = board.bus.clone(); + assert!(!Arc::ptr_eq(cpu.cached_bus(), &sysad), "sysad is the uncached connection"); +} + +/// Unmodeled Hub heads retain the PROM compatibility readback without +/// absorbing the adjacent, independently implemented L1 UART window. +#[test] +fn compatibility_hub_heads_do_not_hide_the_l1_uart_window() { + let board = crate::ip34_machine::FuelBoard::new(&[], Arc::new(|| 0)).unwrap(); + assert_ne!(board.bedrock.cell_permits() as u64, 0); + for base in [0x1000_0000, 0x3000_0000] { + for offset in [0, 8, 0x40, 0x78] { + let address = base + offset; + assert_eq!(board.bus.write8(address, 0xa5), BUS_OK); + assert_eq!(board.bus.write16(address, 0xa5a5), BUS_OK); + assert_eq!(board.bus.write32(address, 0xa5a5a5a5), BUS_OK); + assert_eq!(board.bus.write64(address, 0x2000), BUS_OK); + let value = board.bus.read8(address); + assert_eq!(value.status, BUS_OK); + assert_eq!(value.data, 0); + let value = board.bus.read16(address); + assert_eq!(value.status, BUS_OK); + assert_eq!(value.data, 0); + let value = board.bus.read32(address); + assert_eq!(value.status, BUS_OK); + assert_eq!(value.data, 0); + let value = board.bus.read64(address); + assert_eq!(value.status, BUS_OK); + assert_eq!(value.data, 0); + } + assert_eq!(board.bus.read64(base + 0x80).status, BUS_OK); + } +} + +/// A busy host port or unusable transcript path must not prevent a serial +/// machine from starting, including when any of its three ports serves CI. +#[test] +fn unavailable_serial_ports_and_logs_do_not_abort_the_machine() { + use crate::config::{FuelConsole, MachineConfig}; + let listeners: Vec<_> = (0..3).map(|_| std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap()).collect(); + let ports: Vec<_> = listeners.iter().map(|socket| socket.local_addr().unwrap().port()).collect(); + let board = crate::ip34_machine::FuelBoard::new(&[], Arc::new(|| 0)).unwrap(); + let mut config = MachineConfig::default(); + config.fuel.l1_port = Some(ports[0]); + config.serial_port_a = Some(ports[1]); + config.serial_port_b = Some(ports[2]); + config.ci = true; + let missing = std::env::temp_dir().join(format!("iris-fuel-missing-log-{}", std::process::id())).join("absent").join("serial.txt"); + assert!(!missing.parent().unwrap().exists(), "the log parent must be absent"); + config.serial_log = Some(missing.to_string_lossy().into_owned()); + config.fuel.l1_log = config.serial_log.clone(); + for (console, selection) in [FuelConsole::L1, FuelConsole::Ioc3A, FuelConsole::Ioc3B].into_iter().enumerate() { + config.fuel.console = selection; + crate::ip34_machine::FuelMachine::connect_serial_for_test(&config, &board, console).unwrap(); + } + config.ci = false; + crate::ip34_machine::FuelMachine::connect_serial_for_test(&config, &board, 0).unwrap(); +} + +fn fuel_with_graphics(graphics: crate::config::GraphicsBoard) -> crate::ip34_machine::FuelBoard { + crate::ip34_machine::FuelBoard::with_config(crate::ip34_machine::FuelBoardConfig { + payload: &[], clock: Arc::new(|| 0), flash_path: None, + station_mac: [0x08, 0x00, 0x69, 0x21, 0x43, 0x65], + timekeeper: Arc::new(crate::dev::timekeeper::Timekeeper::fuel().unwrap()), + ps2_input: true, + l1: Default::default(), + graphics, + graphics_pool: (2, 32), + graphics_raster: (crate::config::RasterUnit::Cpu, "auto"), + }).unwrap() +} + +/// The VPro's link status in the crossbar, in its slot's data half. +const VPRO_LINK_STATUS: u64 = 0x100 + 5 * 0x40 + 0x10 + 4; +/// A processor access to widget 13 through Bedrock's small window. +const VPRO_PIO: u64 = 13 << 24; + +// The VPro cases follow tests/qtest/sn1-fuel-vpro-test.c in +// jamesbraid/qemu-sgi-origami: widget-identity, profile-identity-and-routing +// and headless-fuel-has-no-board. + +// VPro host-interface registers the tests below touch, as byte offsets in +// the board's widget space. The board's own constants are private to it. +const ODSY_IDENTITY: u64 = 0x00; +const ODSY_CONTROL: u64 = 0x20; +const ODSY_CONTROL_WIDGET_MASK: u32 = 0xf; +const ODSY_INT_DEST_LOWER: u64 = 0x38; +const ODSY_INTR_CLEAR: u64 = 0x101c; +const ODSY_INTR_ENABLE_SET: u64 = 0x1034; +const ODSY_INTR_FIFO_LOW_WATER: u32 = 0x200; +const ODSY_ROUTE_DEST_UPPER: u64 = 0x1044; +const ODSY_ROUTE_DEST_LOWER: u64 = 0x104c; +const ODSY_ROUTE_VECTORS: u64 = 0x1054; +const ODSY_ROUTE_DEST_WIDGET_SHIFT: u32 = 25; + +/// A Fuel IRIX inventory lists its VPro under xtalk/13: the processor must +/// reach the board there, learn its widget number from it, and find the +/// link alive in the crossbar. +#[test] +fn the_vpro_answers_at_widget_13_with_its_link_alive() { + for (graphics, revision) in [(crate::config::GraphicsBoard::V12, 0xa), (crate::config::GraphicsBoard::V10, 0xa)] { + let board = fuel_with_graphics(graphics); + let identity = board.bedrock.pio().read32(VPRO_PIO | (ODSY_IDENTITY + 4)); + assert_eq!(identity.status, BUS_OK); + assert_eq!(identity.data as u32, (revision << 28) | 0x0c01_3047, "{graphics:?}"); + let control = board.bedrock.pio().read32(VPRO_PIO | (ODSY_CONTROL + 4)).data as u32; + assert_eq!(control & ODSY_CONTROL_WIDGET_MASK, 13); + assert_ne!(board.xbridge.crossbar().read32(VPRO_LINK_STATUS).data & (1 << 31), 0); + } +} + +/// The monitor's `vpro` commands resolve on a Fuel that has a VPro, and are +/// unknown on one that has none. +#[test] +fn the_monitor_resolves_vpro_commands_on_a_fuel_with_a_vpro() { + let resolves = |board: &crate::ip34_machine::FuelBoard, command: &str| { + board.monitor_devices().iter().any(|d| d.register_commands().iter().any(|(c, _)| c == command)) + }; + let with = fuel_with_graphics(crate::config::GraphicsBoard::V12); + assert!(resolves(&with, "vpro"), "a Fuel with a V12 answers `vpro`"); + assert!(resolves(&with, "ps2"), "the other board commands stay"); + let without = fuel_with_graphics(crate::config::GraphicsBoard::None); + assert!(!resolves(&without, "vpro"), "a Fuel with no VPro has no `vpro`"); + assert!(resolves(&without, "ps2")); +} + +/// The board's interrupt packets cross the XBridge to Bedrock as the PCI +/// heads' do: a raised source sets its vector's pending bit in the +/// processor interface the destination names, and a cleared one clears it. +#[test] +fn vpro_interrupt_packets_set_and_clear_a_processor_interface_pending_bit() { + let board = fuel_with_graphics(crate::config::GraphicsBoard::V12); + // Register space (bit 47), the node's hub window and PI0's pending + // modify register, at the Bedrock's crossbar port 0xa. + let packet = (1u64 << 47) | 0x180_0000 | 0x90; + let vector = 5; + let write = |offset: u64, value: u64| { + assert_eq!(board.bedrock.pio().write32(VPRO_PIO | offset, value as u32), BUS_OK, "{offset:#x}"); + }; + write(ODSY_ROUTE_DEST_UPPER, (0xa << ODSY_ROUTE_DEST_WIDGET_SHIFT) | (packet >> 32)); + write(ODSY_ROUTE_DEST_LOWER, packet & 0xffff_ffff); + write(ODSY_ROUTE_VECTORS, vector); + write(ODSY_INTR_ENABLE_SET, ODSY_INTR_FIFO_LOW_WATER as u64); + let pending = || pi_regs(&board, 0 as u64).read64(0x98).data & (1 << vector); + assert_ne!(pending(), 0); + write(ODSY_INTR_CLEAR, ODSY_INTR_FIFO_LOW_WATER as u64); + assert_eq!(pending(), 0); +} + +/// A link reset of the graphics port resets the board behind it, as the +/// PCI port's does for the functions behind its head. +#[test] +fn a_graphics_link_reset_resets_the_vpro() { + let board = fuel_with_graphics(crate::config::GraphicsBoard::V12); + let destination = VPRO_PIO | (ODSY_INT_DEST_LOWER + 4); + assert_eq!(board.bedrock.pio().write32(destination, 0x55 as u32), BUS_OK); + assert_eq!(board.bedrock.pio().read32(destination).data, 0x55); + let link_reset = 0x100 + 5 * 0x40 + 0x30 + 4; + assert_eq!(board.xbridge.crossbar().write32(link_reset, 1), BUS_OK); + assert_eq!(board.bedrock.pio().read32(destination).data, 0); +} + +/// Without graphics nothing answers at widget 13 and its link is down, as +/// on a headless Fuel. +#[test] +fn a_fuel_without_graphics_has_no_widget_13() { + let board = fuel_with_graphics(crate::config::GraphicsBoard::None); + assert!(board.vpro.is_none()); + assert_ne!(board.bedrock.pio().read32(VPRO_PIO | 4).status, BUS_OK); + assert_eq!(board.xbridge.crossbar().read32(VPRO_LINK_STATUS).data & (1 << 31), 0); +} + +/// IRIX's VPro driver writes each processor interface's graphics credit +/// counter at attach and tears the board down if it reads back zero; the +/// registers must keep what the guest writes, per interface. +#[test] +fn processor_interface_graphics_registers_keep_their_writes() { + let board = fuel_with_graphics(crate::config::GraphicsBoard::V12); + // Slot A's credit counter and bias; PI1 keeps its own block. + assert_eq!(pi_regs(&board, 0 as u64).write64(0x308, 100), BUS_OK); + assert_eq!(pi_regs(&board, 0 as u64).write64(0x310, 0x1000 | 68), BUS_OK); + assert_eq!(pi_regs(&board, 0 as u64).read64(0x308).data, 100); + assert_eq!(pi_regs(&board, 0 as u64).read64(0x310).data, 68); + assert_eq!(pi_regs(&board, 1 as u64).read64(0x308).data, 0); + board.bedrock.power_on(); + assert_eq!(pi_regs(&board, 0 as u64).read64(0x308).data, 0); +} + +/// The reset PROM fetches uncached at the same plain address that cached RAM +/// answers once memory covers its window. The board presents that fetch at the +/// PROM's own address (`uncached_fetch_alias`), so the JIT keys the page by +/// address and compiles the PROM's bytes; it used to read the cached view and +/// compile the RAM's bytes in place of the PROM's. +#[cfg(feature = "jitv2")] +#[test] +fn uncached_fetch_compiles_the_prom_not_the_ram_over_it() { + // The executor and the code generator are large; the default test stack is not. + std::thread::Builder::new() + .stack_size(64 << 20) + .spawn(uncached_fetch_compiles_the_prom_body) + .unwrap() + .join() + .expect("thread panicked"); +} + +#[cfg(feature = "jitv2")] +fn uncached_fetch_compiles_the_prom_body() { + use crate::cpu::jitv2::jitv2::CompileRequest; + use crate::cpu::mips_isa::{JIT_REGION_BOUNDARY_SENTINEL, OP_ADDIU}; + + /// Every word the same; a region-boundary word compiles to nothing. + struct Words { + word: u32, + generation: AtomicU64, + } + impl Words { + fn new(word: u32) -> Arc { + Arc::new(Self { word, generation: AtomicU64::new(0) }) + } + } + impl BusDevice for Words { + fn read32(&self, _: u64) -> BusRead32 { + BusRead32::ok(self.word) + } + fn read64(&self, _: u64) -> BusRead64 { + BusRead64::ok((self.word as u64) << 32 | self.word as u64) + } + fn gen_ptr(&self, _: u64) -> *const AtomicU64 { + &self.generation + } + } + + let reset_vector = 0x1fc0_0000u64; + let addiu_r1 = (OP_ADDIU << 26) | (1 << 16) | 1; + let rom = Words::new(addiu_r1); + let ram = Words::new(JIT_REGION_BOUNDARY_SENTINEL); + let physical = Physical::new(Vec::new()); + let mut bus = Ip35Bus::new(physical.clone()); + map_device(&physical, reset_vector, 0x1000, ram.clone(), "RAM over the reset window"); + bus.map(0, reset_vector, 0x1000, rom.clone()); + bus.map(0, RESET_WINDOW_FETCH, 0x1000, rom.clone()); + let bus = Arc::new(bus); + let code_bus = bus.code_view(bus.cached_view(ram.clone(), RamExtent::new(&[1024]))); + // The two views really disagree here. + assert_eq!(bus.read32(reset_vector).data, addiu_r1); + assert_eq!(bus.cached_view(ram.clone(), RamExtent::new(&[1024])).read32(reset_vector).data, JIT_REGION_BOUNDARY_SENTINEL); + + let mut cpu = alias_cpu(bus.clone()); + alias_the_reset_window(&mut cpu); + cpu.set_cached_bus(code_bus.clone()); + // The reset vector, KSEG1: uncached, plain physical address 0x1fc00000. + cpu.core.pc = 0xffff_ffff_bfc0_0000; + let _ = cpu.step_int(); + let page = cpu.jitv2_current_page(); + assert!(!page.is_null(), "the uncached fetch must land on a code page"); + let page = unsafe { &mut *page }; + assert_eq!(page.pfn as u64, RESET_WINDOW_FETCH / 4096, "an uncached fetch of the reset window keys the PROM's own frame"); + + page.mark_requested(0); + let req = CompileRequest { page: page as *mut _, compiled_for_fr1: true }; + let mut analyzer = crate::cpu::jitv2::analyzer::Analyzer::new(); + let mut codegen = crate::cpu::jitv2::codegen::Codegen::new(); + #[cfg(feature = "developer")] + crate::cpu::jitv2::comp::handle_request(&req, &cpu.cached_bus(), &mut analyzer, &mut codegen, &crate::cpu::jitv2::JitStats::default()); + #[cfg(not(feature = "developer"))] + crate::cpu::jitv2::comp::handle_request(&req, &cpu.cached_bus(), &mut analyzer, &mut codegen); + assert!(page.is_runnable(0), "the page must compile the PROM's ADDIU, not the RAM's boundary word"); + + // The cached view of the same frame is another page, with RAM's counter and bytes. + let jitv2 = cpu.jitv2.clone(); + let mut jit = jitv2.lock(); + let slot = jit.page_for(0x1fc00, 0x1fc0_0000, bus.as_ref(), false).unwrap(); + let cached = unsafe { &mut *jit.page_ptr(slot) }; + assert_ne!(cached as *mut _, page as *mut _); + cached.mark_requested(0); + let req = CompileRequest { page: cached as *mut _, compiled_for_fr1: true }; + #[cfg(feature = "developer")] + crate::cpu::jitv2::comp::handle_request(&req, &code_bus, &mut analyzer, &mut codegen, &crate::cpu::jitv2::JitStats::default()); + #[cfg(not(feature = "developer"))] + crate::cpu::jitv2::comp::handle_request(&req, &code_bus, &mut analyzer, &mut codegen); + assert!(!cached.is_runnable(0), "the cached page compiles RAM's boundary word to nothing"); + + // A write through one view invalidates only the pages that view's bytes feed. + ram.generation.fetch_add(1, std::sync::atomic::Ordering::Release); + assert!(page.is_runnable(0), "a cached-view write does not change the PROM's bytes"); + rom.generation.fetch_add(1, std::sync::atomic::Ordering::Release); + assert!(!page.is_runnable(0), "a write to the PROM's bytes invalidates its code"); +} + +/// The R16000 is the part that takes an uncached fetch alias. +#[cfg(feature = "jitv2")] +type AliasCpu = crate::cpu::mips_exec::MipsExecutor; + +#[cfg(feature = "jitv2")] +fn alias_cpu(bus: Arc) -> Box { + use crate::cpu::mips_exec::{MipsCpuConfig, MipsExecutor}; + use crate::cpu::mips_tlb::MipsTlb; + let cfg = MipsCpuConfig::for_model::(Arc::new(memory_map(&RamExtent::new(&[1024])))) + .with_prid(crate::cpu::parts::R16000_700_EARLY.prid()); + let mut cpu = Box::new(MipsExecutor::new(bus as Arc, MipsTlb::new(cfg.tlb_entries), &cfg)); + cpu.phys_mask = CPU_PHYS_MASK; + // KSEG0 cached, as the PROM's boot mode leaves it. + cpu.core.cp0_config = (cpu.core.cp0_config & !7) | 3; + cpu +} + +#[cfg(feature = "jitv2")] +fn alias_the_reset_window(cpu: &mut AliasCpu) { + cpu.set_uncached_fetch_alias(RESET_WINDOW, RESET_WINDOW_SIZE, RESET_WINDOW_FETCH); +} + +#[cfg(feature = "jitv2")] +mod code_views { + use super::*; + use crate::cpu::jitv2::jitv2::CompileRequest; + use crate::cpu::mips_isa::{JIT_REGION_BOUNDARY_SENTINEL, OP_ADDIU}; + + struct Words(u32, AtomicU64); + impl BusDevice for Words { + fn read32(&self, _: u64) -> BusRead32 { + BusRead32::ok(self.0) + } + fn read64(&self, _: u64) -> BusRead64 { + BusRead64::ok((self.0 as u64) << 32 | self.0 as u64) + } + fn gen_ptr(&self, _: u64) -> *const AtomicU64 { + &self.1 + } + } + + const RESET: u64 = 0x1fc0_0000; + const ADDIU_R1: u32 = (OP_ADDIU << 26) | (1 << 16) | 1; + + /// A bus whose uncached attribute-0 view of the reset frame holds an ADDIU + /// and whose cached view holds a region boundary, as RAM over the PROM does. + fn bus_and_cpu(aliased: bool) -> (Arc, Box) { + let rom = Arc::new(Words(ADDIU_R1, AtomicU64::new(0))); + let ram = Arc::new(Words(JIT_REGION_BOUNDARY_SENTINEL, AtomicU64::new(0))); + let physical = Physical::new(Vec::new()); + let mut bus = Ip35Bus::new(physical.clone()); + map_device(&physical, RESET, 0x1000, ram.clone(), "RAM over the reset window"); + bus.map(0, RESET, 0x1000, rom.clone()); + // Code that is fetchable in the other views and past the end of memory, so only + // the JIT's own range check keeps it out of a code page. + for attribute in 1..4 { + bus.map(attribute, RESET, 0x1000, rom.clone()); + } + bus.map(0, RESET_WINDOW_FETCH, 0x1000, rom.clone()); + bus.map(0, 0x3_0000_0000, 0x1000, rom); + let bus = Arc::new(bus); + let mut cpu = alias_cpu(bus.clone()); + if aliased { + let cached = bus.cached_view(ram, RamExtent::new(&[1024])); + alias_the_reset_window(&mut cpu); + cpu.set_cached_bus(bus.code_view(cached)); + } + (bus, cpu) + } + + /// The executor and the code generator are large; the default test stack is not. + fn big_stack(body: fn()) { + std::thread::Builder::new().stack_size(64 << 20).spawn(body).unwrap().join().expect("thread panicked"); + } + + fn wait_runnable(page: &crate::cpu::jitv2::PhysicalCodePage) -> bool { + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(20); + while std::time::Instant::now() < deadline { + if page.is_runnable(0) { return true; } + std::thread::sleep(std::time::Duration::from_millis(5)); + } + false + } + + /// After a page-pool flush the restarted compile pool must still read the + /// PROM's page through the compile bus. Restarted on the CPU's own bus it + /// rejects the view address and the page never compiles. + #[test] + fn a_pool_restart_after_a_flush_keeps_the_compile_bus() { + big_stack(|| { + let (_bus, mut cpu) = bus_and_cpu(true); + { + let jit = cpu.jitv2.clone(); + let mut g = jit.lock(); + let stats = g.stats.clone(); + g.compile_queue.set_thread_count(1); + g.compile_queue.start(cpu.cached_bus().clone(), stats); + } + cpu.jitv2_flush_from_here(false); + cpu.core.pc = 0xffff_ffff_bfc0_0000; + let _ = cpu.step_int(); + let page = unsafe { &mut *cpu.jitv2_current_page() }; + assert_eq!(page.pfn as u64, RESET_WINDOW_FETCH / 4096); + page.mark_requested(0); + let jit = cpu.jitv2.clone(); + let g = jit.lock(); + let stats = g.stats.clone(); + assert!(g.compile_queue.send(CompileRequest { page: page as *mut _, compiled_for_fr1: true }, &stats)); + drop(g); + assert!(wait_runnable(page), "the restarted pool must compile the PROM's page"); + jit.lock().compile_queue.stop(); + }); + } + + /// With no alias an uncached fetch keys the frame it names, as on every + /// other machine; and with it a cached fetch of the same window still does. + #[test] + fn a_cached_fetch_and_an_unaliased_executor_key_the_frame_they_name() { + big_stack(|| { + let (_bus, mut cpu) = bus_and_cpu(false); + cpu.core.pc = 0xffff_ffff_bfc0_0000; + let _ = cpu.step_int(); + let page = unsafe { &*cpu.jitv2_current_page() }; + assert_eq!(page.pfn, 0x1fc00); + + let (_bus, mut cpu) = bus_and_cpu(true); + // KSEG0: cached, the same physical address. + cpu.core.pc = 0xffff_ffff_9fc0_0000; + let _ = cpu.step_int(); + let page = unsafe { &*cpu.jitv2_current_page() }; + assert_eq!(page.pfn, 0x1fc00, "a cached fetch is not presented at the PROM's frame"); + }); + } + + /// Uncached attributes 1 to 3 and anything past the end of memory stay in the interpreter. + #[test] + fn attributes_one_to_three_and_high_addresses_get_no_code_page() { + big_stack(|| { + for pc in [ + 0x9000_0000_0000_0000u64 | RESET, // attribute 0: a page + 0x9200_0000_0000_0000u64 | RESET, // attribute 1 + 0x9400_0000_0000_0000u64 | RESET, // attribute 2 + 0x9600_0000_0000_0000u64 | RESET, // attribute 3 + 0x9000_0003_0000_0000u64, // past the end of memory, attribute 0 + ] { + // A fresh executor each time: a tracked page would survive a + // fetch that wrongly kept the same frame. + let (_bus, mut cpu) = bus_and_cpu(true); + // KX, so XKPHYS addresses translate. + let status = cpu.core.cp0_status | 0x80; + cpu.set_cp0_status(status); + cpu.core.pc = pc; + let _ = cpu.step_int(); + let page = cpu.jitv2_current_page(); + if pc == 0x9000_0000_0000_0000u64 | RESET { + assert!(!page.is_null(), "attribute 0 below 4 GiB is compiled"); + } else { + assert!(page.is_null(), "{pc:#x} must not get a code page"); + } + } + }); + } +} diff --git a/src/ip53_machine.rs b/src/ip53_machine.rs new file mode 100644 index 000000000..580b943c3 --- /dev/null +++ b/src/ip53_machine.rs @@ -0,0 +1,1281 @@ +//! The Tezro (IP53 board, IP35 software platform): R16000s on one Bedrock +//! node, a PIC host bridge and the IO9 board's IOC4, ISP12160 and BCM5701, +//! with Iris's CPU thread, monitor and serial console. +//! +//! Selected with `[machine] profile = "tezro_ip53"`. Shared keys select the PROM, +//! NVRAM, memory and SCSI devices; `[tezro]` selects the processor +//! population, the serial console and writable flash. The node itself is +//! the one the Fuel uses ([`Node`]); this board adds its widgets and its +//! inputs. With `[graphics] board = "v12"` a VPro V12 sits at widget 12 +//! behind the PXBOW; otherwise there is no graphics. +//! +//! The board follows the QEMU Tezro compositions: +//! +//! - The one-processor board has one PIC at XIO widget 15; the two- and +//! four-processor board adds a second at widget 11, with nothing on its +//! buses, and a V12 slot at widget 12 that is left empty. The PIC at +//! widget 15 is the IO9's: its PXBOW answers as widget 0 and forwards +//! the processor's accesses to widget 15. On the larger board it carries +//! widget 11 too, and shows that link present and alive (unlike QEMU's +//! composition, below). Its head 0 carries the IO9 PCI bus: the IOC4 at +//! device 1 (input 0), the ISP12160 at device 3 (input 2) and the +//! BCM5701 at device 4 (input 3), with REQ64# wired on the BCM5701's +//! slot. The PIC's interrupts and DMA go to Bedrock IO, the host widget +//! 10. +//! - The IOC4 carries the DS1744-compatible clock and NVRAM in its BAR and +//! the PS/2 keyboard and mouse; its four ports are serial streams beside +//! the L1's, and the L1 is the console by default. +//! - Bedrock board word 0x2001, the Tezro firmware record (the chip's clock, +//! an authored configuration block), and an L1 that reports +//! brick type 0x39, revision 1.48.1, the first port-speed integer 1, the +//! router `brick` reply and, in the node board's record, the CPU +//! configuration the flash block holds. +//! +//! # What is approximated here +//! +//! - Every identity above (board word, L1 type and revision, PIC revision 3 +//! profile, PCI devices, revisions and interrupt routes, REQ64# on the +//! BCM5701's slot) is a software input the QEMU profiles choose, not a +//! value measured on a Tezro. +//! - The one-processor board has one PIC. The tower guide gives that board +//! two PCI-X buses and one XIO connector; that it is one PIC at widget 15 +//! is inferred from them. +//! - Every fitted processor runs from reset, each on its own thread, as in +//! QEMU's IP35 node; CPU enable is stored and starts or stops nothing. The +//! PROM's reset barrier is ordered by Bedrock's LB; the first of a pair +//! to post its liveness is held until its partner posts, and a presence +//! publisher until the other interface publishes ([`PresenceHold`]): +//! approximations with no hardware counterpart. +//! - The second PIC is a port of the IO9 PIC's PXBOW, beside the host's +//! and the IO9 PIC's links: the PROM and IRIX learn which widgets exist +//! only from widget 0's link status and auxiliary status, so one +//! crossbar per node carries both PICs. That is a modelling consequence +//! of the discovery path, not a known route: whether the second PIC has +//! a crossbar of its own in front of it, and its part (0xd102 or 0xd112) +//! and link reset values beyond alive and present, are unknown. QEMU's +//! composition still gives the second PIC its own PXBOW, whose widget 0 +//! nothing maps, and leaves link 11 absent on the IO9 PIC's. The second +//! PIC's interrupts and DMA reach Bedrock IO directly, as the IO9 PIC's +//! do, rather than through the crossbar. +//! - PIC DMA aimed at a widget other than Bedrock IO is refused and logged, +//! as the Fuel's XBridge path refuses it. +//! - No EEPROM image is fitted for the IO9, the node board or the BCM5701: +//! no published source gives their contents. The L1's component 0 record +//! stands in for the node board's: it names the IP53 board Tezro +//! inventories print for the processor count ([`node_board`]), and its +//! CPU configuration is the flash record's, in the encoding QEMU gives (a +//! software input the stock PROM accepts, not a read record). No I/O +//! board record is given, as in QEMU. A BCM5701 EEPROM transfer does not +//! complete. +//! +//! Board composition ported from hw/mips/sgi/sn-machine.c and +//! hw/mips/sgi/sn1-node.c in jamesbraid/qemu-sgi-origami. + +use crate::config::{scsi_label, GraphicsBoard, MachineConfig, TezroConsole}; +use crate::cpu::mips_cache_v2::{CpuModel as CacheCpuModel, MipsCache}; +use crate::cpu::parts::{self, Part}; +use crate::cpu::mips_exec::{CpuDevice, MipsCpu, MipsCpuConfig, MipsExecutor}; +use crate::cpu::cpu_monitor::CpuMonitor; +use crate::cpu::mips_tlb::MipsTlb; +use crate::cpu::smp::World; +use crate::dev::bcm5701::{Bcm5701, Bcm5701Config}; +use crate::dev::bedrock::{Bedrock, Population, SLOTS}; +use crate::dev::bedrock_flash::{FirmwareRecord, Flash}; +use crate::dev::ds1744::{Ds1744, YearBase}; +use crate::dev::heart::{host_clock, Clock}; +use crate::dev::ioc4::{Ioc4, Ioc4Config}; +use crate::dev::isp12160::Isp12160; +use crate::dev::l1::{BoardIdentity, CpuContext, L1Config}; +use crate::dev::odyssey::{Odyssey, PacketSender}; +use crate::dev::pic::{Function, Pic, PicPeer, Refused}; +use crate::dev::pxbow::Pxbow; +use crate::dev::prom_image; +use crate::dev::ps2::Ps2Controller; +use crate::dev::serial_host::{null_serial_backend, open_log, tcp_serial_backend, CiSerialBackend, LogMode, SerialBackend, TeeBackend}; +use crate::dev::l1::L1; +use crate::devlog::{init_devlog, LogModule}; +use crate::gdb_stub::CpuDebug; +use crate::gfx_display::GfxDisplay; +use crate::hptimer::TimerManager; +use crate::ip35::{Ip35Bus, CPU_PHYS_MASK}; +use crate::ip35::node::{connect_cpus, Node, NodeConfig}; +use crate::ip35::presence::PresenceHold; +use crate::machine_state::GuestState; +use crate::monitor::{Monitor, DEFAULT_PORT as DEFAULT_MONITOR_PORT}; +use crate::net::{GatewayConfig, Link, NatControl}; +use crate::perf_monitor::PerfMonitor; +use crate::scsi::{attach_from_config, ScsiBus}; +use crate::traits::{BusDevice, Device, MachineEvent, Resettable, BUS_OK}; +use crate::workstation::{register_system_controller, DiscChanger, MachineRef, MachineSlot, Workstation}; +use parking_lot::Mutex; +use std::net::{Ipv4Addr, SocketAddr}; +use std::path::{Path, PathBuf}; +use std::sync::mpsc::{Receiver, Sender}; +use std::sync::{atomic::{AtomicBool, AtomicU64, Ordering}, Arc, Weak}; + +/// The Config bits 24:0 a chip's reset word defines (K0, DN, CT, PE, PM, EC, +/// SB, SK, BE, SS, SC and the bits above them). The primary-cache sizes above +/// them are the family's, fixed by the cache type. +const CONFIG0_MODE_MASK: u32 = 0x01ff_ffff; + +/// The Bedrock board word the QEMU Tezro profiles select. +const BOARD_ID: u64 = 0x2001; +/// The IO9's PIC, the second PIC of the two- and four-processor board, and +/// the widget Bedrock IO answers as. Widget 12 on that board is the V12's +/// XIO slot, which is left empty. +const IO9_WIDGET: u8 = 15; +const SECOND_PIC_WIDGET: u8 = 11; +const HOST_WIDGET: u8 = 10; +/// IO9's PCI functions: configuration device on head 0. +const IOC4_DEVICE: u8 = 1; +const SCSI_DEVICE: u8 = 3; +const ETH_DEVICE: u8 = 4; + +/// A DIMM size the Tezro's IP53 board takes, which is also the size of each +/// locational bank it provides: each DIMM pair provides two banks of the +/// DIMM's size (Origin 350 guide 007-4566-001, Figure 3-4, p. 118). +/// +/// 512 MB and 1 GB DIMMs only: established for the IP53 node board (Origin +/// 350 guide 007-4566-001, Table 3-3, p. 118); inferred for the Tezro, whose +/// guide (007-4564-001 pp. 27-28) names no size but whose inventories show +/// the same IP53 boards and only 512 and 1024 MB banks. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum BankSize { + Mib512, + Mib1024, +} +impl BankSize { + pub const fn mib(self) -> u32 { + match self { + Self::Mib512 => 512, + Self::Mib1024 => 1024, + } + } + fn from_mib(mib: u32) -> Result { + match mib { + 512 => Ok(Self::Mib512), + 1024 => Ok(Self::Mib1024), + _ => Err(format!("a Tezro bank is 512 or 1024 MiB, not {mib} MiB")), + } + } +} + +/// The banks a Tezro fits, locational bank 0 upward: up to eight banks of 512 +/// or 1024 MiB with none left out between them. What the board needs in total +/// depends on its processors; see [`Self::check_for_cpus`]. +/// +/// The minimums come from the Tezro manuals as the QEMU assembly layer +/// records them. The default, the 4 GiB layout of 1024, 1024 and four +/// 512 MiB banks, follows QEMU and matches a captured Tezro. +/// +/// Approximated: the board takes DIMMs in pairs of one size (Origin 350 +/// guide p. 116; Tezro guide p. 28), so banks 2k and 2k+1 are present +/// together and equal. The model also accepts unpaired and unequal bank +/// pairs, which the board cannot hold. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct MemoryPopulation(Vec); +impl Default for MemoryPopulation { + fn default() -> Self { + Self::four_gib() + } +} +impl MemoryPopulation { + pub fn new(banks: &[BankSize]) -> Result { + if banks.is_empty() || banks.len() > SLOTS { + return Err(format!("memory: {} banks; a Tezro fits 1 to {SLOTS}", banks.len())); + } + Ok(Self(banks.to_vec())) + } + /// From sizes in MiB, rejecting any a Tezro does not take. + pub fn from_mib(banks: &[u32]) -> Result { + let sizes = banks.iter().map(|&mib| BankSize::from_mib(mib)).collect::, _>>()?; + Self::new(&sizes) + } + /// 1 GiB as two banks of 512 MiB. The manual's minimum of 1 GB for two or + /// four processors permits it, but no real Tezro has been observed with + /// less than 4 GiB, and nothing here claims the PROM or IRIX boot at it. + pub fn one_gib() -> Self { + Self(vec![BankSize::Mib512; 2]) + } + /// 4 GiB: 1024, 1024 and four 512 MiB banks, the default. Banks 4 and 5 + /// sit at 4 to 6 GiB. + pub fn four_gib() -> Self { + let mut banks = vec![BankSize::Mib1024; 2]; + banks.extend([BankSize::Mib512; 4]); + Self(banks) + } + pub fn banks(&self) -> &[BankSize] { + &self.0 + } + /// The sizes in MiB, as [`BedrockConfig::banks_mib`](crate::dev::bedrock::BedrockConfig) takes them. + pub fn banks_mib(&self) -> Vec { + self.0.iter().map(|bank| bank.mib()).collect() + } + pub fn total_mib(&self) -> u32 { + self.0.iter().map(|bank| bank.mib()).sum() + } + /// One processor needs at least 512 MB; two or four need at least 1 GB. + pub fn check_for_cpus(&self, cpus: usize) -> Result<(), String> { + let needed = match cpus { + 1 => 512, + 2..=4 => 1024, + _ => return Err(format!("{cpus} processors; a Tezro fits 1 to 4")), + }; + if self.total_mib() < needed { + return Err(format!( + "{} MiB of memory; {cpus} processor{} need at least {needed} MiB", + self.total_mib(), if cpus == 1 { "" } else { "s" })); + } + Ok(()) + } +} + +/// The processors a Tezro takes, the default first: the 700 MHz R16000 a bare +/// `cpu = "r16000"` or `"r16000_700"` selects. Each part's clock, revision, +/// PRId and secondary-cache size are the `Part`'s; `with_cpu!` picks the +/// matching cache type (4 MB for 700 and 800 MHz, 8 MB for 900, 16 MB for 1 GHz). +pub const CPUS: &[&Part] = + &[&parts::R16000_700_EARLY, &parts::R16000_800, &parts::R16000_900, &parts::R16000_1000]; + +/// Processors fitted when `[machine] cpus` is unset: the four-processor tower, +/// as in the Tezro inventories seen. +pub const DEFAULT_CPUS: u8 = 4; + +/// What the Tezro board knows about one of its chips beyond the chip's own +/// identity: the flash record the PROM checks the processor against (which +/// also gives the low 25 bits of the Config word the processor resets with), +/// the processor counts the chip is fitted with, and the cache clock its +/// inventory prints. +/// +/// What is established (E), inferred (I), approximated (A) or unknown (U), +/// by the kind of capture behind it: +/// +/// - PRId 0x0f50 for the 1 GHz part: E (PROM banner `f50` and hinv 3.0 on +/// the same machine). 0x0f41 (700 MHz, rev 2.1), 0x0f42 (800 MHz, rev 2.2) +/// and the 900 MHz part's 0x0f50: I. The displayed revisions are printed +/// by inventories and the raw major follows the display rule, which Linux, +/// OpenBSD and PROM 6.210's printers apply (`Part::prid`). No raw read of a +/// 2.x part is on record. +/// +/// - Config words 0x009bad85 (700 MHz), 0x009baf85 (800 MHz) and 0x009cb185 +/// (900 MHz): E. They are the low 25 bits of the mode words in rows of the +/// stock IP35 PROM's CPU speed table, which the PROM writes to Config at +/// reset and reads back (bits 15:0 and 24:16). That the R16000 lays Config +/// out as the PROM's editor does is E for the PROM and I for the chip: no +/// R16000 manual is retained. SB=1 (128-byte lines) is a model choice the +/// table agrees with. Which row a given machine's flash holds is inferred; +/// Tezro inventories' cache taps (0xc, 0xa, 0xb) agree with these rows. +/// - Config word 0x00edb385 (1 GHz): A. The table stops at 900 MHz and no +/// capture records the word; see `FirmwareRecord::TEZRO_1000`. +/// - FIR 0x0900, every chip: U. It is the one R14000 value on record; a model +/// choice. The PROM never reads or prints it and no R16000 FIR was found +/// in the retained evidence. +/// - Config bit 23 and the HSTL bits of the mode word: their meaning is U. +/// Bit 23 is set in every row of 500 MHz and up, and the model copies it. +/// - Processor counts: the PROM has no count field. It lists IP53_4CPU, +/// IP53_2CPU and IP53_1CPU assembly names with no clock paired to any (E +/// that the names exist), which neither forbids nor shows a two-processor +/// part at any clock. The counts follow the boards captured with each +/// processor (the IP53 and IP59 processor-assemblies datasheet). The 700 +/// MHz chip is fitted as one, two or four processors, from the board's +/// population and those names. The 800 MHz chip (revision 2.2, 4 MB) as +/// four only: E, every captured board with it is `IP53_4CPU` (030-1956-002 +/// and -003); the two-processor 800 MHz board (`IP53_2CPU` 030-2029-001, on +/// an Origin 350) carries revision 3.0 processors, another PRId. The 900 +/// MHz chip as four only: A, a model choice: no IP53 or IP59 board at 900 +/// MHz is captured (the 900 MHz parts on record are on Fuel boards). The +/// 1 GHz chip as two or four: E for the counts, from `IP59_4CPU` +/// (030-1989-003) and `IP59_2CPU` (030-2059-002, in a rack Tezro). +/// - The 900 MHz chip: the PROM's speed table has its row, but no IP53 board +/// at 900 MHz is captured, so that a Tezro was fitted with it is not +/// established. +/// - The 1 GHz chip: captured 1 GHz Tezros carry IP59 node boards +/// (`IP59_4CPU`, `IP59_2CPU`), not IP53. The model builds them as the IP53 +/// board with a 1 GHz chip, as QEMU does; the PROM's socket masks for IP59 +/// equal IP53's. +/// - Cache clock (350, 400, 450 and 333 MHz) is as the inventory prints it +/// and not modelled; the cache taps are only part of the flash records' +/// mode words. +/// +/// An 800 MHz part with an 8 MB cache is captured in a Tezro (`IP53_4CPU` +/// 030-1959-003 C, revision 2.2, four processors, tap 0xa, PROM row 21 +/// 0x549caf85); it is not modelled yet. That is a scope choice, not an +/// evidence gap. +pub(crate) struct TezroChip { + pub chip: &'static Part, + pub firmware: FirmwareRecord, + /// Secondary-cache clock in MHz, as the chip is inventoried. + pub scache_mhz: u32, + pub cpus: &'static [u8], + /// Why `cpus` is what it is when that is not established: a clause for + /// the refusal message. + pub cpus_basis: &'static str, +} + +pub(crate) const CHIPS: [TezroChip; 4] = [ + TezroChip { + chip: &parts::R16000_700_EARLY, + firmware: FirmwareRecord::TEZRO, + scache_mhz: 350, + cpus: &[1, 2, 4], + cpus_basis: ", from the board's population and the PROM's assembly names, which pair with no clock (inferred)", + }, + TezroChip { + chip: &parts::R16000_800, + firmware: FirmwareRecord::TEZRO_800, + scache_mhz: 400, + cpus: &[4], + cpus_basis: ", as on every captured board with it; the two-processor 800 MHz board carries revision 3.0 processors", + }, + TezroChip { + chip: &parts::R16000_900, + firmware: FirmwareRecord::TEZRO_900, + scache_mhz: 450, + cpus: &[4], + cpus_basis: ", a model choice: no IP53 or IP59 board at 900 MHz is captured (approximated)", + }, + TezroChip { + chip: &parts::R16000_1000, + firmware: FirmwareRecord::TEZRO_1000, + scache_mhz: 333, + cpus: &[2, 4], + cpus_basis: ", as the captured IP59_4CPU and IP59_2CPU boards are", + }, +]; + +/// The board's data for `chip`. Panics for a part the Tezro does not take; +/// `MachineProfile::part` has refused one before a board is built. +pub(crate) fn chip(chip: &Part) -> &'static TezroChip { + CHIPS.iter().find(|c| *c.chip == *chip).expect("a Tezro takes only its own R16000 parts") +} + +impl TezroChip { + /// Whether the chip is fitted as `cpus` processors, and the refusal if + /// not. + pub fn accepts_cpus(&self, cpus: u8) -> Result<(), String> { + if self.cpus.contains(&cpus) { + return Ok(()); + } + let list: Vec = self.cpus.iter().map(u8::to_string).collect(); + Err(format!( + "cpus = {cpus}: the {} is fitted with {} processors only{}", + self.chip.label(), + list.join(", "), + self.cpus_basis + )) + } +} + +/// The CPU configuration the node board's L1 record carries: the flash +/// record's mode word and processor clock and the 200 MHz Bedrock. The PROM +/// checks the flash block against it at every start; without it the PROM +/// checks against the table row it picks for board word 0x2001, a 400 MHz +/// part, and prints "CPU configuration values mismatch". A real board's +/// record has not been read, so this is approximated: what QEMU's Tezro gives. +pub(crate) fn cpu_context(firmware: &FirmwareRecord) -> CpuContext { + CpuContext { mode_word: firmware.cpu_mode_word, cpu_mhz: firmware.cpu_clock_hz / 1_000_000, hub_mhz: 200 } +} + +/// The node board the L1's component 0 record names for a board of `cpus` +/// processors. IP53_4CPU part 030-1868-001 revision -C and IP53_2CPU part +/// 030-1869-001 revision -C are the boards Tezro inventories print +/// (established that such boards are fitted in Tezros); that a modelled +/// four- or two-processor Tezro carries them is inferred. The +/// one-processor board is named IP53_1CPU with its part and revision empty: +/// the PROM's socket-mask table holds that name, but no captured board +/// prints it (inferred). QEMU's Tezro profiles name the two- and +/// four-processor boards the same way (hw/mips/sgi/sn-machine.c and the +/// Tezro product layers in jamesbraid/qemu-sgi-origami) and leave the +/// one-processor board's name empty. The record's other fields are the +/// model's, not read from a board. +/// +/// The name matters to the PROM: it tries only the processor sockets in +/// the mask of the product-name row that matches (IP53_4CPU 0xf, IP53_2CPU +/// 0x5, IP53_1CPU 0x1) and all four when no row matches, power-cycling the +/// module three times for each empty socket it tries (Origami IP59 +/// datasheet, the PROM's per-socket bring-up). +fn node_board(cpus: usize) -> BoardIdentity { + match cpus { + 4 => BoardIdentity { name: "IP53_4CPU", part: "030-1868-001", revision: "-C" }, + 2 => BoardIdentity { name: "IP53_2CPU", part: "030-1869-001", revision: "-C" }, + _ => BoardIdentity { name: "IP53_1CPU", part: "", revision: "" }, + } +} + +/// The serial connections: the L1, then IOC4 ports 0 to 3. +const SERIAL_PORTS: usize = 5; + +pub struct TezroMachine { + board: Arc, + /// The processors, CPU 0 first, in the board's slot order. + cpus: Vec>, + timers: Arc, + monitor: Arc, + monitor_port: u16, + ci_serial: Option>, + console: usize, + ci: bool, + scratch: Option, + nvram: String, + frames: Option>, + nat: Option<(Arc, (Ipv4Addr, Ipv4Addr, Ipv4Addr))>, + events: Option>, + slot: Option, + lifecycle: GuestState, +} + +impl TezroMachine { + pub fn new(cfg: &MachineConfig) -> Result { + cfg.validate()?; + let prom = prom_image::load(Path::new(&cfg.prom), 8 * 1024 * 1024).map_err(|e| format!("Tezro {e}"))?; + let clock = Arc::new(Ds1744::new(YearBase::From(1968))?); + clock.set_nvram_file(Path::new(&cfg.nvram))?; + let memory = cfg.tezro_memory()?; + let chip = chip(cfg.cpu()); + let board = Arc::new(TezroBoard::with_config(TezroBoardConfig { + payload: &prom, + clock: host_clock(), + flash_path: cfg.tezro.flash.as_deref().map(Path::new), + station_mac: cfg.network().mac, + timekeeper: clock, + memory: &memory, + cpus: usize::from(cfg.machine.cpus()), + chip, + v12: cfg.graphics_board() == GraphicsBoard::V12, + graphics_pool: (cfg.graphics.threads, cfg.graphics.tile), + graphics_raster: (cfg.graphics.raster, cfg.graphics.gpu_backend.as_str()), + })?); + let timers = Arc::new(TimerManager::new()); + board.ioc4.set_timer_manager(timers.clone()); + let count_hz = cfg.clock.fixed_mhz.map_or(chip.chip.count_hz(), |mhz| (mhz * 1_000_000.0) as u64).max(1); + let cpus = build_cpus(&board, &timers, count_hz); + let cpu = cpus[0].clone(); + board.status.set_cpu_cycles(cpu.cycles_ptr()); + board.status.set_count_hz(cpu.count_hz_atomic()); + for c in cpus.iter().skip(1) { + board.status.add_cpu_stats(c.cycles_ptr(), c.fasttick_count()); + } + // One page pool and one body of compiled code, reaching each + // processor's state through the core it is called with. + let world = Arc::new(World::new()); + world.set_slots(cpus.iter().map(|c| c.park_slot()).collect()); + #[cfg(feature = "jitv2")] + if cpus.len() > 1 { + if cfg!(feature = "jitv2_lockstep") { + return Err("[machine] cpus above 1 cannot run with jitv2_lockstep".into()); + } + let jit = cpu.jitv2(); + for c in &cpus { + c.share_jitv2(jit.clone(), world.clone()); + } + } + board.presence.set_cpus(&cpus); + connect_cpus("Tezro", &board.bedrock, &board.l1, &board.events, &cpus)?; + board.bedrock.set_timer_manager(&timers); + let rx = board.receiver.lock().take().expect("the board's events are taken once"); + // After connect_cpu set the windows: each compile worker takes the + // cache's geometry and the CPU model's policies once, as it starts. + // Workers read plain physical code addresses through the view the CPU + // fetched each page by: the cached plane for cached pages, the + // uncached attribute-0 view for the reset PROM's uncached fetches. + #[cfg(feature = "jitv2")] + { + let cpu_device: Arc = cpu.clone().as_device(); + let jit = cpu.jitv2(); + let mut g = jit.lock(); + g.compile_queue.set_thread_count(cfg.jitv2.threads.max(1)); + g.compile_queue.set_cpu(Arc::downgrade(&cpu_device)); + g.compile_queue.set_owner(Arc::downgrade(&jit)); + if cpus.len() > 1 { + g.compile_queue.set_world(world.clone()); + } + #[cfg(not(feature = "jitv2_lockstep"))] + { + let stats = g.stats.clone(); + g.compile_queue.start(board.code_bus(), stats); + } + // The compile pool keeps only a weak handle on the CPU. + std::mem::forget(cpu_device); + } + let scratch = Self::attach_scsi(cfg, &board)?; + let console = cfg.tezro.console.index(); + let ci_serial = cfg.ci.then(|| board.serial[console].clone()); + Self::connect_serial(cfg, &board, console)?; + let monitor = Monitor::new(); + monitor.register_device(init_devlog()); + monitor.register_device(Arc::new(CpuMonitor::new(cpus.clone()))); + monitor.register_device(PerfMonitor::new( + cpu.running_flag(), cpu.cycles_ptr(), cpu.fasttick_count(), None, None, + )); + for device in board.monitor_devices() { + monitor.register_device(device); + } + for c in &cpus { + c.register_locks(); + } + board.scsi.register_locks(); + board.ioc4.register_locks(); + board.eth.register_locks(); + for pic in &board.pics { + pic.register_locks(); + } + board.flash.register_locks(); + board.bedrock.register_locks(); + let (mut nat, mut frames) = (None, None); + let net = GatewayConfig::from_network(&cfg.network()); + if let Some(link) = Link::with_gateway(net.clone())? { + frames = Some(link.frames()); + nat = Some((link.nat(), (net.client_ip, net.gateway_ip, net.netmask))); + board.eth.attach(link); + } + eprintln!( + "iris: Tezro IP53, {} R16000 {} MHz, {} MiB, {}, serial console {:?}", + cpus.len(), + chip.chip.mhz, + memory.total_mib(), + if board.v12.is_some() { cfg.graphics_board().label() } else { "no graphics" }, + cfg.tezro.console + ); + Ok(Self { + board, + cpus, + timers, + monitor: Arc::new(monitor), + monitor_port: cfg.monitor_port.unwrap_or(DEFAULT_MONITOR_PORT), + ci_serial, + console, + ci: cfg.ci, + scratch, + nvram: cfg.nvram.clone(), + nat, + frames, + events: Some(rx), + slot: None, + lifecycle: GuestState::Idle, + }) + } + + fn attach_scsi(cfg: &MachineConfig, board: &TezroBoard) -> Result, String> { + let attached = attach_from_config(cfg, Some("tezro"), &|channel| { + board.scsi.bus_for_channel(channel) + .ok_or_else(|| format!("controller {channel} (a Tezro has channels 0 and 1)")) + })?; + if let Some(device) = attached.daynaports.first() { + return Err(format!("{}: kind = \"daynaport\" is not available on the Tezro", + scsi_label(device.controller, device.id))); + } + Ok(attached.scratch) + } + + fn connect_serial(cfg: &MachineConfig, board: &TezroBoard, console: usize) -> Result<(), String> { + let [a, b] = cfg.serial_ports(); + let ioc4 = |port: usize| -> Box) + '_> { + Box::new(move |backend| board.ioc4.set_serial_backend(port, backend)) + }; + let ports = [ + SerialPort { + label: "L1", + port: cfg.tezro.l1_port.unwrap_or(8882), + setting: "tezro.l1_port", + ci: &board.serial[0], + log: cfg.tezro.l1_log.clone(), + connect: Box::new(|backend| board.l1.set_serial_backend(backend)), + }, + SerialPort { label: "IOC4 0", port: a, setting: "serial_port_a", ci: &board.serial[1], log: None, connect: ioc4(0) }, + SerialPort { label: "IOC4 1", port: b, setting: "serial_port_b", ci: &board.serial[2], log: None, connect: ioc4(1) }, + SerialPort { + label: "IOC4 2", + port: cfg.tezro.serial_port_c.unwrap_or(8883), + setting: "tezro.serial_port_c", + ci: &board.serial[3], + log: None, + connect: ioc4(2), + }, + SerialPort { + label: "IOC4 3", + port: cfg.tezro.serial_port_d.unwrap_or(8884), + setting: "tezro.serial_port_d", + ci: &board.serial[4], + log: None, + connect: ioc4(3), + }, + ]; + connect_serial_ports(cfg, "Tezro", &ports, console) + } +} + +/// One of the board's serial connections: what it is called, the TCP port +/// and the setting that names the port, the CI backend it uses when it is +/// the console, the log it keeps when it is not, and where the host end goes. +struct SerialPort<'a> { + label: &'static str, + port: u16, + setting: &'static str, + ci: &'a Arc, + log: Option, + connect: Box) + 'a>, +} + +/// Connect the board's serial ports to the host. The console takes +/// `serial_log` and, under CI, the in-process backend; the others listen on +/// their TCP ports, and a port that is taken leaves its stream unconnected +/// rather than stopping the machine. +fn connect_serial_ports(cfg: &MachineConfig, name: &str, ports: &[SerialPort<'_>], console: usize) -> Result<(), String> { + for (i, a) in ports.iter().enumerate() { + if let Some(b) = ports[i + 1..].iter().find(|b| b.port == a.port) { + return Err(format!("{name} {} and {} serial ports must be distinct", a.label, b.label)); + } + } + let mut log_config = cfg.clone(); + for (index, port) in ports.iter().enumerate() { + log_config.serial_log = if index == console { cfg.serial_log.clone() } else { port.log.clone() }; + let tag = format!("{name} {}", port.label); + let log = open_log(&log_config, &tag, LogMode::Fresh); + let backend: Arc = if cfg.ci && index == console { + if let Some(log) = log { + port.ci.set_log(log); + } + port.ci.clone() + } else { + let mut backend = null_serial_backend(); + match tcp_serial_backend(port.port) { + Some((socket, actual_port)) => { + eprintln!("iris: {tag} on 127.0.0.1:{actual_port}"); + backend = socket; + } + None => eprintln!( + "iris: {tag}: port {} unavailable; is another iris using it? ({})", + port.port, port.setting + ), + } + match log { + Some(log) => Arc::new(TeeBackend::new(backend, log)), + None => backend, + } + }; + (port.connect)(backend); + } + Ok(()) +} + +impl Workstation for TezroMachine { + fn start(&mut self) { + self.lifecycle = GuestState::Starting; + self.monitor.clone().start_server(format!("127.0.0.1:{}", self.monitor_port)); + self.restart_peripherals(); + if !self.ci { + self.cpu_start(); + } + } + fn stop(&mut self) { + for c in &self.cpus { + c.clone().as_device().stop(); + } + self.board.stop(); + self.board.timekeeper.flush(); + if self.lifecycle != GuestState::Idle { + self.lifecycle = GuestState::Halted; + } + } + fn reset(&mut self) { + self.stop(); + for c in &self.cpus { + c.power_on(); + } + self.board.power_on(); + self.restart_peripherals(); + } + fn restart_peripherals(&mut self) { + self.board.start(); + } + /// Every processor starts at once, as from the node's reset. + fn cpu_start(&self) { + for c in &self.cpus { + c.clone().as_device().start(); + } + } + fn cpu_is_running(&self) -> bool { + self.cpus.iter().any(|c| c.clone().as_device().is_running()) + } + fn machine_slot(&mut self) -> MachineSlot { + let ptr = self as *mut Self as *mut dyn Workstation; + self.slot.get_or_insert_with(|| Arc::new(Mutex::new(MachineRef(ptr)))).clone() + } + fn register_system_controller(&mut self) { + let slot = self.machine_slot(); + let events = self.events.take().expect("Tezro system controller registered twice"); + register_system_controller(&self.monitor, slot, Some(events)); + } + fn monitor_addr(&self) -> Option { + self.monitor.bound_addr() + } + fn lifecycle(&self) -> GuestState { + self.lifecycle + } + fn set_lifecycle(&mut self, state: GuestState) { + self.lifecycle = state; + } + fn get_display(&self) -> Option> { + self.board.v12.as_ref().map(|v12| v12.display()) + } + fn register_monitor_device(&self, device: Arc) { + self.monitor.register_device(device); + } + fn get_ps2(&self) -> Arc { + self.board.ioc4.ps2() + } + fn get_timer_manager(&self) -> Arc { + self.timers.clone() + } + fn get_cpu_debug(&self) -> Arc { + self.cpu().clone().debug_adapter() + } + fn fasttick_count(&self) -> Arc { + self.cpu().fasttick_count() + } + fn cpu_counters(&self) -> Vec<(crate::cpu::mips_core::CyclesPtr, Arc)> { + self.cpus.iter().map(|c| (c.cycles_ptr(), c.fasttick_count())).collect() + } + fn get_ci_serial(&self) -> Option> { + self.ci_serial.clone() + } + fn inject_serial_console(&self, bytes: &[u8]) { + self.board.serial[self.console].push_host(bytes); + } + fn as_any_mut(&mut self) -> &mut dyn std::any::Any { + self + } + fn save_nvram(&self, path: Option<&str>) -> Result { + if path.is_some_and(|p| p != self.nvram) { + return Err("Tezro NVRAM save uses the configured path".into()); + } + self.board.timekeeper.flush(); + Ok(self.nvram.clone()) + } + fn scratch_path(&self) -> Option<&Path> { + self.scratch.as_deref() + } + fn disc_changer(&self) -> Option> { + Some(self.board.scsi.clone() as Arc) + } + fn pending_chd_sync_count(&self) -> usize { + self.scsi_buses().iter().map(|bus| bus.pending_chd_sync_count()).sum() + } + fn flush_disks(&self) -> std::io::Result<()> { + self.board.flush_disks() + } + fn sync_chd_disks( + &self, + only: Option, + progress: &mut dyn FnMut(usize, usize, f32), + cancel: &dyn Fn() -> bool, + ) -> std::io::Result { + ScsiBus::sync_chd_disks(&self.scsi_buses(), only, progress, cancel) + } + fn net_frames(&self) -> Option> { + self.frames.clone() + } + fn nat_control(&self) -> Option> { + self.nat.as_ref().map(|(n, _)| n.clone()) + } + fn nat_expected(&self) -> Option<(Ipv4Addr, Ipv4Addr, Ipv4Addr)> { + self.nat.as_ref().map(|(_, a)| *a) + } +} + +impl TezroMachine { + /// CPU 0, the one the front ends drive. + fn cpu(&self) -> &Arc { + &self.cpus[0] + } + + fn scsi_buses(&self) -> Vec<&ScsiBus> { + (0..2).filter_map(|channel| self.board.scsi.bus_for_channel(channel)).collect() + } +} + +impl Drop for TezroMachine { + fn drop(&mut self) { + if let Some(slot) = self.slot.take() { + slot.lock().0 = MachineRef::null(); + } + self.stop(); + self.monitor.shutdown(); + } +} + +/// The processors the machine runs, of the board's chip: one for each +/// fitted slot, CPU 0 first, in the board's slot order. +fn build_cpus(board: &Arc, timers: &Arc, count_hz: u64) -> Vec> { + let count = board.processors.count(); + crate::with_cpu!(board.chip.chip, true, false, C => { + (0..count).map(|id| start_cpu::(board, timers, count_hz, id as u8, count)).collect() + }) +} + +/// Processor `id` of `count`, of type `C`, with its timers; CPU 0 also +/// runs the board's poll, and on a board of several processors every one +/// takes part in the PROM's holds at its batch boundary. A chip with a +/// secondary cache of another size is another type, instantiated here. +fn start_cpu(board: &Arc, timers: &Arc, count_hz: u64, id: u8, count: usize) -> Arc +where + C: CacheCpuModel + From> + 'static, + MipsCpu: CpuDevice, +{ + let mut exec = build_cpu::(board); + // CPU 0's Compare matches blink the graphics status bar's LED. + exec.core.fasttick_count = if id == 0 { board.fasttick.clone() } else { Arc::new(AtomicU64::new(0)) }; + exec.core.set_timer_manager(timers.clone()); + exec.core.set_count_hz(count_hz); + exec.core.count_write_samples = true; + exec.core.cpu_id = id; + exec.core.smp = count > 1; + let poll_board = board.clone(); + let cpu = usize::from(id); + exec.batch_hook = match (id, count > 1) { + (0, false) => Some(Box::new(move |_| poll_board.poll())), + (0, true) => Some(Box::new(move |_| { + poll_board.poll(); + poll_board.presence.at_batch(cpu); + })), + (_, true) => Some(Box::new(move |_| poll_board.presence.at_batch(cpu))), + (_, false) => None, + }; + Arc::new(MipsCpu::new(*exec)) +} + +/// A processor of type `C` with the board chip's identity and reset-mode +/// inputs, on the board's bus. +fn build_cpu>>(board: &TezroBoard) -> Box> { + let chip = board.chip.chip; + let cfg = MipsCpuConfig::for_model::(Arc::new(crate::ip35::memory_map(&board.bedrock.ram_extent()))) + .with_prid(chip.prid()); + let mut cpu = Box::new(MipsExecutor::new( + board.bus.clone() as Arc, + MipsTlb::new(cfg.tlb_entries), + &cfg, + )); + cpu.phys_mask = CPU_PHYS_MASK; + // The Config word of the chip's flash record over the low 25 bits the + // cache type builds (FIR is the type's). The flash record's mode word and + // the processor's reset mode must agree, or the PROM enters its error + // routine before GP is set. + cpu.core.cp0_config = (cpu.core.cp0_config & !CONFIG0_MODE_MASK) | board.chip.firmware.reset_mode(); + cpu.cache = C::from(board.cached_bus()); + // The PROM fetches its own code uncached; with memory over its window + // the cached plane answers RAM there, so the fetch is presented at the + // PROM's own address and compiled from the view that holds it. + cpu.set_uncached_fetch_alias(crate::ip35::RESET_WINDOW, crate::ip35::RESET_WINDOW_SIZE, crate::ip35::RESET_WINDOW_FETCH); + #[cfg(feature = "jitv2")] + cpu.set_cached_bus(board.code_bus()); + #[cfg(not(feature = "jitv2"))] + cpu.set_cached_bus(board.cached_bus()); + cpu +} + +pub(crate) struct TezroBoardConfig<'a> { + pub payload: &'a [u8], + pub clock: Clock, + pub flash_path: Option<&'a Path>, + pub station_mac: [u8; 6], + /// The DS1744-compatible clock and NVRAM in the IOC4's BAR. + pub timekeeper: Arc, + pub memory: &'a MemoryPopulation, + /// Processors fitted. + pub cpus: usize, + /// The R16000 chip fitted. + pub chip: &'static TezroChip, + /// A VPro V12 at widget 12. + pub v12: bool, + /// Rendering threads and tile size for the V12. + pub graphics_pool: (usize, u32), + /// Which unit draws and, for the GPU, the wgpu backend (`[graphics]`). + pub graphics_raster: (crate::config::RasterUnit, &'a str), +} + +/// The Tezro's devices around its Bedrock node. +pub(crate) struct TezroBoard { + pub bus: Arc, + pub bedrock: Arc, + pub l1: Arc, + pub flash: Arc, + /// Cacheable memory at plain physical addresses: what the caches fill + /// from. + cached: Arc, + /// What the JIT compiles code through: `cached`, and the reset PROM's + /// uncached-fetch window ([`Ip35Bus::code_view`]). + #[cfg(feature = "jitv2")] + code: Arc, + /// The node's resets and power events, and their receiver until the + /// system controller takes it. + events: Sender, + receiver: Mutex>>, + /// The slots whose processors run: every fitted one. + pub processors: Population, + /// The holds on the PROM's pair liveness check and presence publisher + /// across them. + pub presence: Arc, + /// The PICs in QEMU's host order, which numbers their PI level sources: + /// widget 11's first when the board has it, then the IO9's at widget 15. + pub pics: Vec>, + pub ioc4: Arc, + pub scsi: Arc, + pub eth: Arc, + pub timekeeper: Arc, + pub serial: [Arc; SERIAL_PORTS], + /// The R16000 chip fitted, whose flash record the node holds. + pub chip: &'static TezroChip, + /// The VPro V12, when the board has one. + pub v12: Option>, + /// The status bar's activity bits and the CPU's Compare matches, which + /// the graphics board shows. + pub heartbeat: Arc, + pub fasttick: Arc, + /// What the graphics board's status bar shows: the activity bits, the + /// Compare matches and the CPU's cycle count and Count rate. + pub status: Arc, + clock: Clock, + last_clock: AtomicU64, + running: AtomicBool, +} + +impl TezroBoard { + /// The board's devices that take monitor commands: the physical map, the + /// L1, the SCSI controller, the keyboard and mouse (`ps2 type` and + /// friends) and, when one is fitted, the VPro (`vpro ...`). + pub fn monitor_devices(&self) -> Vec> { + let mut devices: Vec> = vec![ + self.bedrock.physical().clone(), + self.l1.clone(), + self.scsi.clone(), + self.ioc4.ps2(), + ]; + if let Some(v12) = &self.v12 { + devices.push(v12.clone()); + } + devices + } + + /// Flush the disks on both channels of the storage controller. + pub fn flush_disks(&self) -> std::io::Result<()> { + (0..2).try_for_each(|channel| self.scsi.bus_for_channel(channel).unwrap().flush()) + } + + #[cfg(test)] + pub fn new(cpus: usize) -> Result { + Self::with_config(TezroBoardConfig { + payload: &[], + clock: Arc::new(|| 0), + flash_path: None, + station_mac: L1Config::default().mac, + timekeeper: Arc::new(Ds1744::new(YearBase::From(1968))?), + memory: &MemoryPopulation::default(), + cpus, + chip: &CHIPS[0], + v12: false, + graphics_pool: (2, 32), + graphics_raster: (crate::config::RasterUnit::Cpu, "auto"), + }) + } + + pub fn with_config(config: TezroBoardConfig<'_>) -> Result { + let TezroBoardConfig { payload, clock, flash_path, station_mac, timekeeper, memory, cpus, chip, v12, graphics_pool, graphics_raster } = config; + if ![1, 2, 4].contains(&cpus) { + return Err(format!("{cpus} processors; a Tezro board fits 1, 2 or 4")); + } + memory.check_for_cpus(cpus)?; + let running = Population::fitted_cpus(cpus)?; + let node = Node::build(NodeConfig { + name: "Tezro", + payload, + clock: clock.clone(), + flash_path, + banks_mib: &memory.banks_mib(), + processors: running, + firmware: chip.firmware, + board_id: BOARD_ID, + l1: L1Config { + type_code: 0x39, + revision: Some([1, 48, 1]), + mac: station_mac, + port_speed_first: 1, + router: true, + cpu_context: Some(cpu_context(&chip.firmware)), + node_board: Some(node_board(cpus)), + ..Default::default() + }, + })?; + let Node { + bus, + bedrock, + l1, + flash, + cached, + #[cfg(feature = "jitv2")] + code, + events, + receiver, + } = node; + let serial: [Arc; SERIAL_PORTS] = std::array::from_fn(|_| Arc::new(CiSerialBackend::new())); + l1.set_serial_backend(serial[0].clone()); + + let io9 = Pic::new(IO9_WIDGET); + let pxbow = io9.enable_pxbow(HOST_WIDGET, clock.clone()).map_err(|e| format!("Tezro PXBOW: {e:?}"))?; + bedrock.connect_widget(0, pxbow.widget_zero()); + bedrock.connect_widget(IO9_WIDGET, pxbow.route(IO9_WIDGET)); + let presence = PresenceHold::new(running, clock.clone(), &bedrock); + let heartbeat = Arc::new(AtomicU64::new(0)); + let fasttick = Arc::new(AtomicU64::new(0)); + let status = crate::disp::StatusSource::new(heartbeat.clone(), fasttick.clone()); + let v12 = v12.then(|| fit_v12(&bedrock, &pxbow, &status, graphics_pool, graphics_raster)).transpose()?; + let pics = if cpus == 1 { + vec![io9.clone()] + } else { + // The PROM finds widget 11 only as a live link of widget 0's + // crossbar, so the second PIC is a port of the same switch. + let second = Pic::new(SECOND_PIC_WIDGET); + pxbow + .attach_endpoint(SECOND_PIC_WIDGET, second.widget()) + .map_err(|e| format!("Tezro: second PIC on the PXBOW: {e:?}"))?; + bedrock.connect_widget(SECOND_PIC_WIDGET, pxbow.route(SECOND_PIC_WIDGET)); + vec![second, io9.clone()] + }; + for (index, pic) in pics.iter().enumerate() { + pic.connect_peer(Arc::new(BedrockPeer(Arc::downgrade(&bedrock)))) + .map_err(|_| "Tezro: PIC peer connected twice".to_string())?; + // Each PIC owns its own 16 PI level sources, as QEMU numbers them. + pic.connect_node(&bedrock, index * 2 * crate::dev::pic::PINS); + } + + let ioc4 = Arc::new(Ioc4::new(Ioc4Config::tezro())); + ioc4.attach_clock(timekeeper.clone()); + for (port, backend) in serial[1..].iter().enumerate() { + ioc4.set_serial_backend(port, backend.clone()); + } + let scsi = Arc::new(Isp12160::new(Isp12160::new_bus(), Isp12160::new_bus())); + scsi.set_heartbeat(heartbeat.clone()); + // The Ethernet slot drives REQ64#, which the chip samples as reset ends. + let eth = Bcm5701::new(Bcm5701Config { req64: true, ..Bcm5701Config::new(l1.station_mac()) })?; + io9.attach(0, IOC4_DEVICE, ioc4.clone(), Function { self_ordered_pio: true, req64: false }); + io9.attach(0, SCSI_DEVICE, scsi.clone(), Function::default()); + io9.attach(0, ETH_DEVICE, eth.clone(), Function { self_ordered_pio: false, req64: true }); + Ok(Self { + bus, + bedrock, + l1, + flash, + cached, + #[cfg(feature = "jitv2")] + code, + events, + receiver: Mutex::new(Some(receiver)), + processors: running, + presence, + pics, + ioc4, + scsi, + eth, + timekeeper, + serial, + chip, + v12, + heartbeat, + fasttick, + status, + last_clock: AtomicU64::new(clock()), + clock, + running: AtomicBool::new(false), + }) + } + + pub fn cached_bus(&self) -> Arc { + self.cached.clone() + } + + /// The bus the JIT compiles code through; see [`Ip35Bus::code_view`]. + #[cfg(feature = "jitv2")] + pub fn code_bus(&self) -> Arc { + self.code.clone() + } + + /// Everything back to its power-on state. + pub fn power_on(&self) { + self.presence.reset(); + self.bedrock.power_on(); + self.l1.power_on(); + self.flash.power_on(); + self.ioc4.power_on(); + self.scsi.power_on(); + self.eth.power_on(); + for pic in &self.pics { + pic.power_on(); + } + if let Some(v12) = &self.v12 { + v12.power_on(); + } + } + + pub fn start(&self) { + if let Some(v12) = &self.v12 { + v12.start(); + } + self.last_clock.store((self.clock)(), Ordering::Release); + self.running.store(true, Ordering::Release); + self.bedrock.start(); + self.l1.start(); + self.ioc4.start(); + self.eth.start(); + self.scsi.start(); + } + + /// The boot flash's command timing, on CPU 0's batch boundary. + pub fn poll(&self) { + if self.running.load(Ordering::Acquire) { + let now = (self.clock)(); + let previous = self.last_clock.swap(now, Ordering::AcqRel); + self.flash.advance_ns(now.saturating_sub(previous)); + } + } + + pub fn stop(&self) { + self.poll(); + if let Some(v12) = &self.v12 { + v12.stop(); + } + self.running.store(false, Ordering::Release); + self.scsi.stop(); + self.eth.stop(); + self.ioc4.stop(); + self.l1.stop(); + self.bedrock.stop(); + } +} + +impl Drop for TezroBoard { + fn drop(&mut self) { + self.stop(); + } +} + +/// Where the V12 sits: widget 12 of the crossbar behind the PIC that owns +/// widget 0. On the two- and four-processor board this is QEMU's graphics +/// slot. On the one-processor board the tower guide gives one XIO connector +/// for a VPro board but names neither its widget nor its route; widget 12 +/// behind the same PXBOW is inferred, not established. +const V12_WIDGET: u8 = 12; + +/// Fit the V12 behind the PXBOW: its interrupt packets and DMA reach +/// Bedrock IO as the PIC's do. Nothing counts the processors' stores into +/// its graphics page or returns credits for them. +fn fit_v12( + bedrock: &Arc, + pxbow: &Arc, + status: &Arc, + graphics_pool: (usize, u32), + graphics_raster: (crate::config::RasterUnit, &str), +) -> Result, String> { + let io = Arc::downgrade(bedrock); + let packets: PacketSender = Arc::new(move |widget, address, vector, set| { + if widget != HOST_WIDGET { + dlog!(LogModule::Bedrock, "Tezro: V12 interrupt to widget {widget} refused; only Bedrock IO is reachable"); + return false; + } + let Some(io) = io.upgrade() else { return false }; + let payload = u64::from(vector) | if set { 1 << 8 } else { 0 }; + io.inbound_write(V12_WIDGET, Some(INBOUND_DEVICE), address, 8, payload) == BUS_OK + }); + let dma = bedrock.widget_dma(V12_WIDGET); + let v12 = Odyssey::new_widget_relative( + V12_WIDGET, + GraphicsBoard::V12.as_str(), + packets, + dma, + status.clone(), + graphics_pool, + graphics_raster, + )?; + pxbow + .attach_endpoint(V12_WIDGET, v12.clone() as Arc) + .map_err(|e| format!("Tezro V12: {e:?}"))?; + bedrock.connect_widget(V12_WIDGET, pxbow.route(V12_WIDGET)); + Ok(v12) +} + +/// The far end of a PIC's link: Bedrock IO. Interrupt packets and DMA arrive +/// there as inbound packets from the PIC's widget, device 0, as QEMU's +/// Bedrock IO takes them by default. DMA aimed at another widget is refused: +/// peer-to-peer XIO is not modelled. +struct BedrockPeer(Weak); + +const INBOUND_DEVICE: u8 = 0; + +impl BedrockPeer { + fn io(&self) -> Result, Refused> { + self.0.upgrade().ok_or(Refused) + } + + fn host(widget: u8) -> Result<(), Refused> { + if widget == HOST_WIDGET { + Ok(()) + } else { + dlog!(LogModule::Bedrock, "Tezro: PIC DMA to widget {widget} refused; only Bedrock IO is reachable"); + Err(Refused) + } + } + + /// The legs of a transfer: each the largest naturally aligned size of + /// eight bytes or fewer that fits, so word accesses keep their width. + fn legs(address: u64, len: usize) -> impl Iterator { + let mut offset = 0; + std::iter::from_fn(move || { + if offset == len { + return None; + } + let at = address + offset as u64; + let size = [8usize, 4, 2, 1] + .into_iter() + .find(|&n| at.is_multiple_of(n as u64) && n <= len - offset) + .unwrap_or(1); + let leg = (offset, size); + offset += size; + Some(leg) + }) + } +} + +impl PicPeer for BedrockPeer { + fn interrupt(&self, source_widget: u8, address: u64, payload: u64) -> Result<(), Refused> { + let io = self.io()?; + (io.inbound_write(source_widget, Some(INBOUND_DEVICE), address, 8, payload) == BUS_OK) + .then_some(()) + .ok_or(Refused) + } + + fn dma_read(&self, source_widget: u8, widget: u8, address: u64, data: &mut [u8]) -> Result<(), Refused> { + Self::host(widget)?; + let io = self.io()?; + for (offset, size) in Self::legs(address, data.len()) { + let value = io.inbound_read(source_widget, Some(INBOUND_DEVICE), address + offset as u64, size as u8); + if value.status != BUS_OK { + return Err(Refused); + } + data[offset..offset + size].copy_from_slice(&value.data.to_be_bytes()[8 - size..]); + } + Ok(()) + } + + fn dma_write(&self, source_widget: u8, widget: u8, address: u64, data: &[u8]) -> Result<(), Refused> { + Self::host(widget)?; + let io = self.io()?; + for (offset, size) in Self::legs(address, data.len()) { + let value = data[offset..offset + size].iter().fold(0u64, |v, b| (v << 8) | u64::from(*b)); + if io.inbound_write(source_widget, Some(INBOUND_DEVICE), address + offset as u64, size as u8, value) != BUS_OK { + return Err(Refused); + } + } + Ok(()) + } +} + +#[cfg(test)] +mod tests; diff --git a/src/ip53_machine/tests.rs b/src/ip53_machine/tests.rs new file mode 100644 index 000000000..538e57452 --- /dev/null +++ b/src/ip53_machine/tests.rs @@ -0,0 +1,659 @@ +use super::*; +use crate::traits::{BUS_ERR, BUS_OK}; +use std::sync::atomic::AtomicU64; + +/// A PIO offset in Bedrock IO's small window: widget in bits 27:24. +/// The II's PIO face as the processor reaches it, a `size`-byte access at +/// a time. +struct Pio(Arc); +impl Pio { + fn pio_read(&self, a: u64, size: u8) -> crate::traits::BusRead64 { + match size { + 8 => self.0.read64(a), + 4 => { + let r = self.0.read32(a); + crate::traits::BusRead64 { data: u64::from(r.data), status: r.status } + } + _ => unreachable!("tests read words and doublewords"), + } + } + fn pio_write(&self, a: u64, size: u8, value: u64) -> u32 { + match size { + 8 => self.0.write64(a, value), + 4 => self.0.write32(a, value as u32), + _ => unreachable!("tests write words and doublewords"), + } + } +} +fn pio_read(bedrock: &Arc, a: u64, size: u8) -> crate::traits::BusRead64 { + Pio(bedrock.pio()).pio_read(a, size) +} + +fn widget(widget: u64, offset: u64) -> u64 { + (widget << 24) | offset +} + +/// Configuration register 0 of `device` on the IO9 PIC's head 0. +fn config_id(device: u64) -> u64 { + widget(15, 0x2_0000 + (device << 12)) +} + +/// The processor finds the PXBOW at widget 0 and the IO9's PIC at widget +/// 15, as the QEMU profile maps them; the one-processor board has no widget +/// 11 and no other widget answers. +#[test] +fn widget_0_is_the_pxbow_and_widget_15_the_io9_pic() { + let board = TezroBoard::new(1).unwrap(); + let io = Pio(board.bedrock.pio()); + let pxbow = io.pio_read(widget(0, 4), 4); + assert_eq!(pxbow.status, BUS_OK); + assert_eq!((pxbow.data >> 12) & 0xffff, u64::from(crate::dev::pxbow::PART)); + let pic = io.pio_read(widget(15, 0), 8); + assert_eq!(pic.status, BUS_OK); + assert_eq!(pic.data, u64::from(crate::dev::pic::identity(0))); + for absent in [8, 9, 11, 12, 13, 14] { + assert_eq!(io.pio_read(widget(absent, 0), 8).status, BUS_ERR, "widget {absent}"); + } + assert_eq!(board.pics.len(), 1); +} + +/// IO9's functions sit at PCI devices 1, 3 and 4 of head 0 with the +/// revisions the QEMU profile selects, and device 2 is empty. +#[test] +fn the_io9_bus_holds_the_ioc4_the_isp12160_and_the_bcm5701() { + let board = TezroBoard::new(1).unwrap(); + let io = Pio(board.bedrock.pio()); + for (device, id, revision) in [(1, 0x100a_10a9, 0x53), (3, 0x1216_1077, 0x06), (4, 0x1645_14e4, 0x00)] { + assert_eq!(io.pio_read(config_id(device), 4).data, id, "device {device}"); + assert_eq!(io.pio_read(config_id(device) + 8, 4).data & 0xff, revision, "device {device}"); + } + assert_eq!(io.pio_read(config_id(2), 4).status, BUS_ERR, "device 2 is empty"); +} + +/// The BCM5701's slot drives REQ64#, so the chip reports a 64-bit bus, +/// and still does after the board is reset. +#[test] +fn the_bcm5701_sees_req64_before_and_after_a_reset() { + use crate::pci::PciFunction; + const PCISTATE: usize = 0x70 / 4; + const BUS_32BIT: u32 = 1 << 4; + let board = TezroBoard::new(1).unwrap(); + assert_eq!(board.eth.config_dword(PCISTATE) & BUS_32BIT, 0); + board.power_on(); + assert_eq!(board.eth.config_dword(PCISTATE) & BUS_32BIT, 0); +} + +/// The board identity the PROM reads: Bedrock board word 0x2001 and the +/// L1's Tezro station address. +#[test] +fn the_board_reports_the_tezro_board_word() { + let board = TezroBoard::new(1).unwrap(); + let id = board.bus.read64(0x1000_0000 + 0x400_0000); + assert_eq!((id.status, id.data), (BUS_OK, BOARD_ID)); + assert_eq!(board.l1.station_mac(), L1Config::default().mac); +} + +/// The fields of an L1 EEPROM area after its six-byte header, up to its +/// end mark: each a descriptor byte whose low six bits give the length. +fn area_fields(area: &[u8]) -> Vec<&[u8]> { + let mut fields = Vec::new(); + let mut at = 6; + while area[at] != 0xc1 { + let len = usize::from(area[at] & 0x3f); + fields.push(&area[at + 1..at + 1 + len]); + at += 1 + len; + } + fields +} + +/// The PROM's start-up check compares the boot flash's configuration block +/// with the CPU configuration in the node board's L1 record (component 0's +/// board area): the mode word at +0x04 with Config0, the clocks at +0x10 and +/// +0x18 with the CPU and Hub MHz, and the word at +0x30 with the class it +/// derives from the board word's top nibble (0x4 gives 2, 0x8 gives 1, any +/// other 0). Any difference prints "CPU configuration values mismatch". +#[test] +fn the_node_board_record_agrees_with_the_flash_configuration() { + let class = match BOARD_ID & 0xf000 { + 0x4000 => 2u32, + 0x8000 => 1, + _ => 0, + }; + for cpus in [1, 2, 4] { + let board = TezroBoard::new(cpus).unwrap(); + let area = board.l1.record(0, 2).expect("component 0's board area"); + assert_eq!(area.iter().fold(0u8, |s, b| s.wrapping_add(*b)), 0, "the area's checksum"); + assert_eq!(area[0], 2, "the format that carries CPU fields"); + let fields = area_fields(&area); + let identity = node_board(cpus); + assert_eq!([fields[1], fields[3], fields[5]], [identity.name, identity.part, identity.revision].map(str::as_bytes), + "{cpus} CPUs: the node board's name, part and revision"); + assert_eq!(board.l1.record(1, 2), None, "{cpus} CPUs: no I/O board record"); + assert_eq!(fields[8], [3], "the CPU-field selector"); + let word = |f: &[u8]| u32::from_be_bytes(f.try_into().unwrap()); + let (config0, hub_mhz, cpu_mhz) = (word(fields[10]), word(fields[14]), word(fields[15])); + for empty in [9, 11, 12, 13, 16, 17, 18] { + assert!(fields[empty].is_empty(), "field {empty} is unknown and left empty"); + } + let block = &board.flash.contents()[0x60..0xb0]; + let at = |o: usize, n: usize| block[o..o + n].iter().fold(0u64, |v, b| v << 8 | u64::from(*b)); + assert_eq!(at(0x04, 4), u64::from(config0), "{cpus} CPUs: the mode word"); + assert_eq!(at(0x10, 8), u64::from(cpu_mhz) * 1_000_000, "{cpus} CPUs: the CPU clock"); + assert_eq!(at(0x18, 8), u64::from(hub_mhz) * 1_000_000, "{cpus} CPUs: the Hub clock"); + assert_eq!(at(0x30, 4), u64::from(class), "{cpus} CPUs: the board word's class"); + } + for chip in &CHIPS { + let context = cpu_context(&chip.firmware); + assert_eq!(context.mode_word, chip.firmware.cpu_mode_word, "{}", chip.chip.label()); + assert_eq!(context.cpu_mhz * 1_000_000, chip.firmware.cpu_clock_hz, "{}", chip.chip.label()); + assert_eq!(chip.firmware.machine_type, class, "{}", chip.chip.label()); + } +} + +/// PIC DMA reaches memory through Bedrock IO in naturally aligned legs, and +/// DMA aimed at any widget but Bedrock IO is refused. +#[test] +fn pic_dma_reaches_memory_through_bedrock_io_only() { + let board = TezroBoard::new(1).unwrap(); + // Enable bank pair 0, as the Fuel's DMA tests do. + assert_eq!(board.bedrock.registers().write64(crate::dev::bedrock::reg::MD_BASE + 8, 0x33), BUS_OK); + let peer = BedrockPeer(Arc::downgrade(&board.bedrock)); + let data: Vec = (1..=19).collect(); + assert_eq!(peer.dma_write(15, HOST_WIDGET, 0x1003, &data), Ok(())); + let mut back = vec![0; data.len()]; + assert_eq!(peer.dma_read(15, HOST_WIDGET, 0x1003, &mut back), Ok(())); + assert_eq!(back, data); + let memory = board.bedrock.dma_view(); + assert_eq!(memory.read64(0x1000).data, 0x0000_0001_0203_0405); + assert_eq!(peer.dma_write(15, 12, 0x1003, &data), Err(Refused)); + assert_eq!(peer.dma_read(15, 0, 0x1003, &mut back), Err(Refused)); +} + +#[test] +fn transfer_legs_keep_aligned_widths() { + let legs: Vec<_> = BedrockPeer::legs(0x1003, 19).collect(); + assert_eq!(legs, [(0, 1), (1, 4), (5, 8), (13, 4), (17, 2)]); + assert_eq!(BedrockPeer::legs(0x2000, 16).collect::>(), [(0, 8), (8, 8)]); + assert_eq!(BedrockPeer::legs(0x2000, 0).count(), 0); +} + +/// The two- and four-processor board adds a second PIC at widget 11 with +/// nothing on its buses, keeps the IO9 on widget 15 behind widget 0, and +/// leaves the V12's widget 12 empty. +#[test] +fn two_or_four_processors_bring_the_second_pic_at_widget_11() { + for cpus in [2, 4] { + let board = TezroBoard::new(cpus).unwrap(); + let io = Pio(board.bedrock.pio()); + assert_eq!(board.pics.iter().map(|p| p.widget_number()).collect::>(), [11, 15]); + assert_eq!(io.pio_read(widget(11, 0), 8).data, u64::from(crate::dev::pic::identity(0))); + assert_eq!(io.pio_read(widget(15, 0), 8).data, u64::from(crate::dev::pic::identity(0))); + assert_eq!(io.pio_read(widget(12, 0), 8).status, BUS_ERR, "no V12"); + assert_eq!((io.pio_read(widget(0, 4), 4).data >> 12) & 0xffff, u64::from(crate::dev::pxbow::PART)); + for device in 1..=4 { + assert_eq!(io.pio_read(widget(11, 0x2_0000 + (device << 12)), 4).status, BUS_ERR, "widget 11 device {device}"); + } + assert_eq!(io.pio_read(config_id(1), 4).data, 0x100a_10a9); + } +} + +// Widget 0's crossbar as PROM 6.210 and the IP35 kernel walk it, from how the PROM finds +// widgets 0xb and 0xf. +// QEMU's sn/builder/tezro-two-four-processor-board in +// tests/qtest/sn-builder-test.c still expects link 11 absent; this board +// follows the PROM's walk instead. + +/// Link `n`'s register block in widget 0, as the 32-bit word address the +/// PROM issues: the low word of each doubleword. +fn link_word(n: u64, register: u64) -> u64 { + widget(0, 0x100 + (n - 8) * 0x40 + register + 4) +} +const LINK_CONTROL: u64 = 0x08; +const LINK_STATUS: u64 = 0x10; +const LINK_ARB_UPPER: u64 = 0x18; +const LINK_ARB_LOWER: u64 = 0x20; +const LINK_STATUS_CLEAR: u64 = 0x28; +const LINK_RESET: u64 = 0x30; +const LINK_AUX: u64 = 0x38; +const ALIVE: u64 = 1 << 31; +const PRESENT: u64 = 1 << 5; +const FAILURE_MODE: u64 = 1 << 6; + +fn read_word(board: &TezroBoard, address: u64) -> u64 { + let r = pio_read(&board.bedrock, address, 4); + assert_eq!(r.status, BUS_OK, "{address:#x}"); + r.data +} + +/// Widget 0's part and manufacturer, from its identity word. +fn widget_zero_part(board: &TezroBoard) -> (u64, u64) { + let id = read_word(board, widget(0, 4)); + ((id >> 12) & 0xffff, (id >> 1) & 0x7ff) +} + +fn alive_and_present(board: &TezroBoard, n: u64) -> (bool, bool) { + (read_word(board, link_word(n, LINK_STATUS)) & ALIVE != 0, read_word(board, link_word(n, LINK_AUX)) & PRESENT != 0) +} + +/// The PROM's per-link test: alive and present, except that link 15 is +/// taken regardless when widget 0 is part 0xd100. +fn prom_uses(board: &TezroBoard, n: u64) -> bool { + let (alive, present) = alive_and_present(board, n); + (alive && present) || (n == 15 && widget_zero_part(board).0 == 0xd100) +} + +/// The kernel's: alive, and present unless widget 0 is part 0xd100 with +/// manufacturer 0 and the link is 15. +fn kernel_uses(board: &TezroBoard, n: u64) -> bool { + let (alive, present) = alive_and_present(board, n); + alive && (present || (n == 15 && widget_zero_part(board) == (0xd100, 0))) +} + +/// Links 8 to 15 that pass both tests on their own bits, without either +/// guest's link-15 exception, as a widget mask. +fn live_links(board: &TezroBoard) -> u16 { + (8..16).filter(|&n| alive_and_present(board, n) == (true, true)).fold(0, |m, n| m | 1 << n) +} + +/// A board whose clock the test advances. +fn board_with_clock(cpus: usize) -> (TezroBoard, Arc) { + let now = Arc::new(AtomicU64::new(0)); + let clock = now.clone(); + let board = TezroBoard::with_config(TezroBoardConfig { + payload: &[], + clock: Arc::new(move || clock.load(Ordering::SeqCst)), + flash_path: None, + station_mac: L1Config::default().mac, + timekeeper: Arc::new(Ds1744::new(YearBase::From(1968)).unwrap()), + memory: &MemoryPopulation::default(), + cpus, + chip: &CHIPS[0], + v12: false, + graphics_pool: (2, 32), + graphics_raster: (crate::config::RasterUnit::Cpu, "auto"), + }) + .unwrap(); + (board, now) +} + +/// On the two- and four-processor board widget 0's crossbar shows the +/// hub's link 10 and both PICs' links 11 and 15 alive and present, with no +/// failure mode and arbitration bits 0 and 2 clear, so the PROM and the +/// kernel both take 11 and 15 without leaning on the link-15 exception. +/// Every other link reads empty. A power-on keeps the same picture. +#[test] +fn widget_0_shows_both_pic_links_live_on_the_larger_board() { + for cpus in [2, 4] { + let board = TezroBoard::new(cpus).unwrap(); + for pass in ["built", "after power-on"] { + assert_eq!(widget_zero_part(&board), (0xd100, 0), "{cpus} CPUs, {pass}"); + assert_eq!(read_word(&board, widget(0, 4)) & 1, 1); + assert_eq!(live_links(&board), 1 << 10 | 1 << 11 | 1 << 15, "{cpus} CPUs, {pass}"); + for n in [11, 15] { + assert!(prom_uses(&board, n) && kernel_uses(&board, n), "link {n}, {cpus} CPUs, {pass}"); + assert_eq!(read_word(&board, link_word(n, LINK_AUX)) & FAILURE_MODE, 0, "link {n}"); + assert_eq!(read_word(&board, link_word(n, LINK_ARB_UPPER)) & 0b101, 0, "link {n}: unclaimed"); + } + for n in [8, 9, 12, 13, 14] { + assert_eq!(alive_and_present(&board, n), (false, false), "link {n}, {cpus} CPUs, {pass}"); + assert!(!prom_uses(&board, n) && !kernel_uses(&board, n)); + } + board.power_on(); + } + } +} + +/// The one-processor board has no second PIC: link 11 reads empty and +/// widget 11 does not answer, while link 15 stays usable. +#[test] +fn the_one_processor_board_shows_no_link_11() { + let board = TezroBoard::new(1).unwrap(); + assert_eq!(live_links(&board), 1 << 10 | 1 << 15); + assert_eq!(alive_and_present(&board, 11), (false, false)); + assert!(!prom_uses(&board, 11) && !kernel_uses(&board, 11)); + assert!(prom_uses(&board, 15) && kernel_uses(&board, 15)); + assert_eq!(pio_read(&board.bedrock, widget(11, 4), 4).status, BUS_ERR); +} + +/// Widget 11 answers as a bridge-class PIC with both heads, the identity +/// the PROM's "Found PIC" path needs (manufacturer 0x24, bit 0 set, part +/// 0xd102 on head 0), and takes the PROM's control-word writes on both +/// heads. Widget 15 keeps its own. +#[test] +fn widget_11_answers_as_a_pic_with_both_heads() { + use crate::dev::pic::HEAD_OFFSET; + let board = TezroBoard::new(4).unwrap(); + let io = Pio(board.bedrock.pio()); + for (head, part) in [(0, 0xd102), (HEAD_OFFSET, 0xd112)] { + let id = read_word(&board, widget(11, head + 4)); + assert_eq!((id & 1, (id >> 1) & 0x7ff, (id >> 12) & 0xffff), (1, 0x24, part), "head at {head:#x}"); + // Widget control, as the PROM's table-driven field writes it. + let control = widget(11, head + 0x24); + let before = read_word(&board, control); + assert_eq!(io.pio_write(control, 4, (before & !0xf000) | 0x4000), BUS_OK); + assert_eq!(read_word(&board, control) & 0xf000, 0x4000, "head at {head:#x}"); + } + assert_eq!(read_word(&board, widget(15, 4)) >> 12 & 0xffff, 0xd102); +} + +/// The PROM's bring-up of link 11 (read the status clear, write credit 2, +/// clear both arbitration registers, then the 0xd102 credit 3) leaves the +/// link alive and present and the credit field reading back. +#[test] +fn the_proms_link_bring_up_keeps_link_11_alive() { + let board = TezroBoard::new(4).unwrap(); + let io = Pio(board.bedrock.pio()); + read_word(&board, link_word(11, LINK_STATUS_CLEAR)); + assert_eq!(io.pio_write(link_word(11, LINK_CONTROL), 4, 0x0008_0000), BUS_OK); + assert_eq!(io.pio_write(link_word(11, LINK_ARB_UPPER), 4, 0), BUS_OK); + assert_eq!(io.pio_write(link_word(11, LINK_ARB_LOWER), 4, 0), BUS_OK); + assert_eq!(alive_and_present(&board, 11), (true, true)); + let control = read_word(&board, link_word(11, LINK_CONTROL)); + assert_eq!((control >> 18) & 0x1f, 2); + assert_eq!(io.pio_write(link_word(11, LINK_CONTROL), 4, (control & !(0x1f << 18)) | 3 << 18), BUS_OK); + assert_eq!((read_word(&board, link_word(11, LINK_CONTROL)) >> 18) & 0x1f, 3); + assert_eq!(alive_and_present(&board, 11), (true, true)); + assert_eq!(io.pio_read(widget(11, 4), 4).status, BUS_OK); +} + +/// The processor reaches widget 11 through the crossbar, not around it: a +/// reset of link 11 or of the hub's link 10 takes widget 11 out of reach +/// for the pulse, and it answers again, registers kept, once it ends. A +/// reset of link 15 leaves widget 11 alone. +#[test] +fn widget_11_is_routed_through_widget_0s_crossbar() { + let (board, now) = board_with_clock(4); + let io = Pio(board.bedrock.pio()); + let route = widget(11, 0x130); + assert_eq!(io.pio_write(route, 8, 0x1122_3344_5566_7788), BUS_OK); + for link in [11, 10] { + now.fetch_add(100, Ordering::SeqCst); + assert_eq!(io.pio_write(link_word(link, LINK_RESET), 4, 0), BUS_OK); + assert_eq!(io.pio_read(route, 8).status, BUS_ERR, "down during link {link}'s pulse"); + now.fetch_add(20, Ordering::SeqCst); + let r = io.pio_read(route, 8); + assert_eq!((r.status, r.data), (BUS_OK, 0x1122_3344_5566_7788), "after link {link}'s pulse"); + assert_eq!(alive_and_present(&board, 11), (true, true)); + } + now.fetch_add(100, Ordering::SeqCst); + assert_eq!(io.pio_write(link_word(15, LINK_RESET), 4, 0), BUS_OK); + assert_eq!(io.pio_read(route, 8).status, BUS_OK, "link 15's pulse"); + assert_eq!(io.pio_read(widget(15, 4), 4).status, BUS_ERR); + assert_eq!(board.pics[1].pxbow().unwrap().endpoint_widgets(), 1 << 11 | 1 << 15); + assert!(board.pics[0].pxbow().is_none(), "the second PIC has no crossbar of its own"); +} + +/// Every fitted processor slot reads present, in the board's order (CPU 1 +/// in PI1/A, CPUs 2 and 3 in the B slots), and every fitted processor runs +/// from reset with its enable set, as QEMU's node grants them. +#[test] +fn fitted_processors_read_present_and_start_enabled() { + use crate::dev::bedrock::reg::{pi::{CPU_A_ENABLE, CPU_B_ENABLE, INTERFACE_SLOT_A, INTERFACE_SLOT_B}, PI1_BASE}; + for (cpus, present) in [(1, [[1, 0], [0, 0]]), (2, [[1, 0], [1, 0]]), (4, [[1, 1], [1, 1]])] { + let board = TezroBoard::new(cpus).unwrap(); + assert_eq!(board.processors.count(), cpus); + let regs = board.bedrock.registers(); + for interface in 0..2u64 { + let pi = |at: u64| regs.read64(interface * PI1_BASE + at); + let slots = [pi(INTERFACE_SLOT_A).data, pi(INTERFACE_SLOT_B).data]; + assert_eq!(slots, present[interface as usize], "{cpus} CPUs, PI{interface}"); + let enabled = [pi(CPU_A_ENABLE).data & 1, pi(CPU_B_ENABLE).data & 1]; + assert_eq!(enabled, present[interface as usize], "{cpus} CPUs, PI{interface}"); + } + } +} + +/// Only the counts a board fits are accepted. +#[test] +fn the_board_refuses_a_population_it_cannot_fit() { + assert!(TezroBoard::new(3).is_err()); + assert!(TezroBoard::new(0).is_err()); + assert!(TezroBoard::new(5).is_err()); +} + +/// A board of one processor and one GiB of memory with `chip` fitted. +fn board_with(chip: &Part) -> TezroBoard { + TezroBoard::with_config(TezroBoardConfig { + payload: &[], + clock: Arc::new(|| 0), + flash_path: None, + station_mac: L1Config::default().mac, + timekeeper: Arc::new(Ds1744::new(YearBase::From(1968)).unwrap()), + memory: &MemoryPopulation::one_gib(), + cpus: 1, + chip: self::chip(chip), + v12: false, + graphics_pool: (2, 32), + graphics_raster: (crate::config::RasterUnit::Cpu, "auto"), + }) + .unwrap() +} + +/// Run `body` on a thread with the stack a debug build's processor needs. +fn on_big_stack(body: impl FnOnce() + Send + 'static) { + if let Err(panic) = std::thread::Builder::new().stack_size(64 << 20).spawn(body).unwrap().join() { + std::panic::resume_unwind(panic); + } +} + +/// Each chip resets with its own PRId, FIR and Config word, and keeps them +/// across a CPU reset, which reloads PRId and FIR from the reset values. The +/// board's flash record and Count rate follow the chip's clock. These are the +/// facts the chips grade as inferred or approximated, so each is pinned here. +#[test] +fn each_tezro_chip_resets_with_its_identity_and_clock() { + fn check>>(model: &Part, prid: u32, config0: u32, mhz: u32) { + let board = board_with(model); + let mut cpu = build_cpu::(&board); + let name = model.name(); + assert_eq!((cpu.core.cp0_prid, cpu.core.reset_prid), (prid, prid), "{name}"); + assert_eq!(model.prid(), prid, "{name}: the chip's PRId"); + assert_eq!((cpu.core.fpu_fir, cpu.core.reset_fir), (0x900, 0x900), "{name}"); + assert_eq!(cpu.core.cp0_config & CONFIG0_MODE_MASK, config0, "{name}"); + assert_eq!(cpu.core.cp0_config >> 26, 0b011_011, "{name}: 32 KB primary caches"); + let config = cpu.core.cp0_config; + cpu.core.fpu_fir = 0; + cpu.core.cp0_prid = 0; + cpu.core.reset(false); + assert_eq!((cpu.core.cp0_prid, cpu.core.fpu_fir, cpu.core.cp0_config), (prid, 0x900, config), "{name}: after reset"); + assert_eq!(model.mhz, mhz); + assert_eq!(model.count_hz(), u64::from(mhz) * 500_000, "{name}: Count at half the clock"); + assert_eq!(chip(model).firmware.cpu_clock_hz, mhz * 1_000_000, "{name}: the PROM's clock record"); + } + use crate::cpu::mips_cache_shadow::{R16000Scache16ShadowCache, R16000Scache8ShadowCache, R16000ShadowCache}; + on_big_stack(|| { + check::(&parts::R16000_700_EARLY, 0x0f41, 0x009b_ad85, 700); + check::(&parts::R16000_800, 0x0f42, 0x009b_af85, 800); + check::(&parts::R16000_900, 0x0f50, 0x009c_b185, 900); + check::(&parts::R16000_1000, 0x0f50, 0x00ed_b385, 1000); + }); +} + +/// Every chip's Config word decodes, by the PROM editor's formulas, to the +/// fields its inventory states: SS gives the cache size, SC the printed cache +/// clock (2 * CPU / (SC + 1)) and EC the 200 MHz Bedrock (code 2 * CPU / +/// Bedrock - 1), with K0 5, PM 3, SB and BE set. The 1 GHz word is the only +/// approximated one, and this is what it is built from: SS 5, SC 5, EC 9. +#[test] +fn each_tezro_configs_fields_decode_to_the_stated_sizes_and_clocks() { + for c in &CHIPS { + let word = c.firmware.reset_mode(); + let name = c.chip.name(); + let field = |lo: u32, bits: u32| (word >> lo) & ((1 << bits) - 1); + let mhz = c.chip.mhz; + let bytes = crate::with_cpu!(c.chip, true, false, C => ::L2_SIZE as u32); + assert_eq!((field(0, 3), field(7, 2), field(13, 1), field(15, 1)), (5, 3, 1, 1), "{name}: K0 PM SB BE"); + assert_eq!(field(9, 4), 2 * mhz / 200 - 1, "{name}: EC"); + assert_eq!(2 * mhz / (field(19, 3) + 1), c.scache_mhz, "{name}: SC gives the cache clock"); + assert_eq!(512u32 << 10 << field(16, 3), bytes, "{name}: SS gives the cache size"); + assert_eq!((word >> 23) & 1, 1, "{name}: bit 23 is set in every row of 500 MHz and up"); + assert_eq!(word >> 24, 0, "{name}: bit 24 clear"); + } + let word = chip(&parts::R16000_1000).firmware.reset_mode(); + assert_eq!(((word >> 16) & 7, (word >> 19) & 7, (word >> 9) & 15, (word >> 22) & 1), (5, 5, 9, 1), "1 GHz SS SC EC and bit 22"); +} + +/// A part's revision, as hinv prints it, and the PRId register's raw one +/// differ for an R16000: the register's major is the displayed one plus 2 (the +/// rule Linux and OpenBSD apply). The raw bytes are written out here, not +/// derived, so this ties the Tezro's "2.1", "2.2", "3.0" and "3.0" to 0x0f41, +/// 0x0f42 and 0x0f50, the 1 GHz one also printed by the PROM banner as `f50`. +#[test] +fn every_parts_revision_is_the_one_its_prid_displays() { + for (part, shown, raw) in + CPUS.iter().zip(["2.1", "2.2", "3.0", "3.0"]).zip([0x41, 0x42, 0x50, 0x50]).map(|((p, s), r)| (p, s, r)) + { + assert_eq!(format!("{}.{}", part.rev.0, part.rev.1), shown, "{}", part.label()); + assert_eq!(part.prid(), 0x0f00 | raw, "{}", part.label()); + } +} + +/// A chip and its flash record are two statements of one chip: the clock, +/// and the Config word the PROM compares against the processor's, must agree. +#[test] +fn each_tezro_chip_agrees_with_its_flash_record() { + for c in &CHIPS { + let record = c.firmware; + let name = c.chip.name(); + assert_eq!(u64::from(record.cpu_clock_hz), u64::from(c.chip.mhz) * 1_000_000, "{name}"); + assert_eq!(record.cpu_mode_word & CONFIG0_MODE_MASK, record.reset_mode(), "{name}"); + assert_eq!(board_with(c.chip).chip.chip, c.chip); + assert!(c.cpus.iter().all(|&n| n <= 4) && !c.cpus.is_empty(), "{name}"); + } + assert_eq!(CHIPS.len(), CPUS.len()); + assert!(CHIPS.iter().zip(CPUS).all(|(c, &model)| c.chip == model), "CPUS lists the chips in table order"); +} + +/// The cache type `with_cpu!` builds follows the chip: the secondary-cache +/// size the chip is inventoried with, and nothing else about the type is the +/// chip's. +#[test] +fn the_processors_take_the_cache_type_of_the_chips_size() { + for (model, mib) in [ + (&parts::R16000_700_EARLY, 4u32), + (&parts::R16000_800, 4), + (&parts::R16000_900, 8), + (&parts::R16000_1000, 16), + ] { + let (name, bytes) = crate::with_cpu!(model, true, false, C => (::NAME, ::L2_SIZE as u32)); + assert_eq!((name, bytes >> 20), ("R16000", mib), "{}", model.name()); + } +} + +/// A bare family name builds the 700 MHz chip, and a chip name its own: +/// PRId, flash record and Count rate follow, end to end from a configuration. +#[test] +fn the_family_name_builds_the_700_mhz_chip_and_a_chip_name_its_own() { + on_big_stack(|| { + for (cpu, prid, mhz, mib) in [("r16000", 0x0f41, 700, 4u32), ("r16000_800", 0x0f42, 800, 4), ("r16000_900", 0x0f50, 900, 8), ("r16000_1000", 0x0f50, 1000, 16)] { + let cfg: MachineConfig = toml::from_str(&format!("[machine]\nprofile = \"tezro_ip53\"\ncpu = \"{cpu}\"\n")).unwrap(); + let part = cfg.cpu_part().unwrap(); + let board = board_with(part); + let (core_prid, core_config, l2) = crate::with_cpu!(part, true, false, C => { + let cpu_core = build_cpu::(&board); + (cpu_core.core.cp0_prid, cpu_core.core.cp0_config, ::L2_SIZE as u32) + }); + assert_eq!(core_prid, prid, "{cpu}"); + assert_eq!(l2 >> 20, mib, "{cpu}: secondary cache"); + assert_eq!((core_config >> 16) & 7, 1 + mib.trailing_zeros(), "{cpu}: Config SS"); + assert_eq!(board.chip.firmware.cpu_clock_hz, mhz * 1_000_000, "{cpu}: the flash record"); + assert_eq!(board.chip.firmware.reset_config0(), core_config, "{cpu}: record and processor agree"); + } + }); +} + +/// IOC4 reads its 32-bit registers in the processor's byte order through +/// the IO9 PIC's self-ordered slot: what the processor writes is the +/// register's value, as on the QEMU Tezro, where PROM 6.210 writes the +/// serial DMA control 0x100201ff and the TX producer 0x3f8. +#[test] +fn the_processor_reads_and_writes_ioc4_registers_in_its_own_byte_order() { + let board = TezroBoard::new(1).unwrap(); + let io = Pio(board.bedrock.pio()); + // IOC4's BAR at PCI memory 0x200000, decoding memory, reached through + // head 0's first short window. + assert_eq!(io.pio_write(config_id(1) + 0x10, 4, 0x20_0000), BUS_OK); + assert_eq!(io.pio_write(config_id(1) + 4, 4, 2), BUS_OK); + assert_eq!(io.pio_write(widget(15, 0x200), 8, (1 << 12) | 2), BUS_OK); + let reg = |offset: u64| widget(15, 0x20_0000 + offset); + assert_eq!(io.pio_read(reg(0x24), 4).data, 1, "MISC: the 66 MHz clock"); + assert_eq!(io.pio_write(reg(0x314), 4, 0x3f8), BUS_OK); + assert_eq!(io.pio_read(reg(0x314), 4).data, 0x3f8, "port 0 TX producer"); +} + +// VPro host-interface registers the tests below touch, as byte offsets in +// the board's widget space. The board's own constants are private to it. +const ODSY_IDENTITY: u64 = 0x00; + +fn board_with_v12(cpus: usize) -> TezroBoard { + TezroBoard::with_config(TezroBoardConfig { + payload: &[], + clock: Arc::new(|| 0), + flash_path: None, + station_mac: L1Config::default().mac, + timekeeper: Arc::new(Ds1744::new(YearBase::From(1968)).unwrap()), + memory: &MemoryPopulation::default(), + cpus, + chip: &CHIPS[0], + v12: true, + graphics_pool: (2, 32), + graphics_raster: (crate::config::RasterUnit::Cpu, "auto"), + }) + .unwrap() +} + +/// The V12 answers as widget 12 behind the PXBOW on either board, and +/// widget 12 stays empty without one. (On the one-processor board the +/// placement is inferred; see V12_WIDGET.) +#[test] +fn the_v12_answers_at_widget_12_on_both_boards() { + for cpus in [1, 4] { + let board = board_with_v12(cpus); + let id = pio_read(&board.bedrock, widget(12, ODSY_IDENTITY), 4); + let direct = board.v12.as_ref().unwrap().read32(ODSY_IDENTITY); + assert_eq!((id.status, id.data), (BUS_OK, u64::from(direct.data)), "{cpus} CPUs"); + assert_eq!(board.pics[board.pics.len() - 1].pxbow().unwrap().endpoint_widgets() & (1 << 12), 1 << 12); + } + assert_eq!(pio_read(&TezroBoard::new(1).unwrap().bedrock, widget(12, ODSY_IDENTITY), 4).status, BUS_ERR); +} + +/// The monitor's `vpro` commands resolve on a Tezro that has a V12, and are +/// unknown on one that has none. +#[test] +fn the_monitor_resolves_vpro_commands_on_a_tezro_with_a_v12() { + let resolves = |board: &TezroBoard, command: &str| { + board.monitor_devices().iter().any(|d| d.register_commands().iter().any(|(c, _)| c == command)) + }; + let with = board_with_v12(4); + assert!(resolves(&with, "vpro"), "a Tezro with a V12 answers `vpro`"); + assert!(resolves(&with, "ps2"), "the other board commands stay"); + let without = TezroBoard::new(1).unwrap(); + assert!(!resolves(&without, "vpro"), "a Tezro with no V12 has no `vpro`"); + assert!(resolves(&without, "ps2")); +} + +/// The cached view the processors' caches sit on swaps RAM above 4 GiB, so +/// SC stays atomic there too. +#[test] +fn the_cached_view_swaps_ram_above_4_gib() { + let board = TezroBoard::new(4).unwrap(); + // Banks 0 to 5: two 1 GiB banks and four of 512 MiB, as MD's tests set them. + assert_eq!(board.bedrock.registers().write64(crate::dev::bedrock::reg::MD_BASE + 8, 0x53 | (0x43 << 8) | (0x43 << 16)), BUS_OK); + let cached = board.cached_bus(); + let at = (5u64 << 30) + 0x80; + assert_eq!(cached.write64(at, 1), BUS_OK); + assert_eq!(cached.cas64(at, 1, 2), Some(true)); + assert_eq!(cached.cas64(at, 1, 3), Some(false)); + assert_eq!(cached.read64(at).data, 2); +} + +/// A fresh flash holds no environment records: with the node board named +/// for its processor count the PROM tries only the fitted positions, so no +/// board needs a Disable record to boot straight through. +#[test] +fn a_fresh_flash_starts_with_no_environment_records() { + for cpus in [1, 2, 4] { + let bytes = TezroBoard::new(cpus).unwrap().flash.contents(); + assert!(bytes[0x9e0100..0x9e0200].iter().all(|&b| b == 0xff), "{cpus} CPUs"); + } +} diff --git a/src/irq.rs b/src/irq.rs index 6760ddbd2..a5810be61 100644 --- a/src/irq.rs +++ b/src/irq.rs @@ -15,6 +15,15 @@ pub trait InterruptController: Send + Sync { /// Drive input `line` to `level`. An input the controller latches takes /// a rising level as its edge. fn set_line(&self, line: u32, level: bool); + + /// A crosstalk interrupt packet that widget `source` sent to `address`, + /// naming vector `line`; whether the host took it. HEART and the Hub + /// decode the vector alone, as this default does. A host whose packet + /// address picks the destination overrides it. + fn packet(&self, _source: u8, _address: u64, line: u32, level: bool) -> bool { + self.set_line(line, level); + true + } } /// One input of one controller, as a device sees it. diff --git a/src/lib.rs b/src/lib.rs index e09369076..5772763c7 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -190,6 +190,8 @@ pub mod ip27_onyx2; pub mod ip27_origin2000; /// Fuel has an IP34 board and uses the IP35 software platform and processor views. pub mod ip35; +pub mod ip34_machine; +pub mod ip53_machine; pub mod machine_state; pub mod platform; pub mod physical; diff --git a/src/machine.rs b/src/machine.rs index 62566f107..0cb6cf311 100644 --- a/src/machine.rs +++ b/src/machine.rs @@ -53,7 +53,7 @@ pub fn set_emulator_profile(profile: MachineProfile) { fn emulator_profile() -> Option { let raw = EMULATOR_PROFILE.load(std::sync::atomic::Ordering::Relaxed); - [MachineProfile::IndyIp24, MachineProfile::Indigo2Ip22, MachineProfile::Indigo2Ip28, MachineProfile::O2Ip32, MachineProfile::OctaneIp30, MachineProfile::Origin200Ip29, MachineProfile::Origin2000Ip27, MachineProfile::Onyx2Ip27] + [MachineProfile::IndyIp24, MachineProfile::Indigo2Ip22, MachineProfile::Indigo2Ip28, MachineProfile::O2Ip32, MachineProfile::OctaneIp30, MachineProfile::Origin200Ip29, MachineProfile::Origin2000Ip27, MachineProfile::Onyx2Ip27, MachineProfile::FuelIp34, MachineProfile::TezroIp53] .into_iter().find(|p| *p as u8 == raw) } @@ -71,6 +71,8 @@ fn emulator_name_thirds(profile: Option) -> &'static [&'static s Some(MachineProfile::Origin200Ip29) => &["IRIX", "Iris", "IPsomething", "IP29"], Some(MachineProfile::Origin2000Ip27) => &["IRIX", "Iris", "IPsomething", "IP27"], Some(MachineProfile::Onyx2Ip27) => &["IRIX", "Iris", "IPsomething", "IP27"], + Some(MachineProfile::FuelIp34) => &["IRIX", "Iris", "IPsomething", "IP34"], + Some(MachineProfile::TezroIp53) => &["IRIX", "Iris", "IP53"], None => &["IRIX", "Iris", "IPsomething"], } } @@ -2166,11 +2168,13 @@ mod emulator_name_tests { Some(MachineProfile::Origin200Ip29) => &["Indy", "Indigo2", "IP24", "IP22", "IP28", "IP30", "IP32", "O2"], Some(MachineProfile::Origin2000Ip27) => &["Indy", "Indigo2", "IP24", "IP22", "IP28", "IP30", "IP32", "O2"], Some(MachineProfile::Onyx2Ip27) => &["Indy", "Indigo2", "IP24", "IP22", "IP28", "IP30", "IP32", "O2"], + Some(MachineProfile::FuelIp34) => &["Indy", "Indigo2", "IP24", "IP22", "IP28", "IP30", "IP32", "O2"], + Some(MachineProfile::TezroIp53) => &["Indy", "Indigo2", "IP24", "IP22", "IP28", "IP30", "IP32", "IP34", "O2"], None => &["Indy", "Indigo2", "IP24", "IP22", "IP28", "IP30", "IP32", "O2"], } }; for p in [None, Some(MachineProfile::IndyIp24), Some(MachineProfile::Indigo2Ip22), - Some(MachineProfile::Indigo2Ip28), Some(MachineProfile::O2Ip32), Some(MachineProfile::OctaneIp30), Some(MachineProfile::Origin200Ip29), Some(MachineProfile::Origin2000Ip27), Some(MachineProfile::Onyx2Ip27)] { + Some(MachineProfile::Indigo2Ip28), Some(MachineProfile::O2Ip32), Some(MachineProfile::OctaneIp30), Some(MachineProfile::Origin200Ip29), Some(MachineProfile::Origin2000Ip27), Some(MachineProfile::Onyx2Ip27), Some(MachineProfile::FuelIp34), Some(MachineProfile::TezroIp53)] { let thirds = emulator_name_thirds(p); assert!(!thirds.is_empty()); for w in thirds { @@ -2182,7 +2186,7 @@ mod emulator_name_tests { #[test] fn profile_round_trips_through_the_global() { - for p in [MachineProfile::IndyIp24, MachineProfile::Indigo2Ip22, MachineProfile::Indigo2Ip28, MachineProfile::O2Ip32, MachineProfile::OctaneIp30, MachineProfile::Origin200Ip29, MachineProfile::Origin2000Ip27, MachineProfile::Onyx2Ip27] { + for p in [MachineProfile::IndyIp24, MachineProfile::Indigo2Ip22, MachineProfile::Indigo2Ip28, MachineProfile::O2Ip32, MachineProfile::OctaneIp30, MachineProfile::Origin200Ip29, MachineProfile::Origin2000Ip27, MachineProfile::Onyx2Ip27, MachineProfile::FuelIp34, MachineProfile::TezroIp53] { set_emulator_profile(p); assert_eq!(emulator_profile(), Some(p)); } diff --git a/src/pci/bus.rs b/src/pci/bus.rs index 55f116203..b4ae1304f 100644 --- a/src/pci/bus.rs +++ b/src/pci/bus.rs @@ -206,7 +206,8 @@ impl PciBus { } /// A configuration write of `width` bytes at byte `off`: only those bytes - /// change, and the header decides which of their bits stick. False if + /// change, and the header decides which of their bits stick. A function + /// may take a whole-dword write itself ([`PciFunction::config_write`]). False if /// nothing answers on a root bus; behind a bridge the write is discarded by /// master abort, as a read of nothing is (see [`PciBus::config_read`]), and /// counts as answered. The mappings are updated when the command register, @@ -217,6 +218,9 @@ impl PciBus { crate::dlog!(LogModule::Pci, "pci: bus {} {}.{} write {off:#04x}/{width} {val:#x}{}", self.0.number, at.device, at.function, if f.is_some() { "" } else { ", no answer" }); let Some(f) = f else { return self.master_abort() }; + if width == 4 && f.config_write(off / 4, val) { + return true; + } let (cfg, reg, shift) = (f.state().config(), off / 4, (off & 3) * 8); let mask = (u32::MAX >> (32 - 8 * width)) << shift; let old = cfg.read(reg); @@ -544,6 +548,39 @@ mod tests { assert!(b.occupied(3) && !b.occupied(4)); } + /// A function whose dword 0x1e is a window into the chip: a whole-dword + /// write goes to the chip every time, even when it repeats a value, and + /// the header stores nothing; a narrower one takes the header's path. + #[test] + fn a_function_takes_a_whole_dword_configuration_write_itself() { + struct Window(crate::pci::PciFunctionState, Mutex>); + impl PciFunction for Window { + fn state(&self) -> &crate::pci::PciFunctionState { + &self.0 + } + fn bar_read(&self, _: usize, _: u32, _: usize) -> u32 { + 0 + } + fn bar_write(&self, _: usize, _: u32, _: u32, _: usize) {} + fn config_write(&self, reg: usize, val: u32) -> bool { + (reg == 0x1e).then(|| self.1.lock().unwrap().push(val)).is_some() + } + } + let (_h, b) = bus(); + let w = Arc::new(Window(crate::pci::PciFunctionState::new(PciConfig::new(1, 2, 0, 1, bars(), PCI_INTA)), Mutex::new(Vec::new()))); + b.attach(at(1, 0), w.clone()); + assert!(b.config_write(at(1, 0), 0x78, 4, 0x1234_5678)); + assert!(b.config_write(at(1, 0), 0x78, 4, 0x1234_5678)); + assert_eq!(*w.1.lock().unwrap(), [0x1234_5678, 0x1234_5678], "a repeated value reaches the chip again"); + assert_eq!(b.config_read(at(1, 0), 0x78, 4), Some(0), "the header stored nothing"); + assert!(b.config_write(at(1, 0), 0x78, 2, 0xffff)); + assert_eq!(w.1.lock().unwrap().len(), 2, "a halfword write is the header's"); + // Another dword is the header's too. + assert!(b.config_write(at(1, 0), 0x3c, 4, 0x0000_00ab)); + assert_eq!(b.config_read(at(1, 0), 0x3c, 1), Some(0xab)); + assert_eq!(w.1.lock().unwrap().len(), 2); + } + #[test] fn config_size_probe_and_masks() { let (_h, b) = bus(); diff --git a/src/pci/function.rs b/src/pci/function.rs index d239d1e62..a7d12e7d3 100644 --- a/src/pci/function.rs +++ b/src/pci/function.rs @@ -73,6 +73,15 @@ pub trait PciFunction: Send + Sync { /// shared core, and the SCSI controllers drive their pins. fn attached(&self) {} + /// A configuration write of the whole dword `reg`, offered before the + /// header sees it. A function whose chip computes the register (the + /// indirect windows of a NIC) takes it here, returns true, and the header + /// stores nothing; every other function leaves it to the header. Narrower + /// writes always take the header's path. Called with no header lock held. + fn config_write(&self, _reg: usize, _val: u32) -> bool { + false + } + /// The bus wrote configuration dword `reg` of this function's header and /// it changed from `old` to `new`. A PCI-to-PCI bridge acts on bridge /// control here; every other function has nothing to do. diff --git a/src/ppmem/ppmem.rs b/src/ppmem/ppmem.rs index 4ec2c6ad9..e15af6d79 100644 --- a/src/ppmem/ppmem.rs +++ b/src/ppmem/ppmem.rs @@ -703,6 +703,9 @@ struct SpaceState { bits: u64, /// Each CPU's cached view of memory: see [`PpMemSpace::add_cached_view`]. views: Vec, + /// Ranges where a held page may stand with no memory mapped behind it: + /// see [`PpMemSpace::allow_unbacked_lines`]. + unbacked: Vec<(u64, u64)>, } /// One CPU's cached view of memory: a window of its own, with every mapping @@ -777,6 +780,7 @@ impl PpMemSpace { sink2, bits: 0, views: Vec::new(), + unbacked: Vec::new(), }), }) } @@ -964,7 +968,7 @@ impl PpMemSpace { let st = unsafe { &mut *self.state.get() }; let gran = granularity() as u64; let page = pa & !(gran - 1); - if !st.mappings.iter().any(|m| m.at <= page && m.at + m.len >= page + gran) { + if !self.backs(page) && !st.unbacked.iter().any(|&(a, b)| page >= a && page + gran <= b) { return None; } let shift = window_shift(st.window); @@ -991,6 +995,23 @@ impl PpMemSpace { Some(true) } + /// Whether memory is mapped behind the whole page holding `pa`. A page + /// [`Self::hold_line_page`] takes without it starts as zeros, for the + /// caller to fill. + pub fn backs(&self, pa: u64) -> bool { + let st = unsafe { &*self.state.get() }; + let gran = granularity() as u64; + let page = pa & !(gran - 1); + st.mappings.iter().any(|m| m.at <= page && m.at + m.len >= page + gran) + } + + /// Let held pages in `[start, end)` stand with no memory mapped behind + /// them: a PROM that keeps its stack and code in cache lines before it + /// has set up memory. + pub fn allow_unbacked_lines(&self, start: u64, end: u64) { + unsafe { (*self.state.get()).unbacked.push((start, end)) }; + } + /// One line of [`Self::hold_line_page`]'s page is no longer valid. /// Returns true when it was the page's last: the view shows memory there /// again, and what the page held is dropped, as a line invalidated @@ -2060,4 +2081,21 @@ mod tests { space.release_line_page(b, 0x2040); assert_eq!(gen(at()) / GEN_HELD, 0); } + + /// With no memory behind it a page is refused unless the machine allows + /// lines there, and then its region is the view's until the page goes. + #[test] + fn an_unbacked_page_needs_the_machines_leave() { + let (space, _banks) = PpMemSpace::with_bank_sizes(&[64]).unwrap(); + let (v, field) = view_of(&space); + assert_eq!(space.hold_line_page(v, 0x10_0000), None); + space.allow_unbacked_lines(0, 64 * MB as u64); + assert!(!space.backs(0x10_0000)); + assert_eq!(space.hold_line_page(v, 0x10_0000), Some(true)); + assert_eq!(*field, 1); + assert_eq!(space.mapped_bitmap(), 0, "the shared window is unchanged"); + assert_eq!(word(space.view_base(v), 0x3ff_fffc), 0, "the region is safe to touch"); + space.release_line_page(v, 0x10_0000); + assert_eq!(*field, 0); + } } diff --git a/src/scsi.rs b/src/scsi.rs index bfb2ffda8..eb82e3347 100644 --- a/src/scsi.rs +++ b/src/scsi.rs @@ -162,6 +162,7 @@ pub struct ScsiDevice { kind: DeviceKind, buffer: Vec, // Internal buffer for WRITE_BUFFER/READ_BUFFER commands pending_sense: [u8; 18], // Stored sense data, served on REQUEST_SENSE + pending_attention: Option, /// Physical sector size: always 2048 for CD-ROM (Yellow Book), 512 for HDD. /// Used for TOC lead-out LBA (CD spec requires TOC in physical sectors). phys_block_size: u64, @@ -222,6 +223,7 @@ impl ScsiDevice { kind: if is_cdrom { DeviceKind::Cdrom } else { DeviceKind::Disk }, buffer: vec![0u8; SCSI_BUFFER_SIZE], pending_sense: [0u8; 18], + pending_attention: None, phys_block_size: if is_cdrom { CDROM_SECTOR } else { 512 }, // CD-ROM drives default to 2048-byte logical blocks (Sony CDU-76S behaviour). // HDD defaults to 512. dksc switches CD-ROM to 512 for EFS, back to 2048 for ISO. @@ -242,6 +244,7 @@ impl ScsiDevice { kind: DeviceKind::Cdrom, buffer: vec![0u8; SCSI_BUFFER_SIZE], pending_sense: [0u8; 18], + pending_attention: None, phys_block_size: CDROM_SECTOR, logical_block_size: 2048, identity, @@ -326,6 +329,10 @@ impl ScsiDevice { ScsiResponse { status: 0x02, data: vec![] } } + fn take_attention(&mut self) -> Option { + self.pending_attention.take().or_else(|| self.media.take_changed().then_some(0x28)) + } + fn exec_test_unit_ready(&mut self, _cdb: &[u8]) -> Result { if self.media.take_changed() { // Sense key 0x06 UNIT_ATTENTION, ASC 0x28 "Not Ready to Ready Transition / Medium Changed" @@ -1118,11 +1125,10 @@ impl ScsiTarget for ScsiDevice { } // Unit attention fires before any other command (except REQUEST_SENSE/INQUIRY). - if req.cdb[0] != scsi_cmd::REQUEST_SENSE - && req.cdb[0] != scsi_cmd::INQUIRY - && self.media.take_changed() - { - return Ok(self.check_condition(0x06, 0x28, 0x00)); + if req.cdb[0] != scsi_cmd::REQUEST_SENSE && req.cdb[0] != scsi_cmd::INQUIRY { + if let Some(asc) = self.take_attention() { + return Ok(self.check_condition(0x06, asc, 0x00)); + } } let mut response = match req.cdb[0] { @@ -1268,6 +1274,12 @@ impl ScsiTarget for ScsiDevice { fn block_size(&self) -> Option { Some(self.logical_block_size) } + fn bus_reset(&mut self) { + self.pending_sense = [0; 18]; + self.pending_sense[0] = 0x70; + self.pending_attention = Some(0x29); + } + fn reset_block_size(&mut self, cdrom_512: bool) { if cdrom_512 && self.is_cdrom() { self.logical_block_size = 512; @@ -1867,12 +1879,38 @@ mod tests { cd.changer().unwrap().load(&chd.display().to_string()).unwrap(); assert!(cd.has_media()); assert_eq!(cd.size(), 16 * 2048, "the CHD's size, not the file's"); - // The medium change is reported once, then the disc reads. + // A reset is reported before a media change that was already pending. + crate::scsi::ScsiTarget::bus_reset(&mut cd); + let reset = cd.request(&request(&[0, 0, 0, 0, 0, 0], None)).unwrap(); + assert_eq!(reset.status, 0x02, "unit attention after the reset"); + let reset_sense = cd.request(&request(&[0x03, 0, 0, 0, 18, 0], None)).unwrap(); + assert_eq!(reset_sense.data[12], 0x29, "reset attention comes first"); + let tur = cd.request(&request(&[0, 0, 0, 0, 0, 0], None)).unwrap(); - assert_eq!(tur.status, 0x02, "unit attention after the change"); + assert_eq!(tur.status, 0x02, "the pending media change follows"); + let sense = cd.request(&request(&[0x03, 0, 0, 0, 18, 0], None)).unwrap(); + assert_eq!(sense.data[12], 0x28, "media change keeps its ASC"); let read = cd.request(&request(&[0x28, 0, 0, 0, 0, 3, 0, 0, 1, 0], None)).unwrap(); assert_eq!(read.status, 0x00); assert_eq!(read.data, vec![0x42u8; 2048]); let _ = std::fs::remove_dir_all(dir); } + #[test] + fn bus_reset_reports_unit_attention_once_and_keeps_block_size() { + let mut dev = cdrom(); + dev.logical_block_size = 512; + dev.set_sense(0x05, 0x20, 0x00); + crate::scsi::ScsiTarget::bus_reset(&mut dev); + + let old_sense = run(&mut dev, &[0x03, 0, 0, 0, 18, 0]); + assert_eq!((old_sense.data[2], old_sense.data[12]), (0, 0), "reset clears previous sense"); + let response = run(&mut dev, &[0x00, 0, 0, 0, 0, 0]); + assert_eq!(response.status, 0x02, "a bus reset arms unit attention"); + let sense = run(&mut dev, &[0x03, 0, 0, 0, 18, 0]); + assert_eq!(sense.data[12], 0x29, "the reset attention has the reset ASC"); + + let response = run(&mut dev, &[0x00, 0, 0, 0, 0, 0]); + assert_eq!(response.status, 0x00, "the attention is reported once"); + assert_eq!(dev.block_sizes().1, 512, "bus reset leaves the selected block size alone"); + } } diff --git a/src/scsi/bus.rs b/src/scsi/bus.rs index 4283d763f..cb978d19a 100644 --- a/src/scsi/bus.rs +++ b/src/scsi/bus.rs @@ -51,8 +51,15 @@ pub trait ScsiTarget: Send { /// The logical block size, for a target that has one. fn block_size(&self) -> Option { None } - /// A controller's bus reset: block size back to what the controller - /// leaves, with a CD-ROM at 512 bytes if `cdrom_512`. + /// A SCSI bus reset reached this target, which owes the initiator a + /// unit attention on its next command. The ISP12160 calls this when the + /// guest resets either of its buses; the WD33C93A, AIC-7880 and ISP1040 + /// models never told their targets about a reset and still do not. + fn bus_reset(&mut self) {} + + /// The AIC-7880's bus reset: block size back to what that controller + /// leaves, with a CD-ROM at 512 bytes if `cdrom_512`. Only the AIC-7880 + /// calls this. fn reset_block_size(&mut self, _cdrom_512: bool) {} /// Put the mode parameters a guest can change back to what the target diff --git a/src/workstation.rs b/src/workstation.rs index 2bd5798da..63cc1472e 100644 --- a/src/workstation.rs +++ b/src/workstation.rs @@ -40,6 +40,8 @@ use crate::traits::{Device, MachineEvent}; pub fn build(cfg: MachineConfig) -> Result, String> { crate::machine::set_emulator_profile(cfg.machine.profile); match cfg.machine.profile { + crate::config::MachineProfile::FuelIp34 => Ok(Box::new(crate::ip34_machine::FuelMachine::new(&cfg)?)), + crate::config::MachineProfile::TezroIp53 => Ok(Box::new(crate::ip53_machine::TezroMachine::new(&cfg)?)), // The O2 is its own machine (CRIME, MACE, GBE), not an MC/HPC3 one. crate::config::MachineProfile::O2Ip32 => Ok(Box::new(crate::ip32_machine::O2Machine::new(&cfg)?)), // So is the Octane (HEART, Xbow, Bridge). diff --git a/tezro.toml.example b/tezro.toml.example new file mode 100644 index 000000000..e3829b156 --- /dev/null +++ b/tezro.toml.example @@ -0,0 +1,43 @@ +# SGI Tezro (IP53 board, IP35 software platform), serial-only setup. +# Copy to tezro.toml and run from the repository root: iris --config tezro.toml +# Create ignore/tezro/ and supply your decoded raw IP35 PROM 6.210 payload +# and an IRIX disk installed on a Tezro: a disk installed on another machine +# carries that machine's device numbering. +# No PROM or disk images are included. Convert QCOW2 disks to raw first. + +prom = "ignore/tezro/ip35prom.payload.bin" +nvram = "ignore/tezro/nvram.bin" +# banks left unset: 4 GiB as 1024, 1024 and four 512 MiB banks. +headless = true +no_audio = true +monitor_port = 18888 +serial_port_a = 18880 +serial_port_b = 18881 + +[machine] +profile = "tezro_ip53" +# Optional; left out, or as the family "r16000", it is the 700 MHz R16000, +# "r16000_700". "r16000_800", "r16000_900" and "r16000_1000" are the others (four +# processors only); other values are refused. +cpu = "r16000_700" +# Processors fitted: 1, 2 or 4 (the default). Each runs on its own host thread. +cpus = 4 + +[graphics] +board = "none" + +[tezro] +flash = "ignore/tezro/flash.bin" +console = "l1" +l1_port = 18882 + +[network] +# Disconnected cable. For NAT, use mode = "nat". +mode = "off" + +[scsi.1] +path = "ignore/tezro/irix.raw" +cdrom = false +# Keep the base disk unchanged. Guest writes go to irix.raw.overlay. +overlay = true +# write_cache = false # default true. false stores every write before it completes, like SGI's drives (slower); true can lose ~5 s of writes in a host crash. See STORAGE.md