Repository navigation
ip27: add the Origin 200, Origin 2000 and Onyx2 - #455
Merged
Merged
Conversation
PCI functions need different writable and write-one-to-clear header fields. Describe those fields in one construction policy while keeping the current header defaults for existing functions. Forward narrow Bridge configuration writes with their byte enables so write-one-to-clear bits are handled correctly when a function supplies them. Existing Octane functions do not use that status policy, so their behavior stays unchanged.
Fuel fits a DP83843 at MII management address 1, where the Octane has an ICS1890. The IOC3 Ethernet controller needs the same packet and link machinery with either PHY, so keep that machinery shared: move the Octane's ICS1890 register model behind a PHY interface, and supply the DP83843's register and negotiation behavior through it. The MAC keeps its existing packet path. The interface carries the snapshot hooks too: the ICS1890 keeps the register save and load it had in the IOC3 model, and the DP83843 saves its registers and negotiation timers the same way, so the shared Ethernet block saves whichever PHY it holds.
Fuel stores its clock and nonvolatile configuration in an M48T35 on the IOC3 byte bus. Keep the register overlay and battery-backed bytes together so latching and persistence share one owner.
Fuel attaches an M48T35 and has different PCI header policy from the Octane. Carry those choices in the population configuration and wire the timekeeper once so byte-bus accesses need no mutable attachment slot. The Fuel's onboard IOC3 also carries the PS/2 keyboard and mouse controller: OpenBSD configures a keyboard controller through the onboard IOC3 (pci/ioc.c, dev/iockbc.c), and the controller core is the one the Octane already has. Turn the controller on in the Fuel's population, as the Octane's is, and test it by sending a host key through the IOC3 registers: it latches the keyboard interrupt and arrives in the K_RX register, and a mouse command answers from its own port, both before and after a chip reset.
Fuel boot flash uses AMD unlock, program and erase commands. Keep that state machine separate from array storage and bus decode so its command handling can serve each fitted flash array. The Intel and AMD command sets need the same rules for seeding and writing flash images, so share the file storage while each command engine keeps its own command state.
Fuel PROM 6.210's memory test fails when uncached SB and SH reach the bus carrying only their low bits. Its L1 console shows: mtest_bank_main failed: Walking Address RSLT mtest_bank_mai FAIL diag_rc = 43 Walking Address In both memory banks the expected halfwords 0xa55a, 0x5aa5, 0x9999 and 0x6666 read back from the ECC check-bit view as 0x0000. The PROM stores each check word with SH from a register whose low 16 bits are zero; the word sits higher in the register, at a position that depends on the tested address. With the whole register visible to the ECC view and the same PROM configuration it reports ALL DIAGS PASSED and enables all 512 MiB. Add BusDevice::write_narrow_from, whose default stores the architectural byte or halfword exactly as write8/write16 do, and call it for every uncached SB and SH. Ordinary devices therefore still see the low bits of the register, and only a device that models a path observing the other lanes overrides it; Bedrock's ECC view does, in its own commit. A partial-word write drives all of SysAD[63:0] in its data cycle, but the R10000 User's Manual (v2.0, p. 104) defines only the lanes the address and size select and says nothing of the others. Debug writes keep the ordinary-width path, and the JIT callouts pass the same source register. The call replaces the one to write8 or write16 and adds only a size branch.
The module system controller keeps a 2048-byte store on the module's I2C bus. It answers eight consecutive addresses, one 256-byte block each, with a one-byte word address, as a 24C16-class part does. The top block holds the controller's own fields: validity marker, password, debug switches, configuration byte, module number and clock select. A store with no backing file starts as a master controller leaves it after first power-up. A backing file must be exactly 2048 bytes and its contents win; a missing one is created holding the fresh image. The file is replaced whole when a transfer ends with changed bytes, so the store survives the process. The controller reads and writes its own fields through a direct access that moves the address counter as a real transfer would.
The Origin 200 Hub drives its I2C bus through a Philips PCF8584, two registers eight bytes apart. Model its register file, the ESO/ES1/ES2 selection, the receive pipeline that returns the held byte before fetching the next, repeated starts, the programmed own address and the general call, in front of an untimed private bus. A byte completes inside the register access that starts it. Every access width is one register access, since the PROM reads the data register with 8-byte loads and a split would advance the pipeline twice. Targets are serviced around each access, so a grant a target publishes lands in the data register the way the arbitration protocol expects. The interrupt output is driven with no lock held.
The Origin 200's PROM and kernel talk to a module system controller over I2C: checksummed ASCII commands for power, reset, NMI, the heartbeat watchdog and console selection, arbitration grants that tell each processor when it may use the bus, and an AUX serial port with an operator command line. Without it a guest cannot even learn the module number or power the machine off. No specification of the protocol exists, so every timing, the version text, the grant schedule and the reply text are model choices, labelled as such in the module doc. Time is virtual and run lazily from a Clock, so tests drive it with a manual clock and never wait. Power, reset and NMI are reported as events for the machine to drain. The Origin 2000 build, the second module's controller and the shared four-adapter bus are left for a module-management device.
The SN0 node controller's unsupported-access messages need a module of their own to switch on from the monitor. Bedrock's would mislabel them, so add one.
The Hub is the node controller of the Origin 200, Origin 2000 and Onyx2. Its processor interface carries what the PROM and the kernels poll first: the two pending-interrupt banks and their masks, the 1.25 MHz real-time counter with one comparator per processor, processor presence, enable, NMI and soft reset, the cached-alias size, the graphics flow-control registers and the error record of a failed processor read. The polled registers (pending banks, counter origin, comparator pending and enable) are atomics, so a waiting processor takes no lock. The interrupt lines are levels recomputed on every change and published under a dispatch lock. The graphics registers reuse the Odyssey PiGfx block and count no stores. hub.rs holds the hooks a machine implements and an optional observer for the stores a processor rendezvous would watch. Reset values, the effect of a cleared enable and the other approximations are listed in the module header. The cases follow the register contract of the model this is ported from.
Firmware configures memory and reads the node's board record through this block, and the PROM reads the slot identity and writes progress codes to the indicators. The configuration register is masked storage, the error cluster reads zero because the model latches no memory error, and the DIMM registers are write-only. The one-wire master drives the shared w1 bus and completes a slot one microsecond after it starts, so a guest polling for done never sees it inside the starting write. The slot identity is whatever the board drives and is supplied by the machine.
The Hub's XIO port is where the Bridge or Crossbow hangs. This adds its widget registers, access and link control, the seven big-window translation entries and the two block transfer engines. The translation entries are the decode table for the big windows: an access reads its entry with one atomic load and the widget slots are write-once, so re-aiming a window needs no lock and no mode flag. A window aimed at an absent widget reads zero and logs, since guests disable windows that way. The engines run a transfer inside the store that launches it, through a node fabric trait the Hub implements over its claim map, and complete with a notification write or an interrupt-modify on the Hub holding the named node number.
The network interface holds the node number the boot master assigns, the link state, vector PIO and the routing tables. Firmware uses vector PIO to reach a Hub before node numbers exist. Two Hubs joined by a direct cable are the only fabric: a zero route reaches the neighbour, and a nonzero route completes with the address error type because there is no Router to carry it. A request nothing answers leaves the status untouched, so the requester's poll times out. Link up, width and the never-came-out-of-reset reason follow the cable, because the guest kernel treats a down link without that reason as a lost CrayLink. The two ends never hold their locks across a call to each other. Protection, ageing and the diagnostic bits other than the two that act are stored and never consulted.
The sections only become a Hub behind one 8 MiB window. Doubleword accesses dispatch by section, a 32-bit access names half of the aligned doubleword as on a big-endian bus, and the I2C controller and bring-up UART answer narrow accesses over the memory block. A node's address decode is rebuilt when something programs it, never tested per access. The PROM numbers the nodes after reset, so each Hub keeps a table from node number to the Hub that answers it (itself first, then the neighbour across the cable). The claim, processor memory and I/O views read that table, and the cached alias is a size compare. For the Bridge the Hub offers the minimum: an inbound XIO face for DMA and interrupt-modify writes, and an interrupt target whose lines are interrupt numbers. Failed processor reads are recorded through a wrapper over the downstream I/O space. The observer reports the stores a processor rendezvous would watch without changing any value.
The Origin 200 Hub reaches its system controller and the controller's NVRAM over I2C, and nothing in iris models that bus. Add a small one: a master starts at an address, the bus selects the target that claims it (every target that takes a general call for address 0), and bytes then move one call at a time. A repeated start to the selected address keeps the selection. Targets lock their own state and never call back into the master, so the master's lock is the only one held across a transfer. A target can also ask its master to put an address-only transaction on the wire or to end the open transfer, which the controller's arbitration grants and its deferred reset and power commands need.
An SN0 node's RAM sits at fixed bank strides, its boot flash at the reset address, and the processor reaches four uncached spaces selected by address bits 58:57. This adds the node's memory on a decode table (with the ppmem window the cache and JIT read), the Hub-special space with its memory and flash windows and the ECC and directory back doors, and the bus that indexes the four spaces with no per-access decision.
One Hub node with an R10000 or R12000, the node flash and memory, the module controller on the Hub's I2C, and the IP29 System Board's I/O: a Bridge straight on the Hub's XIO port with two ISP1040s, an IOC3 and the board's one-wire record. Selected with profile = "origin200_ip29". Ip27Machine is generic over an Ip27Board so another SN0 product can reuse the node; the Origin 200 is the one board. The Hub's CPU enable is set at power-on because the stock PROM never stores it. The module controller's configuration store comes from the controller's own constructor through one function the machine does not look inside. The Bridge reaches the Hub through its inbound face and interrupt target, the smallest adapter it needs until the Bridge learns the Hub as a master widget.
The Origin 200 System Board's Ethernet part is a DP83840A, which the PROM's transceiver test checks register by register: identity, the negotiation and PCS reset values, and the three loopback switches. The existing DP83843 model has another identity and other reset values, so the test failed against it. Out of reset the link is down; the first status read arms a two-second settle, and a control write links at once because there is no cable to negotiate against.
The System Board's Ethernet transceiver answers at management address 31, not at the address the Octane and Fuel boards use, so the PROM's register probe read an absent part. Ioc3Eth::new_origin200 puts the DP83840A there.
Add the sample configuration, a guide with the machine's limits (one node, the Bridge's master widget, the logged Hub stores), and the feature-list and changelog entries.
A multi-node SN0 machine is several of the same node: a Hub, memory, boot flash, module controller and processor each. The machine held exactly one, so a board could not describe a second. Make a board return one BoardSpec per node, including the files that keep each node's flash and controller store, and wire all its nodes at once so it can join them. The machine keeps a Unit per node and starts, stops and resets them together; the first is the one the front ends address. The PCI population the Origin 200 and the Origin 2000 BaseIO share moves to a function both boards call. The processors are marked as sharing memory when the board has more than one node, so a store-conditional is a compare-and-swap. No behaviour changes for the Origin 200.
An Origin 2000 module starts from a different store than an Origin 200: the configuration byte is 0, with no automatic power-up bit, the module number is the one the machine selects, and the clock-select byte at 0x70b is 0. The marker and the password are the same. Offer it as ElscNvram::origin2000 so the machine supplies the image and never builds its contents. The 0x70b value is an approximation. The controller writes 0x78 there on the full-clock midplane type and 0 on the others, and that this module is the full-clock type is only inferred, so the byte stays 0 as the model has always presented it.
An Origin 2000 module puts up to four adapters in front of one controller and one NVRAM. The NVRAM keeps a single address counter, so two nodes' transfers must not interleave. Add a BusOwner that the adapters share: a first START that finds another adapter holding the bus waits and runs after the holder's STOP, a STOP or clearing ESO withdraws it, and the status BB bit reads busy while another adapter holds the bus. The holder's STOP hands the bus to the lowest waiter, so a holder that STARTs again cannot overtake it. What a START on a busy bus does is unspecified, so waiting is a model choice. The clocked shared wire, with bitwise arbitration loss and clock stretching, is not modelled.
The Origin 2000 and Onyx2 controller is the same program built a second way. It answers dsp, dsc, key and vlm, which the Origin 200 build refuses with err cmd, and not aut. Its version field is VER 3.2, it holds the module in reset 4.4 s at power-up, it has an eight-character display that status messages replace, and its role bit in rsw is not the master flag. What a module owns, not the controller, is reached through a ModuleOps trait: whether the module is powered, whether a reset or power action can happen, the module's NMI, the serial turn on the bus and the midplane one-wire line. A command the module cannot carry out is answered err cmd, because a caller takes ok as proof. Calls that reach back into the controller are queued and delivered with its lock released. The status message texts and the supply margins' electrical effect are unknown and not invented. The Origin 200 master/slave forwarding stays out of both builds.
A deskside module has up to four node adapters in front of one controller, one NVRAM and the UST control port. Build that: four Pcf8584 adapters sharing a bus, the controller in its Origin 2000 build, the NVRAM, the UST port at 0x20, the midplane one-wire net the controller's nic line drives, and the scheduler. The scheduler publishes one grant per millisecond, eight processor codes, to every adapter; hands the bus to a controller or serial turn ahead of a grant; counts a bus that stays busy for five seconds and reinitialises the controller; announces a controller message in every code for a second; drives 100 ms reset and NMI pulses; and sequences module power. The machine drains reset, NMI and power changes as events. Timings, the recovery count, the UST port and the waiting START are model choices, named in the module doc. The clocked wire, with bitwise arbitration, mid-byte STOPs and clock stretching, is not modelled.
The IP27 PROM runs from a byte-wide 1 MiB AMD-command part behind the Hub's flash controller. The existing Bedrock flash is two 16-bit 8 MiB arrays and cannot stand in for it. The part is seen as the chip itself (the reset view) and through the controller window, which repeats the image and takes commands on 64-bit writes only. Program and erase finish inside the store that starts them. initial_image() builds the first contents: the jump-table template, the PROM payload over it, and the product and processor-module words patched into the configuration block with its byte sum kept.
A processor reaches its own Hub's registers at a fixed I/O offset with a node field of zero, whatever number the node holds. The processor's I/O view claimed every address by node number, so once the PROM numbered a second node 1, its stores to its own registers went to the node still holding 0, and its reads came from that node. The second node of a pair then ran the PROM against the first one's Hub and never finished. Give the processor an I/O view that keeps node-zero addresses in that alias on its own node, and leave every other address, including the alias offset under another node's number, to the claim map. I/O references are not on the memory fast path.
The Crossbow routes a request by widget number and passes it only while the link to that widget is up. The Octane keeps that rule in its address map. A Hub's I/O space has no such map: it holds a device per widget number, so a Crossbow in front of it has nothing to gate the access. Add Xbow::port, a wrapper that hands an access to the widget while the link carries traffic and fails it otherwise. A machine puts the wrapper where it would have put the widget. It costs one atomic load per access and only on a machine that uses it. Add Xbow::packets beside it: the face a Hub's big window reaches a peer's I/O view through, which routes a packet by the widget named in address bits 51:48 while that widget's link is up.
A node's cached references go straight to its own memory, which holds only node-relative addresses. With a second node, a reference that names another node number fell into unfitted memory: reads gave zero and writes vanished, so the PROM's memory test of the second node failed. Let a board route the windows of node numbers 1 to 255 to the Hub's node-qualified memory, which picks the Hub by the number the address carries, across the cable, and fails a number no Hub holds. Node zero's window is the node's own memory only while the node holds zero. Once it holds another number, a node-zero reference inside the cached alias still stays on the node, and one beyond the alias goes to the Hub holding zero. The second node's PROM merges the first node's tables that way; read from its own memory instead, the merge copied the second node's empty table over itself and left the first node's entry unset, and the PROM then faulted on it. The node's RAM sits in the ppmem window the cache and the JIT read directly, so moving the window's reference to the Hub takes that mapping away while the number is nonzero. A node numbered zero, which is a single node and the first node of a pair, keeps the direct mapping and takes no new decision on the hot path. That rerouting is the processor's own cached path only. Memory and Hub-special accesses, which name a node's banks by offset, go to the banks themselves through NodeMemory::home, so renumbering a node does not move what the Hub and the PROM's window reach. A store-conditional to another node's memory has to be as atomic as one to the processor's own, so the Hub's node-qualified views and that home view forward compare-and-swap to the node's banks.
Add the second SN0 product: one or two node boards on the deskside midplane, selected with machine.profile = "origin2000_ip27" and [origin2000] nodes. The board is data (slot identity 7 and 6, Hub 2.3, the flash with no machine type) plus its wiring: each node hosts a Crossbow whose port to the Hub is gated by link state, the first carries the BaseIO (a Bridge with the Origin 200's PCI population) at widget 8, the module has one controller with an I2C adapter per node, and two Hubs are joined by the direct cable with their cached windows routed through each other. The Bridge is built with the Hub that owns its Crossbow, so it reaches one Hub. One or two nodes boot the stock PROM to the menu and IRIX 6.5.30 to a serial login. Both are in the guide.
Add the Onyx2 deskside as a second board on the platform, selected with machine.profile = "onyx2_ip27" and the same [origin2000] section. Its midplane drives slot identity 3 and 2 into the Hubs, which the PROM reads as the deskside's node slots and so picks the single-Crossbow widget map. Both Hubs are hosts of the one Crossbow: node ports at widgets 10 and 9, the BaseIO-G at 15 and a graphics slot at 8. A packet into a peer's I/O view is routed by widget through the Crossbow, which is how the PROM reads the other Hub's node number. The graphics slot holds a test widget until a pipe is modelled; it logs and fails everything but its identification. Either Hub reaches the BaseIO, and the Bridge delivers DMA and interrupts to the Hub at the widget the guest programs, each behind its Crossbow port's link. One node boots an installed IRIX 6.5.30 to a serial login.
The Bridge had one host fixed at build time and dropped any transfer or interrupt packet whose widget was not that host. On a machine with two host chips behind one Crossbow the guest programs its own node's widget in the direct map, the translation entries and the interrupt destination, and with the other node as the host both the transfer and its completion interrupt were refused. Bridge::new now takes an Upstream: the host chips it can reach, indexed by widget number, each with its addressing, memory view and interrupt inputs. The widget the guest programmed selects the entry, so on the single-host path the transfer swaps one compare against a constant for one indexed load, with no lock and no dyn call beyond the memory access it already made. A widget with no entry is logged and dropped, as before. A machine with one host (HEART, a Hub with no Crossbow) builds Upstream::single, so the Octane behaves as it did. A host behind a Crossbow carries that port's link (Xbow::upstream): the Bridge asks it once per transfer chunk and once per interrupt packet, never per bus word, and a dropped read is timed out as the Crossbow's drop is. That check is one branch per 4 KB chunk on the single-host path (no link to ask) and a dyn call per chunk behind a Crossbow. No machine uses Xbow::upstream yet; the Octane and the Origin 200 build Upstream::single. Tests put two hosts behind a Crossbow and move DMA, translation-entry transfers, device and chip-error interrupts between them by reprogramming the target widget, and check that a host with its link down gets nothing and times out a read. Measured on the transfer path with a bridge test loop (4 KB DMA write plus 4 KB read through the direct map, ship profile, every function aligned to 64 bytes on both sides, A B B A x4, host load about 100), ns per 4 KB transfer, median of the per-run minimums: before 7119 after 6930 (-2.7%, inside run-to-run noise of about +-5%) An Octane IRIX guest booted to the login prompt on the new build, ran hinv and df against its SCSI disk, and halted cleanly. The Origin 200 shares this transfer path (Upstream::single), so the same figure applies; no Origin 200 machine was run.
The Crossbow sent widget 0's interrupt packets to "widget 8" by
constant. Reuse XtalkHost, which the Bridge now takes the same way, and
replace connect_heart with connect_host(host, irq): the packet goes to
the host's interrupt inputs when the destination register names the
host's widget and that link is up. The delivery path is unchanged; the
compare is against a field of the OnceLock already read there.
The Octane passes HEART_HOST, so its behaviour does not change. A test
names a different host widget and checks that a packet addressed to
widget 8 is dropped and one addressed to the host arrives.
Measured with iris-bench run --quick --cpu r10000, ship profile, A B B A
x4 per side. Host instructions per guest instruction (perf stat
instructions:u / icount, median of 4), base being the Bridge change
underneath:
base branch
interpreter 551.8 550.9 (-0.15%)
interpreter, functions
aligned to 64 bytes 549.0 550.2 (+0.23%)
jitv2 could not be measured: the host load average was about 100 and
the compile threads swamp the count. The bench kernels do not reach the
Crossbow; the packet path gains one field compare per widget 0 interrupt.
Load a boot PROM image from a file, whether it is a version-six single-segment container or a raw payload, and reject a container that is malformed instead of retrying it as raw bytes. The machines that boot a stock PROM file read it through this, bounded by the size of the flash they fill. The file is the one the Fuel uses; carrying it here lets a second machine use the loader before the Fuel lands.
Map a device into a processor view as an exact-bounds window: a read or write outside the device's own size is refused even where the 4 KiB page the decode table rounds to would otherwise answer it. SparseWindowMap keeps a short sorted list of such windows for a view that has a handful of apertures, so a view needs no full decode table and no entry in the process-wide fault-handler registry. This is the part of the Fuel's processor bus the SN0 node board's uncached views share. The Fuel's copy has the rest of the bus beside it.
A Hub or Bedrock processor interface counts graphics credits for the stores the processor sends to a graphics head, and exposes the counters as a small register block: a page, a credit count, a bias and an interrupt count and compare for each of two slots. Add that block as PiGfx, with the register decode and the starvation rule, as pure state the interface embeds. This is the register block of the Fuel's pi_gfx.rs without the credit-link trait and the store-mark helpers, which only the Fuel's graphics board uses and which land with it. The file is therefore not byte-identical to the Fuel's.
The Bridge hard-coded HEART's widget number and interrupt-packet offset
to decide where its interrupts and DMA go. Interrupt delivery already
goes through InterruptController and memory through a BusDevice, so no
new trait is needed; what is missing is how the host chip is addressed.
Another host (the Hub on the Origin boards) differs only in those
values: its widget number, taking a packet at any address, and 128
vectors instead of 64.
Add XtalkHost (widget number, packet address match, vector count) and
have the machine pass it to Bridge::new. The Octane passes HEART_HOST,
so the comparisons that were against constants are against a field of
the same value, and nothing about the Octane's behaviour moves. IOC3
needs no change: it reaches the host through the Bridge's PCI pins and
DMA. Tests build a Bridge with other hosts and check that packets
follow them.
Measured with iris-bench run --quick --cpu r10000, ship profile, A B B A
x4 per side. Host instructions per guest instruction (perf stat
instructions:u / icount, median of 4):
base branch
interpreter 551.8 550.9 (-0.15%)
jitv2 could not be measured: the host load average was about 100 and
the compile threads swamp the count. The bench kernels do not reach the
Bridge, so this shows the build is not disturbed, not the packet path
itself; that gained one mask and compare per interrupt packet and none
per DMA byte. The numbers predate the final packet-address mask.
Owner
|
oh this looks mergeable. |
Collaborator
Author
|
haha, nice. There might be some cleanups to follow but at least it's merged |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Iris runs the Octane, O2 and the Indy family. I wanted it to run SGI's IP27 machines too, so this adds the Origin 200, the Origin 2000 deskside and the Onyx2 deskside. IRIX 6.5.30 boots to a login on all three, including two nodes and two processors on the Origin 2000 and Onyx2.
It's a port of my QEMU work (jamesbraid/qemu-sgi-origami), which runs these machines today. Each ported file says which QEMU file it came from.
What's in it
ip35.rshere holds only the window map, so take Fuel's file on that conflict.Things to look at
Tested
hinv,unameanddf, on: the Origin 200 with R10000 and with R12000, the Origin 2000 on one and two nodes, and the Onyx2 on one and two nodes.cargo test --workspacepasses at each.jitv2feature, and timing.