diff --git a/flow/designs/asap7/tage_update/README.md b/flow/designs/asap7/tage_update/README.md new file mode 100644 index 0000000000..37e7663948 --- /dev/null +++ b/flow/designs/asap7/tage_update/README.md @@ -0,0 +1,151 @@ +# tage_update + +This design adds to the variety, relevance and fast turnaround of the +ORFS tests: generated SRAMs at scale (6 / 40 / 160 macros through +`AUTO_MEMORIES`) on ORFS's default flow settings, the structure of a +TAGE branch predictor as in XiangShan, and a size ladder whose +10-minute base variant already shows the effect. + +One cycle of a branch predictor's second stage: the TAGE's tables and +the main BTB are read from the same pc, and their responses decide the +fetch block's prediction. When that prediction overrides the first +stage's, the next fetch starts from its target, and the TAGE and main +BTB banks that pc hashes to are read again, so the path ends at the +bank being read: its SRAM's read address or enable, or the write +buffer beside a TAGE table, which cannot drain while the bank is read. + +The decision logic needs both sets of SRAMs' outputs, so placement +puts it between them, and a path from one TAGE table back to a TAGE +table goes out to that logic and comes back. The design exists to +measure that detour: how far the path travels compared with the +distance between its ends, and how much of its delay is repeaters and +wire rather than logic. + +## The path + +```mermaid +flowchart LR + subgraph TAGE[TAGE: TABLES x BANKS x WAYS] + TS[entry and useful SRAMs] --> TR[s2 response regs] + WB[write buffers] --> TS + end + subgraph MBTB[main BTB: 2 halves x MBTB_BANKS x MBTB_WAYS] + MS[entry SRAMs] --> MR[s2 slots and their order] + end + TR -->|tag compare| D[provider and alternate: direction] + D --> C + MR --> C[first taken slot: direction, position, target] + C --> O[override of stage 1?] + O -->|s0 pc| WB + O -->|s0 pc| MS +``` + +The critical path starts at a stage-2 register, decides the TAGE's +direction, combines it with the main BTB's slots into the block's +prediction, compares that with the first stage's, picks the next pc, +hashes it to a bank, and ends at that bank: the SRAM's read address or +enable, or the write buffer, which does not drain while the bank is +read. + +```mermaid +sequenceDiagram + participant S as stage control + participant T as TAGE tables + participant B as main BTB + S->>T: s0 read at pc + S->>B: s0 read at pc + T->>S: s2 responses (gather) + B->>S: s2 slots + S->>S: same cycle: direction, first taken slot, override, next pc + S->>T: next s0 read, at the bank that pc hashes to: no drain there + S->>B: next s0 read +``` + +## Where it comes from + +XiangShan's Frontend (Kunminghu), its branch predictor: +`xiangshan/frontend/bpu/Bpu.scala` for the stage-2 prediction and the +override, `bpu/tage/` for the TAGE, `bpu/mbtb/` for the main BTB and +`bpu/WriteBuffer.scala` for the write buffers. The sources cite the +lines each piece models. + +The SRAMs keep XiangShan's shapes: 512x17 for the TAGE's entries +(`array_512x17`), 64x16 for its useful counters (`array_64x16`, eight +2-bit counters to a row) and 256x46 for the main BTB's entries +(`array_256x46`). All use firtool's port convention, so +`AUTO_MEMORIES` turns them into generated macros. + +What is left out: the TAGE predicts one direction per fetch block +where XiangShan's predicts one per slot (wider, not deeper); the +targets' upper-bit correction; the Ftq, which in XiangShan returns a +just-predicted branch's metadata to training and here is one wire. + +## Variants + +| FLOW_VARIANT | TAGE | main BTB | macros | +|---|---|---|---| +| small | 2 tables, 1 bank, 1 way | 1 bank, 1 way | 6 | +| medium (= base, the default) | 4 tables, 2 banks, 2 ways | 2 banks, 2 ways | 40 | +| large | 8 tables, 4 banks, 2 ways | 4 banks, 4 ways | 160 | + +large is XiangShan's size for both. base, the variant ORFS runs when +FLOW_VARIANT is not set, is the smallest that shows the effect. + +## What it shows + +The worst path in every variant leaves stage 2's pc and ends at an +SRAM's read address or enable: the override has decided where the next +fetch starts, and the banks that pc hashes to have to be read. The +decision needs both the TAGE's and the main BTB's outputs, so it sits +between the two groups of SRAMs, and the path back from it to a bank +grows with the field of SRAMs around it. + +- small is logic. The decision sits next to everything it reads, and + repeaters and wire are a small part of the path. +- medium starts to travel: a few repeaters appear, and the path is + longer than the distance between its ends. +- large is the size of XiangShan's predictor. The main BTB's SRAMs sit + along one side of the die and the TAGE's along the other, the path is + about a millimetre long, and close to half of its delay is repeaters + and wire. That half is what this design is for. + +Two things decide how much of that half is paid: + +- The metal stack. With resistance-aware global routing (asap7's + default) the critical nets go to M8 and M9, which this design routes + to, as XiangShan does. +- Detailed routing. It makes the long nets longer than global route + estimated, so large is worse at final than at global route. + +A change to the flow for this class of path shows on large as a shorter +path or a smaller repeater and wire share. Look at the worst path after +global route: + +```sh +make DESIGN_CONFIG=./designs/asap7/tage_update/config.mk FLOW_VARIANT=large gui_grt +``` + +XiangShan's Frontend has the same structure on a die 2.4 times wider, +the rest of the Frontend surrounding the predictor, so the effect is +larger there. + +## Constraints + +`constraint.sdc` sets a 473 ps clock (see its comments for why) and +sources `$PLATFORM_DIR/constraints.sdc`: the block's ports are budgeted +with `set_max_delay`, and only register-to-register paths can fail. + +## Try this + +```sh +make DESIGN_CONFIG=./designs/asap7/tage_update/config.mk +make DESIGN_CONFIG=./designs/asap7/tage_update/config.mk FLOW_VARIANT=small +make DESIGN_CONFIG=./designs/asap7/tage_update/config.mk FLOW_VARIANT=large +``` + +Then look at the worst register-to-register path and where its cells +are: + +```sh +make DESIGN_CONFIG=./designs/asap7/tage_update/config.mk gui_grt +``` diff --git a/flow/designs/asap7/tage_update/config.mk b/flow/designs/asap7/tage_update/config.mk new file mode 100644 index 0000000000..568d3d24c0 --- /dev/null +++ b/flow/designs/asap7/tage_update/config.mk @@ -0,0 +1,40 @@ +# tage_update: a branch predictor's stage-2 decision between two sets of +# SRAMs, and back. README.md says what it is for and where it comes from. +export PLATFORM = asap7 +export DESIGN_NAME = tage_update +export DESIGN_NICKNAME = tage_update + +export VERILOG_FILES = $(sort $(wildcard $(DESIGN_HOME)/src/$(DESIGN_NICKNAME)/*.sv)) +export SDC_FILE = $(DESIGN_HOME)/$(PLATFORM)/$(DESIGN_NICKNAME)/constraint.sdc + +# Readable, parameterised SystemVerilog, read by slang. +export SYNTH_HDL_FRONTEND = slang + +# The tables' SRAMs are behavioural modules in the firtool port +# convention; AUTO_MEMORIES turns each into a generated macro. +export AUTO_MEMORIES = 1 + +# All three threshold voltages, so the resizer can trade leakage for +# speed per path, as a 3 GHz-class core does. +export ASAP7_USE_VT = RVT LVT SLVT + +export CORE_UTILIZATION = 30 +export CORE_MARGIN = 2 + +# The full metal stack, as XiangShan's own configs route: asap7's power +# grid uses M1, M2, M5 and M6 only, so M8 and M9 are free for signals. +export MAX_ROUTING_LAYER = M9 + +# A ladder, FLOW_VARIANT=small|medium|large: small to iterate on, large +# at the size of XiangShan's TAGE and main BTB. base, ORFS's default +# variant, is medium: the smallest size that shows the effect (README.md, +# Variants). +ifeq ($(FLOW_VARIANT),small) +export VERILOG_TOP_PARAMS = TABLES 2 BANKS 1 WAYS 1 MBTB_BANKS 1 MBTB_WAYS 1 +else ifeq ($(FLOW_VARIANT),medium) +export VERILOG_TOP_PARAMS = TABLES 4 BANKS 2 WAYS 2 MBTB_BANKS 2 MBTB_WAYS 2 +else ifeq ($(FLOW_VARIANT),large) +export VERILOG_TOP_PARAMS = TABLES 8 BANKS 4 WAYS 2 MBTB_BANKS 4 MBTB_WAYS 4 +else +export VERILOG_TOP_PARAMS = TABLES 4 BANKS 2 WAYS 2 MBTB_BANKS 2 MBTB_WAYS 2 +endif diff --git a/flow/designs/asap7/tage_update/constraint.sdc b/flow/designs/asap7/tage_update/constraint.sdc new file mode 100644 index 0000000000..d105b3b329 --- /dev/null +++ b/flow/designs/asap7/tage_update/constraint.sdc @@ -0,0 +1,15 @@ +# 473 ps is XiangShan's synthesis period on this library: its published +# goal is 3 GHz (333 ps) on 7 nm, 41.1 FO4 at ASAP7's published 8.1 ps, +# which is 591 ps at this flow's 14.37 ps FO4, and 0.8 of that. +set clk_name clk +set clk_port_name clock +set clk_period 473 + +# A macro: only reg2reg can fail closure. The boundaries are +# optimisation targets, set_max_delay in the platform's constraints.sdc, +# not set_input_delay/set_output_delay. +set in2reg_max [expr { $clk_period * 0.8 }] +set reg2out_max [expr { $clk_period * 0.8 }] +set in2out_max [expr { $clk_period * 0.6 }] + +source $::env(PLATFORM_DIR)/constraints.sdc diff --git a/flow/designs/src/tage_update/bpu_ctrl.sv b/flow/designs/src/tage_update/bpu_ctrl.sv new file mode 100644 index 0000000000..e5440d85ea --- /dev/null +++ b/flow/designs/src/tage_update/bpu_ctrl.sv @@ -0,0 +1,94 @@ +// The branch predictor's top: the stage-2 prediction of a fetch block, +// and from it which pc stage 0 reads next. +// +// XiangShan: xiangshan/frontend/bpu/Bpu.scala. A slot of the main BTB's +// result is taken if it is a jump, or a conditional branch the TAGE says +// is taken (s2_takenMask, Bpu.scala:356-360); the block's branch is the +// first taken slot (s2_firstTakenBranchOH, :362), its target the block's +// target, and with no taken slot the block falls through (:367). Stage 2 +// overrides stage 1 when the direction or the position differs from the +// fast stage-1 prediction (s2_override, :374-380); an override flushes +// stage 1, and stage 0 reads from the stage-2 target (s0_startPc, +// :533-541). Stage 0 is the next read of the TAGE and of the main BTB, +// at banks and sets hashed from that pc, so the stage-2 response of both +// decides, within the same cycle, which of their banks are read next. +module bpu_ctrl #( + parameter int SLOTS = 8 +) ( + input logic clock, + input logic enable, // s1_ready && sramResetDone, registered + // a redirect from the back end, registered at the boundary + input logic redirect_valid, + input logic [31:0] redirect_pc, + // the stage-1 prediction, from the fast predictor (uBTB) + input logic s1_valid, + input logic [31:0] s1_target, + // what stage 1 predicted for the block now in stage 2 + input logic s2_valid, + input logic s2_s1_taken, + input logic [ 3:0] s2_s1_position, + input logic [31:0] s2_pc, + // the stage-2 results: the TAGE's direction, the main BTB's slots + input logic tage_taken, + input logic slot_hit [SLOTS], + input logic slot_cond [SLOTS], + input logic slot_jump [SLOTS], + input logic [ 3:0] slot_position[SLOTS], + input logic [31:0] slot_target [SLOTS], + input logic slot_before [SLOTS][SLOTS], + // the prediction, and stage 0 + output logic s2_taken, + output logic s0_fire, + output logic [31:0] s0_pc, + output logic s1_flush +); + // ---- the stage-2 prediction --------------------------------------------- + // The TAGE here predicts one direction per block; XiangShan's predicts + // one per slot (fastTakenVec), which is wider and no deeper. + logic taken[SLOTS], first[SLOTS]; + always_comb begin + for (int i = 0; i < SLOTS; i++) + taken[i] = slot_hit[i] && (slot_jump[i] || (slot_cond[i] && tage_taken)); + // CompareMatrix.getLeastElementOH + for (int i = 0; i < SLOTS; i++) begin + first[i] = taken[i]; + for (int j = 0; j < SLOTS; j++) + if (j != i) first[i] = first[i] && (!taken[j] || slot_before[i][j]); + end + end + + logic [ 3:0] s2_position; + logic [31:0] s2_target; + always_comb begin + s2_taken = 1'b0; + s2_position = '0; + s2_target = s2_pc + 32'd64; // fall through: the next fetch block + for (int i = 0; i < SLOTS; i++) begin + s2_taken = s2_taken || taken[i]; + if (first[i]) begin // one-hot: Mux1H + s2_position = slot_position[i]; + s2_target = slot_target[i]; + end + end + end + + // ---- stage 0 ----------------------------------------------------------------- + logic s2_override; + assign s2_override = s2_valid && (s2_taken != s2_s1_taken || + (s2_taken && s2_position != s2_s1_position)); + assign s1_flush = redirect_valid || s2_override; + + logic [31:0] s0_pc_q; + always_ff @(posedge clock) if (s0_fire) s0_pc_q <= s0_pc; + + always_comb begin + if (redirect_valid) s0_pc = redirect_pc; + else if (s2_override) s0_pc = s2_target; + else if (s1_valid) s0_pc = s1_target; + else s0_pc = s0_pc_q; + end + + // s0_fire = s1_ready && sramResetDone (Bpu.scala:260); with the Ftq + // always ready, stage 1 is always ready + assign s0_fire = enable; +endmodule diff --git a/flow/designs/src/tage_update/mbtb.sv b/flow/designs/src/tage_update/mbtb.sv new file mode 100644 index 0000000000..adb2dce5bd --- /dev/null +++ b/flow/designs/src/tage_update/mbtb.sv @@ -0,0 +1,113 @@ +// The main BTB: which slots of a fetch block hold branches, of what +// kind, at which position, with which target. Read at stage 0 from the +// same pc as the TAGE; its stage-2 result is what the TAGE's directions +// are combined with to decide the block's prediction (bpu_ctrl.sv). +// +// XiangShan: xiangshan/frontend/bpu/mbtb/MainBtb.scala, +// MainBtbAlignBank.scala and MainBtbInternalBank.scala. A 64-byte fetch +// block is two 32-byte halves, each read from its own align bank; an +// align bank has internal banks, each a set of one-way SRAMs, and one +// internal bank is read per access. Every way of a read set is a slot of +// the result, so a fetch block has ALIGN * WAYS slots. +// +// A slot's tag is checked and its target assembled at stage 2; the +// order of the slots' positions is compared at stage 1 and registered +// (s2_compareMatrix, Bpu.scala:347), as in XiangShan. The entry's target +// carry, which corrects the upper target bits, is left out: it is +// arithmetic on the target and not on the decision. +module mbtb #( + parameter int BANKS = 4, // internal banks per align bank + parameter int WAYS = 4, + localparam int ALIGN = 2, + localparam int SLOTS = ALIGN * WAYS, + localparam int BW = BANKS > 1 ? $clog2(BANKS) : 1, + localparam int WW = WAYS > 1 ? $clog2(WAYS) : 1 +) ( + input logic clock, + // stage 0 + input logic rd_valid, + input logic [31:0] rd_pc, + // the stage-2 pc, for the tag check and the target's upper bits + input logic [31:0] s2_pc, + // a training write, registered upstream + input logic wr_valid, + input logic wr_align, + input logic [BW-1:0] wr_bank, + input logic [WW-1:0] wr_way, + input logic [ 7:0] wr_set, + input logic [45:0] wr_entry, + // stage 2: one result per slot + output logic s2_hit [SLOTS], + output logic s2_cond [SLOTS], + output logic s2_jump [SLOTS], + output logic [ 3:0] s2_position[SLOTS], + output logic [31:0] s2_target [SLOTS], + output logic s2_before [SLOTS][SLOTS] // slot i before slot j +); + // ---- stage 0: the set of each half, and the internal bank ------------ + // The block starts in half rd_pc[5]; the other half is the next one. + logic [7:0] set_of[ALIGN]; + logic [BW-1:0] bank; + always_comb begin + for (int a = 0; a < ALIGN; a++) + set_of[a] = rd_pc[13:6] + 8'(a < 32'(rd_pc[5])); + bank = rd_pc[2+:BW]; + end + + // ---- the SRAMs --------------------------------------------------------- + logic [45:0] rdata[ALIGN][BANKS][WAYS]; + for (genvar a = 0; a < ALIGN; a++) begin : g_align + for (genvar b = 0; b < BANKS; b++) begin : g_bank + for (genvar w = 0; w < WAYS; w++) begin : g_way + logic read, write; + assign read = rd_valid && bank == BW'(b); + assign write = wr_valid && wr_align == 1'(a) && wr_bank == BW'(b) && wr_way == WW'(w); + mbtb_entry_sram sram ( + .RW0_clk (clock), + .RW0_en (read || write), + .RW0_wmode(!read), + .RW0_addr (read ? set_of[a] : wr_set), + .RW0_wdata(wr_entry), + .RW0_rdata(rdata[a][b][w]) + ); + end + end + end + + // ---- stage 1: the read bank's entries, and the order of the slots ------ + logic [BW-1:0] s1_bank; + always_ff @(posedge clock) s1_bank <= bank; + + logic [45:0] s1_entry[SLOTS]; + always_comb + for (int a = 0; a < ALIGN; a++) + for (int w = 0; w < WAYS; w++) + s1_entry[a*WAYS+w] = rdata[a][s1_bank][w]; + + // an entry is {tag[45:30], branch type[29:28], ras action[27:26], + // position[25:22], target carry[21:20], target[19:0]}; a slot in the + // second half comes after every slot in the first + function automatic logic [4:0] order(logic [45:0] e, int slot); + return {1'(slot >= WAYS), e[25:22]}; + endfunction + + logic [45:0] s2_entry[SLOTS]; + always_ff @(posedge clock) begin + for (int i = 0; i < SLOTS; i++) begin + s2_entry[i] <= s1_entry[i]; + for (int j = 0; j < SLOTS; j++) + s2_before[i][j] <= order(s1_entry[i], i) < order(s1_entry[j], j); + end + end + + // ---- stage 2: the slots ------------------------------------------------ + always_comb begin + for (int i = 0; i < SLOTS; i++) begin + s2_hit[i] = s2_entry[i][29:28] != 2'd0 && s2_entry[i][45:30] == s2_pc[29:14]; + s2_cond[i] = s2_entry[i][29:28] == 2'd1; + s2_jump[i] = s2_entry[i][29:28] >= 2'd2; + s2_position[i] = s2_entry[i][25:22]; + s2_target[i] = {s2_pc[31:21], s2_entry[i][19:0], 1'b0}; + end + end +endmodule diff --git a/flow/designs/src/tage_update/mbtb_entry_sram.sv b/flow/designs/src/tage_update/mbtb_entry_sram.sv new file mode 100644 index 0000000000..3376695db9 --- /dev/null +++ b/flow/designs/src/tage_update/mbtb_entry_sram.sv @@ -0,0 +1,28 @@ +// One way of one internal bank of one align bank of the main BTB: 256 +// sets of a 46-bit entry, single-ported, synchronous read. XiangShan's +// Frontend has 32 of exactly this macro (array_256x46), generated from +// SRAMTemplate in xiangshan/frontend/bpu/mbtb/MainBtbInternalBank.scala. +// +// An entry (MainBtbEntry, mbtb/Bundles.scala) is {tag[15:0], +// attribute[3:0], position[3:0], target carry[1:0], target[19:0]}. +// +// Same firtool port convention and fixed size as tage_entry_sram.sv, +// for the same reasons. +module mbtb_entry_sram ( + input logic RW0_clk, + input logic RW0_en, + input logic RW0_wmode, // 1 write, 0 read + input logic [ 7:0] RW0_addr, + input logic [45:0] RW0_wdata, + output logic [45:0] RW0_rdata +); + logic [45:0] mem[256]; + logic ren_q; + logic [ 7:0] raddr_q; + always_ff @(posedge RW0_clk) begin + ren_q <= RW0_en && !RW0_wmode; + raddr_q <= RW0_addr; + if (RW0_en && RW0_wmode) mem[RW0_addr] <= RW0_wdata; + end + assign RW0_rdata = ren_q ? mem[raddr_q] : 'x; +endmodule diff --git a/flow/designs/src/tage_update/tage_entry_sram.sv b/flow/designs/src/tage_update/tage_entry_sram.sv new file mode 100644 index 0000000000..71ef88fd1b --- /dev/null +++ b/flow/designs/src/tage_update/tage_entry_sram.sv @@ -0,0 +1,32 @@ +// One bank of one way of a TAGE table: 512 sets of a 17-bit entry +// (valid, 13-bit tag, 3-bit taken counter), single-ported, synchronous +// read. XiangShan's Frontend has 64 of exactly this macro +// (array_512x17), generated from SRAMTemplate in +// xiangshan/frontend/bpu/tage/TageTable.scala. +// +// Every port follows the firtool memory convention (RW0_), so +// ORFS's AUTO_MEMORIES detects this module and replaces it with a +// generated macro; the body below is its behavioural model. The size is +// fixed on purpose: the detector names the macro after the module, and +// a parameterised module would be renamed by the frontend. +module tage_entry_sram ( + input logic RW0_clk, + input logic RW0_en, + input logic RW0_wmode, // 1 write, 0 read + input logic [ 8:0] RW0_addr, + input logic [16:0] RW0_wdata, + output logic [16:0] RW0_rdata +); + // firtool's own form for a read-write port: the address is registered + // on the clock and the array is read from the registered address, which + // is a clocked read port to yosys and to AUTO_MEMORIES's detector. + logic [16:0] mem[512]; + logic ren_q; + logic [ 8:0] raddr_q; + always_ff @(posedge RW0_clk) begin + ren_q <= RW0_en && !RW0_wmode; + raddr_q <= RW0_addr; + if (RW0_en && RW0_wmode) mem[RW0_addr] <= RW0_wdata; + end + assign RW0_rdata = ren_q ? mem[raddr_q] : 'x; +endmodule diff --git a/flow/designs/src/tage_update/tage_table.sv b/flow/designs/src/tage_update/tage_table.sv new file mode 100644 index 0000000000..963999cb97 --- /dev/null +++ b/flow/designs/src/tage_update/tage_table.sv @@ -0,0 +1,145 @@ +// One TAGE table: BANKS x WAYS entry SRAMs and as many useful-counter +// SRAMs, the stage-2 read-response registers beside them, and one write +// buffer per bank. +// +// XiangShan: xiangshan/frontend/bpu/tage/TageTable.scala. A table is read +// by prediction (readReq(0)) and re-read by training (readReq(1), +// Tage.scala:252). Training writes go into a per-bank write buffer +// (TageTable.scala:142-150), which drains into its bank only in a cycle +// when that bank is not being read (TageTable.scala:192, 199): the +// SRAMs are single-ported. +// +// That drain is the end of the path this design is about. Whether a +// bank is read this cycle depends on `rerd_*`, the update re-read the +// central decision in tage_update.sv computes from every table's +// stage-2 response. +module tage_table #( + parameter int BANKS = 4, + parameter int WAYS = 2, + parameter int WB_SIZE = 4, + localparam int BW = BANKS > 1 ? $clog2(BANKS) : 1, + localparam int WW = WAYS > 1 ? $clog2(WAYS) : 1 +) ( + input logic clock, + // prediction read, stage 0 + input logic rd_valid, + input logic [ BW-1:0] rd_bank, + input logic [ 8:0] rd_set, + // the update re-read: the scatter, from the central decision + input logic rerd_valid, + input logic [ BW-1:0] rerd_bank, + input logic [ 8:0] rerd_set, + // a training write, registered upstream + input logic wr_valid, + input logic [ BW-1:0] wr_bank, + input logic [ WW-1:0] wr_way, + input logic [ 8:0] wr_set, + input logic [ 16:0] wr_entry, + input logic [ 1:0] wr_useful, + // the gather: this table's stage-2 read response, one entry and one + // useful counter per way + output logic [16:0] s2_entry [WAYS], + output logic [ 1:0] s2_useful[WAYS] +); + // ---- which bank is read this cycle, and from where ---------------- + logic [BANKS-1:0] read_b; + logic [8:0] read_set[BANKS]; + always_comb begin + for (int b = 0; b < BANKS; b++) begin + read_b[b] = (rd_valid && rd_bank == BW'(b)) || (rerd_valid && rerd_bank == BW'(b)); + read_set[b] = (rerd_valid && rerd_bank == BW'(b)) ? rerd_set : rd_set; + end + end + + // ---- per-bank write buffer ----------------------------------------- + // `valid` is the buffer's dirty bit: the entry is still to be written. + logic wb_valid[BANKS][WB_SIZE]; + logic [ 8:0] wb_set [BANKS][WB_SIZE]; + logic [WW-1:0] wb_way [BANKS][WB_SIZE]; + logic [ 16:0] wb_entry[BANKS][WB_SIZE]; + logic [ 1:0] wb_useful[BANKS][WB_SIZE]; + + // the oldest dirty entry of each bank, and whether it drains now + logic [$clog2(WB_SIZE)-1:0] head[BANKS]; + logic [BANKS-1:0] drain; + always_comb begin + for (int b = 0; b < BANKS; b++) begin + head[b] = '0; + drain[b] = 1'b0; + for (int i = WB_SIZE - 1; i >= 0; i--) begin + if (wb_valid[b][i]) begin + head[b] = i[$clog2(WB_SIZE)-1:0]; + drain[b] = !read_b[b]; + end + end + end + end + + // A write to a set and way already in the buffer updates that entry, + // so an older write never lands after a newer one; any other write + // takes the first free slot, and a full buffer overwrites slot 0 + // (WriteBuffer.scala: hitMask, then PriorityEncoder of the clean slots). + // The drain is the lowest dirty slot, PriorityEncoder(dirty), as above. + always_ff @(posedge clock) begin + for (int b = 0; b < BANKS; b++) begin + if (drain[b]) wb_valid[b][head[b]] <= 1'b0; + if (wr_valid && wr_bank == BW'(b)) begin + automatic int slot = 0; + automatic logic hit = 1'b0; + for (int i = WB_SIZE - 1; i >= 0; i--) begin + if (wb_valid[b][i] && wb_set[b][i] == wr_set && wb_way[b][i] == wr_way) begin + hit = 1'b1; + slot = i; + end + end + if (!hit) for (int i = WB_SIZE - 1; i >= 0; i--) if (!wb_valid[b][i]) slot = i; + wb_valid[b][slot] <= 1'b1; + wb_set[b][slot] <= wr_set; + wb_way[b][slot] <= wr_way; + wb_entry[b][slot] <= wr_entry; + wb_useful[b][slot] <= wr_useful; + end + end + end + + // ---- the SRAMs ------------------------------------------------------- + logic [16:0] rdata[BANKS][WAYS]; + logic [15:0] urdata[BANKS][WAYS]; + for (genvar b = 0; b < BANKS; b++) begin : g_bank + for (genvar w = 0; w < WAYS; w++) begin : g_way + logic write; + assign write = drain[b] && wb_way[b][head[b]] == WW'(w); + tage_entry_sram sram ( + .RW0_clk (clock), + .RW0_en (read_b[b] || write), + .RW0_wmode(!read_b[b]), + .RW0_addr (read_b[b] ? read_set[b] : wb_set[b][head[b]]), + .RW0_wdata(wb_entry[b][head[b]]), + .RW0_rdata(rdata[b][w]) + ); + // the useful counters of eight sets share a row; a write here + // writes its counter into every lane of the row, which keeps the + // timing structure and not the bit-exact folding + tage_useful_sram useful ( + .RW0_clk (clock), + .RW0_en (read_b[b] || write), + .RW0_wmode(!read_b[b]), + .RW0_addr (read_b[b] ? read_set[b][8:3] : wb_set[b][head[b]][8:3]), + .RW0_wdata({8{wb_useful[b][head[b]]}}), + .RW0_rdata(urdata[b][w]) + ); + end + end + + // ---- stage 1 to stage 2: the response registers ---------------------- + logic [BW-1:0] s1_bank; + logic [2:0] s1_lane; + always_ff @(posedge clock) begin + s1_bank <= rerd_valid ? rerd_bank : rd_bank; + s1_lane <= rerd_valid ? rerd_set[2:0] : rd_set[2:0]; + for (int w = 0; w < WAYS; w++) begin + s2_entry[w] <= rdata[s1_bank][w]; + s2_useful[w] <= urdata[s1_bank][w][2*s1_lane+:2]; + end + end +endmodule diff --git a/flow/designs/src/tage_update/tage_update.sv b/flow/designs/src/tage_update/tage_update.sv new file mode 100644 index 0000000000..a97b3a313d --- /dev/null +++ b/flow/designs/src/tage_update/tage_update.sv @@ -0,0 +1,278 @@ +// tage_update: a branch predictor's stage-2 decision, between the TAGE's +// tables and the main BTB, and back to the tables in the same cycle. +// README.md (beside config.mk) tells the story; this file is the logic. +// +// XiangShan's TAGE (xiangshan/frontend/bpu/tage/Tage.scala at aa6b520): +// +// gather every table's stage-2 read response is registered beside its +// SRAMs (s2_readResp, Tage.scala:117); +// decide the provider is the longest-history table that hits, the +// alternate the next one (Tage.scala:147-173); whether training +// must re-read the tables follows from it (t0_needRead, +// Tage.scala:228-236); +// scatter the re-read is a read request to one bank of every table +// (Tage.scala:252), and a bank being read cannot drain its +// write buffer this cycle (TageTable.scala:192, 199). +// +// The same response also decides the next read (bpu_ctrl.sv): when the +// TAGE's direction overrides the stage-1 prediction, stage 0 fetches +// from the stage-2 target, and the bank that pc hashes to is read, so +// it cannot drain. The block's direction is not the TAGE's alone: it is +// the first taken slot of the main BTB's result (mbtb.sv), its own SRAMs +// read from the same pc. The logic that decides sits between the two +// sets of SRAMs, and the loop goes out to it and back, as in XiangShan. +// +// In XiangShan the training decision also reaches the tables through +// the Ftq, which hands a just-predicted branch's metadata straight back; +// here that round trip is one wire. +module tage_update #( + parameter int TABLES = 4, + parameter int BANKS = 2, + parameter int WAYS = 2, + parameter int MBTB_BANKS = 4, // main BTB internal banks per half + parameter int MBTB_WAYS = 4, + localparam int SLOTS = 2 * MBTB_WAYS, + localparam int MBW = MBTB_BANKS > 1 ? $clog2(MBTB_BANKS) : 1, + localparam int MWW = MBTB_WAYS > 1 ? $clog2(MBTB_WAYS) : 1, + localparam int BW = BANKS > 1 ? $clog2(BANKS) : 1, + localparam int WW = WAYS > 1 ? $clog2(WAYS) : 1, + localparam int TW = TABLES > 1 ? $clog2(TABLES) : 1 +) ( + input logic clock, + input logic fetch_enable, // stage 1 ready, SRAMs out of reset + input logic redirect_valid, // a redirect from the back end + input logic [31:0] redirect_pc, + input logic ubtb_taken, // the fast stage-1 prediction + input logic [ 3:0] ubtb_position, // its branch's slot position + input logic [31:0] ubtb_target, // and its target + // a main BTB training write + input logic mbtb_wr_valid, + input logic mbtb_wr_align, + input logic [MBW-1:0] mbtb_wr_bank, + input logic [MWW-1:0] mbtb_wr_way, + input logic [ 7:0] mbtb_wr_set, + input logic [45:0] mbtb_wr_entry, + input logic train_taken, // the resolved direction + output logic pred_taken, + output logic [TW-1:0] pred_provider +); + // ---- the boundary, registered ---------------------------------------- + logic en_q, redirect_valid_q, ubtb_taken_q, train_taken_q; + logic [ 3:0] ubtb_position_q; + logic [31:0] redirect_pc_q, ubtb_target_q; + logic mbtb_wr_valid_q, mbtb_wr_align_q; + logic [MBW-1:0] mbtb_wr_bank_q; + logic [MWW-1:0] mbtb_wr_way_q; + logic [ 7:0] mbtb_wr_set_q; + logic [45:0] mbtb_wr_entry_q; + always_ff @(posedge clock) begin + en_q <= fetch_enable; + ubtb_target_q <= ubtb_target; + ubtb_position_q <= ubtb_position; + mbtb_wr_valid_q <= mbtb_wr_valid; + mbtb_wr_align_q <= mbtb_wr_align; + mbtb_wr_bank_q <= mbtb_wr_bank; + mbtb_wr_way_q <= mbtb_wr_way; + mbtb_wr_set_q <= mbtb_wr_set; + mbtb_wr_entry_q <= mbtb_wr_entry; + redirect_valid_q <= redirect_valid; + redirect_pc_q <= redirect_pc; + ubtb_taken_q <= ubtb_taken; + train_taken_q <= train_taken; + end + + // each table hashes the pc its own way (XiangShan folds in a history of + // a different length per table; a pc rotation keeps the tables apart) + function automatic logic [8:0] set_of(logic [31:0] pc, int t); + logic [31:0] r = (pc >> (2 + t)) | (pc << (30 - t)); + return r[8:0] ^ r[20:12]; + endfunction + function automatic logic [12:0] tag_of(logic [31:0] pc, int t); + logic [31:0] r = (pc >> (5 + 2 * t)) | (pc << (27 - 2 * t)); + return r[12:0] ^ r[31:19]; + endfunction + + // ---- stage 0, from the stage control ----------------------------------- + logic s0_fire, s1_flush, s2_pred_taken; + logic [31:0] s0_pc; + logic s1_valid, s2_valid, s1_taken, s2_taken, s2_s1_taken; + logic [ 3:0] s2_s1_position; + logic [31:0] s1_pc, s2_pc; + + // ---- the main BTB -------------------------------------------------------- + logic slot_hit[SLOTS], slot_cond[SLOTS], slot_jump[SLOTS]; + logic [ 3:0] slot_position[SLOTS]; + logic [31:0] slot_target[SLOTS]; + logic slot_before[SLOTS][SLOTS]; + mbtb #( + .BANKS(MBTB_BANKS), + .WAYS (MBTB_WAYS) + ) btb ( + .clock (clock), + .rd_valid (s0_fire), + .rd_pc (s0_pc), + .s2_pc (s2_pc), + .wr_valid (mbtb_wr_valid_q), + .wr_align (mbtb_wr_align_q), + .wr_bank (mbtb_wr_bank_q), + .wr_way (mbtb_wr_way_q), + .wr_set (mbtb_wr_set_q), + .wr_entry (mbtb_wr_entry_q), + .s2_hit (slot_hit), + .s2_cond (slot_cond), + .s2_jump (slot_jump), + .s2_position(slot_position), + .s2_target (slot_target), + .s2_before (slot_before) + ); + + logic s2_block_taken; + bpu_ctrl #( + .SLOTS(SLOTS) + ) ctrl ( + .clock (clock), + .enable (en_q), + .redirect_valid(redirect_valid_q), + .redirect_pc (redirect_pc_q), + .s1_valid (s1_valid), + .s1_target (ubtb_target_q), + .s2_valid (s2_valid), + .s2_s1_taken (s2_s1_taken), + .s2_s1_position(s2_s1_position), + .s2_pc (s2_pc), + .tage_taken (s2_pred_taken), + .slot_hit (slot_hit), + .slot_cond (slot_cond), + .slot_jump (slot_jump), + .slot_position (slot_position), + .slot_target (slot_target), + .slot_before (slot_before), + .s2_taken (s2_block_taken), + .s0_fire (s0_fire), + .s0_pc (s0_pc), + .s1_flush (s1_flush) + ); + + logic [BW-1:0] s0_bank; + assign s0_bank = s0_pc[2+:BW]; + + // stages 1 and 2 carry the request alongside the tables' read; an + // override or a redirect flushes stage 1 (Bpu.scala:248-270) + always_ff @(posedge clock) begin + s1_valid <= s0_fire; + if (s0_fire) s1_pc <= s0_pc; + s1_taken <= train_taken_q; + s2_valid <= s1_valid && !s1_flush; + s2_pc <= s1_pc; + s2_s1_taken <= ubtb_taken_q; + s2_s1_position <= ubtb_position_q; + s2_taken <= s1_taken; + end + + // ---- the tables ------------------------------------------------------ + logic [16:0] s2_entry[TABLES][WAYS]; + logic [ 1:0] s2_useful[TABLES][WAYS]; + logic [ 1:0] t1_useful; + logic rerd_valid; + logic [BW-1:0] rerd_bank; + logic t1_valid; + logic [TW-1:0] t1_table; + logic [WW-1:0] t1_way; + logic [31:0] t1_pc; + logic [16:0] t1_entry; + logic [BW-1:0] t1_bank; + + for (genvar t = 0; t < TABLES; t++) begin : g_table + tage_table #( + .BANKS(BANKS), + .WAYS (WAYS) + ) table_i ( + .clock (clock), + .rd_valid (s0_fire), + .rd_bank (s0_bank), + .rd_set (set_of(s0_pc, t)), + .rerd_valid(rerd_valid), + .rerd_bank (rerd_bank), + .rerd_set (set_of(s2_pc, t)), + .wr_valid (t1_valid && t1_table == TW'(t)), + .wr_bank (t1_bank), + .wr_way (t1_way), + .wr_set (set_of(t1_pc, t)), + .wr_entry (t1_entry), + .wr_useful (t1_useful), + .s2_entry (s2_entry[t]), + .s2_useful (s2_useful[t]) + ); + end + + // ---- gather: which table and way hit --------------------------------- + // an entry is {valid, tag[12:0], taken counter[2:0]} + logic [TABLES-1:0] hit; + logic [WW-1:0] hit_way[TABLES]; + always_comb begin + for (int t = 0; t < TABLES; t++) begin + hit[t] = 1'b0; + hit_way[t] = '0; + for (int w = WAYS - 1; w >= 0; w--) begin + if (s2_entry[t][w][16] && s2_entry[t][w][15:3] == tag_of(s2_pc, t)) begin + hit[t] = 1'b1; + hit_way[t] = WW'(w); + end + end + end + end + + // ---- decide: provider and alternate ---------------------------------- + logic has_provider, has_alt; + logic [TW-1:0] provider, alt; + always_comb begin + has_provider = 1'b0; + has_alt = 1'b0; + provider = '0; + alt = '0; + for (int t = 0; t < TABLES; t++) begin // the last hit wins: longest history + if (hit[t]) begin + has_alt = has_provider; + alt = provider; + has_provider = 1'b1; + provider = TW'(t); + end + end + end + + logic [2:0] provider_ctr, alt_ctr; + logic [1:0] provider_useful; + assign provider_ctr = s2_entry[provider][hit_way[provider]][2:0]; + assign alt_ctr = s2_entry[alt][hit_way[alt]][2:0]; + assign provider_useful = s2_useful[provider][hit_way[provider]]; + + // a weak provider defers to the alternate, and a provider that is not + // useful may be replaced: either way training must re-read the tables + logic provider_weak; + assign provider_weak = provider_ctr == 3'd3 || provider_ctr == 3'd4; + + // ---- scatter: the update re-read, to one bank of every table --------- + assign rerd_valid = s2_valid && has_provider && + ((has_alt && provider_weak) || provider_useful == 2'd0); + assign rerd_bank = s2_pc[2+:BW]; + + // ---- the prediction, and the training write it leads to -------------- + logic use_provider; + assign use_provider = has_provider && !(provider_weak && has_alt); + // no table hits: not taken, the base prediction's job in XiangShan + assign s2_pred_taken = use_provider ? provider_ctr[2] : has_alt && alt_ctr[2]; + always_ff @(posedge clock) begin + pred_taken <= s2_block_taken; // the block's prediction + pred_provider <= provider; + t1_valid <= s2_valid && has_provider; + t1_table <= provider; + t1_way <= hit_way[provider]; + t1_bank <= s2_pc[2+:BW]; + t1_pc <= s2_pc; + t1_useful <= (provider_ctr[2] == s2_taken) ? + (provider_useful == 2'd3 ? 2'd3 : provider_useful + 2'd1) : provider_useful; + t1_entry <= {1'b1, tag_of(s2_pc, int'(provider)), + s2_taken ? (provider_ctr == 3'd7 ? 3'd7 : provider_ctr + 3'd1) + : (provider_ctr == 3'd0 ? 3'd0 : provider_ctr - 3'd1)}; + end +endmodule diff --git a/flow/designs/src/tage_update/tage_useful_sram.sv b/flow/designs/src/tage_update/tage_useful_sram.sv new file mode 100644 index 0000000000..54ad6cd0b5 --- /dev/null +++ b/flow/designs/src/tage_update/tage_useful_sram.sv @@ -0,0 +1,25 @@ +// One bank of one way of a TAGE table's useful counters: 64 rows of +// eight 2-bit counters, the 512 sets folded eight to a row. XiangShan's +// Frontend has 64 of this macro (array_64x16) beside the 64 entry SRAMs +// (NumUsefulCtrSramFolds = 8 in xiangshan/frontend/bpu/tage/Parameters.scala). +// +// Same firtool port convention and fixed size as tage_entry_sram.sv, +// for the same reasons. +module tage_useful_sram ( + input logic RW0_clk, + input logic RW0_en, + input logic RW0_wmode, // 1 write, 0 read + input logic [ 5:0] RW0_addr, + input logic [15:0] RW0_wdata, + output logic [15:0] RW0_rdata +); + logic [15:0] mem[64]; + logic ren_q; + logic [ 5:0] raddr_q; + always_ff @(posedge RW0_clk) begin + ren_q <= RW0_en && !RW0_wmode; + raddr_q <= RW0_addr; + if (RW0_en && RW0_wmode) mem[RW0_addr] <= RW0_wdata; + end + assign RW0_rdata = ren_q ? mem[raddr_q] : 'x; +endmodule