diff --git a/.gitignore b/.gitignore index e73378ca..e23d392e 100644 --- a/.gitignore +++ b/.gitignore @@ -58,6 +58,7 @@ dump.fst # Compiled software (regenerated on demand by test/build/load scripts) sw.mem +sw64.mem sw.txt sw_ddr.mem sw_ddr.txt diff --git a/ROADMAP.md b/ROADMAP.md index 7600cb1c..1e6d29ad 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -22,7 +22,7 @@ exit (see "Standing invariants" and "Side quests"). - Documentation moves with the change that makes it stale — READMEs, module docs, registry descriptions — never as a follow-up. -## Phase 0 — Harden the Linux substrate (current) +## Phase 0 — Harden the Linux substrate (done) Until 2026-07-26 the no-MMU boot depended on a post-link binary mutation of the kernel (in what is now `patch_linux_image.py`) that rewrote the @@ -91,13 +91,23 @@ Exit: N consecutive unpatched-kernel boots pass the deepened CI on Verilator and QEMU, and a hardware boot is demonstrated on at least one board. -## Phase 1 — RV64 (RV64GCB, still M/U, still no-MMU) +## Phase 1 — RV64 (RV64GCB, still M/U, still no-MMU) (current) Widen the core to XLEN=64 before touching the privilege architecture, so the datapath disruption is validated while the environment is still one-dimensional, and so the MMU is built once, directly for Sv39. The struct plumbing is already XLEN-parameterized (`riscv_pkg`); the FP -regfile, LQ/SQ data paths, and CDB already carry 64-bit values for D. +regfile and CDB already carry 64-bit values for D. (The phase-entry audit +qualified the rest of that assumption: below the queues the memory tier is +32-bit per transaction — FLD/FSD are two-phase word pairs — so a native +64-bit data tier is in scope, sequenced first and proven under rv32.) + +Execution is planned in [docs/rv64/phase1_plan.md](docs/rv64/phase1_plan.md) +(strategy, decisions D1–D15, milestones M0–M8 with gates), grounded in the +[XLEN=64 readiness audit](docs/rv64/xlen_audit.md) — 365 file:line findings +against `9b76e39`, including 106 silent-misbehavior hazards, three +adversarially-verified readiness claims, and the empirical tool-behavior +checks. Scope: RV64I W-instructions and 64-bit shifts/compares/AGU, RV64 M (64×64 MUL/MULH, 64-bit DIV), RV64 A (LR.D/SC.D/AMO*.D), RV64 F/D diff --git a/docs/rv64/m1_data_tier.md b/docs/rv64/m1_data_tier.md new file mode 100644 index 00000000..58456497 --- /dev/null +++ b/docs/rv64/m1_data_tier.md @@ -0,0 +1,147 @@ + + +# M1 — the 64-bit data tier (design note) + +Interface contract for [phase1_plan.md](phase1_plan.md) Milestone M1 / +decision D2: the data-memory tier below the load/store queues becomes +native 64-bit single-beat, implemented and proven **while the core is +still rv32** (the tier width is deliberately XLEN-independent). The +FLD/FSD two-phase machinery is deleted, not generalized — the +[audit](xlen_audit.md) established that phased dwords cannot serve RV64 +(torn `mtime` reads, unphaseable AMO*.D) and that reusing them for INT +dwords is throwaway work. + +Everything here is provable by the existing rv32 suites: FLD/FSD exercise +every widened path single-beat, rv32 word/half/byte ops exercise the +lane/strobe machinery, and the Linux boot lanes exercise the CLINT. + +## The bus contract + +New `riscv_pkg` constants (data-tier width, not XLEN): + +```systemverilog +localparam int unsigned MemDataBits = 64; // data-tier beat width +localparam int unsigned MemStrbBits = MemDataBits/8; // 8 byte lanes +``` + +- **Aligned-dword view.** Every data-side bus (BRAM tier, cached tier, + MMIO, router, adapter responses) carries the aligned dword at + `addr[31:3]`; byte lane *i* is byte address `{addr[31:3], i}`. Consumers + extract by `addr[2:0]`; producers position by `addr[2:0]`. +- **Write positioning by replication.** Store data is replicated across + the beat (`{8{byte}}`, `{4{half}}`, `{2{word}}`, dword pass-through) and + the 8-lane strobe selects: `BYTE = 8'h01 << addr[2:0]`, + `HALF = 8'h03 << {addr[2:1],1'b0}`, `WORD = addr[2] ? 8'hF0 : 8'h0F`, + `DOUBLE = 8'hFF`. Replication keeps the data mux shallow (no + byte-lane shifter) — the same shape the 32-bit tier uses today, one + level wider. +- **Read extraction is uniform.** `load_unit` gains an `addr[2]` word + select ahead of its existing half/byte selects; FLD consumes the full + beat. The MMIO mux positions register values in their address-matching + lanes so extraction needs no MMIO special case (a 32-bit register at + offset +4 appears in lanes [63:32]). +- **Misalignment is unchanged.** The existing size-cased checks already + implement the 8-byte class (`|addr[2:0]` for DOUBLE); a dword access + never spans beats, so no crossing logic exists anywhere. + +## Memory-side changes (M1.1) + +- **`data_mem_request_router`**: all data ports `MemDataBits`, strobes + `MemStrbBits`; AMO writes (word-sized until RV64A in M3) drive + `addr[2] ? 8'hF0 : 8'h0F` with `{2{amo_data}}` instead of hardcoded + `4'b1111`. +- **`cached_tier_adapter`**: beat width 64 — word select becomes the + dword index `addr[4:3]` (4 dwords per 32 B line), wstrb placement + `addr[4:3]*8 +: 8`. Line geometry and the single-outstanding contract + are untouched (that is Phase 2). +- **dmem (BRAM tier)**: one 64-bit-wide byte-enabled BRAM at half depth. + Init comes from a new `sw64.mem` (64-bit `$readmemh` tokens) emitted by + the same objcopy step that produces `sw.mem`; `sw.mem` itself is + unchanged (imem stays 32-bit-word organized for the predecode sideband, + and every loader/image format stays 32-bit-word per the audit's + keep-the-formats recommendation). The instruction-programming /JTAG + Port A keeps its 32-bit face through a lane adapter (`addr[2]` steers + the strobe nibble) so `load_software`/`file_to_ddr` are untouched. +- **MMIO**: `mmio_read_data` becomes 64-bit with lane positioning; + 32-bit register writes honor the strobe lanes (so the existing lo/hi + word decodes keep working unchanged). The **CLINT** gains what RV64 + requires and rv32 can already verify: `mtime` at `+0xBFF8` is + dword-aligned, so a 64-bit read returns the whole counter in one beat + (single-copy atomic — no more hi/lo/hi loop tearing exposure) and a + 64-bit `mtimecmp` write lands atomically; the 32-bit lo/hi aliases + remain as strobed half-writes with identical rv32 semantics. UART/FIFO + registers are 32-bit-access-max, documented in the MMIO map (an SD to + them writes the addressed word lanes only). +- **`FROST_XILINX_PRIMS`** FDRE loop already follows `$bits` (M0). + +## Queue-side changes (M1.2) + +- **LQ**: delete the FLD two-phase state (`fp64_phase`, the `+4` second + beat, phase-advance/re-issue arms, response lo/hi steering); merge the + split lo/hi data RAMs into one FLEN-wide RAM; the `is_fp && + MEM_SIZE_DOUBLE` conjunctions collapse to size-only tests (the audit's + hazard list — un-gating these is exactly what RV64 LD reuses in M3). + MMIO loads ride the same aligned-dword view. +- **`load_unit`**: input becomes the 64-bit beat; extraction order + `addr[2]` word select → existing half/byte selects → existing + sign/zero extension. FLD passes the beat through. (RV64 LW/LWU/LD + semantics arrive in M3; at rv32 this is bit-identical behavior.) +- **`lq_l0_cache`**: dword-granule lines — index `addr[3 +: IW]`, 64-bit + data, tag `addr[31:3+IW]` (D3 keeps tags physical-width). Fills come + from full beats; stores/AMOs invalidate the containing dword + (conservative for sub-dword stores — same policy as today, one + granule coarser). FLD becomes L0-eligible for free. +- **SQ**: delete the FSD two-phase drain (`sq_fp64_phase`, `+4` leg, the + completes-gate) and the "doubles fly alone" pipelining exclusion — + DOUBLE joins the plain fast-drain; `gen_byte_en`/`gen_write_data` + follow the bus contract above. +- **`sq_forwarding_unit`**: overlap model moves to dword granule — + `store_off` becomes `addr[2:0]` (3 bits), byte masks 8-lane, image + reconstruction shifts by `{store_off,3'b0}` up to 56, the + size-compatibility matrix becomes: exact-dword forward + (DOUBLE→DOUBLE), covered-subset forward for any load whose 8-lane mask + is a subset of the store's, and the `double_hi_match` special cases + disappear (a dword store fully covers both its words by construction). + The hand-tiled equality comparators re-tile at `[31:3]` granule. +- **AMO/LR/SC**: word-sized semantics unchanged (RV64A is M3); only the + strobe/lane positioning adapts. The reservation granule stays word for + now (M3 widens it); the interrupt-shield/orphaned-write machinery is + untouched in shape — `restore_window_stress` must pass unmodified. + +## Verif mirror (M1.3) + +`memory_model` driver/monitor takes 8-lane strobes + 64-bit data; +`memory_utils.calculate_byte_mask_for_store` gains the 8-lane forms; +`test_directed_multicycle`'s FLD lo/hi split-word modeling and +`test_directed_atomics` init patterns update to single-beat. All keyed +off `config` widths (M0 centralization). + +## Gate (before M1 is done) + +rv32, all green: rv32ud + rv32uf riscv-tests; arch F/D bram batches; +`fpu_assembly_test`, `ddr_test`, `ddr_heap_test`, `ddr_atomic_test`, +`ddr_smc_test`; torture (FLD/FSD-heavy) both tiers; `frost_cache` bench; +`restore_window_stress`; `directed_atomics` (CLI); LQ/SQ/L0 formal +targets re-proven (depths/timeouts re-measured per the audit's formal +budget warning); the Linux boot-health cocotb leg (CLINT compatibility); +plus a synthesis-only timing probe (Yosys UltraScale+ target) to size +the widened BRAM write cascade before M2's flip. + +Documentation moving with this change: `hw/rtl/README.md` MMIO map +(CLINT 64-bit access + UART/FIFO 32-bit-max), `linux/README.md` counters +section (mtime single-copy-atomic 64-bit read), store_queue/load_queue +READMEs (two-phase sections retire), tomasulo README FP-phasing rows. diff --git a/docs/rv64/phase1_plan.md b/docs/rv64/phase1_plan.md new file mode 100644 index 00000000..9e6fff19 --- /dev/null +++ b/docs/rv64/phase1_plan.md @@ -0,0 +1,484 @@ + + +# Phase 1 execution plan — RV64GCB + +Execution plan for [ROADMAP Phase 1](../../ROADMAP.md): widen FROST from +RV32GCB to RV64GCB (still M/U-only, no MMU, FLEN=64, physical map unchanged +below 4 GiB). Grounded in the [XLEN=64 readiness audit](xlen_audit.md) +(365 findings against `main` @ `9b76e39`); read that document's verdicts +section first — two of its results shape this plan's spine. + +Exit criteria (from the roadmap, made concrete in Milestone M8 below): +rv64 suites green in CI, an rv64 no-MMU Buildroot Linux boots in CI, X3 +timing re-closed at 300 MHz post-route, and the rv32-configuration +keep-or-freeze decision recorded with measured costs. + +## Strategy + +1. **rv32 stays the referee until rv64 can testify.** Until rv64 suites + run, the rv32 matrix is the only regression evidence, so every landing + keeps rv32 CI green and rv64 capability accretes behind a build define. + The roadmap's freeze-vs-maintain decision stays open at zero incremental + cost until phase exit, when its true price (RTL generate branches + CI + lanes) has been measured rather than estimated. + +2. **The 64-bit data tier lands first, under rv32.** The audit refuted the + assumption that FLD/FSD imply an 8-byte memory path: doubles are + two-phase word pairs end-to-end, the AMO/reservation machinery is + word-granular, and a phased 64-bit read of the rolling `mtime` counter + tears — so RV64 requires a native 64-bit single-beat data tier + regardless. That widening is XLEN-*independent*: FLD/FSD and the + existing rv32 suites (riscv-tests rv32ud, arch F/D batches, torture, + ddr_test/ddr_atomic_test, the frost_cache bench, LQ/SQ formal targets) + exercise every widened structure. Landing it as its own rv32-green + milestone (M1) removes the largest, most bug-prone RV64 dependency from + the flip itself — and deletes the two-phase FSM complexity and its + "doubles fly alone" drain restriction as a bonus. + +3. **Substrate before semantics.** A behavior-preserving parameterization + pass (M0) kills the audit's hazard classes — hardcoded sign-extension + replication counts, 32-bit cause/overflow constants, `[31:0]` struct + fields, XLEN-relative region decodes — while everything still runs + rv32. Each fix is mechanically reviewable and the full rv32 matrix + gates it. + +4. **Then climb the ISA in test-suite rungs.** Decode+execute for one + extension group at a time, each gated by its rv64 riscv-tests / + arch-test batch before the next begins: I → B/K at 64 → M → A → F/D + converts → C recode → CSR/trap surface → torture → programs → Linux. + +5. **Timing risk is concentrated, not diffuse — and mostly avoidable.** + The known-critical rename/wakeup/CDB structures are tag-indexed or + already FLEN-wide: they do not change. The 64-bit exposure is (a) + ALU/shifter/compare datapaths, (b) LQ/SQ address CAMs, (c) BTB tag + compare, (d) AGU adders, (e) mul/div. The address-side exposure is + neutralized by the canonicalization policy (D3): PCs and effective + addresses are constrained to the sub-4-GiB physical space at their + producers, so synthesis constant-sweeps the upper 32 bits of every + downstream PC/address register, compare, and prediction RAM. What + remains is the arithmetic datapath itself, probed early with + synthesis-only runs (M1/M3 checkpoints) and closed for real at M8. + +## Decisions + +Numbered so later commits can reference them (e.g. "per D3"). Each is a +recommendation adopted by this plan unless a probe overturns it; overturning +one updates this file in the same change. + +- **D1 — XLEN mechanism: build-time define, package remains the single + source of truth.** `riscv_pkg` derives `XLEN` from a `FROST_RV64` + preprocessor define (default off → 32). Module-level `parameter XLEN` + defaults change from `32` to `riscv_pkg::XLEN` (the audit found the + bare-32 defaults are a live footgun for standalone unit benches); + `tomasulo_wrapper` has no XLEN parameter and bakes package structs into + its ports, so per-instance dual-width elaboration is illusory — the + define is the only honest dual-XLEN mechanism. Where RV32/RV64 + *semantics* differ (decode legality, RVC tables, CSR legality, misa), + code branches on `XLEN == 64` in ordinary generate/ternary form. The + define is plumbed through tests/Makefile (`COMPILE_ARGS`), the Yosys + targets, formal `.sby` files, and the Vivado build as one switch. + +- **D2 — Data tier: widen to native 64-bit single-beat.** 64-bit + data buses and 8-lane byte enables end-to-end: SQ drain / LQ fill / + router / cached_tier_adapter (dword lane select `addr[3+:...]`) / BRAM + tier / MMIO. Delete the FLD/FSD two-phase FSMs, the `+4` second-beat + legs, and the DOUBLE drain-pipelining exclusion. The CLINT gains native + 8-byte access at `mtime`/`mtimecmp` (single-copy-atomic, as RV64 Linux + expects) while keeping the 32-bit lo/hi aliases for rv32 compatibility. + UART/FIFO MMIO registers are declared 32-bit-access-max and documented as + such in the MMIO map. dmem organization: decided at implementation + between one 64-bit byte-enabled BRAM (needs a Port-A 32→64 lane adapter + for the JTAG loader and a second `$readmemh` init format) and two + interleaved 32-bit banks (keeps `sw.mem` and the loader byte-exact); + bias toward whichever produces the smaller verified diff — the image + *file formats* stay 32-bit-word either way (audit: they are memory-side, + not XLEN-side). imem stays 32-bit-word organized (predecode sideband is + per-word). + +- **D3 — Addresses: full-width architectural state, producer-side + canonicalization to the 4-GiB physical space.** CSRs that hold + addresses (mepc/mtvec/mscratch/mtval) keep full 64-bit storage — + arch-test privilege suites probe their WARL behavior against Spike, so + masking *storage* risks signature mismatches. Instead, addresses are + canonicalized where they become fetch/memory addresses: the PC-redirect + producers (branch/JALR resolution, trap/mret targets entering the PC + mux) and the AGU outputs mask bits [63:32] to zero. Consequences: + every downstream PC register, BTB/RAS/L0 tag, ROB PC RAM, forwarding + CAM, and region decode ([31:30] MMIO quadrant, bit-31 tier select) sees + structurally-zero upper bits and synthesis sweeps them — the audit's + address-side storage/timing growth largely evaporates, and the + hand-tiled SQ comparators keep their current tiling. Region decodes are + simultaneously fixed to physical-bit form (`addr[31]`/`addr[31:30]`, + never `[XLEN-1]` — closing the `if_stage.sv:884` dead-guard hazard). + Simulation-only assertions flag any nonzero [63:32] reaching the seams, + so nonconformant software is caught loudly in sim instead of aliasing + silently. Software that jumps to a non-canonical address observes + aliasing rather than an access fault; a real fetch/access-fault path is + deliberately deferred to Phase 3's PMA work. + + Why this is signature-safe against Spike (independently cross-checked + against the vendored test-suite and toolchain sources): standard rv64 + codegen for this map is PC-relative (`-mcmodel=medany`, `la`, `auipc`), + and both riscv-tests and riscv-arch-test form control-flow and data + addresses that way, so with a canonical entry PC every suite-reachable + address — including the ones landing in mtval on misaligned-access + tests — is already sub-4-GiB positive and masking is the identity. RV64 + `lui` does sign-extend, but a bare `lui`-materialized DDR-region + address is exactly the class that would raise an access fault on + Spike's own memory map (nothing exists at 0xFFFFFFFF_8xxxxxxx), so no + portable signature test can rely on it; gas's `li` pseudo materializes + exact positive constants, so the boot shim's `li` addresses stay + canonical (verified again at M7 as part of the shim review). Recorded + invariant: the physical map stays below 4 GiB through Phase 1 (also + pins cache-tier `ADDR_WIDTH=32` and the fixed `[31:0]` AXI wiring as + documented contract, per audit). + +- **D4 — Prediction storage: keep derived widths, rely on D3 sweeping.** + No explicit tag/target compression work. If post-M3 synthesis probes + show the BTB tag compare or LUTRAM growth surviving the sweep (they + should not), revisit with explicit 32-bit stored targets. + +- **D5 — Immediates: widen `from_id_to_ex_t` fields to `[XLEN-1:0]` and + fix sign-extension in `immediate_decoder` (XLEN-relative replication + counts).** One point of truth beats auditing every consumer for + mandatory `signed'` casts; the cost (~160 flops/slot, wider RS payload) + is inherent to carrying 64-bit immediates and lands inside the M0 + substrate where rv32 equivalence is checkable. The PD-stage B-immediate + replications get the same one-line fixes. + +- **D6 — `store_op_e` gains `STD` (widen to 3 bits, `STN` stays 0 for + Verilator 2-state init).** The audit resolved the apparent conflict: the + SQ data path keys on `mem_size_e` (already has `MEM_SIZE_DOUBLE`), so + `STD` is decode/dispatch-side classification only. `instruction_type_decoder` + additionally gains a double-word load class (LD must not classify as a + word load). + +- **D7 — Multiplier: rebuild as 65×65→128 via `dsp_tiled_multiplier_unsigned` + plus a sign-correction wrapper, uniform latency.** The bespoke 33×33 unit + retires. Pipeline depth becomes whatever closes 300 MHz (expect 5–6); + `MulPipeDepth` in `int_muldiv_shim` is derived from a shared localparam + exported by the multiplier, never hand-copied (audit hazard). MULW takes + the same pipe (no early-out — variable latency breaks the shift-register + tracker); result select becomes a tracked 3-way (low64 / high64 / + sext-low32). + +- **D8 — Divider: stay 2-bits/stage at 64 (33-cycle latency), W-forms + share the 64-bit path with a tracked sext flag.** Uniform latency keeps + the tracker simple. If the doubled per-stage subtract chain misses + 300 MHz, fall back to 1 bit/stage (65 cycles) rather than restructuring. + +- **D9 — Dual-XLEN through the phase; fate decided at exit with measured + costs.** rv32 CI lanes run unchanged throughout; rv64 lanes are added + per milestone. At M8, record: lines of `XLEN==64` generate branching, + CI wall-time of the dual matrix (~71 sim jobs today; dual roughly + doubles), and the maintenance friction observed — then decide + keep-dual / demote-rv32-to-nightly / freeze-rv32, and reword the docs' + ISA claims accordingly. + +- **D10 — Spike gets pinned into the Docker image.** A + riscv-isa-sim build stage joins the Dockerfile so golden-reference + regeneration (all arch-test + torture references, both XLENs) is + containerized and reproducible. This unblocks and gates every rv64 + signature suite; the currently-unpinned host-Spike provenance is + retired. + +- **D11 — Signatures stay 4-byte/8-hex; references are namespaced per + XLEN.** Matches riscv-arch-test granularity conventions and minimizes + extractor churn; fixes the audit's reference-path collision + (`rv64i_m/I/add-01` currently resolves to the rv32 golden). Torture's + signature layout is re-derived for 64-bit GPRs (64 GPR words) in the + same change as its rv64 generator config. + +- **D12 — Linux: add an rv64 image alongside rv32; both boot lanes run + during the phase.** rv64 defconfig derived from Buildroot's + `qemu_riscv64_nommu_virt` (BR2_RISCV_64, uClibc + elf2flt bFLT — the + upstream-precedented rv64-nommu userspace; verified in the vendored + trees: `ARCH_HAS_BINFMT_FLAT` is unconditional for riscv, Buildroot's + own riscv64-virt nommu kernel config uses `CONFIG_BINFMT_FLAT`, and + nommu kernels default `RELOCATABLE`, i.e. PC-relative and + D3-canonical). One watch item: Buildroot's arch config gates some + hard-float ABI choices on MMU support, so whether userspace is lp64d + or falls back to lp64 soft-float is confirmed when the defconfig is + re-derived — the kernel side is unaffected either way. Kernel base + config regenerated (CONFIG_ARCH_RV64I), rv64 DTS strings emitted by + `build_fpga_boot.py`, the boot shim's address materialization verified + per D3, `patch_linux_image.py`'s rv32-pinned byte offsets re-derived or + gated off for rv64, `frost-stress` gains an `__riscv_xlen==64` counter + path (single 64-bit `csrr`; counters-unavailable stays a FROST-side + failure), and the QEMU lane runs `qemu-system-riscv64 -cpu rv64,mmu=off` + (binary already in the image). + +- **D13 — FreeRTOS demo freezes at rv32 during the phase.** Its + FROST-local port (`port_frost_asm.S`, 4-byte context frames) needs a + full sd/ld rewrite for rv64; not on the critical path. Ported at exit + only if D9 resolves to keep-dual; otherwise its fate is decided with + D9. + +- **D14 — Custom `mperfdata`/`mperfdatah` keep the split 32-bit pair.** + Custom CSR space, zero software churn; documented as a deliberate + rv64 inconsistency with the architectural counters. + +- **D15 — `mstatus.FS` gets a real (still minimal) implementation in + M5.** The audit surfaced that FS is absent-hardwired-0 while F/D + execute — a latent nonconformance that rv64 Linux with hard-float + userspace actively depends on. Cross-checking the pinned 6.18.7 tree + confirmed the exact mechanism: `fstate_save` saves FP state only when + the trapped task's saved status shows FS==Dirty, and `fstate_restore` + skips when FS==Off (`arch/riscv/include/asm/switch_to.h`), so + *hardware Dirty-setting reflected into the trap-time mstatus image* is + the load-bearing behavior — without it every task's FP context is + silently never saved. Scheme: FS a writable 2-bit field, hardware sets + Dirty on any FP-architectural-state write (FP regfile or fcsr), SD + mirrors FS==Dirty at bit 63, **and FP instructions raise + illegal-instruction when FS==Off** — the Off-trap is not optional: + riscv-tests' machine-mode `csr` test asserts an `fsw` with FS clear + either traps or has no effect (it is plausibly why `csr` sits in + today's skip list; landing D15 is what makes it un-skippable on both + XLENs). Verified by directed test, the un-skipped `csr` test, and the + rv64 Linux boot itself. (This also retro-fixes the latent rv32 + exposure.) + +## Workstreams and milestones + +Workstreams are the units of review; milestones M0–M8 are the ordered +landing sequence on `main`, each with an acceptance gate. Standing gate for +*every* milestone: the full rv32 matrix (cocotb both tiers, riscv-tests, +arch, torture, formal, Yosys targets, Linux lanes) stays green. + +### M0 — Substrate: XLEN-clean at rv32 (Workstream A) + +The behavior-preserving parameterization pass over the audit's hazard + +mechanical inventory. Highlights: immediate_decoder/pd_stage replication +counts → XLEN-relative (D5, including the struct-field widening); +`IntMachine*` → XLEN-wide with the interrupt bit at `XLEN-1`; 64-bit +DIV-overflow constants added alongside the 32-bit ones; region decodes to +physical-bit form (D3 producer masks + sim assertions); module XLEN +defaults → `riscv_pkg::XLEN`; `c_ext_state`/aligner/`cpu_ooo.i_served_addr` +port widths; `cpu_tb` taps re-expressed in `riscv_pkg::XLEN`; the +`FROST_RV64` define plumbed through every build entry point (D1); verif +width constants centralized (one shared module replacing the ~16 private +`XLEN=32` copies feeding struct packers); `FROST_XILINX_PRIMS` FDRE loop +bound by `$bits`; retire-trace/hang-triage formats widened. + +Gate: rv32 matrix green (bit-identical behavior expected); with +`FROST_RV64=1` the tree *elaborates* (Verilator lint + Yosys generic) even +though rv64 semantics are absent. Zero-width-replication cleanup is +explicitly optional (audit proved all pinned tools accept it) — done only +where touched anyway. + +### M1 — 64-bit data tier, proven under rv32 (Workstream C) + +D2 in full: buses/strobes widened, LQ/SQ single-beat DOUBLE (two-phase +FSMs deleted), forwarding matrix reworked at dword granule (3-bit store +offset, 8-lane masks, SD→LD/FLD exact-forward cases), L0 dword geometry, +AMO strobe derivation, CLINT 8-byte access, MMIO 64-bit read mux, dmem +reorganization + loader adapter, `memory_model`/`memory_utils` widened to +match. + +Gate: full rv32 matrix green — with attention called to rv32ud, arch F/D +batches, `ddr_test`/`ddr_heap_test`/`ddr_atomic_test`, torture (FLD/FSD +heavy), `restore_window_stress` (its DDR-draining-SC shape exercises the +drain path), the frost_cache bench, and re-tuned `load_queue`/`store_queue` +formal targets (depth/timeout re-measured per audit). rv32 Linux boot lanes +green (CLINT alias compatibility). First synthesis-only timing probe +(Yosys UltraScale+ target + Vivado synth) to size the widened BRAM write +cascade and forwarding CAMs. + +### M2 — First rv64 instruction retires (Workstreams B+E minimum) + +Smallest end-to-end rv64 slice: `common.mk` march/mabi axis +(`rv64…/lp64d`) + `standalone_asm.mk` emulation + the 9 per-app +ARCH/ABI Makefiles parameterized; a hand-written rv64 asm smoke app +(W-op/LD/SD/shamt6 sanity, no libc) through the cocotb custom-program +flow; decode/ALU/load-unit minimum to run it (full WS-B lands in M3). +Predecode init generator consumed per-XLEN (C-table still rv32-shaped +until M4 — the smoke app avoids compressed code, built `-march=rv64i…` +no-C). + +Gate: smoke app passes in cocotb under `FROST_RV64=1`; rv32 matrix green. + +### M3 — RV64 integer ISA complete (Workstream B) + +- Decode: OP-IMM-32/OP-32 arms, LD/LWU/SD, 6-bit shamt legality + restructure (funct3=001/101 collision matrix per audit), ZIP/UNZIP → + illegal, REV8 immediate re-key, ZEXT.H → PACKW alias, all four + op-classification copies + dispatch mem_size/mem_signed/imm-select + cases updated in lockstep — with the audit-recommended simulation + assertions that convert silent-default misclassification into loud + failures (e.g. any op with `mem_needs_lq/sq` must resolve a non-default + `mem_size`). +- ALU: 64-bit shifts/rotates (6-bit shamt channel through decode → RS → + shims), W-op family with single final sext mux, Zbs 6-bit indices, + SEXT_B/H fixes, 64-bit ORC_B/REV8/BREV8/PACK, ctz64/cpop64 helpers, + CLZW/CTZW/CPOPW; dead `ENABLE_MULDIV` legacy path deleted. +- M: D7 multiplier + D8 divider, shim trackers re-derived. +- A: LR.D/SC.D/AMO*.D decode + LQ AMO width bit (32-bit sub-ALU + + sext for .W forms — a semantic fix for .W at 64), 8-byte reservation + granule everywhere (sc_pending, wrapper snoop, LQ invalidate — audit + lists the `[XLEN-1:2]` sites), wrapper `make_lq_alloc`/`make_sq_alloc`/ + `is_sc` classification extended. +- F/D: fp_convert INT_WIDTH split (W vs L bounds; sign bit `XLEN-1` fix), + the 10 new convert/FMV ops through the three classification expressions + + stage-4 case + shim routing with a latched int-width flag, + W-form results sign-extended (including WU and FMV.X.W — RV64 semantic), + FEQ/FLT/FLE boxing fix in `fpu_compare_unit`, missing `.XLEN` override + at `fp_add_shim:340`, LQ NaN-box concats made explicit `[31:0]`. +- Verif mirror (Workstream E, interleaved per rung): encoders (shamt6 + layout, W-op factories, AMO funct3 param), alu/branch/fp model widening + (parametric to_signed/to_unsigned, 64-bit helpers, W evaluators, + LW-sign-extends semantic), suite runners re-keyed (rv64 suites + + re-derived skip lists — including *removing* the rv32ud `move` skip: + FMV.X.D/FMV.D.X become the new coverage), reference namespacing + D10 + Spike-in-Docker + full golden regeneration, K-filter re-derivation + (packw in, zip/unzip out). + +Gate (cumulative, bram tier first then ddr): riscv-tests rv64ui → rv64um → +rv64ua → rv64uf/ud green; arch-test rv64 I/M/A/F/D/B/K/Zicond batches +green; branch-unit and id_stage cocotb suites extended with +64-bit-discriminating vectors (audit's vacuous-pass list). Second +synthesis-only timing probe (64-bit ALU/mul/div in place). + +### M4 — RV64C recode (Workstream D) + +The five-site lockstep recode: `rvc_decompressor` (C.ADDIW, C.LD/C.SD/ +C.LDSP/C.SDSP with 8-scaled immediates, C.SUBW/C.ADDW, 6-bit C-shamts, +reserved-rd checks), `riscv_pkg::imem_compressed_control` + +`imem_rvc_source_hot`, the aligner's `slot1_branch_compressed`, +`frontend_validity_tracker`'s classifier, `ras_detector`'s C.JAL gate, and +the offline generator `generate_imem_predecode_init.py` — plus regenerated +Vivado init images. Unit-test policy: RV32-reserved-now-valid vectors are +replaced by positive RV64 vectors (decompressor/aligner/predecode/ +ras_detector suites per the audit's inverted-test list); the +RTL-vs-Python cross-checks and the independently-derived fast-replica +model are re-derived from the RV64 spec, not copy-pasted. + +Gate: rv64uc + arch C batch green; predecode/decompressor/aligner/ +ras_detector cocotb suites green with the new positive vectors; rv32 +equivalents still green (dual C-table per D1/D9). + +### M5 — CSR/trap/counter surface (Workstream F) + +misa (MXL=2, correct letter set), mstatus 64-bit rebuild (UXL=2 hardwired, +SD at 63, D15 FS implementation), mcause bit-63 plumbing end-to-end +(constants landed in M0; trap_unit/formal properties verified), counters as +single 64-bit CSRs with `*h` addresses illegal at every privilege (csr_file +read mux + the ROB `ucounter_onehot` addr[7] re-restriction + mcounteren +gate unchanged for the base three), mepc/mtvec write masks reviewed under +D3, sw `csr.h`/`trap.h`/`limits.h` widened (`unsigned long` accessors, +`MCAUSE_INTERRUPT_BIT` at `__riscv_xlen-1`, rd*64 single-csrr on rv64), +umode_test/csr gating matrix rewritten for rv64 semantics (base counters +gated by mcounteren; `*h` trap unconditionally). + +Gate: rv64mi + arch privilege batch green; directed traps + umode_test +rv64 variants green; the riscv-tests `csr` skip re-evaluated now that +D15's FS lands (expected: un-skippable on both XLENs); formal +csr_file/trap_unit targets re-proven at 64. + +### M6 — Randomized + program-level convergence + +Torture rv64 (generator config, 64-bit signature layout per D11, new +Spike goldens), cocotb real-program suite at rv64 both tiers (isa_test +rv64 expectations fork, c_ext_test rv64 variant, new W-op/LD/SD/AMO.D +directed apps registered in `TEST_REGISTRY`), coremark/coremark-pro +rebuilt lp64d (stack/ROM budgets re-measured per audit's link.ld note), +`cpu_random`/multicycle registry notes updated. FreeRTOS demo marked +rv32-only per D13. + +Gate: torture rv64 green both tiers; full program suite green both tiers +at rv64; rv32 matrix still green. + +### M7 — rv64 no-MMU Linux (Workstream G) + +D12 in full. Bring-up order: QEMU rv64 boots the image first (validates +kernel/userspace/DT independent of RTL), then cocotb bounded boot-health +(the 22M-cycle checkpoint regression re-baselined for rv64), then +hardware. `restore_window_stress` re-run at rv64 (sret comes in Phase 3; +the M-mode window shape is unchanged but W-op/64-bit restore images are +new). Both Linux lanes (rv32 + rv64) in CI. + +Gate: `linux-boot-qemu` (rv64) asserts `FROST_USERSPACE_STRESS_PASS` + +login prompt + counter deltas present; `linux-boot-cocotb` (rv64) reaches +its checkpoint; rv32 lanes unaffected. + +### M8 — Timing closure, hardware, and the exit decision (Workstream H) + +Full X3 Vivado place+route at 300 MHz (per the Vivado run policy this is +the feature-final run; interim probes were synthesis-only), WNS ≥ 0; +iterate on the audit's predicted hot cones (BRAM write cascade, forwarding +CAMs, branch-target equality, divider stages) as needed. Genesys2: build +attempted; if the 64-bit datapath does not fit/close at 133 MHz on the +~69%-full Kintex-7, Genesys2 is documented rv32-only pending a slimming +side-quest — this plan treats rv64 as X3-first and does not gate the phase +on Genesys2. Hardware: 10-boot rv64 soak on X3 via `linux_boot_soak.py`; +`hw_regression.py` BASELINE_SCORES reset to None at the switchover and +re-recorded from phase-exit hardware runs; README utilization/CoreMark +tables regenerated. Record the D9 decision with measured costs; update +ROADMAP Phase 1 to done; sweep the audit's documentation-staleness list +(README ISA claims, per-module READMEs' latency/phasing/reservation text, +sw/CONTRIBUTING templates, linux/README ABI - ISA string, counter section). + +Gate = the roadmap exit criteria, plus: every audit finding is either +closed by a commit or explicitly recorded as deferred-with-rationale in +this directory. + +## Verification additions beyond the mirrored matrices + +Directed coverage the mirrored suites don't guarantee (tracked as new +cocotb tests/apps as each area lands): W-op sign-extension corners +(negative 32-bit results into 64-bit rd), LW-vs-LWU extension, shamt +32–63 boundary set, SLLIW-with-bit-25 illegal, C.ADDIW-vs-C.JAL decode +(including rd=0 reserved), C.ADDIW-not-a-RAS-call regression, compressed +negative-offset PD-redirect (the audit found the compressed arm has zero +coverage today), LR.D/SC.D granule (SC.W-vs-LR.D pairing, snoop kills +across the dword), AMO*.W sext at 64, FCVT.L rounding/saturation corners, +FEQ-writes-exactly-0-or-1 at 64-bit rd, NaN-box checks for FCVT.S.L/LU, +counter `*h`-illegal traps from both privileges, mcause bit-63 interrupt +classification, mtime 8-byte single-copy-atomic read (torn-read +regression), 64-bit-discriminating branch-compare vectors, and D3 seam +assertions (no nonzero [63:32] reaches fetch/memory). + +## Risks and watch items + +- **Coverage debt in the giant files.** load_queue/store_queue/RS/wrapper + were audited grep-driven; M1/M3 implementation re-reads surrounding + logic as it changes, and the loud-assertion policy (M3) is the backstop + for what grep missed. +- **Suite-runner drift.** The skip/filter tables are re-derived (not + renamed) per audit — several rv32 skips invert meaning at rv64. +- **Formal budgets.** LQ/SQ/csr_file/trap_unit state grows; depths and + `SBY_TASK_TIMEOUT_S` re-measured at M1/M5 rather than discovered flaky. +- **Yosys/CI wall time.** The UltraScale+ synthesis lane already measures + ~2750 s against a 3600 s timeout; re-budget after the first rv64 + synthesis. Dual CI lanes roughly double ~71 sim jobs — acceptable during + the phase, revisited at D9. +- **Genesys2 fit.** Explicitly non-gating (M8). +- **lp64 stack/ROM growth.** link.ld budgets re-measured in M6. +- **`instr_op_e` enum ordinals.** id_stage's cocotb bench pins ordinals; + new ops are appended (documented append-only discipline) and the bench + table is regenerated in the same commit. + +## Working agreements for this phase + +- Every landing keeps the rv32 matrix green; rv64 gates accrete per + milestone. Feature branches per milestone (or finer), landing to `main` + sequentially. +- Documentation moves with each change (repo policy); the audit's + staleness inventory is the checklist, M8 sweeps the remainder. +- Line-referenced audit findings are checked off in commit messages by + content, not line number (the tree drifts). +- Timing evidence: synthesis-only probes at M1/M3 checkpoints; the one + full place+route at M8 per the Vivado run policy. diff --git a/docs/rv64/xlen_audit.md b/docs/rv64/xlen_audit.md new file mode 100644 index 00000000..d6fb0443 --- /dev/null +++ b/docs/rv64/xlen_audit.md @@ -0,0 +1,1326 @@ + + +# XLEN=64 readiness audit + +Point-in-time audit of everything that must change — or would silently +misbehave — if `riscv_pkg`'s `localparam XLEN` were flipped from 32 to 64. +Audited tree: `main` @ `9b76e39` (2026-07-28). Line numbers reference that +commit and will drift as Phase 1 lands; the finding descriptions are written +to stay locatable by content. + +This document is the working inventory for +[Phase 1 of the roadmap](../../ROADMAP.md) — the execution plan that +sequences it is [`phase1_plan.md`](phase1_plan.md). Scope: RV64GCB +(+Zicntr/Zifencei/Zicond/Zbkb/Zihintpause), still M/U-only, no MMU, FLEN +stays 64, physical memory map unchanged (sub-4-GiB). + +**Method.** Fifteen subsystem auditors swept every RTL, verification, +software, and build file (plus four gap-fill auditors dispatched by a +completeness pass); three adversarial verifiers attacked the load-bearing +"already 64-bit-ready" claims; findings are cited as `file:line` against the +audited commit. Every finding is classified: + +- **hazard** — compiles clean at XLEN=64 and silently misbehaves at runtime. + The most valuable class; there are 106 of them. +- **design** — new logic or semantics (W-ops, RVC recode, 64-bit mul/div, + CSR legality, …). +- **policy** — needs a project decision before code changes. +- **mechanical** — width/literal/comment hygiene fixed by parameterization. + +Totals: **365 findings** — 106 hazard, 104 design, 31 policy, 124 +mechanical. Coverage caveat from the completeness pass: the largest files +(`load_queue.sv`, `store_queue.sv`, `reservation_station.sv`, +`tomasulo_wrapper.sv`, `cpu_ooo.sv`) were audited by targeted (grep-driven) +reading rather than line-by-line; residual risk concentrates exactly where +findings are already densest, so implementation work in those files should +re-scan surrounding code as it lands. + +## The three load-bearing verdicts + +**1. "FLD/FSD exist, so the 8-byte LD/SD data path below dispatch is +already there" — REFUTED.** The entire memory system below the queues is +32-bit-per-transaction with 4-bit byte enables: FLD/FSD are synthesized by +two-phase state machines in the LQ/SQ that issue two independent word +transactions (`store_queue.sv:30/356/370/761`, `load_queue.sv:29`). The +load-side phase machinery is `is_fp`-gated at ≥6 sites, so an integer +`MEM_SIZE_DOUBLE` load would silently complete with a zero-extended low +word; the INT result-return path is structurally 32-bit at four formatting +sites; the word-granular L0 would serve a bogus single-word hit for an INT +dword; the AMO datapath and LR/SC reservation granule are word-wide; and the +CLINT is architected as lo/hi word pairs, so a phased 64-bit `LD` of the +rolling `mtime` counter would tear (RV64 software does single 8-byte CLINT +accesses and does not guard against tearing). What *is* already 64-bit: MEM +RS payloads, SQ entry storage and the forward payload struct, LQ entry +lo/hi halves, the CDB, and the 8-byte misalignment checks. Consequence: the +data tier below the queues must be widened to native 64-bit single-beat — +two-phase reuse is not a viable RV64 fallback — and this widening is +XLEN-independent, provable under the existing rv32 suites (see the plan's +Workstream C). + +**2. "Integer results ride FLEN=64 carriers end-to-end with +correct-at-64 extension points" — HOLDS with exceptions.** Every carrier +struct and RAM (RS src values, CDB, ROB value, RAT lookup, commit, regfile +buses) is FLEN-wide, extension points are the identity-at-64 +`{{(FLEN-XLEN){1'b0}}, x}` idiom, and no double-extension exists. The +exceptions are localized: `int_muldiv_shim.sv` hard-codes the 32-bit +MUL/DIV datapath widths (operand capture, product, result select — all +being rebuilt for RV64M anyway), and the four FLW NaN-boxing concatenations +in `load_queue.sv` (`{32'hFFFF_FFFF, xlen_data}`) silently drop the box at +XLEN=64. + +**3. "Front-end PCs widen coherently; only prediction storage has explicit +widths" — HOLDS, with the exception list inverted.** The in-core front end +(pc_controller, if_stage datapath, aligner handoff, all five prediction +structures) is XLEN-parametric — BTB tag/target widths are *derived* +(`TagBits = XLEN-9`), so prediction storage widens automatically; the cost +is LUTRAM growth and a 23→55-bit tag compare, not recode. The real porting +boundary is the fetch seam: `fetch_provider.sv` is a wholesale +hardcoded-32 island, `cpu_and_mem.sv` truncates the PC at the instantiation +boundary, and bit 31 doubles as the cached-tier select. One semantic +hazard: `if_stage.sv:884` guards a served-window desync check with +`pc_reg[XLEN-1]` meaning "cached region" — at XLEN=64 bit 63 is never set, +the guard silently dies, and the exact boot-Oops class it was added to stop +returns. There is no hardcoded reset vector (PC resets to `'0`). + +## Cross-cutting facts established empirically + +- **Zero-width replication is a non-issue.** `{{(FLEN-XLEN){1'b0}}, x}` + with FLEN=XLEN=64 was tested against every pinned tool: Verilator 5.050 + (`-Wall` clean), Yosys 0.64 (clean; SAT proves exact pass-through), + Verible (clean), Vivado 2025.2 xvlog (clean) and synth_design (benign + `Synth 8-693` warning, correct netlist). The ~15 sites flagged across + subsystems are cosmetics; rewrite as `FLEN'(x)` casts only for a + warning-free Vivado log. +- **The pinned container toolchain is rv64-ready.** xPack + riscv-none-elf-gcc 15.2.0-1 ships `rv64imafdc_zicsr_zaamo_zalrsc/lp64d` + multilibs; `qemu-system-riscv64` is already in the image. +- **Spike is NOT in the container.** Every arch-test/torture golden was + generated by an unpinned host Spike; regenerating all references at rv64 + is gated on pinning Spike (recommended: build riscv-isa-sim into the + Dockerfile). +- **Reference-path collision:** `rv64i_m/I/add-01.S` resolves to the + committed *rv32* golden signature today (`test_arch_compliance.py:196` + keys references by extension + stem only). + +## Top hazard clusters (what actually bites on a bare XLEN flip) + +1. **Sign-extension replication counts hardcoded for 32-bit results** — + `immediate_decoder.sv` (all I/S/B/J/U formats), `pd_stage.sv` (both + PD-redirect immediate arms). Every negative immediate, backward-branch + target, and LUI with bit 19 set silently zero-extends. The single + highest-blast-radius hazard class. +2. **mcause interrupt bit baked at 31** — `IntMachine*` constants, + `trap_unit`, sw `csr.h`, verif checks. Every interrupt would read as a + giant synchronous exception; self-consistent constants mean sim and + formal stay green while Linux misroutes every interrupt. +3. **RVC C-table reinterpretation** — C.JAL→C.ADDIW (decompressor + 3 + sideband/classifier copies + the offline Python generator + ras_detector, + which would push a garbage RAS entry on every C.ADDIW), + C.FLW/C.FSW/C.FLWSP/C.FSWSP→C.LD/C.SD/C.LDSP/C.SDSP (8-scaled + immediates, integer regfile), C.SUBW/C.ADDW/shamt6 un-reserving. +4. **5-bit shamt truncation** — ALU shifts/rotates/Zbs indices, both int + shims, the encoder's `(sh & 0x1F) | (f7 << 5)` packing, and the verif + ALU model. Shift amounts 32–63 silently alias 0–31 everywhere. +5. **32-bit DIV/REM special-case constants** — INT_MIN/-1 and div-by-zero + detection silently stops firing at 64 (RTL and Python model both). +6. **Word-shaped address comparators** — the SQ/forwarding hand-tiled + comparators ignore diff bits above [29]/[31]; `lq_l0_cache`'s + XLEN-relative MMIO decode makes MMIO cacheable at 64; `sq_forwarding`'s + 2-bit store offset misplaces bytes in a 64-bit image. +7. **Counter CSR surface** — cycleh/timeh/instreth (and mcycleh/minstreth) + must become illegal; the ROB's `ucounter_onehot` deliberately aliases + addr[7] so they'd stay mcounteren-gated-legal; `csr_file` returns + `[31:0]` slices of the 64-bit counters; the just-landed frost-stress + counter phase reads `0xc80-0xc82` and would silently report + `counters=unavailable` on rv64 while still printing PASS. +8. **Silent-default classification whitelists** — the op-classification + lists exist 4× (id_stage slot-1/slot-2 + sim-only pkg helpers) plus + dispatch's mem_size/imm-select cases; any of the ~45 new ops missed in + any copy dispatches as a no-dest RS_INT op and "completes" without + executing. The audit recommends assertion guards to convert these to + loud failures during bring-up. +9. **Test suites that would pin wrong behavior** — `test_ras_detector` + asserts C.JAL-classifies-as-call (green against un-recoded RTL); + decompressor tests assert C.SUBW/shamt6 are illegal; branch-unit vectors + all fit in 32 bits (vacuous pass if a comparator stays [31:0]). The + recode policy is: RV32-reserved-now-valid vectors are *replaced by + positive RV64 vectors*, never deleted. + +--- + +# Findings by subsystem + +Each subsystem section opens with the auditor's summary, then findings +grouped hazard → design → policy → mechanical. + +## instruction decode + riscv_pkg types + +The decode subsystem splits cleanly into three risk tiers. Tier 1 (silent-wrong hazards on a bare XLEN flip): immediate_decoder's sign-extension replications are hardcoded to 32-bit results ({20{}}/{19{}}/{11{}} and the U-type concatenation), so at XLEN=64 every negative I/S/B/J immediate and every LUI/AUIPC with bit 31 set zero-extends — and these wires feed branch_target_precompute directly, corrupting every backward branch/JAL precomputed target; riscv_pkg's from_id_to_ex_t immediates are hardcoded [31:0] and get silently truncated at the id_stage pipeline register; the interrupt-cause constants bake the mcause interrupt bit at 31; and the 32-bit DIV-overflow magic constants stop matching 64-bit operands. Tier 2 (accept-when-must-trap): ZIP/UNZIP and the RV32-form REV8 immediate would still decode as valid on RV64 because instr_decoder contains zero XLEN references — no widening pass will touch it. Tier 3 (loud missing decode): instr_decoder is a strict whitelist (every unmatched encoding hits o_illegal), so LD/LWU/SD, the .D atomics, all W-ops, FCVT.L forms, FMV.X.D/D.X, and 6-bit shamts simply trap today — safe, but the shamt6 restructure has real encoding-collision complexity in the funct3=001/101 families. The most dangerous integration hazard is the 4-way duplication of op-classification whitelists (id_stage slot-1 + slot-2 + eight riscv_pkg sim-only helpers): any of the ~45 new instr_op_e members missed in any copy silently defaults to RS_INT/no-dest/no-sources, e.g. an LD dispatching to the integer RS and 'completing' without touching memory. instruction_type_decoder additionally needs a double-word load class (LD would silently classify as a 32-bit word load; LWU works by accident), and store_op_e is full at 2 bits and needs an STD member with STN kept at 0. branch_target_precompute is the one genuinely clean-parametric file, contingent on the immediate fix. Parameter plumbing is verified live: cpu_ooo passes riscv_pkg::XLEN down the whole id hierarchy. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/id_stage/immediate_decoder.sv:43`** [high] I/S/B/J sign-extension replication counts are hardcoded for 32-bit results ({20{...}} at L43-45 and L49-51, {19{...}} at L56-63, {11{...}} at L72-79) while outputs are [XLEN-1:0]. At XLEN=64 each 32-bit concatenation ZERO-extends into the 64-bit output, so every negative immediate is silently corrupted (e.g. imm=-4 becomes 64'h00000000_FFFFFFFC). These wires feed branch_target_precompute directly (id_stage.sv L197-199), so every backward branch/JAL target, ras_expected_rs1, and btb_expected_rs1 goes wrong; the XLEN'(signed') casts in branch_target_precompute are no-ops on already-64-bit inputs and cannot repair this. Fix: replication counts must be XLEN-12/XLEN-13/XLEN-21 (e.g. {{(XLEN-12){i_instruction.funct7[6]}}, ...}). + - *Action:* Parameterize all sign-extend replication counts on XLEN +- **`hw/rtl/cpu_and_mem/cpu/id_stage/immediate_decoder.sv:67`** [high] U-type: o_immediate_u_type = {i_instruction[31:12], 12'h0} is exactly 32 bits and zero-extends into the 64-bit output. RV64 LUI/AUIPC sign-extend bit 31 to XLEN, so any LUI with imm[19]=1 (e.g. lui x,0x80000) silently produces a positive 64-bit value instead of a sign-extended negative one. + - *Action:* Change to {{(XLEN-32){i_instruction[31]}}, i_instruction[31:12], 12'h0} +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:927`** [high] from_id_to_ex_t immediate_i/s/b/u/j_type are hardcoded logic [31:0] (L927-931) while immediate_decoder outputs and all other datapath fields are [XLEN-1:0]. At XLEN=64, id_stage L834-838/L1523-1527 silently truncate 64-bit wires into these 32-bit fields. All five formats DO fit in 32 signed bits, and at least one consumer (ex_stage/branch_jump_unit.sv:83) already applies XLEN'(signed'(...)), so keep-32+sign-extend-at-every-use is viable — but any consumer that zero-extends instead breaks silently. Needs an explicit decision plus an audit of every field reader. + - *Action:* Decide widen-to-XLEN vs keep-[31:0]-with-mandatory-signed-extension; audit all consumers either way +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:742`** [high] IntMachineSoftware/Timer/External = 32'h8000_0003/7/B (L742-744) bake the mcause interrupt bit at bit 31. On RV64 the interrupt bit is bit 63; if these bit[31:0] constants are assigned/compared against a 64-bit mcause they zero-extend with bit 63 clear, so interrupts would be silently reported as (nonexistent) synchronous exceptions with huge cause codes. Compile-clean, wrong at runtime. + - *Action:* Rebuild as {1'b1, (XLEN-2)'0, cause} or widen to XLEN with the interrupt bit at XLEN-1 +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:808`** [high] SignedInt32Min/SignedInt32Max/UnsignedInt32Max/NegativeOne are bit[31:0] magic constants used for DIV/REM overflow and divide-by-zero special cases. At XLEN=64, equality of a 64-bit operand against zero-extended 32'h8000_0000 NEVER matches INT64_MIN and (a == NegativeOne) never matches 64-bit -1, so the DIV/REM overflow path silently stops firing. 64-bit counterparts (SignedInt64Min etc.) are needed for native ops; the 32-bit ones are still needed for DIVW/REMW. + - *Action:* Add 64-bit variants; re-audit every use site (ALU/divider) for which width applies +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:1419`** [medium] Comment in reorder_buffer_alloc_req_t documents the dispatch pattern value={{FLEN-XLEN{1'b0}}, link_addr}. At XLEN=64, FLEN-XLEN=0 → zero-width replication (legal SV inside a concatenation but a per-tool lint hazard; this repo runs Verilator 5.050 + Yosys 0.64 which differ on it). The actual code lives in dispatch (another agent's file) but this comment is the contract statement and goes stale — every {{FLEN-XLEN{...}}} occurrence must be rewritten as FLEN'(...) or guarded. + - *Action:* Rewrite the documented pattern (and comment) as a width cast; grep-sweep FLEN-XLEN repo-wide +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:1851`** [medium] Sim-only (`ifndef SYNTHESIS) helpers get_rs_type/has_int_dest/has_fp_dest/uses_fp_rs1/uses_fp_rs2/uses_fp_rs3/uses_int_rs1/uses_int_rs2 (L1851-2084) enumerate ops by whitelist with defaults. Every new RV64 op omitted from a list silently falls to the default (RS_INT, no dest, no source) in simulation-side classification, diverging from the synthesized inlined copies in id_stage. ZIP/UNZIP must also be dropped. Also stale doc: L1394 comment says mcause interrupt bit is 'bit 31'. + - *Action:* Extend all eight helper lists in lockstep with the id_stage inlined copies; fix L1394 comment +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv:134`** [medium] ZIP (L133-135) and UNZIP (L156-159) are RV32-only Zbkb encodings but would still decode as valid on RV64 — the core would silently execute nonexistent instructions instead of trapping. Nothing about flipping XLEN changes this decoder (it has no XLEN reference), so this is an explicit edit that will be missed by a pure-widening pass. + - *Action:* Make ZIP/UNZIP decode o_illegal=1 on RV64 +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv:152`** [medium] REV8 decodes as funct7=0110100, rs2=11000 (imm 011010011000) — that is the RV32 encoding. The RV64 REV8 immediate is 011010111000 (funct7=0110101, rs2=11000). As written, RV64 code's REV8 traps illegal AND the RV32-only encoding is silently accepted. BREV8 (rs2=00111, same funct7 block) is encoding-stable across XLEN. + - *Action:* Move REV8 match to funct7=0110101 on RV64; make the 0110100+11000 form illegal +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instruction_type_decoder.sv:80`** [high] Load class flags have no double-word class: o_is_load_byte/o_is_load_halfword (L80-83) plus implicit 'word otherwise'. Once LD (funct3=011) decodes, it classifies as a WORD load here — silently truncating 64-bit loads to 32. LWU (110) accidentally classifies correctly (not byte/half + funct3[2]=1 via o_is_load_unsigned at L84 -> word, unsigned). An o_is_load_double output (or a 2-bit size) is required, and downstream consumers of these flags must map to MEM_SIZE_DOUBLE. Also RV64 LW must be treated as SIGNED word (funct3[2]=0 already gives that). + - *Action:* Add double-word load classification output and plumb through from_id_to_ex_t +- **`hw/rtl/cpu_and_mem/cpu/id_stage/id_stage.sv:834`** [medium] The [XLEN-1:0] immediate wires (L82-86, L866-870) are registered into the hardcoded [31:0] from_id_to_ex_t fields at L834-838 and L1523-1527 — a silent truncation at XLEN=64. Benign only if the keep-32+signed-extend-at-use policy is adopted AND immediate_decoder is fixed (truncating a correctly sign-extended 64-bit value to 32 preserves all information for these formats); otherwise a live bug. Pairs with the riscv_pkg L927 finding. + - *Action:* Resolve together with from_id_to_ex_t immediate-width decision +- **`hw/rtl/cpu_and_mem/cpu/id_stage/id_stage.sv:382`** [high] The pre-decoded operand-classification case lists exist FOUR times (slot-1 L382-660, slot-2 L1100-1376, plus the two sim-only pkg helper sets) and are whitelists with silent defaults. Every new RV64 op missed in any copy silently gets rs_type=RS_INT, no dest, no sources — e.g. an LD absent from the RS_MEM list (L424-436/L1142-1154) dispatches to the integer RS as a no-dest op and 'completes' without loading; a MULW absent from RS_MUL (L418-422) does the same. Required insertions: RS_MEM += LWU/LD/SD/LR_D/SC_D/AMO*_D; RS_MUL += MULW/DIVW/DIVUW/REMW/REMUW; RS_INT += all W-ALU/W-shift/Zba.UW/Zbb-W ops; is_int_store (L471/L1189) += SD; has_int_dest (L529-569/L1246-1287) += all of the above with int rd plus FCVT_L_*/FMV_X_D; has_fp_dest/uses_fp_rs1 += FCVT_S_L(U)/FCVT_D_L(U)/FMV_D_X (int rs1!)/FMV_X_D (fp rs1); uses_int_rs2 (L634-659/L1350-1375) += W R-type ops, SD, SC_D, AMO*_D; ZIP/UNZIP removed from all lists (L411/L550/L650/L1129/L1268/L1366). + - *Action:* Update all four copies in lockstep; consider generating them from one table to kill the divergence risk + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:766`** [medium] store_op_e is a FULL 2-bit enum {STN,STB,STH,STW}; SD needs a new member and the enum must widen to 3 bits with STN kept at 0 (Verilator 2-state init requirement). Note FSD today routes as STN + 'FP64 store unit' (instr_decoder.sv L329-332), so SD could alternatively reuse that 64-bit store path — but the flag would then not say 'store nothing' for a real store, so a proper STD member is cleaner. + - *Action:* Widen enum to bit[2:0], add STD, keep STN=0; update every store_op_e consumer +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:443`** [medium] instr_op_e needs ~45 new members: LWU, LD, SD; ADDIW/SLLIW/SRLIW/SRAIW; ADDW/SUBW/SLLW/SRLW/SRAW; MULW/DIVW/DIVUW/REMW/REMUW; LR_D/SC_D/AMO{SWAP,ADD,XOR,AND,OR,MIN,MAX,MINU,MAXU}_D; FCVT_L_S/LU_S/S_L/S_LU/L_D/LU_D/D_L/D_LU; FMV_X_D/FMV_D_X; ADD_UW/SLLI_UW/SH1ADD_UW/SH2ADD_UW/SH3ADD_UW; CLZW/CTZW/CPOPW/ROLW/RORW/RORIW; PACKW. ZIP (L543) and UNZIP become RV32-only. The enum is unsized (implicit int), so no width saturation — but every case statement over instr_op_e in the core must be revisited. + - *Action:* Add members; decide fate of ZIP/UNZIP members (delete vs keep-unreachable) +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:74`** [medium] opc_e has no OP-IMM-32 (7'b0011011) or OP-32 (7'b0111011) members; all RV64 W-form ALU decode needs them (and pd/if predecode class functions elsewhere compare raw 7-bit literals that may also need these opcodes classified). + - *Action:* Add OPC_OP_IMM_32 and OPC_OP_32 to opc_e +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:640`** [medium] CsrCycleH/CsrTimeH/CsrInstretH (0xC80-0xC82, L640-642) and CsrMcycleH/CsrMinstretH (0xB80/0xB82, L646/648) must become ILLEGAL CSR addresses on RV64, and cycle/time/instret/mcycle/minstret become single 64-bit CSRs. The constants likely survive for the illegality check, but their '(high 32 bits)' comments and every CSR-file consumer change. mcounteren (L655) stays a 32-bit CSR — no change here. + - *Action:* Keep addresses for illegal-decode; rework CSR module legality and comments +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:717`** [medium] mstatus bit-position localparams are RV32-only (comment says so): MstatusMieBit/MpieBit/MppLo/MprvBit keep their positions on RV64, but SD moves 31->63 and UXL appears at [33:32] (reads 2, can be read-only) — neither has a constant here. misa MXL moves to [63:62] (no misa layout constant in this package; CSR module owns it). + - *Action:* Add MstatusSdBit=63, MstatusUxlLo=32 constants for the CSR module +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv:119`** [high] 6-bit shamt legality: SLLI/BSETI/BCLRI/BINVI (L119-122, funct3=001) and SRLI/SRAI/BEXTI/RORI (L142-145, funct3=101) decode on FULL funct7 equality, so instruction bit 25 (shamt[5]) set falls to default:o_illegal. On RV64 these must match funct7[6:1] only. This is entangled with same-funct3 encodings that keep full-funct7/rs2 decode (ORC_B 0010100+rs2=00111 at L146-149, REV8/BREV8 at L150-155, ZIP/UNZIP 0000100+rs2=01111, CLZ/CTZ/CPOP/SEXT_* under 0110000 at L123-131 which collides with RORI's 011000x range on funct3=001... actually 0110000 funct3=001 is the Zbb unary block vs RORI funct3=101) — the case restructure needs a careful collision matrix. W-form shifts (new OP-IMM-32 arm) keep 5-bit shamt with bit 25 set ILLEGAL. + - *Action:* Restructure shift-family decode to funct7[6:1] for native forms; keep full-funct7 for W-forms and unary/byte ops +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv:216`** [high] OPC_LOAD arm (L216-224) lacks funct3=011 (LD) and funct3=110 (LWU) — both currently trap illegal (loud, not silent). LW (L220) keeps its enum but its SEMANTICS change on RV64 (sign-extend 32->64), which is a load-unit/LQ change plumbed from here. LWU is the only zero-extending word load. + - *Action:* Add LD and LWU decode; plumb size=DOUBLE and sign flags +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv:227`** [high] OPC_STORE arm (L227-242) lacks funct3=011 (SD); needs the new store_op_e member (o_store_op=STD) and RS_MEM size=DOUBLE routing. Currently traps illegal. + - *Action:* Add SD decode with new store_op_e member +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv:293`** [medium] OPC_AMO gate is `if (funct3 == 3'b010)` (word forms only, L293); funct3=011 (.D forms: LR.D, SC.D, AMO*.D) traps illegal. RV64 needs the full .D set, with AMO*.W results sign-extending to 64 downstream and an 8-byte reservation-granule decision for LR/SC. + - *Action:* Add funct3=011 arm mapping to new *_D enum members +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv:94`** [high] ZEXT.H is decoded as PACK rd,rs1,x0 (L93-94) — correct for RV32 only. On RV64 ZEXT.H is PACKW rd,rs1,x0 in the new OP-32 opcode, and plain PACK becomes a 64-bit pack of 32-bit halves (semantic change in ALU). The new OP-32 arm must decode PACKW (plus CLZW/CTZW/CPOPW/ROLW/RORW/RORIW, ADD.UW/SLLI.UW/SH[123]ADD.UW, ADDW/SUBW/SLLW/SRLW/SRAW, MULW/DIVW/DIVUW/REMW/REMUW). + - *Action:* Write complete OP-IMM-32/OP-32 case arms; today they hit default:o_illegal at L494 +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv:418`** [medium] FP conversion decode: FCVT arms (L417-442) only accept rs2 field 00000/00001 (W/WU); RV64 adds rs2=00010/00011 (L/LU) for all four funct7 blocks. FMV.X.D (funct7=1110001, funct3=000) is currently illegal — L461-467 only allows FCLASS_D — and FMV.D.X (funct7=1111001) is entirely absent. New FCVT.S.L/LU results must be NaN-boxed like existing single-precision producers. + - *Action:* Add L/LU conversion arms, FMV_X_D, FMV_D_X; reserved-rm check at L503-522 already covers the FCVT funct7 blocks +- **`hw/rtl/cpu_and_mem/cpu/id_stage/instruction_type_decoder.sv:109`** [medium] o_is_lr/o_is_sc require funct3==3'b010 (L109-112), so LR.D/SC.D (funct3=011) would be is_amo=1 but is_lr=is_sc=0 — silently misrouted as a plain AMO once the .D decode is added. Must accept funct3 in {010,011}. + - *Action:* Widen funct3 acceptance when .D atomics land + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:667`** [low] Custom CsrMperfDataH (0xFC1) keeps the split 32-bit high-half read scheme for profiling counters. On RV64 a single 64-bit CsrMperfData read would suffice; keeping the H-half is legal (custom CSR space) but inconsistent with dropping the architectural *H CSRs. + - *Action:* Decide: keep split custom counter reads or collapse to one 64-bit CSR + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:734` [low] ExcIllegalInstr..ExcEcallMmode are bit[31:0] (L734-739). Zero-extension into a 64-bit mcause is semantically correct for these small synchronous causes (unlike the interrupt constants), so this is a width-hygiene widen to XLEN, not a behavior bug. Also new misaligned classes: 8-byte alignment checks for LD/SD/LR.D/SC.D/AMO*.D reuse the same cause codes (4/6). +- `hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:1604` [low] mem_size_e already has MEM_SIZE_DOUBLE=2'b11 (encoding is RV64-ready) but the comment '64-bit (FLD/FSD only)' goes stale: LD/SD/LWU/LR.D/SC.D/AMO*.D all map to it, and lq_alloc_req_t.sign_ext must be driven correctly for LWU (0) vs LW (1, a semantic change: LW sign-extends to 64 on RV64). +- `hw/rtl/cpu_and_mem/cpu/id_stage/instruction_type_decoder.sv:182` [low] RAS coroutine/return checks compare i_immediate_i_type ([XLEN-1:0]) against 32'b0 (L182, L189). At XLEN=64 the literal zero-extends so the comparison is still semantically correct, but it is a width-mismatch lint hit and hides intent. +- `hw/rtl/cpu_and_mem/cpu/id_stage/id_stage.sv:36` [low] parameter XLEN=32 default is correctly overridden: cpu_ooo.sv:31 declares XLEN=riscv_pkg::XLEN and passes it at cpu_ooo.sv:626, and id_stage forwards it to immediate_decoder/instruction_type_decoder/branch_target_precompute. So the package flip propagates — no dead default. PC/link-address adders (L123-126, L901-904) and WB-bypass data paths are already XLEN-parametric. +- `hw/rtl/cpu_and_mem/cpu/id_stage/branch_target_precompute.sv:74` [low] Fully XLEN-parametric (adders L74-75, subtractors L82/L88, comparator L99). The XLEN'(signed'(...)) casts become no-ops once the immediate inputs are already 64-bit — correctness is entirely contingent on immediate_decoder producing properly sign-extended values (see that file's hazard). Widening triples: 5 carry chains of 64 bits + one 64-bit equality per instance, two instances (2-wide). + +## fetch front end and PC management + +The fetch front end is in very good shape for RV64: all six files were read in full, and pc_controller, pc_increment_calculator, pc_reg_precompute, control_flow_tracker, and the if_stage datapath are essentially fully XLEN-parametric -- PC registers, all five fetch-advance candidates, the pc_reg precompute adders, redirect/trap/fence/PD-target muxes, halfword pending-prediction compares, link-address computation, and the served-window guard all widen correctly by flipping the parameter. Instruction-word carriers ([31:0] instr_buffer, effective_instr, assembled_instr, the 64-bit i_instr fetch window, and the [47:32]/[15:0] spanning parity selection) are correctly width-invariant and need no change. The one high-risk hazard is if_stage.sv:884, where the served-window desync guard uses pc_reg[XLEN-1] as the cached-region predicate: at XLEN=64 with the sub-4-GiB map, bit 63 is never set, so the guard silently goes dead and the exact mid-instruction-byte pc_reg corruption it was added to stop (the workqueue_init_early boot Oops) returns -- and it diverges from fetch_provider.sv, which hardcodes bit 31 and stays correct. c_ext_state.sv has the only true port bug: i_pc/i_pc_reg are hardcoded [31:0] against XLEN-wide drivers (lint-fatal truncation at 64, behaviorally benign since only bit 1 is consumed and i_pc is dead). pc_increment_calculator.sv:267's ~32'd3 mask actually evaluates correctly at 64 bits under SV context-extension rules, but it sits on a permanently dead path (mid-32bit correction is tied off) and should be rewritten or deleted. No RV64 instruction-semantics work lands in these files -- they track only instruction sizes, so the RVC recode belongs to the aligner/PD decompressor. The real Phase-1 cost here is timing, not logic: seven word-index adders grow to 62 bits, six pending-prediction NEQ compares and the halfword magnitude compares grow to 63/64 bits, and the 9-arm final PC muxes double in width, all inside cones that needed keep/dont_touch surgery to close 300 MHz at 32 bits. Decisions needed: the canonical region-decode idiom, the PC[63:32] invariant, and BTB/RAS target-storage compression. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv:884`** [high] window_cannot_serve_pc_reg gates the served-window desync guard on pc_reg[XLEN-1] as the 'cached region (>= CACHED_BASE 0x8000_0000)' predicate. At XLEN=64 with the unchanged sub-4-GiB physical map, bit 63 is never set for DDR-region PCs, so the guard NEVER fires: the exact pc_reg-lands-on-a-mid-instruction-byte desync class it was added to prevent (the workqueue_init_early epc 0x8038d7fa boot Oops, documented at lines 800-806) silently returns. Note fetch_provider.sv uses a hardcoded [31] for the same region test (fetch_provider.sv:219,294,381,407), so the two idioms diverge at XLEN=64 -- the provider still classifies correctly while IF's guard goes dead, which is worse than both being wrong. + - *Action:* Replace pc_reg[XLEN-1] with an explicit region decode on the physical bits (pc_reg[31] given the sub-4-GiB map, or a shared riscv_pkg CachedBase compare used by both if_stage and fetch_provider). Tie to the project-wide high-address-bit policy decision. + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu/if_stage/pc_controller.sv:669`** [low] Reset arms drive next_pc/next_pc_reg to '0 (lines 669, 733): the reset PC remains address 0 at any XLEN, parametric and consistent. All redirect sources (i_trap_target, i_branch_target, i_fence_i_target, i_pd_redirect_target, prediction targets) are already [XLEN-1:0] ports and widen for free -- but they inherit whatever width their producers (CSR mepc/mtvec, EX target adders, PD) actually drive, so correctness of PC[63:32] is owned upstream, not here. + - *Action:* Decide the PC[63:32] invariant (always-zero enforced at redirect producers vs. fetch-side access-fault check); pc_controller itself needs no edit either way. + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/c_ext_state.sv:59` [low] Ports i_pc and i_pc_reg (lines 59-60) are hardcoded [31:0] although the module takes an XLEN parameter (unused for these ports). if_stage connects XLEN-wide pc/pc_reg (if_stage.sv:637-638), so at XLEN=64 the connection implicitly truncates. Behavior is benign today -- the body consumes only i_pc_reg[1] (lines 182, 187, 209, 280) and i_pc is entirely unused -- but CI Verilator 5.050 width lint will reject the 64-to-32 port truncation, and the dead i_pc port invites future misuse. +- `hw/rtl/cpu_and_mem/cpu/if_stage/pc_increment_calculator.sv:267` [low] pc_mid_32bit_correction = ((i_pc_reg + IncC) & ~32'd3) + Inc4 uses a 32-bit-sized mask literal on what becomes a 64-bit operand. Per SV context-determined sizing the operand of unary ~ is extended to 64 bits BEFORE inversion, so it evaluates correctly (~64'd3), but this is the classic pattern tools lint on and reviewers misread; the whole expression is also currently dead logic (o_mid_32bit_correction is tied to 1'b0 at pc_controller.sv:276, seq_sel_mid_32bit can never assert), so a wrong belief about it would go untested. +- `hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv:816` [low] ServedP1LowBits=16 split of the served-window S=P+1 compare was sized for 30-bit word addresses (16 low + 14 upper). At XLEN=64 the word address is 62 bits, so the upper equality (line 835-836) and upper increment (line 831) become 46-bit structures while the low arm stays 2 CARRY8s -- functionally correct (all slicing is XLEN-parametric and the SYNTHESIS-guarded reference model at lines 849-877 stays equivalent), but the 'two CARRY8s per arm' balance the comment promises no longer holds. +- `hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv:962` [low] Comments encode 32-bit PC assumptions that go stale at XLEN=64: line 962 'word(pc_reg[31:2]+1)' and the spanning parity discussion around lines 955-963 describe [31:2] word indexing. Code itself is parametric; only documentation drifts. +- `hw/rtl/cpu_and_mem/cpu/if_stage/pc_controller.sv:439` [medium] Pending-prediction crossing logic uses [XLEN-2:0] halfword magnitude comparators (pc_reg_hw < pending_prediction_pc_hw, seq_next_pc_reg_hw_q >= pending_prediction_pc_hw at lines 439-448) plus full-width equality (o_pc_reg == pending_prediction_pc, lines 456-459, 537-540). All parametric and correct at 64 including the modular-arithmetic prev_pc capture (line 656, o_pc - PcIncrementCompressed wraps identically mod 2^64, and the SYNTHESIS assertions at 785-796 remain valid). Flagged because these become 63/64-bit compare trees inside a cone that already required keep/max_fanout attributes and PC-mux-local duplication for 300 MHz timing. +- `hw/rtl/cpu_and_mem/cpu/if_stage/pc_increment_calculator.sv:316` [medium] The six parallel NEQ comparators (neq_hold/mid/plus2/plus4/plus6/plus8, lines 316-323) feeding o_seq_next_pc_reg_neq_pc become 64-bit compares. This bit feeds prediction_needs_pending in pc_controller (line 417-423), a path that previously measured -1.303ns before the compare-then-mux restructure; the compare trees double in depth by one LUT level each at 64 bits. Functionally parametric and clean -- the boot-hang-critical full-compare semantics (not a bit1 proxy) are preserved at any width. +- `hw/rtl/cpu_and_mem/cpu/if_stage/pc_reg_precompute.sv:31` [medium] Comment contract 'outputs settle ~0.3 ns into the cycle, well before BRAM data arrives at ~0.9 ns' (also pc_increment_calculator.sv:195-196) is a 32-bit-era measurement. At XLEN=64 the word-index adders become 62-bit CARRY8 chains (~8 additional CARRY8 stages), eroding the settle margin that the dont_touch/keep_hierarchy timing architecture is built on. Logic itself is fully parametric. +- `hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv:413` [low] slot2_pc_plus2_for_btb / slot2_pc_plus4_for_btb (pc_reg + 2 / + 4) become 64-bit adders driving the dual-port BTB lookup addresses combinationally, and link_address (line 1194) becomes a 64-bit adder feeding the RAS push path. Parametric and correct; flagged solely because both feed prediction structures whose index/tag/storage widths are a separate policy decision (BTB target compression) and both sit near known-critical fetch paths. + +## RVC decompression + instruction-memory predecode (RV64 C-table recode blast radius) + +This subsystem carries no XLEN-width-bearing datapath — instructions, parcels, and the 18-bit sideband are all fixed-width — so flipping riscv_pkg's XLEN localparam changes nothing here, which is exactly the danger: every RV64 C-table semantic shift lands as a silent misdecode. The worst hazards are the reinterpreted encodings: rvc_decompressor.sv still expands q01 funct3=001 as C.JAL (RV64: C.ADDIW), q00 funct3=011/111 as C.FLW/C.FSW (RV64: C.LD/C.SD with 8-byte immediate scaling and integer register files), and q10 funct3=011/111 as C.FLWSP/C.FSWSP (RV64: C.LDSP/C.SDSP), so RV64 binaries would execute FP loads and taken jumps in place of integer loads and ADDIWs. The same C.JAL misclassification is replicated in three more places that must be recoded in lockstep: riscv_pkg::imem_compressed_control (feeds every stored AllowsSlot2After/Pairable sideband bit across BRAM init, port-A writes, LUTRAM replicas, and L1I fill), instruction_aligner's slot1_branch_compressed (would report C.ADDIW as a branch to pc_controller), and the Python generator's compressed_control (Vivado power-up init images). imem_rvc_source_hot and its Python mirror also change for C.ADDIW (rs1/rs2 metadata is genuinely different) and need new C.SUBW/C.ADDW arms; I verified the C.LD/C.LDSP load recodes are coincidentally bit-identical in the stored {rs2[1], rs1[2:1]} because bit 1 of both the scale-4 and scale-8 immediates is constant zero, so those arms need only comment updates. New-logic design work is confined to the decompressor: C.SUBW/C.ADDW expansion to OP-32, 6-bit shamts with bit12 as shamt[5] (dropping three bit12-illegal checks), OP-IMM-32/OP-32 opcode constants, and reserved-rd checks for C.ADDIW/C.LDSP. imem_predecode.sv and imem_predecode_line.sv are structurally unaffected because they derive everything from riscv_pkg's functions, but the offline-generated init images must be regenerated and the cocotb RTL-vs-Python cross-check extended to RV64 encodings to keep the mirror honest. Policy items are the front-end PC/address port widths (only low bits are consumed under the sub-4-GiB map) and the dual-XLEN-vs-RV64-only shape of the shared C-table across its four RTL sites and one offline mirror. Timing exposure is modest: the recode slightly deepens the slot-2 decompressor case trees on the documented post-BRAM WNS cone, while the sideband recode itself is width-neutral and removes a minterm from the PC-critical compressed-control predicate. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:216`** [high] Quadrant-01 funct3=001 unconditionally expands to JAL x1 (C.JAL). On RV64 this encoding is C.ADDIW: expand to {imm_ci, rd_full, 3'b000, rd_full, OP-IMM-32(0011011)}, with rd=0 reserved (illegal). If XLEN flips without this recode, every C.ADDIW in an RV64 binary silently becomes a taken JAL. + - *Action:* Recode to ADDIW expansion; add rd==0 illegal check; imm_ci already computed +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:192`** [high] Quadrant-00 funct3=011 expands to C.FLW (OpcLoadFp, funct3=010, imm_lw_sw scale-4). On RV64 it is C.LD: OpcLoad (integer), funct3=011, imm_ld_sd scale-8. Silent misdecode: writes the FP regfile with a 4-byte-scaled offset instead of an 8-byte integer load. + - *Action:* Expand as {imm_ld_sd, rs1_prime, 3'b011, rd_prime, OpcLoad}; imm_ld_sd already computed for C.FLD +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:201`** [high] Quadrant-00 funct3=111 expands to C.FSW (OpcStoreFp, funct3=010, imm_lw_sw). On RV64 it is C.SD: OpcStore, funct3=011, imm_ld_sd scale-8. Silent misdecode of every C.SD. + - *Action:* Expand as {imm_ld_sd[11:5], rs2_prime, rs1_prime, 3'b011, imm_ld_sd[4:0], OpcStore} +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:314`** [high] Quadrant-10 funct3=011 expands to C.FLWSP (OpcLoadFp, funct3=010, imm_lwsp). On RV64 it is C.LDSP: OpcLoad, funct3=011, imm_ldsp scale-8, and rd=0 is reserved (currently no rd check because FP rd=0 is legal) — reserved encodings would be silently accepted. + - *Action:* Expand as {imm_ldsp, 5'd2, 3'b011, rd_full, OpcLoad} with rd==0 illegal; imm_ldsp already computed for C.FLDSP +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:346`** [high] Quadrant-10 funct3=111 expands to C.FSWSP (OpcStoreFp, funct3=010, imm_swsp scale-4). On RV64 it is C.SDSP: OpcStore, funct3=011, imm_sdsp scale-8. Silent misdecode of every C.SDSP. + - *Action:* Expand as {imm_sdsp[11:5], rs2_full, 5'd2, 3'b011, imm_sdsp[4:0], OpcStore}; imm_sdsp already computed for C.FSDSP +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/instruction_aligner.sv:516`** [high] slot1_branch_compressed counts quadrant-01 funct3=001 as a branch (comment: 'C.JAL (RV32)'). On RV64 that encoding is C.ADDIW — an ALU op would be reported as o_slot1_is_branch to pc_controller and c_ext_state, silently terminating bundles and driving branch-side state for a non-branch. This predicate is a third copy of the compressed-control class (alongside riscv_pkg::imem_compressed_control and cpu_ooo's if_stage_has_control_flow per the comment at line 488) — all copies must be recoded in lockstep. + - *Action:* Drop funct3==3'b001 from the q01 branch set; audit the sibling copy in cpu_ooo (out of this audit's scope) and historical comments at lines 516/565 +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:151`** [high] imem_compressed_control includes q01 funct3==3'b001 (C.JAL) in the compressed-control class. On RV64 that is C.ADDIW: the stored AllowsSlot2After/EvenLocalPairValid/Pairable* sideband bits silently classify every C.ADDIW as control flow (bundle-killing, and inconsistent with a recoded aligner/decompressor). Affects BRAM init, port-A writes, the LUTRAM fast replicas (bit 6 of *_compressed), and the L1I fill path — all derived from this one function. + - *Action:* Drop funct3==3'b001 from the q01 control set (RV64); keep C.J/C.BEQZ/C.BNEZ/C.JR/C.JALR +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:275`** [high] imem_rvc_source_hot lumps funct3 001 and 101 (q01) as C.JAL/C.J with rs1={5{imm_j[11]}}, rs2={imm_j[4:1],imm_j[11]}. On RV64, funct3=001 is C.ADDIW whose expansion has rs1=rd_full and rs2(imm field)=imm_ci[4:0]; the stored {rs2[1], rs1[2:1]} = {expanded[21], expanded[17:16]} would be silently wrong for every C.ADDIW, corrupting the exact-replica rename/source timing endpoints these bits replace. + - *Action:* Split funct3=001: rs1=rd_full, rs2=imm_ci[4:0] (mirror the C.ADDI arm); keep 101 as C.J +- **`sw/common/generate_imem_predecode_init.py:93`** [high] compressed_control() includes funct3 0b001 in the q01 control set — the offline mirror of riscv_pkg::imem_compressed_control. Must change in lockstep or the Vivado power-up sideband init files silently diverge from the RTL write/fill paths (the cocotb bench cross-checks imem_predecode_line against this script, so a one-sided change is catchable there, but only if the bench runs with RV64 encodings). + - *Action:* Drop 0b001 from the control set in lockstep with riscv_pkg +- **`sw/common/generate_imem_predecode_init.py:190`** [high] rvc_source_hot() lumps funct3 0b001/0b101 as C.JAL/C.J (rs1 from imm_j sign, rs2 from imm_j bits). RV64 C.ADDIW needs rs1=rd_full, rs2=imm_ci&0x1F — mirror of the riscv_pkg finding at line 275. + - *Action:* Split funct3=0b001 into an ADDIW arm in lockstep with riscv_pkg::imem_rvc_source_hot + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:169`** [high] shamt is [4:0] = parcel[6:2] ('5-bit for RV32' comment). RV64 C.SLLI/C.SRLI/C.SRAI take 6-bit shamt {parcel[12], parcel[6:2]}; the expanded instruction's bit 25 must carry shamt[5]. Fixed [4:0] here plus the bit12-illegal checks means RV64's upper-half shifts are unreachable. + - *Action:* Widen to 6-bit; place shamt[5] at expanded[25] (funct7 becomes 0000000/0100000 with bit25=shamt[5]) +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:231`** [medium] C.SRLI (and C.SRAI at line 235) assert o_illegal when bit12=1. On RV64 bit12 is shamt[5] and the encoding is valid. Currently loud (traps), not silent, but must be removed with the shamt widening. + - *Action:* Drop bit12 illegal checks for C.SRLI/C.SRAI; route bit12 into shamt[5] +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:305`** [medium] C.SLLI asserts o_illegal when bit12=1 ('shamt[5] still reserved on RV32'). Valid 6-bit shamt on RV64. + - *Action:* Drop bit12 illegal check; route bit12 into shamt[5] +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:241`** [medium] Quadrant-01 funct3=100, bits[11:10]=11, bit12=1 is wholesale illegal (comment even says 'RV64-only op encodings'). On RV64, funct2=00 is C.SUBW and funct2=01 is C.ADDW, expanding to OP-32 (0111011) with funct7 0100000/0000000 funct3=000; funct2 10/11 remain reserved. + - *Action:* Add C.SUBW/C.ADDW arms targeting a new OpcOp32 localparam; keep funct2 10/11 illegal +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv:54`** [low] Opcode localparam list (OpcLui..OpcOp) lacks OP-IMM-32 (0011011) and OP-32 (0111011) needed by the C.ADDIW/C.SUBW/C.ADDW expansions. + - *Action:* Add OpcOpImm32 and OpcOp32 localparams +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:299`** [medium] imem_rvc_source_hot leaves q01 funct3=100 [11:10]=11 bit12=1 as rs1=rs2=0 ('reserved on RV32 and expands to zero'). On RV64 C.SUBW/C.ADDW (funct2 00/01) expand with rs1_prime/rs2_prime — the metadata must match the recoded decompressor output or the source-hot bits silently diverge from the literal expansion. + - *Action:* Add C.SUBW/C.ADDW arm: rs1=rs1_prime, rs2=rs2_prime for funct2 00/01; keep 10/11 zero +- **`sw/common/generate_imem_predecode_init.py:204`** [medium] rvc_source_hot() `elif not sign` leaves bit12=1 (q01 funct3=100 subop=11) as rs1=rs2=0. RV64 C.SUBW/C.ADDW (funct2 00/01) need rs1_prime/rs2_prime — mirror of the riscv_pkg finding at line 299. + - *Action:* Add C.SUBW/C.ADDW arm for funct2 00/01 under bit12=1 + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/instruction_aligner.sv:53`** [low] i_pc_reg is hardcoded [31:0] and the module's XLEN parameter (line 42) is never referenced in the body. Only pc_reg[2:1] are consumed, so behavior is XLEN-independent, but the port width vs a 64-bit PC is a decision (keep 32-bit front-end PC under the sub-4-GiB physical map, or widen to XLEN and drop/use the dead parameter). + - *Action:* Decide front-end PC port width; remove or wire the unused XLEN parameter +- **`hw/rtl/cpu_and_mem/imem_predecode.sv:90`** [low] i_port_a_byte_address / i_port_b_byte_address (line 98) are [31:0]. Only ADDR_WIDTH+2 low bits are consumed, so with the sub-4-GiB physical map nothing misbehaves at XLEN=64; whether these fetch/programming address ports widen to XLEN is the project-wide high-address-bit policy. Everything else in this module (sideband, fast replicas, hi_rd_is_x2, init/write paths) is derived from riscv_pkg::imem_make_sideband and inherits the recode automatically — but the Vivado init images must be regenerated with the updated Python script or power-up sideband will be stale for RV64 binaries. + - *Action:* Decide address-port width; regenerate all *.mem init images with the recoded generator as part of the RV64 build flow + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:251` [low] q00 funct3 010/011 share the C.LW/C.FLW arm (rs2=imm_lw_sw[4:0]). On RV64 funct3=011 is C.LD with rs2=imm_ld_sd[4:0]. VERIFIED coincidentally harmless: bit [1] of both imm_lw_sw and imm_ld_sd is constant 0, so {rs2[1]} is identical under both interpretations and rs1 (rs1_prime) is unchanged. Same situation at line 323 for C.LWSP/C.FLWSP→C.LDSP (imm_lwsp[1]=imm_ldsp[1]=0). Only comments/labels need updating, but this equivalence should be re-proven in the cross-check bench when the table recodes. +- `sw/common/generate_imem_predecode_init.py:181` [low] C.LW/C.FLW shared arm (and C.LWSP/C.FLWSP at line 211) — same coincidental bit-identity as riscv_pkg line 251: imm bit [1] is 0 for both scale-4 and scale-8 immediates, so the packed source-hot value is unchanged for the load recodes. Comment/label update only. + +## branch prediction storage and control (BTB, bimodal direction predictor, RAS, prediction metadata tracking) + +The branch-prediction subsystem is almost entirely XLEN-parametric and needs no datapath rewrites for RV64: every PC, target, and link-address carrier uses riscv_pkg::XLEN or a properly-overridden XLEN parameter, and the index hashes (BTB PC[9:2], tag PC[XLEN-1:10]++PC[1], bimodal PC[10:1]) are written XLEN-relative and keep working unchanged at 64. The single genuine RV64 bug is in ras_detector.sv:155: the C.JAL pattern (funct3=001, op=01) becomes C.ADDIW on RV64, so every C.ADDIW would silently push a garbage return address onto the RAS — functionally recoverable (RAS is checkpointed) but a severe, silent return-prediction poisoning; the module has no XLEN parameter, so an explicit gate must be added. The main policy question is storage width: TagBits = XLEN-BTB_INDEX_BITS-1 auto-grows 23 -> 55 bits across five replicated tag LUTRAMs and targets grow 32 -> 64 across three replicas, roughly doubling BTB LUTRAM (~65 Kbit total growth) for upper PC bits that are architecturally constant given the sub-4-GiB physical map; compressing tags/targets back to RV32 widths with zero-extended readout is safe only once the project fixes its high-address-bit fetch policy. Timing exposure concentrates in the widened 55-bit tag compare on the combinational BTB-lookup -> next-PC path, which this code's comments show has already been through repeated 300 MHz timing surgery. The RAS (8xXLEN LUTRAM), direction predictor (pure 10-bit indexing), prediction_metadata_tracker (pure XLEN passthrough), and controller gating logic are all clean; only stale RV32-specific comments (23-bit tag, PC[31:10], target(32)) need doc updates in the same change. One low-grade latent hazard: several modules default their XLEN parameter to 32, which current instantiations override correctly but would silently truncate 64-bit PCs if a future instantiation or testbench omits the override. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/ras_detector.sv:155`** [high] is_c_jal decodes funct3=001/op=01 as C.JAL (RAS call). On RV64 this encoding is C.ADDIW, so every C.ADDIW instruction is classified as a call (via is_call_c at line 183), pushing a garbage link address onto the RAS. Compiles and runs; silently poisons return prediction — every subsequent 'ret' pops a wrong target, mispredicts, and takes the full EX-recovery penalty. Functionally safe (RAS is a predictor with checkpoint recovery) but a severe silent performance regression on RV64 code, where C.ADDIW is extremely common. The module has no XLEN parameter at all, so nothing flips automatically; needs an explicit XLEN==32 gate (or removal for rv64-only) on is_c_jal. Header comments at lines 45 and 119 already say 'RV32 only' but the code does not enforce it. + - *Action:* Gate is_c_jal with a generate/parameter check (XLEN==32), or delete it in an rv64-only build; add an XLEN or IS_RV64 parameter to ras_detector since it currently has none. Verify C.ADDIW-heavy code no longer perturbs RAS depth in a directed cocotb test. +- **`hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv:73`** [low] parameter int unsigned XLEN = 32 default. The in-tree instantiation (branch_prediction_controller.sv:201-203) overrides it with riscv_pkg::XLEN, so production RTL flips correctly — but any testbench or future instantiation that omits .XLEN() silently builds a 32-bit BTB against 64-bit PCs, truncating without error (implicit port-width coercion). Same latent default exists in direction_predictor.sv:47 and prediction_metadata_tracker.sv:37. + - *Action:* Either change the defaults to riscv_pkg::XLEN (package import in the parameter default) or audit/grep all instantiations after the flip; a width-mismatch lint pass (Verilator -Wall WIDTHEXPAND/WIDTHTRUNC) in CI catches accidental default use. + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv:126`** [medium] TagBits = XLEN - BTB_INDEX_BITS - 1 silently grows 23 -> 55 bits at XLEN=64, and the three XLEN-wide target RAMs (lines 267-301) grow 32 -> 64 bits. Tag storage is replicated across FIVE sdp_dist_ram instances (lookup, lookup_2, lookup_2_alt, update, early_update: lines 202-264) and the compressed/handoff/counter RAMs are unaffected. Net LUTRAM growth: tags 5x256x(55-23) = +40,960 bits, targets 3x256x(64-32) = +24,576 bits — roughly doubling the BTB's distributed-RAM LUT cost. Mechanically correct as written, but with the physical map fixed below 4 GiB (M/U, no MMU), PC bits [63:32] are architecturally constant for all trained branches, so full-width tags/targets buy nothing. + - *Action:* Decide: (a) accept the growth (zero-logic change), or (b) compress — keep TagBits pinned to 32-BTB_INDEX_BITS-1 = 23 (tag from PC[31:10]++PC[1]) and store 32-bit targets zero-extended to 64 on readout. Option (b) is only sound once the project's high-address-bit policy guarantees PCs above 4 GiB can never be fetched/trained (they'd alias into the truncated tag). + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv:451` [medium] Slot-1 BTB hit compare (lookup_tag_stored == lookup_tag) widens 23 -> 55 bits; same for the two slot-2 compares at lines 469-470 and the early/late update-read compares at lines 489 and 508. The slot-1/slot-2 compares sit on the combinational LUTRAM-read -> hit -> predicted_taken -> prediction_used -> next-PC path that this subsystem has been repeatedly timing-optimized around (per comments in branch_prediction_controller lines 361-369). A 55-bit equality adds roughly one 6-LUT compare-tree level plus wider LUTRAM read fanout. +- `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv:29` [low] Header and inline documentation hardcodes the RV32 layout: 'tag (23 bits) + target (32)' (line 29), 'Compare tag (PC[31:10] ++ PC[1])' (line 51), '// 23 bits (includes PC[1])' comment on the TagBits localparam (line 126), and 'Tag: PC[31:10] concatenated with PC[1] (23 bits)' (line 162). All become stale at XLEN=64 (or should be rewritten in XLEN-relative terms). +- `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv:192` [low] update_pc_2_key = i_update_pc - XLEN'(2) and update_pc_2_alt_key = i_update_pc - XLEN'(4) (line 195) become 64-bit subtractors. Update-side only (explicitly documented as off the fetch-PC recurrence), so mechanical and low timing exposure; noted because they are full-width carry chains that double in length. +- `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/return_address_stack.sv:188` [low] RAS storage sdp_dist_ram DATA_WIDTH(riscv_pkg::XLEN): 8x32 -> 8x64 bits. Widens automatically and correctly (i_link_address, i_push_address_after_restore, o_ras_target are all riscv_pkg::XLEN already). Growth is trivial (8 entries), but the same sub-4-GiB compression policy could apply for consistency. Pointer/count arithmetic (RAS_PTR_BITS casts, valid_count saturation at lines 109-110, 253-258, 274-280) is XLEN-independent and unaffected. +- `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_prediction_controller.sv:522` [low] pd_redirect_kills_prediction_metadata compares o_predicted_target_r != i_pd_redirect_target — widens to a 64-bit inequality feeding the synchronous-clear terms of three holdoff/metadata flop groups (lines 541-653). Both operands are registered upstream, so this is a fresh-from-flop compare, but it fans out into control logic the comments identify as timing-sensitive. +- `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/direction_predictor.sv:66` [low] bim_idx(pc) = pc[BIM_BITS:1] (bits [10:1]) is XLEN-independent and correct at 64; o_pred_idx / update-index plumbing (BpDirIdxBits=10 through riscv_pkg commit-capture structs at riscv_pkg.sv:882/920/1008) carries only the 10-bit index, so commit-side training widths stay matched. The pc argument's unused upper bits grow from [31:11] to [63:11] — pre-existing unused-bit situation, may surface a new Verilator UNUSEDSIGNAL width at 64 depending on waiver style. +- `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/ras_detector.sv:43` [low] Header comment block (lines 43-48) documents C.JAL as 'CALL, RV32 only' and the compressed-decode comment at line 119 repeats it; once the RV64 gating is added these comments need to state the C.ADDIW recode explicitly so the gate isn't 'simplified' away later. Doc-only companion to the line-155 hazard. + +## integer execute (ALU, shifter, mul, div, branch compare) + fu shims + riscv_pkg bit-manip helpers + +The integer-execute subsystem splits cleanly into three tiers. Genuinely parametric and near-free at XLEN=64: branch_jump_unit (JALR masking, immediate sext, and comparators all widen correctly - timing is the only concern), divider.sv (fully WIDTH-parametric including the div-by-zero and overflow-by-wraparound behavior; latency goes 17 to 33 cycles and its per-stage double-subtract is the subsystem's biggest timing question), and dsp_tiled_multiplier_unsigned (a ready-made building block for a 65x65 multiply). The dangerous tier is alu.sv: it takes an XLEN parameter, so flipping XLEN=64 COMPILES, but roughly a dozen case arms silently misbehave - 5-bit shamt truncation on all shifts/rotates/Zbs indices, SEXT_B/SEXT_H zero-extending instead of sign-extending, ORC.B/REV8/BREV8/CLZ/CTZ/CPOP operating on only the low 32 bits via truncating helper calls, ROL's hardwired 32-shamt identity, and DIV special-case comparisons against 32-bit constants that zero-extend to wrong 64-bit values. The multiplier is fixed 33x33 and needs a ground-up 65x65->128 redesign with 1-2 extra pipeline stages; its 4-cycle latency is hardcoded a second time as MulPipeDepth=4 in int_muldiv_shim, a silent tag/value-mismatch hazard if the two drift. int_muldiv_shim also hardwires 32-bit operand muxes and a low/high result split at bit 32, and declares 32-bit wires against divider ports that would become 64-bit - all compile-and-truncate hazards. The whole W-op family (ADDW through REMUW, plus Zba .UW, Zbb/Zbkb W-forms) is new design work, best structured as a single word-op qualifier driving operand shaping plus one final sign-extend mux. riscv_pkg already has a tree-based clz64; ctz64, cpop64, 64-bit brev8, and XLEN-wide div-overflow constants must be added, while zip32/unzip32 retire to illegal on RV64. Four zero-width-replication sites ({FLEN-XLEN{1'b0}}) in the two shims become lint hazards at XLEN=64 and should become casts. The alu's dead ENABLE_MULDIV=1 legacy path duplicates the div special-case logic with 32-bit constants and should probably be deleted rather than fixed. Latency contracts to re-document precisely: multiplier 4 cycles (header + shim), divider 17 cycles = XLEN/2+1 (shim derives this correctly from XLEN, but comments hardcode 17). + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:159`** [high] SLL/SRL/SRA use i_operand_b[4:0] as shift amount (lines 159-162). At XLEN=64 the native shifts need shamt[5:0]; bit 5 is silently dropped, so e.g. x<<40 computes x<<8 with no compile error. + - *Action:* Widen register-form shamt to i_operand_b[5:0] for native shifts; add SLLW/SRLW/SRAW variants that keep [4:0] and sign-extend the 32-bit result. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:173`** [high] SLLI/SRLI/SRAI (lines 173-175) take the shamt from i_instruction.source_reg_2, a 5-bit field. RV64 immediate shifts have 6-bit shamt (instruction bit 25 = shamt[5]); it cannot be delivered through this field, so shamt 32-63 silently aliases 0-31. Same field feeds BSETI/BCLRI/BINVI/BEXTI (298-301) and RORI (320). + - *Action:* Add a 6-bit shamt channel (widen the field or a dedicated shamt6 input from the shim/decode) plus SLLIW/SRLIW/SRAIW W-forms with 5-bit shamt and bit-25-set illegal. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:233`** [medium] DIV/REM special cases compare 64-bit operands against 32-bit constants: SignedInt32Min (233, 235, 258), NegativeOne (234, 259), and DIVU div-by-zero returns UnsignedInt32Max (246). At XLEN=64 the 32-bit literals zero-extend, so INT_MIN/-1 detection never fires and div-by-zero returns 0x00000000FFFFFFFF instead of all-ones. Note this ENABLE_MULDIV=1 path is currently dead (only int_alu_shim instantiates alu, with ENABLE_MULDIV=0), but it compiles and would misbehave silently. + - *Action:* Replace with XLEN-wide SignedIntMin/NegativeOne/UnsignedIntMax constants (and 32-bit versions for the W-form special cases), or delete the dead ENABLE_MULDIV=1 path entirely. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:63`** [medium] multiplier_input_a/b are [XLEN:0] and multiplier_result is [2*XLEN-1:0], but the multiplier module has fixed 33-bit inputs / 64-bit output. At XLEN=64 the port connections silently truncate 65->33 and the result slice [127:64] reads zeros (dead ENABLE_MULDIV=1 path; also o_result = multiplier_result[31:0] at line 190 is a hardcoded 32-bit slice). + - *Action:* Delete or fully rework the legacy in-ALU muldiv path; if kept, it needs the same 65x65 multiplier as the shim path. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:293`** [high] Zbs BSET/BCLR/BINV/BEXT use i_operand_b[4:0] as bit index (293-296) and BSETI/etc use the 5-bit source_reg_2 (298-301). At XLEN=64 bit indices 32-63 silently alias 0-31. The 32'd1 literals are contextually widened so the shift datapath itself is fine; only the index truncates. + - *Action:* Widen index to [5:0] for register form and route 6-bit shamt for immediate form; BEXT concat {31'd0,...} at 296/301 is a mechanical width fix. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:315`** [high] ROR/ROL/RORI funnel rotates are hardwired for 32: shamt [4:0], ROL computes 6'd32 - shamt (318), RORI uses 5-bit source_reg_2 (320). At XLEN=64 the funnel {a,a} becomes 128-bit but the rotate amount is truncated to 5 bits and ROL's 32-shamt identity is wrong - silently wrong results. + - *Action:* 6-bit rotate amount, ROL = ROR by (7'd64 - shamt), plus new ROLW/RORW/RORIW arms operating on the low 32 bits with sign-extended result. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:322`** [high] CLZ/CTZ/CPOP call clz32/ctz32/cpop32; passing a 64-bit i_operand_a silently truncates the function argument to [31:0], returning counts of the low word only. + - *Action:* Switch to clz64 (exists in riscv_pkg, returns [6:0]) and add ctz64/cpop64; add CLZW/CTZW/CPOPW arms using the 32-bit helpers with sign-extended (always non-negative, so zero-extended-equivalent) result. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:326`** [high] SEXT_B/SEXT_H build 32-bit concats ({{24{...}},...} / {{16{...}},...}). Assigned to a 64-bit o_result these ZERO-extend, so negative bytes/halfwords lose their upper-32 sign bits - silently wrong. + - *Action:* Replication counts become 56/48 (or XLEN-8/XLEN-16). +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:329`** [high] ORC_B covers only bytes [31:0] (329-335); at XLEN=64 operand bits [63:32] are ignored and result upper half is zero - silently wrong. REV8 (336-337) swaps only 4 bytes; RV64 REV8 is an 8-byte swap and its immediate encoding also changes (decode-side). + - *Action:* Extend both to 8 bytes (parametrize over XLEN/8); coordinate REV8 encoding change with decode. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:347`** [high] BREV8 calls riscv_pkg::brev8 whose argument is [31:0] - a 64-bit operand silently truncates, reversing only the low 4 bytes. ZIP/UNZIP (348-349) are RV32-only and must decode as illegal on RV64; their case arms should be removed/gated. + - *Action:* Widen brev8 to 64-bit (8 bytes); delete or gate ZIP/UNZIP arms; decode must reject their encodings. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_muldiv_shim.sv:125`** [high] MUL operand mux is hardwired 32-bit: mul_operand_a/b are [32:0] built from src_value[31]/[31:0] (125-143). At XLEN=64 this silently computes a 32x32 multiply for 64-bit MUL/MULH operands - compiles clean, wrong results. + - *Action:* Widen to [64:0] with sign/zero-extension per op at 64 bits; add MULW arm (sign-extend low-32 inputs is unnecessary - multiply low 32s and sext result). +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_muldiv_shim.sv:162`** [high] MulPipeDepth = 4 is a hardcoded copy of the multiplier's pipeline latency (also documented at lines 28-33). The 64-bit multiplier will be deeper; if this constant is not updated in lockstep the tracker tail samples the wrong product - silent tag/value mismatch on the CDB. + - *Action:* Export latency as a parameter/localparam from the multiplier module and derive MulPipeDepth from it; update header comments. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_muldiv_shim.sv:396`** [high] div_quotient/div_remainder are declared [31:0] (396-397) but connect to divider ports of WIDTH=riscv_pkg::XLEN (399-411). Flipping XLEN to 64 makes the divider 64-bit while these wires truncate its outputs to 32 - tools warn but typically still build; results silently lose the high 32 bits. + - *Action:* Declare as [riscv_pkg::XLEN-1:0]; div_result_32 (516-518) likewise widens. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_muldiv_shim.sv:267`** [medium] {{(riscv_pkg::FLEN - riscv_pkg::XLEN) {1'b0}}, ...} becomes a zero-width replication when XLEN=64=FLEN (also line 518). Legal SV inside a concat only if another member exists - it does here - but this is a known per-tool lint hazard (Verilator/Yosys/Vivado disagree historically). + - *Action:* Replace with FLEN'(value) casts or generate-guarded concat; audit pattern repo-wide. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_alu_shim.sv:72`** [high] alu_instruction.source_reg_2 = i_rs_issue.imm[4:0] delivers only a 5-bit shamt to the ALU for SLLI/SRLI/SRAI/RORI/Zbs-immediates. RV64's 6-bit shamt (imm bit 5) is silently dropped - shamt 32-63 aliases 0-31. + - *Action:* Deliver imm[5:0] via a widened field or dedicated shamt port; W-form immediate shifts keep 5 bits with bit 5 illegal (decode-side check). +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_alu_shim.sv:139`** [low] {{(riscv_pkg::FLEN - riscv_pkg::XLEN) {1'b0}}, alu_result} zero-width replication at XLEN=64 (also line 158). Value semantics become a pass-through (correct); pure lint hazard. Note: integer results are zero-extended into the FLEN=64 CDB carrier today - at XLEN=64 the full value occupies the carrier, so no NaN-boxing interaction, but new FCVT.S.L/LU results elsewhere must still NaN-box; this shim never produces FP values so it is unaffected. + - *Action:* Replace with FLEN'() cast; also extend the non-M-op assertion list (168-172) to cover MULW/DIVW/etc so W-form M ops cannot leak into the ALU shim. +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:808`** [high] SignedInt32Min/SignedInt32Max/UnsignedInt32Max/NegativeOne are bit [31:0] constants (808-811). Any comparison against a 64-bit signal zero-extends them (NegativeOne becomes 0x00000000FFFFFFFF), silently breaking DIV overflow/div-zero detection wherever used at XLEN=64. + - *Action:* Add XLEN-wide SignedIntMin/NegativeOne/UnsignedIntMax (and keep 32-bit ones, properly named, for W-form special cases); audit all users. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:152`** [medium] The entire W-op family is absent from the case statement: ADDW/SUBW/SLLW/SRLW/SRAW/ADDIW/SLLIW/SRLIW/SRAIW, and Zba/Zbb W-forms. Every W-op must produce sext(result[31:0]). + - *Action:* Add W-op enum arms; cleanest structure is a single is_word_op qualifier that (a) muxes 32-bit-shaped datapaths where needed (shifts, ADDW/SUBW carry chain can reuse the 64-bit adder low half) and (b) drives one final {{32{res[31]}},res[31:0]} output mux stage. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:343`** [medium] PACK packs 16-bit halves ({b[15:0],a[15:0]}); RV64 PACK packs 32-bit halves ({b[31:0],a[31:0]}). Current code at XLEN=64 silently produces a zero-extended 32-bit pack. PACKW is a new op. PACKH (345) stays byte-based and its {16'd0,...} concat zero-extends correctly at 64, so it is only a lint/width cleanup. ZEXT.H's decode changes from PACK to PACKW on RV64 (decode-side dependency). + - *Action:* PACK -> 32-bit halves; add PACKW arm ({b[15:0],a[15:0]} sign-extended); keep PACKH; coordinate ZEXT.H alias with decode. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:289`** [medium] SH1ADD/SH2ADD/SH3ADD are width-parametric and fine at 64, but RV64 Zba adds ADD.UW, SLLI.UW, SH1/2/3ADD.UW (zero-extend rs1[31:0] before the shift-add), which do not exist anywhere. + - *Action:* Add .UW arms: operand_a_uw = {32'b0, a[31:0]}, then existing shift-add; SLLI.UW uses the 6-bit shamt path. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/multiplier.sv:74`** [medium] Multiplier is a fixed 33x33->64 datapath end to end: [32:0] ports (74-75), abs_33 (85-87), 17/16 split (90-91), 34/33/32-bit partial products (132-135), 66-bit final add (216-227), o_product_result[63:0] (77). RV64 MUL/MULH/MULHSU/MULHU need 65x65->128. No silent failure (ports mismatch loudly), but it is a full redesign: 65-bit magnitudes split into 4x17-bit chunks give 16 DSP tiles with a deeper reduction tree, or reuse dsp_tiled_multiplier_unsigned(A=B=65: 3x2=6 tiles of 27x35, 4 reduce stages) plus a sign-correction wrapper. Pipeline depth will grow from the current 4 cycles; the depth is a documented contract (header lines 20-63). + - *Action:* Rebuild as 65x65 unsigned-magnitude multiply + sign fixup, likely 5-6 pipe stages at 300 MHz; consider MULW early-out through a 32-bit path (but see uniform-latency constraint in the shim tracker). +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/dsp_tiled_multiplier_unsigned.sv:36`** [low] Fully parameterized (A_WIDTH/B_WIDTH/tile sizes); at 65x65 it elaborates to 6 tiles, PipelineStages=4, PaddedWidth=160-bit adder tree rows. Viable building block for the 64-bit integer multiplier; no code change needed, only instantiation. MinPipelineStages=3 floor (65) is an FP S/D latency-matching contract - do not disturb when reusing for INT. + - *Action:* Candidate reuse for 65x65 magnitude multiply; verify 160-bit registered adds meet 300 MHz or raise ADD_CHUNK_WIDTH chunking review. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/divider.sv:143`** [medium] Each pipeline stage chains TWO WIDTH-bit subtract-and-select iterations combinationally (143-164). At WIDTH=64 that is two dependent 65-bit subtracts plus muxes per 3.33 ns cycle - plausibly the tightest new arithmetic path in the subsystem. + - *Action:* If timing fails at 300 MHz, fall back to 1 bit/stage (65-cycle latency, DivPipeDepth doubles) or restructure the stage; decide before freezing the shim's tracker depth. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_muldiv_shim.sv:265`** [medium] Result select splits the 64-bit product at bit 32: mul_result_32 = is_low ? product[31:0] : product[63:32] (265-267), and is_mul_low is a 1-bit low/high flag (72, 90). RV64 needs a 128-bit product with MUL=low64, MULH*=high64, MULW=sext(product[31:0]) - a 3-way select tracked per in-flight op. + - *Action:* Widen product to 128, replace mul_trk_is_low with a 2-bit result-select carried through the tracker shift register, add sext mux for MULW. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_muldiv_shim.sv:416`** [medium] DivPipeDepth = riscv_pkg::XLEN/2 + 1 self-adjusts to 33 at XLEN=64 (good), but the '// 17' comment and header ('17-cycle latency', lines 29, 36-38) go stale, and the latency doubling changes the scheduling/performance contract. W-form divides (DIVW/DIVUW/REMW/REMUW) can reuse the 64-bit divider by sign/zero-extending 32-bit operands (overflow, div-by-zero, and remainder cases all fall out correctly after sext of the low 32 result), but need a per-op W flag carried through the 33-deep tracker plus a sext mux at FIFO push; op-decode case lists (76-90, 392, 466) must add the W ops. + - *Action:* Add W ops to decode/routing, carry is_word through div tracker, sext(result[31:0]) at push; decide whether 33-cycle uniform latency is acceptable for W divides or a second 17-stage path is warranted (variable latency breaks the shift-register tracking). +- **`hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv:1151`** [medium] Helper coverage for 64-bit Zbb/Zbkb is incomplete: clz32 (1151), ctz32 (1186), cpop32 (1216), brev8 (1279) are 32-bit-only; clz64 exists (1243, returns [6:0], byte-tree, FPGA-friendly - ready to use); ctz64 and cpop64 do not exist; zip32/unzip32 (1291, 1301) are RV32-only ops that must become illegal encodings on RV64. Calling any 32-bit helper with a 64-bit argument silently truncates. + - *Action:* Add ctz64/cpop64 (mirror clz64's byte tree; cpop64 is one more adder level), widen brev8 to 64, keep 32-bit helpers for CLZW/CTZW/CPOPW, retire zip32/unzip32 from the RV64 decode. + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:36`** [medium] alu has an XLEN parameter and partially-parametric core (ADD/SUB/logic/SLT/difference/sltu at 135-171 widen correctly), but the file mixes parametric and hardcoded-32 code so heavily that 'flip XLEN' produces a silently wrong ALU rather than a broken build. Whether to keep the XLEN parameter (dual-width support) or hard-commit to 64 is a project decision that shapes every fix above. + - *Action:* Decide dual-XLEN parameterization vs rv64-only before starting edits; also decide fate of the dead ENABLE_MULDIV=1 legacy path (78-124, 185-275) which duplicates div special-case logic. + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv:163` [low] SLT/SLTI/SLTU/SLTIU results are cast 32'(...) (163, 164, 170, 171). Value is still correct (0/1 zero-extends) but the hardcoded 32 is a width-mismatch lint at XLEN=64. Same class: 32'h0000_0000 at 260, {31'd0,...} at 296/301, 32'd0 at 339/340. +- `hw/rtl/cpu_and_mem/cpu/ex_stage/alu/divider.sv:74` [low] Divider is cleanly WIDTH-parametric (restoring, 2 bits/stage, WIDTH/2 stages + 1 init). At WIDTH=64: latency goes 17 -> 33 cycles; overflow-by-wraparound argument still holds at 64 (abs(INT64_MIN)/1 negated = INT64_MIN) and div-by-zero all-ones/dividend outputs are WIDTH-wide, so no functional change needed for native 64-bit ops. Header comments hardcode '17 cycles'/'16 stages'/'32 for RV32' (65-66, 74, 105) and go stale. +- `hw/rtl/cpu_and_mem/cpu/ex_stage/branch_jump_unit.sv:83` [low] Clean-parametric: jalr_target = (rs1 + sext(imm)) & ~XLEN'(1) widens correctly (JALR LSB masking already right), i_immediate_i_type [31:0] with XLEN'(signed') cast is per-spec, comparators (90-92) are XLEN-wide. Only impact is timing: 64-bit equality/less-than compares feed branch resolution and the mispredict redirect. + +## FPU - RV64F/D additions (int<->FP conversions, FMV, compare/classify int results, NaN-boxing) + +The FPU conversion subsystem is in better shape than expected: fp_convert.sv is already fully parameterized on XLEN (saturation constants, exponent bounds, LZC, mantissa alignment, and the wide/narrow rounding generate all scale correctly), and the value carriers are already FLEN=64, so the widening is mostly plumbing plus new op arms rather than datapath rework. Three silent hazards dominate. First, fp_add_shim.sv:340 instantiates fpu_convert_unit without an .XLEN override, so flipping riscv_pkg::XLEN leaves the converter at 32 bits and silently truncates 64-bit int operands at the port boundary. Second, fpu_compare_unit.sv:73 NaN-boxes FEQ.S/FLT.S/FLE.S integer results with ones in [63:32]; today the shim's [XLEN-1:0] slice at line 406 strips them, but at XLEN=64 rd receives 0xFFFFFFFF_00000001. Third, fp_convert.sv:145 hardcodes sign-bit index 31, which would make FCVT.S.L/FCVT.D.L silently mishandle 64-bit signed operands. The fourth semantic trap is fp_add_shim.sv:425's zero-extension of fp->int results: on RV64 every W-form conversion result and FMV.X.W must sign-extend (WU included), so the shim needs a latched W-vs-L flag orthogonal to op_is_double. The real design work is decoupling integer width from XLEN in fp_convert so W-forms keep 32-bit bounds while L-forms get 64-bit ones, then threading the 10 new op enums through three op-classification expressions, the stage-4 case, and the shim's two routing cases. NaN-boxing infrastructure is already correct end-to-end (ones-boxing in fpu_convert_unit, fp_convert_sd, and the shim; unbox32 checking in the shim), so FCVT.S.L/LU and FMV.W.X inherit correct boxing for free, and FMV.X.D/FMV.D.X are near-free through the existing D-instance FMV path. FCLASS zero-extension is already RV64-correct. fp_convert_sd, fp_compare, fp_sign_inject, fp_result_assembler, fp_operand_unpacker, and the five arithmetic wrapper units are FLEN-neutral and unaffected. Both zero-width {(FLEN-XLEN){...}} replications live in fp_add_shim (lines 406, 425) and disappear naturally when those muxes are rewritten for RV64 semantics. Timing exposure is confined to the multi-cycle convert FSM (64-bit negate+LZC, 117-bit shifter), away from the known-critical rename/wakeup/CDB paths. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_add_shim.sv:340`** [high] fpu_convert_unit is instantiated with NO .XLEN parameter override, so it stays at its default XLEN=32 even after riscv_pkg::XLEN flips to 64. The shim's own localparam XLEN (line 56) becomes 64, so i_int_operand at line 349 feeds a 64-bit expression into a 32-bit port (silent truncation of FCVT.S.L/D.L operands' upper 32 bits) and the 32-bit o_int_result implicitly zero-extends into the 64-bit convert_int_result net. Compiles with only WIDTH warnings in most tools. + - *Action:* Pass .XLEN(riscv_pkg::XLEN) (and audit every fpu_*_unit instantiation for the same omission) as part of the widening; better, make the parameter mandatory-explicit. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_compare_unit.sv:73`** [high] o_result = valid_s ? box32(result_s) : ... unconditionally NaN-boxes single-precision results with ones in [63:32], INCLUDING the integer 0/1 results of FEQ.S/FLT.S/FLE.S (o_is_compare=1). Today the shim slices compare_result[XLEN-1:0]=[31:0] so the boxing is discarded; at XLEN=64 the slice becomes [63:0] and rd receives 0xFFFFFFFF_0000000{0,1}. Silent compile-and-misbehave. + - *Action:* Mux the boxing on is_compare_s: box only FMIN.S/FMAX.S results; zero-extend compare results. Add a directed test asserting FEQ.S writes exactly 0 or 1 into a 64-bit rd. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv:145`** [high] is_signed_conv && int_operand_reg[31] hardcodes sign-bit index 31. If this instance's XLEN parameter is flipped to 64 for the L-form conversions, FCVT.S.L/FCVT.D.L reads bit 31 instead of bit 63: negative 64-bit operands with bit31=0 are treated as positive, positives with bit31=1 get negated. Silently wrong results, no compile error (bit 31 always exists). + - *Action:* Change to int_operand_reg[XLEN-1]. For W-forms retained at 64-bit XLEN, pre-extend the operand at the wrapper instead (see design items). +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_add_shim.sv:425`** [high] Comment and code implement 'FP->INT: zero-extend int result to FLEN'. On RV64, FCVT.W.S/FCVT.WU.S/FCVT.W.D/FCVT.WU.D and FMV.X.W must all SIGN-extend their 32-bit result into the 64-bit rd (WU included, per spec). Zero-extension here silently produces wrong rd values for negative/W-form results. Also {(FLEN-XLEN){1'b0}} becomes a zero-width replication at XLEN=64 (legal in concat, per-tool lint hazard). + - *Action:* Add a W-vs-L discriminator (from op_reg) and sign-extend bit 31 for all W-form fp->int and FMV.X.W results; pass L-form results through full-width. Remove/guard the zero-width replication. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_add_shim.sv:406`** [medium] {(FLEN-XLEN){1'b0}} zero-width replication once XLEN=64 (lint hazard per tool), and compare_result[XLEN-1:0] widens to [63:0], which is what exposes the fpu_compare_unit boxing corruption (see companion finding). FEQ/FLT/FLE results must be zero-extended from the single bit, not sliced from a possibly-boxed 64-bit bus. + - *Action:* Rewrite as {{(FLEN-1){1'b0}}, compare_result[0]} for is_compare results (after fixing the wrapper boxing), eliminating both the zero-width replication and the aliasing. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv:431`** [medium] move_int_result_s2_comb = fp_operand_reg[XLEN-1:0]: in the S instance (FP_WIDTH=32) with XLEN=64 this is an out-of-range part-select of a 32-bit vector (elaboration error - loud, not silent). Semantically FMV.X.W on RV64 must produce sign_extend(fp_operand[31:0]) anyway; FMV.X.D (new) needs the full 64-bit move in the D instance. + - *Action:* Guard with generate or min(FP_WIDTH,XLEN) slice and add sign-extension for FMV.X.W when XLEN > FP_WIDTH; add FMV_X_D/FMV_D_X arms to the stage-4 case (lines 528-535). +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv:141`** [high] is_signed_conv tests only FCVT_S_W/FCVT_D_W (also line 375 is_signed_conv_s2, and line 241 is_unsigned_conv tests only FCVT_WU_S/FCVT_WU_D). The new L-form ops FCVT_S_L/FCVT_D_L/FCVT_S_LU/FCVT_D_LU are absent; if enums are added but these lists are not updated, FCVT_*_LU would be classified 'signed' and FCVT_*_L 'not signed' - silently wrong conversions. + - *Action:* Extend all three op-classification expressions (lines 141-142, 241-242, 375-376) with the L-form enums; consider deriving signed/unsigned/int-width flags once at decode and passing them as sideband bits instead of re-decoding op enums per stage. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv:475`** [high] Stage-4 unique case enumerates only FCVT_{W,WU}_{S,D}, FCVT_{S,D}_{W,WU}, FMV_X_W, FMV_W_X. FCVT_{L,LU}_{S,D}, FCVT_{S,D}_{L,LU}, FMV_X_D, FMV_D_X fall into the default arm and return all-zero results with no flags - silent if decode routes them here without new arms. + - *Action:* Add case arms for all 10 new ops (L-form conversions in both directions plus FMV.X.D/FMV.D.X), each with correct is_fp_to_int routing. +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv:44`** [high] fp_convert conflates the integer operand/result width with XLEN. On RV64 the same S-format unit must handle both W-forms (32-bit bounds: saturate at 2^31/2^32, then sign-extend to 64 - both signed and unsigned) and L-forms (64-bit bounds 2^63/2^64). The saturation constants IntMax/IntMin/UintMax (lines 84-86) and range checks MaxExpSigned/MaxExpUnsigned (lines 82-83) all key off XLEN, so a single XLEN=64 instance gets W-form bounds wrong; a kept XLEN=32 instance gets L-forms wrong. The good news: everything is already cleanly parameterized on XLEN, so an INT_WIDTH split is mostly plumbing. + - *Action:* Either add a runtime W/L mode input selecting between two bound sets, or split the parameter into INT_WIDTH (32/64) with four converter configurations (S/D fp-width x W/L int-width) - int->fp W-forms can alternatively reuse the 64-bit datapath by pre-extending the operand (sign-extend for .W, zero-extend for .WU) at the wrapper since the numeric result is identical. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_add_shim.sv:131`** [medium] Shim op-decode (lines 131-140) and cvt_use_s/d routing (lines 328-337) enumerate only W-form convert ops. New ops need routing, and the existing op_is_double flag (keyed to FP width, used at lines 428-429 for NaN-boxing) is orthogonal to int width: FCVT.L.S is single-FP but 64-bit int, FCVT.D.W is double-FP but 32-bit int. FMV.X.D/FMV.D.X must route with op_is_double=1 so line 428 skips boxing. + - *Action:* Add all L-form and FMV.D ops to both case statements; introduce a separate int-width flag latched at fire so the result mux can pick sign-extend-32 vs pass-64 independently of FP-width boxing. + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv:425` [low] gen_move_fp_pad zero-pads FMV int->FP with {(FP_WIDTH-XLEN){1'b0}} when FP_WIDTH>XLEN. At XLEN=64 both instances take the nopad branch so no zero-width replication occurs (generate-guarded), and NaN-boxing of FMV.W.X is correctly applied downstream with ones (fpu_convert_unit box32 line 47-49, shim line 429). The pad branch simply becomes dead code. Low risk; noting per the flag-every-occurrence instruction. +- `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv:36` [low] Header comments (lines 21-26, 36) document 32-bit-int-only semantics ('24-bit mantissa for 32-bit int', FCVT.W list); the FracBits>=(XLEN-1) generate split at lines 359-370 already handles the RV64 consequence that FCVT.D.L becomes inexact-capable (52 < 63 selects the narrow/rounding path automatically), so this is doc-only. +- `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_classify.sv:43` [low] o_result is hardcoded [31:0] with the 10-bit class mask zero-padded (line 113), and the shim zero-extends it to FLEN via {(FLEN-32){1'b0}} (fp_add_shim.sv:415, nonzero replication at any XLEN). Zero-extension is exactly the correct RV64 FCLASS semantics, so this path needs no change - recording as verified-correct rather than a defect. + +## rename/dispatch/issue machinery (dispatch, reservation station + issue2 selector, RAT, CDB arbiter, FU-CDB adapter, dispatch-RS router) + +The rename/dispatch/issue machinery is in good shape for RV64: value carriers (RS source values, CDB, RAT lookup, repair channels, bypass masks) are already FLEN=64-wide, and tag/valid/wakeup logic is width-neutral, so cdb_arbiter.sv, fu_cdb_adapter.sv, rs_issue2_selector.sv, and dispatch_rs_router.sv need zero changes. The dangerous concentrations are in dispatch.sv and one line of reservation_station.sv. The single worst hazard is the immediate path: dispatch assigns the hardcoded-[31:0] from_id_to_ex_t immediates into its [XLEN-1:0] imm (8 assignment sites across both slots), which at XLEN=64 silently ZERO-extends sign-extended I/S/U immediates — every negative immediate and every LUI/AUIPC result would be wrong until the package fields are widened with sign-extension in ID. Second, dispatch's mem_signed formula excludes word loads from sign-extension, so RV64 LW would silently stay zero-extended; the mem_size and imm-select case statements likewise default new RV64 ops (LD/SD/LWU/AMO*.D, W-form ALU immediates) to WORD-size/no-immediate — compile-and-misbehave defaults that must be updated in lockstep with the ID-stage rs_type pre-decode. Third, the RS's stage2_is_sc compares against SC_W only, so SC.D would lose its store-conditional sideband. The RAT contains all eight {{(FLEN-XLEN){1'b0}},...} zero-extension points where INT operands enter FLEN carriers; each becomes an identity extension at XLEN=64 (semantically correct) but a zero-width replication (per-tool lint hazard) — replace with FLEN'() casts. RAT rename/checkpoint storage is tag-only and does not grow. Timing exposure is confined to the XLEN-carrying sideband (imm/pc/targets/link_addr): rs_dispatch_t grows 160 bits across 13 fanned-out packets on the historically-critical ID->RS write path, and the RS payload LUTRAM widens accordingly, while the known-critical wakeup/CDB cones are structurally unchanged. csr_write_data's uimm5 zero-extension is parametric and correct as-is. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/dispatch/dispatch.sv:395`** [high] imm ([XLEN-1:0]) is assigned from from_id_to_ex_t.immediate_i_type which is hardcoded [31:0] in riscv_pkg (line 927). At XLEN=64 this implicit width mismatch ZERO-extends a sign-extended 12-bit immediate: ADDI x1,x0,-1 yields imm=64'h0000_0000_FFFF_FFFF. Same pattern at lines 401 (immediate_s_type), 407/412 (immediate_u_type — LUI/AUIPC must sign-extend bit 31 to 63 on RV64), and slot-2 mirrors at 565, 570, 575, 580. + - *Action:* Widen immediate_i/s/u_type to [XLEN-1:0] in riscv_pkg with sign-extension performed in ID; the dispatch assignments then become identity. Verify no Verilator WIDTHEXPAND waiver hides the mismatch in the interim. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/dispatch/dispatch.sv:363`** [high] mem_signed = is_load && !is_load_unsigned && (is_load_byte || is_load_halfword) — word loads are excluded from sign-extension. On RV64, LW must SIGN-extend to 64 bits and LWU zero-extends; with this formula unchanged, LW silently stays zero-extended (wrong result for every negative word load) and LD gets no defined treatment. Slot-2 mirror at lines 535-537. + - *Action:* Rework: mem_signed must cover LW (add is_load_word term or invert to !is_load_unsigned && size MEM_SIZE_DOUBLE and LWU -> MEM_SIZE_WORD to both slot-1 and slot-2 case statements. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/dispatch/dispatch.sv:383`** [high] Immediate-select case lists RV32 ops only; new RV64 ops (ADDIW, SLLIW/SRLIW/SRAIW, LWU, LD -> I-type; SD -> S-type; SLLI_UW, RORIW, W-form B-ext immediates) fall into the default arm which sets use_imm=0 and imm=0 — compiles and silently dispatches with no immediate. Slot-2 mirror at 555-588. + - *Action:* Add every new immediate-form RV64 op to the I-type and S-type arms of both slot-1 and slot-2 case statements when the ops are added to instr_op_e. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/dispatch/dispatch.sv:272`** [medium] rs_type is pre-decoded in the ID stage and carried in from_id_to_ex_t (dispatch just casts it, lines 272-273/457-458). The RS-family routing for new RV64 ops (W-ops -> RS_INT/RS_MUL, LD/SD/LWU/AMO*.D -> RS_MEM, FCVT.L forms -> RS_FP) therefore lives in id_stage, NOT in these files — but dispatch's own case(op) lists (findings above) must be updated in lockstep or they silently diverge from the ID routing. + - *Action:* Coordinate: whoever adds ops to the ID rs_type pre-decode must update dispatch's imm/mem_size/mem_signed cases in the same change. + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/tomasulo/dispatch/dispatch.sv:1204` [low] csr_write_data = {{(XLEN - 5) {1'b0}}, csr_imm} — parametric zero-extension of the CSR uimm5; correct per spec at XLEN=64 (uimm is always zero-extended). Slot-2 mirror at 1253-1255. Non-zero replication at both XLEN values, no lint hazard. Verified clean; listed because it is an extension point the widening must not disturb. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/reservation_station/reservation_station.sv:714` [low] Payload-pack width comments ('// 32 imm', '// 32 branch_target', '// 32 predicted_target', '// 32 pc', '// 32 link_addr' at lines 715, 717, 719, 727, 728 and the slot-2 pack at 739-763) document 32-bit fields; the PayloadWidth expression (lines 661-663) is already XLEN-parametric so function is unaffected, but the comments go stale at XLEN=64. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/reservation_station/reservation_station.sv:661` [low] PayloadWidth grows by 5*32 = 160 bits per entry (imm, branch_target, predicted_target, pc, link_addr), doubling the XLEN share of the 2-write-port distributed-RAM payload and the stage2/stage2b registers. mem_size stays 2 bits (2'(dispatch_mem_size) at 723/749, pl_mem_size_bits at 791) — adequate since mem_size_e already encodes DOUBLE. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/reservation_station/reservation_station.sv:273` [low] int_rs_writes_cdb() excludes only BEQ..BGEU from CDB writeback; RV64 adds no new branch ops, and all new W-ops correctly fall into the default (writes CDB). Verified width-neutral and RV64-correct as-is. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/register_alias_table/register_alias_table.sv:214` [low] RAT storage is tag-only (RatEntryWidth = 2 + ReorderBufferTagWidth; snapshot widths at 218-225 are XLEN-independent). Checkpoint save/restore, commit-clear, and rename write paths carry no data values — fully width-neutral. Only the regfile-passthrough concatenations (separate finding) touch XLEN. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/dispatch/dispatch.sv:300` [low] x0 zero-detection uses '!= 5\'b0' (also line 478) and the intra-bundle RAW compares at 1071-1083 compare 5-bit RegAddrWidth addresses — all register-address width, independent of XLEN. Wakeup/tag-match logic throughout dispatch and the RS compares ReorderBufferTagWidth tags only. Confirmed width-neutral. + +## ROB, commit, branch recovery, pipeline control + +This subsystem is in unusually good shape for the RV64 respin: every assigned file except reorder_buffer.sv and frontend_validity_tracker.sv is either fully XLEN-parametric or width-independent, and the value path is already FLEN=64 end to end (ROB value RAMs, commit structs, regfile write data), so the int-result slice at commit (rob_commit.value[XLEN-1:0] in commit_actions.sv:141/164) becomes exactly right at 64 with no literal-31 anywhere. I found no bit-31 sign checks, no [4:0] shamt truncations, and no addr[1:0] word-index assumptions in these files. The real hazards are three: the FLEN-XLEN zero-width replications at reorder_buffer.sv:1008/1013 (lint-fragile once XLEN=64), the ucounter_onehot addr[7] aliasing that would keep treating cycleh/timeh/instreth as legal mcounteren-gated counters when RV64 requires them to be illegal CSRs, and the IF-stage RVC classifier arm that turns C.ADDIW into phantom control flow (latent today because its aggregate is unconsumed). The NARROW_DATA_WIDTH(XLEN) narrow-port config on all 12 rob_value replicas degenerates safely (the RAM primitive guards NARROW front_end_control_flow_pending; that aggregate currently has no exported consumer, so the impact is latent/perf-only - but it compiles and misclassifies silently). + - *Action:* Remove the c_funct3==3'b001 arm for RV64 (part of the RVC recode: C.JAL disappears, becomes C.ADDIW). PD/ID classifiers are safe - they key on the decompressed opcode. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv:2274`** [low] Exception plumbing is width-clean but interface-coupled: o_trap_cause stays an ExcCauseWidth exc_cause_t code (RAM at 1540-1554), o_trap_pc/o_trap_value are XLEN and correct at 64 (head_value[XLEN-1:0] takes the full FLEN=64 slot, so a 64-bit faulting address reaches mtval). The 64-bit mcause form (interrupt bit 63) and mepc width must be handled by the trap-unit/csr_file consumers - nothing in the ROB touches bit 31/63. + - *Action:* No ROB change; verify the trap-unit consumer of o_trap_cause places the interrupt bit at 63 and zero-extends the code (cross-subsystem check). + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv:1347` [medium] All 12 rob_value RAM replicas set NUM_NARROW_WRITE_PORTS(2)/NARROW_DATA_WIDTH(XLEN) against DATA_WIDTH(FLEN). At XLEN=64 narrow==full width; verified lib/ram/mwp_dist_ram.sv guards this case safely (narrow-contract checks gated by NARROW_DATA_WIDTH < DATA_WIDTH, per-port width select degenerates), but the routability optimization the comment at 1315-1322 relies on (alloc banks store only the low XLEN of FLEN) silently evaporates: alloc-bank LUTRAM doubles across 12 replicas in the documented X3 congestion band. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv:798` [low] head_link_is_compressed = (head_value[XLEN-1:0] == (head_pc + 32'd2)): semantically correct at XLEN=64 (32'd2 zero-extends in the 64-bit self-determined context), but the literal should be XLEN-sized for clarity and the equality becomes a 64-bit compare feeding o_commit_comb.is_compressed; same 32'd2/32'd4 literals at 799, 2322, 2493 in the fallthrough/next-PC adders. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv:2042` [low] Alloc-time JAL misprediction detect (predicted_target != branch_target, also 2054-2056 for slot 2) widens from 32- to 64-bit equality in the dispatch/allocation cycle; correctness is parametric, cost is timing. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv:2760` [low] Sim-only retire trace prints head_value_eff[31:0] with %08x and pc with %08x: at RV64 the trace silently truncates values and PCs, corrupting the debug artifact RV64 bring-up will lean on. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv:1726` [low] Comment 'CSR write data RAM (32-bit, written at allocation)' goes stale; the RAM itself is DATA_WIDTH(XLEN) and widens correctly, carrying 64-bit csr_write_data through the commit record to csr_file. +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/branch_recovery/branch_resolution.sv:219` [medium] Target-misprediction compare branch_target_resolved != rs_issue_int.predicted_target widens to 64-bit equality at the head of the documented 16-level stage2_op -> branch_mispredicted -> early-mispredict-capture critical cone (the file's own comments cite this path at -0.739ns historically). Operand slices at 199-200 ([XLEN-1:0] of FLEN src values) are correct at 64. +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/branch_recovery/early_misprediction_recovery.sv:180` [low] early_mispredict_is_compressed capture: (rs_issue_int.link_addr == rs_issue_int.pc + 32'd2) - the literal zero-extends correctly at 64, but the equality/add become 64-bit in the capture cycle; literal should be XLEN-sized. +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/recovery/ex_comb_synthesizer.sv:137` [low] ras_push_address_after_restore = mispredict_commit_q.pc + (is_compressed ? 32'd2 : 32'd4) (also lines 142-143): correct at 64 via zero-extension, cosmetic literal width only. + +## load/store queues, forwarding, AGU, memory request routing + +The entire LQ/SQ memory subsystem is architected around a 32-bit data bus with FLEN=64 payload carriers, so FLD/FSD work today via explicit two-phase (word + word) state machines; RV64's real cost here is collapsing that to a single-beat 64-bit bus, which touches load_queue, store_queue, the router, and cached_tier_adapter simultaneously. The most dangerous silent-misbehavior hazards found: (1) the hand-tiled address comparators word_addr_eq/full_addr_eq/word_addr_inc_eq in store_queue.sv and sq_forwarding_unit.sv ignore all diff bits above [29]/[31], so at XLEN=64 forwarding and ordering CAMs report false address matches; (2) lq_l0_cache's MMIO decode uses XLEN-relative bits [XLEN-1:XLEN-2], which at 64 makes MMIO cacheable; (3) NaN-boxing concats {32'hFFFF_FFFF, xlen_data} silently truncate away the box at XLEN=64 in four places in load_queue.sv; (4) cached_tier_adapter's word-select addr[2+:] would return the wrong dword from a line; (5) the AMO ALU would run AMO*.W as full 64-bit operations; and (6) sq_forwarding_unit's 2-bit store_off misplaces sub-word store bytes in the 64-bit forwarded image. load_unit needs a full rebuild (8-byte extraction, LW sign-extension semantic change, LWU, LD). sc_pending_unit only matches SC_W and compares reservations at word granule; SC.D would deadlock at the ROB head if not added. Byte-enable plumbing is 4-lane end to end (SQ gen_byte_en, router muxes and hardcoded AMO 4'b1111 strobes, adapter wstrb math) and must become 8-lane. Misalignment detection already handles the 8-byte class, and the SQ data payload/forward mirrors are already FLEN=64, so those are free. Key policy calls: high-address-bit handling for all region decodes, stored-address truncation to keep the forwarding CAM and L0 tag compares narrow (both sit on documented critical paths at 300MHz), reservation granule, and 64-bit MMIO access support (CLINT mtime). lq_issue_selector is pure control logic and is the only assigned file that is clean as-is. Coverage: all eight small/medium files read in full; load_queue.sv and store_queue.sv were grep-driven with all data-path, AMO, reservation, drain, forwarding, and response regions read directly and only control-scan/formal regions skimmed. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_unit.sv:110`** [high] Extraction covers only data[31:0] with addr[1:0]/addr[1] selects; at XLEN=64 LB/LH with addr[2]=1 pick the wrong byte/half from the 64-bit word, and the LW fallthrough (line 125) returns the raw 64-bit bus value with no word select and no sign-extension (RV64 LW must sign-extend); no LWU or LD distinction exists. + - *Action:* Rebuild extractor for 8-byte data: addr[2:0] byte select (8 pre-extended bytes), addr[2:1] half select, new word select on addr[2] with sign/zero mux (LW/LWU), and LD passthrough; add size/unsigned control inputs accordingly. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_l0_cache.sv:94`** [high] lookup_mmio = i_lookup_addr[XLEN-1:XLEN-2]==2'b01 tests bits [63:62] at XLEN=64, which are always 0 for the sub-4GiB map, so MMIO addresses (0x4000_0000 quadrant) silently become cacheable and MMIO loads can be served stale from L0 (formal props at 233/271 have the same relative index). + - *Action:* Decode MMIO at fixed bits [31:30] (subject to the high-bit policy), not XLEN-relative bits. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:1728`** [high] FLW NaN-boxing {32'hFFFF_FFFF, lq_data_lo_rd} concatenates 32 + XLEN bits; at XLEN=64 the 96-bit concat is silently truncated to FLEN=64, keeping only the data and DROPPING the NaN-box — FLW results become unboxed and downstream single-precision consumers misbehave. Same pattern at 1831, 1841, 1859. + - *Action:* Box explicitly as {32'hFFFF_FFFF, data[31:0]} at every producer; new FCVT.S.L/LU results elsewhere must match this convention. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:1732`** [low] {{(FLEN - XLEN){1'b0}}, ...} becomes a zero-width replication at XLEN=64 — legal SV inside a concatenation but a per-tool lint/elaboration hazard; occurrences at 1732, 1834, 1844, 1853, 1875 (and sq_forwarding_unit.sv 557/559). + - *Action:* Replace with direct FLEN-wide assignment (identity when FLEN==XLEN) during the widening pass; audit every FLEN-XLEN replication in the tree. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:686`** [high] amo_compute is XLEN-wide and encode_amo_kind (349-361) maps only AMO*_W ops; at XLEN=64 AMO*.W silently performs 64-bit add/min/max/logic on the full dword instead of a 32-bit operation with the old value sign-extended into rd — wrong results whenever bit 31 or high half matters. + - *Action:* Add a width bit to amo_kind; implement 32-bit sub-ALU + sign-extension for .W forms and full 64-bit ops for new AMO*.D; widen issued_amo_rs2/amo_write_data paths (already XLEN) accordingly. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/store_queue.sv:276`** [high] word_addr_eq hand-tiles the comparator into six 5-bit groups covering only diff[29:0]; at XLEN=64 WordAddrWidth=62 and bits [61:30] of the XOR are silently IGNORED — false address equality, wrong store-to-load ordering/forwarding. full_addr_eq (297, covers [31:0]) and word_addr_inc_eq (337-342) have the same truncation; all three are duplicated in sq_forwarding_unit.sv (139-215). + - *Action:* Re-tile the group structure for the full compare width (or narrow stored addresses to 32-bit physical per the address-width decision, keeping the current tiling). +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/store_queue.sv:363`** [high] gen_write_data returns XLEN-wide data but the lane replication is 32-bit-bus shaped: BYTE={4{..}}, HALF={2{..}}, WORD=data[31:0]; at XLEN=64 SB/SH only populate the low 4 bytes (stores with addr[2]=1 write garbage/zero lanes) and SW is zero-extended into the wrong lane for addr[2]=1 — silent wrong store data once strobes are widened. + - *Action:* Replicate across 8 lanes ({8{byte}}, {4{half}}, {2{word}}) and pass the full 64-bit payload for DOUBLE (INT SD and single-beat FSD). +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/sq_forwarding_unit.sv:552`** [high] Forwarded-image reconstruction shifts raw[XLEN-1:0] by {store_off,3'b000} where store_off is captured as address[1:0] (2 bits, struct field line 110, capture 476); at XLEN=64 a sub-word store at addr[2]=1 lands in the wrong byte lanes of the 64-bit memory image — silent wrong forwarded data. + - *Action:* Capture store_off as addr[2:0] (3 bits) and shift up to 56; widen fwd_load_byte_mask (270) to 8 lanes from i_sq_check_addr[2:0]. +- **`hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/cached_tier_adapter.sv:119`** [high] Word/line lane math hardcodes 4-byte words: read_word_sel = pending_read_addr[2+:WordSelBits] and wstrb placement addr[OffsetBits-1:2]*4+:4 (line 111). If XLEN flips to 64, WordsPerLine halves to 4 and read_word_sel picks addr[3:2] instead of the dword index addr[4:3] — the adapter silently returns the WRONG DWORD from the line as valid data; strobe placement is similarly wrong. + - *Action:* Index by addr[3+:WordSelBits], place strobes at addr[OffsetBits-1:3]*8+:8, widen i_write_byte_en/pending_write_byte_en to [7:0]; line width (256b) and replication are otherwise parametric. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_l0_cache.sv:74`** [medium] Line geometry is 4-byte words (index addr[2+:IndexWidth], addr[1:0] ignored) and TagWidth = XLEN-2-IndexWidth grows to 55 bits at XLEN=64; single-beat 8-byte loads need dword-granule lines and dword-granule invalidation, and the wide tag compare sits on the historically critical lookup-hit cone. + - *Action:* Move to dword lines (index addr[3+:], data 64b) or keep word lines with dual lookup; decide 32-bit physical-address tag truncation to keep the compare narrow. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:1444`** [medium] Two-phase FLD machinery (sq_check_fp64_phase_q +32'd4 second beat at 1444-1451, split lo/hi XLEN data RAMs at 479-512, phase-0/1 write steering 1574-1592, phase advance 2434-2439, re-issue 2330-2338) assumes a 32-bit data bus; with a 64-bit bus it does a redundant second fetch and the hi-half plumbing becomes vestigial. + - *Action:* Collapse FLD (and new INT LD) to a single 64-bit beat: delete fp64_phase state, merge lq_data lo/hi into one FLEN RAM, remove the +4 address leg. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:1442`** [medium] cache_hit_fast_path gates FP loads to WORD size but admits any INT load; a new INT LD (is_fp=0, size=MEM_SIZE_DOUBLE) would pass and consume a word-granule L0 entry. All 'is_fp && MEM_SIZE_DOUBLE' conjunctions (1574, 1580, 1723, 1773, 1799, 2330, 2435) encode 'DOUBLE implies FP' and break for INT LD. + - *Action:* Re-key DOUBLE handling on size alone (single-beat for both INT LD and FLD after the bus widening); L0 eligibility follows the new line geometry. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:707`** [medium] Cached-line invalidation snoop compares issued_addr at word granule [XLEN-1:2]; with single-beat 8-byte accesses word-granule compares here (and the wrapper reservation snoop) miss the second word of a dword access unless moved to dword granule. + - *Action:* Move overlap/invalidate compares to [XLEN-1:3] once accesses are dword-granular; audit every [XLEN-1:2] compare in the LQ/SQ/wrapper for the same reason. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/store_queue.sv:349`** [high] gen_byte_en produces [3:0] strobes from addr_offset[1:0]; a 64-bit data bus needs [7:0] strobes from addr[2:0] (BYTE=1<LD/FLD, and covered-subset forwarding for sub-dword loads. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/atomics/sc_pending_unit.sv:155`** [medium] SC allocation predicate is 'op == riscv_pkg::SC_W' only; a future SC.D would never be tracked, so an SC.D at ROB head never fires and the core deadlocks (exactly the failure mode this table was built to fix). + - *Action:* Match both SC_W and new SC_D (and record the size if success semantics ever need it). +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/atomics/sc_pending_unit.sv:166`** [medium] SC success compares reservation vs SC address at word granule [XLEN-1:2]; RV64 needs at least an 8-byte reservation granule for LR.D/SC.D, and the wrapper's snoop-invalidate compare (tomasulo_wrapper.sv 1305) uses the same word granule. SC result value at 181 is already correct (0/1 zero-extended into FLEN). + - *Action:* Move reservation compares to dword granule [XLEN-1:3] (spec permits a larger granule, so this also stays correct for LR.W/SC.W); keep snoop-invalidate granule consistent. +- **`hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/data_mem_request_router.sv:236`** [high] Byte-enable plumbing is 4-lane throughout (ports 54/82/83/86, muxes 235-259) and AMO writes hardcode 4'b1111 — on a 64-bit bus an AMO.W must drive a lane-shifted 4-byte strobe (addr[2] selects the half) and AMO.D an 8-byte strobe; leaving 4'b1111 silently writes the wrong/partial lanes. + - *Action:* Widen all strobe buses to [7:0]; derive AMO strobes from AMO size + addr[2]; SQ strobes arrive pre-widened from gen_byte_en. + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/store_addr/sq_early_addr_pipeline.sv:458`** [medium] is_mmio decode is a fixed-bit test addr[31:30]==2'b01 (also 463/468/480/485/490); at XLEN=64 a computed effective address with nonzero bits [63:32] (e.g. 0x1_4000_0000) still matches and aliases into the MMIO quadrant — behavior for out-of-map high bits is a project policy decision affecting every region decode (this file, lq_l0_cache, router range compares, is_cached_addr). + - *Action:* Decide high-bit handling: access-fault on addr[63:32]!=0, architectural truncation, or full 64-bit region decode; apply uniformly to all MMIO/cached/BRAM decodes. +- **`hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/data_mem_request_router.sv:148`** [medium] MMIO path is 32-bit-register oriented: range decode + single-word read data mux (357) and address-equality pulse decodes (375-378); a 64-bit MMIO LD (e.g. RV64 code reading CLINT mtime in one load) has no data path, and what LD/SD to MMIO should do (support, split, or fault) is undecided. + - *Action:* Decide MMIO access-width policy: either add a 64-bit MMIO read/write sideband (mtime as one LD) or restrict MMIO to <=32-bit accesses and document/enforce (fault or software convention). + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:70` [low] CACHED_BASE/CACHED_SIZE_BYTES are 32-bit 'int unsigned' parameters later part-selected as CACHED_BASE[XLEN-1:0] (line 343-344); at XLEN=64 that is an out-of-range part-select of a 32-bit value (compile error / tool-dependent). Same pattern: store_queue.sv 63-64/776-784, data_mem_request_router.sv 39-45/131-133/149/159/181/186. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:1625` [medium] lq_data_hi_wd[1] = i_sq_forward.data[FLEN-1:XLEN] becomes the reversed/null range [63:64] at XLEN=64 — a compile error, and the lo/hi split it feeds becomes 128 bits of storage for a 64-bit payload (same range also in sq_forwarding_unit.sv 559 and its FORMAL refs 397/406). +- `hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv:329` [low] is_load_misaligned already implements the 8-byte class (MEM_SIZE_DOUBLE -> |addr[2:0]), so LD/SD/LR.D/SC.D/AMO*.D misalign detection is free once INT ops carry size=DOUBLE; no change needed here. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/store_addr/sq_early_addr_pipeline.sv:424` [low] The four store-AGU adders (424-441) are XLEN-parametric and widen cleanly to 64-bit; imm is [XLEN-1:0] in rs_dispatch_t so base+imm widening is mechanical, but the CARRY8 chains double in length (registered inputs, so slack likely absorbs it). +- `hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_issue_selector.sv:36` [low] Pure control/selection logic over valid/issued/ROB-tag masks with no address or data widths; unaffected by the XLEN flip. + +## CSR file and trap unit (csr_file.sv, trap_unit.sv, plus CSR-facing perf-counter widths) + +Both assigned modules are XLEN-parameterized in their port/datapath declarations, so most architectural registers (mepc, mtval, mscratch, mtvec, exception PC/cause/tval pipeline) widen for free; the danger is concentrated in literals and constants that zero-extend silently. The two highest-value hazards: (1) trap_unit assigns the riscv_pkg IntMachine* constants (bit [31:0], 32'h8000_000x) into the XLEN-wide interrupt_cause, so at XLEN=64 the mcause interrupt bit lands at bit 31 instead of 63 - every interrupt reads as a bogus synchronous exception, and the module's own case-compare and formal properties use the same zero-extended constants so nothing catches it; (2) csr_file's MisaValue (32'h4010_112F) zero-extends to give MXL=0 with a stray reserved bit 30. A third silent failure is the counter read mux, which returns cycle_counter[31:0]/i_mtime[31:0]/instret_counter[31:0] zero-extended - RV64 requires full 64-bit single-CSR reads, and the five high-half addresses (0xC80-0xC82, 0xB80, 0xB82) must newly raise illegal-instruction, which requires touching the ROB's Zicntr pre-decode (the mcounteren fault term at reorder_buffer.sv:761-763) because csr_file has no illegal-CSR path of its own - unknown CSRs currently read as 0. mstatus is composed from individual field flops via a 32-bit concat and needs the design rework: UXL=2 hardwired at [33:32], SD at 63, with the existing MIE/MPIE/MPP/MPRV bit positions and write extraction unchanged. mcounteren correctly stays a 32-bit, 3-bit-storage WARL CSR; its base-counter U-mode gate survives, only the *h address classification changes. Everything else - mepc bit-0 masking, mtvec mode handling, the interrupt hold/arming machinery, WFI, drain-wait - is width-agnostic and mechanical. The perf-counter modules are already 64-bit on their CSR-facing buses and are unaffected; only the split MperfData/MperfDataH readback is a low-stakes policy call. Remaining policy items are the high-address-bit treatment for trap/return targets under the sub-4-GiB map and whether the *h-CSR illegality ride-along becomes a general unimplemented-CSR fault. Timing exposure is modest and mostly in the already-sensitive take_trap/redirect cone and the widened CSR read mux; the instret carry-chain retime rationale is preserved as long as the counter stays 64-bit with the staged increment. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:219`** [high] MisaValue = 32'h4010_112F assigned to logic [XLEN-1:0]. At XLEN=64 this zero-extends to 64'h0000_0000_4010_112F: MXL at [63:62] reads 0 (invalid) and the old RV32 MXL=1 bit lands at reserved bit 30. misa must become 64'h8000_0000_0010_112F (MXL=2, letters A/B/C/D/F/I/M/U unchanged). Compiles clean, silently reports a malformed misa. + - *Action:* Replace literal with XLEN-conditional value: MXL=2 in [63:62], extension letters in [25:0]; drop bit 30. Update the comment block at lines 215-218 and header line 41 (0x4010_112F). +- **`hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv:360`** [high] interrupt_cause_comb is assigned riscv_pkg::IntMachineExternal/Software/Timer, which are bit [31:0] constants 32'h8000_000B/3/7 (riscv_pkg.sv:742-744). At XLEN=64 they zero-extend into the 64-bit interrupt_cause, putting the mcause interrupt bit at bit 31 instead of bit 63. mcause written on interrupt entry reads as a giant synchronous-exception code; any handler testing mcause<0 (Linux does) misroutes every interrupt. The case at lines 382-386 and formal props at 550/555/560 compare the same zero-extended constants so they stay self-consistent and nothing fails in sim/formal - fully silent. + - *Action:* Make IntMachine* constants XLEN-wide with bit XLEN-1 set (1<<63 | code) in riscv_pkg; trap_unit then inherits correct values through interrupt_cause_comb, the interrupt_latched_source_enabled case, and the formal properties. +- **`hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:538`** [high] cycle/mcycle read returns cycle_counter[31:0] (and instret/minstret return instret_counter[31:0] at 542, time returns i_mtime[31:0] at 540). At XLEN=64 these zero-extend into the 64-bit read mux: RV64 requires full 64-bit reads of cycle/time/instret/mcycle/minstret. Compiles clean; Linux timekeeping would silently see a 32-bit-wrapping counter. + - *Action:* Return full cycle_counter, i_mtime, instret_counter (64 bits) for the base addresses. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:539`** [medium] CsrCycleH/CsrMcycleH (539), CsrTimeH (541), CsrInstretH/CsrMinstretH (543-544) decode and return high halves. On RV64 addresses 0xC80/0xC81/0xC82/0xB80/0xB82 do not exist and accesses must raise illegal-instruction. csr_file has no illegal-CSR mechanism (default reads 0, line 564); the counter-CSR fault path lives in the ROB (reorder_buffer.sv:471-475, 761-763), whose Zicntr pre-decode currently classifies cycleh/timeh/instreth as mcounteren-gated U-counters and would keep them U-legal when the enable bit is set. + - *Action:* Delete the *H arms from the read mux here AND change the ROB's Zicntr pre-decode so 0xC80-0xC82/0xB80/0xB82 fault unconditionally (all privileges), while 0xC00-0xC02 keep the CY/TM/IR mcounteren gate. Coordinate with the general illegal-CSR policy decision. +- **`hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:173`** [medium] mstatus composed as a 32-bit concat {14'b0, mprv, 4'b0, mpp, 3'b0, mpie, 3'b0, mie, 3'b0} assigned to logic [XLEN-1:0]. At XLEN=64 it zero-extends: UXL[33:32] reads 0 (reserved/illegal - RV64 with U-mode must read UXL=2) and SD stays 0 at bit 63 (acceptable while FS is unimplemented). Compiles clean, silently non-conformant. + - *Action:* Rebuild as 64-bit: SD=0 at bit 63, WPRI padding, UXL=2'd2 hardwired at [33:32], then the existing low-32 layout; UXL read-only (write path at 422-428 already ignores it since mstatus is rebuilt from fields). MIE/MPIE/MPP/MPRV write extraction (csr_new_value[3]/[7]/[12:11]/[17]) unchanged. + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:558`** [low] Custom mperfdata/mperfdatah keep a split 32-bit pair (reads i_perf_counter_data[31:0] / [63:32]); mperfcount (560) assigns a [31:0] input into the 64-bit mux (zero-extend, fine). The split pair is legal on RV64 (custom CSR space) but MperfData could return the full 64 bits. + - *Action:* Decide: keep split pair (no RTL change, profiling tools unchanged) or make MperfData 64-bit and drop/alias MperfDataH. Update header comment lines 55-60 either way. +- **`hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:482`** [medium] mtvec write {csr_new_value[XLEN-1:2], 1'b0, csr_new_value[0]} and mepc write {csr_new_value[XLEN-1:1], 1'b0} (485) are cleanly parametric, but at XLEN=64 they let software install trap/return targets with bits [63:32] set, which the sub-4-GiB physical map cannot fetch. Whether these CSRs hold full 64 bits or WARL-zero the high bits is the project's high-address-bit policy. + - *Action:* Decide unmapped-high-bit policy (full 64-bit WARL pass-through vs hardwired-zero [63:32]); apply consistently to mtvec/mepc and the fetch redirect they feed. + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:467` [low] Reset literals 32'h0000_0000 assigned to XLEN-wide mtvec/mscratch/mepc/mcause/mtval (467-472). Zero-extends correctly at XLEN=64 but is a 32-vs-64 width-mismatch lint hit per tool. +- `hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:159` [low] Fixed-width zero-pad concats produce 32-bit values assigned/compared to XLEN-wide signals: fcsr {24'b0,...} (159), mie {20'b0,...} (183), mip (213), csr_current_value fflags/frm/mcounteren {27'b0}/{29'b0}/{29'b0} (242/243/249), read-mux equivalents (534/535/536/550), and the formal p_mip_reflects_inputs concat (686). All zero-extend to correct values at XLEN=64 but each is a width-mismatch lint hazard; mcounteren correctly stays a 32-bit CSR whose read zero-extends into 64-bit rd. +- `hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv:459` [low] Vectored target: {i_mtvec[XLEN-1:2], 2'b00} + {26'b0, vectored_offset}. The {26'b0, offset} concat is self-determined 32 bits, zero-extended in the 64-bit addition - value correct at XLEN=64 but a fixed-width pad that no longer matches XLEN (lint hazard). +- `hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv:382` [low] interrupt_latched_source_enabled case labels and formal priority/cover properties (550, 555, 560, 601) use the 32-bit IntMachine* constants against XLEN-wide interrupt_cause. Once the constants are widened (bit 63 set) these follow automatically; today both sides zero-extend identically, which is exactly why the line-360 hazard is invisible to formal. +- `hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv:124` [low] All architectural datapaths (i_mstatus/i_mie/i_mtvec/i_mepc, exception cause/tval/pc registers at 287-308, o_trap_target/o_trap_pc/o_trap_cause/o_trap_value) are [XLEN-1:0] parametric; mie bit extraction uses MieM*iBit indices (172-174), XLEN-independent; mepc bit-0 masking lives in csr_file (485) and the formal assume (601) only constrains bit 0 - IALIGN=16 semantics unchanged at 64. Interrupt hold/arming/drain-wait/WFI machinery is width-agnostic. +- `hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv:29` [low] Header/doc block (29-33, 41, 59) describes cycle/cycleh pairs, 'low 32 bits', and RV32 misa 0x4010_112F; all become stale with the RV64 counter and misa changes. +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/perf/perf_counter_aggregator.sv:76` [low] CSR-facing widths are XLEN-independent: perf data buses [63:0] (76, 78), select [7:0] (73, 77), count [31:0] (79); internal counters all [63:0]. No XLEN references; unaffected by the flip. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/perf/tomasulo_perf_counters.sv:100` [low] CSR-facing width is o_perf_counter_data [63:0] (100) with [63:0] internal counters; select [7:0]. No XLEN dependence. + +## OoO top-level glue and register files (tomasulo_wrapper, cpu_ooo, ooo_register_files, generic_regfile) + +This glue layer is in better shape than expected: the FLEN=64 value plumbing (CDB, ROB values, RS dispatch, bypass network, FP regfile) already moves 64-bit data, and ooo_register_files.sv plus generic_regfile.sv are verified fully clean-parametric - flipping riscv_pkg::XLEN widens them correctly with roughly 2x INT-regfile LUTRAM (mwp_dist_ram LVT primitive, 8 read ports) and 2x bypass-mux width as the only costs. The dangerous findings cluster in three places. First, the atomics glue in tomasulo_wrapper hardcodes the W-form op set: LR_W/SC_W/AMO*_W classification (2841-2850, 3046, 1377/1382) and a 4-byte reservation granule compare (1306) would let future LR.D/SC.D/AMO*.D silently bypass the atomics machinery or hold stale reservations. Second, the memory-side glue mixes XLEN-wide data buses with hardcoded [3:0] byte enables (wrapper 443, cpu_ooo 72/79/84/1038) and decodes memory tiers from bits [31:30] alone (2877/3119) - at XLEN=64 the data path widens automatically but the enables and tier decode do not, which is exactly the compile-and-misbehave class; the 32-bit CACHED_/MMIO_ parameters need a deliberate high-address-bit policy. Third, the three {(FLEN-XLEN){1'b0}} concatenations (886/946/1419) become zero-width replications - semantically identity, but a per-tool lint hazard that must be rewritten. One genuine width mismatch exists: cpu_ooo's i_served_addr port is hardcoded [31:0] against if_stage's [XLEN-1:0] declaration. The AMO ALU does not live in these files (it is in load_queue; the wrapper only forwards operands/ports), there is no pending-write FIFO between commit and regfile (direct 2-port XLEN-wide writes plus a delayed-CSR arm in commit_actions), boot-PC injection is in if_stage, and nothing here touches NaN-boxing. Timing exposure concentrates where the audit brief predicted: the CDB value replicas and wakeup fanout, the MEM_RS effective-address adders, and the regfile bypass network all sit on documented sub-nanosecond-margin paths and double in width. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:886`** [medium] cdb_bus_int_rs_qualified.value = {{(FLEN - XLEN){1'b0}}, cdb_bus_int_rs_value}: at XLEN=64 this is a zero-width replication inside a concatenation. Semantically it becomes an identity copy (correct), but zero-width replication is a per-tool lint/compile hazard (Verilator/Yosys/Vivado disagree). Same pattern at line 946 (lane 1). + - *Action:* Guard with generate (if FLEN>XLEN) or replace with FLEN'(cdb_bus_int_rs_value) cast; audit both lanes (886, 946). +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:1419`** [medium] store_misalign_fu_complete.value = {{(FLEN - XLEN){1'b0}}, sq_effective_addr}: third zero-width-replication site (parks misaligned store address for mtval). Becomes identity at XLEN=64 but same lint hazard. + - *Action:* Same fix as lines 886/946: FLEN'() cast or generate guard. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:2877`** [high] lq_addr_is_mmio = (lq_effective_addr[31:30] == 2'b01) (same at 3119 for stores) inspects only bits [31:30] of the now-64-bit effective address. An address with nonzero bits [63:32] but [31:30]==01 silently classifies as MMIO; any high-half garbage aliases into the 4 GiB map. MmioBase localparam (2875) is XLEN-wide from a 32-bit literal (zero-extends, currently unused in logic). + - *Action:* Decide the [63:32] policy (fault, or require-zero and qualify tier decode with |addr[63:32]); today's quadrant decode compiles and misbehaves silently. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:1306`** [high] LR/SC reservation snoop-invalidate compares sq_cache_invalidate_addr[XLEN-1:2] == lq_reservation_addr[XLEN-1:2] - a 4-byte reservation granule. With LR.D/SC.D the granule must cover 8 bytes; keeping word-granule comparison lets an SD or SW to the other word of a reserved doubleword fail to invalidate the reservation, so SC.D succeeds after an intervening store (silent atomicity violation). + - *Action:* Widen reservation granule to at least 8 bytes ([XLEN-1:3]) or make it size-aware; decide granule size (8B vs cache line) as part of LR.D/SC.D support. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:2841`** [high] make_lq_alloc: r.is_lr = (op == LR_W) only; r.is_amo enumerates only AMO*_W ops (lines 2842-2850). New LR_D/AMO*_D ops would allocate as plain loads and bypass the entire atomics machinery (interrupt shield, single-outstanding gate, AMO write path) - silent misbehavior once RV64A decode exists. + - *Action:* Add LR_D and all nine AMO*_D enum values to the classification; r.amo_op = dispatch.op pass-through is fine. The AMO ALU itself lives in load_queue (not this file) and needs .D width plus AMO*.W sign-extension there. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:3046`** [high] make_sq_alloc: r.is_sc = (op == SC_W) only; SC_W-only exclusions also at lines 1377 and 1382 (store_misalign_issue / store_issue_fire). SC.D would take the plain-store completion path instead of the sc_pending_unit hand-off - silent wrong result register value and broken atomicity. + - *Action:* Extend all three SC_W comparisons to also match SC_D. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:443`** [high] o_sq_mem_write_byte_en is hardcoded [3:0] while o_sq_mem_write_data (442) is [XLEN-1:0]. At XLEN=64 the data bus doubles but only 4 byte lanes exist: SD/FSD drain of the upper 4 bytes is silently impossible (today FSD double-pumps two 32-bit beats through the SQ). Mirrored in cpu_ooo.sv:1038 (sq_mem_write_byte_en), cpu_ooo.sv:72/79/84 (o_data_mem_per_byte_wr_en, o_data_mem_bram_byte_wr_en, o_data_mem_cached_byte_wr_en). + - *Action:* Widen byte enables to [XLEN/8-1:0] end-to-end (wrapper, cpu_ooo, data_mem_request_router, BRAM/cached tiers) and rework the SQ 8-byte double-pump into single-beat 64-bit writes; likewise i_lq_mem_read_data (455) becomes single-beat for LD/FLD, obsoleting the LQ beat-assembly. + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:52`** [medium] CACHED_BASE/CACHED_SIZE_BYTES are 'parameter int unsigned' with 32'h8000_0000 / 32'h4000_0000 literals, forwarded to load_queue/store_queue for cached-tier decode against XLEN-wide addresses. At XLEN=64 the compares zero-extend; high address bits make cached-tier decode alias (same policy question as MMIO decode). cpu_ooo.sv has the identical params at lines 33-40 (plus MMIO_ADDR/MMIO_SIZE_BYTES). + - *Action:* Pick param type (longint unsigned or logic [XLEN-1:0]) and the high-bit policy once; apply to wrapper, cpu_ooo, and the router they feed. +- **`hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv:31`** [medium] cpu_ooo declares 'parameter XLEN = riscv_pkg::XLEN' and forwards .XLEN() to if_stage/pd_stage/id_stage/ooo_register_files/commit_actions/trap_unit/csr_file/router, but tomasulo_wrapper (line 1168 instantiation) has no XLEN parameter and all riscv_pkg-typed struct ports bake in riscv_pkg::XLEN. Overriding cpu_ooo's XLEN independently of the package would elaborate mismatched widths - the dual-XLEN parameterization is illusory. + - *Action:* Decide: rv64-only flip of riscv_pkg::XLEN (module params become aliases, simplest) vs true dual-XLEN build (requires parameterizing every riscv_pkg struct - large). Recommend documenting the package localparam as the single source of truth. +- **`hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv:147`** [low] perf_counter_count is [31:0] feeding csr_file i_perf_counter_count; fixed 32-bit event count read through a CSR that becomes 64-bit. Zero-extends on read - functional, but wraps at 2^32 events. + - *Action:* Decide whether custom perf-count CSRs stay 32-bit-wrapping on RV64 or widen; cosmetic either way. + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:742` [medium] cdb_bus_int_rs_value (and cdb_bus_2_int_rs_value at 752, loads at 860/932) are deliberately XLEN-wide dont_touch/max_fanout=64 local CDB value replicas placed to fix the X3 congestion hotspot. At XLEN=64 each replica doubles to 64 live flops with full RS-snoop fanout; the comment documents these nets were among the worst failing endpoints of the routed design. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:2869` [medium] lq_effective_addr = src1_value[XLEN-1:0] + o_mem_rs_issue.imm (sq twin at 3115): rs_dispatch_t.imm is already [XLEN-1:0] so the math widens cleanly, but this is the adder the code comments call the critical RAT->dispatch->adder->SQ path (sq_early_addr_pipeline exists solely to break it at 32 bits). It becomes a 64-bit CARRY8 chain feeding the misalign check and store_misalign CDB packet. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:1252` [low] is_mem_access_misaligned already implements MEM_SIZE_DOUBLE (|addr[2:0]) for FLD/FSD; LD/SD/LR.D/SC.D/AMO*.D reuse it unchanged. No change needed - recorded so it is not re-flagged. +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv:50` [medium] input logic [31:0] i_served_addr is hardcoded 32-bit but if_stage.sv:104 declares the matching port [XLEN-1:0] and compares it against pc_reg_word[XLEN-3:0]; the pass-through connection at line 463 silently zero-extends at XLEN=64, truncating the fetch-window address tag the provider supplies (fetch_provider side also needs widening). +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv:2357` [low] interrupt_resume_pc <= rob_trap_pc + 32'd4 for the WFI mepc seed: the 32-bit literal context-extends correctly in a 64-bit addition, so behavior is right; flagged only as a width-literal cleanliness item ('d4 or XLEN'(4)). +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv:263` [low] The verilator public_flat_rd debug taps (dbg_commit_pc 263, dbg_commit_value 1949, dbg_port0_int_data 322, dbg_trap_cause_internal 317, etc.) are all [XLEN-1:0] and widen automatically to 64. RTL is clean, but every hang_triage/cocotb consumer that reads them as 32-bit Python ints or formats %08x (e.g. the $error strings at 2370/2376) will truncate or misprint. +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/register_files/ooo_register_files.sv:184` [low] Integer regfile: generic_regfile DATA_WIDTH=XLEN, 8 read ports, 2 write ports via mwp_dist_ram (LVT-steered distributed LUTRAM, one RAM instance per read port). At XLEN=64 the INT file's LUTRAM cost doubles (8 ports x 32 additional bits) and the 8 XLEN-wide write-back bypass 2:1 muxes plus final bypass selects (lines 254-266, 301-313) double in width on the ID/dispatch/RAT operand path. +- `hw/rtl/cpu_and_mem/cpu/cpu_ooo/register_files/ooo_register_files.sv:36` [low] Whole module audited line-by-line: every data slice is N*XLEN / N*FpW form, all compares are 5-bit addresses, no literals, no sign-extends, no FLEN-XLEN sites. Verified clean-parametric; FP side untouched (FpW=64 stays). +- `hw/rtl/cpu_and_mem/cpu/wb_stage/generic_regfile.sv:36` [low] Fully parameterized (DATA_WIDTH/NUM_READ_PORTS/NUM_WRITE_PORTS/HARDWIRE_ZERO/DEPTH); primitives sdp_dist_ram / mwp_dist_ram per read port. Only staleness: header comment (line 21) says integer file is '32-bit'. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv:792` [low] cdb_arb_in_7.value[XLEN-1:0] three-arm mux (and its assertions at 3446-3462): at XLEN=64 the slice covers the entire FLEN value, so the 'FP-width upper half retains the generic choice' comment (786) becomes vacuous but behavior stays correct - the XLEN mux simply overrides everything. Identity, not wrong. + +## SoC top, memory tiers, caches, peripherals, sim harness + +This subsystem is largely address-map plumbing, and its RV64 exposure concentrates in two places: the 32-bit data tier and the physical-address policy. The core<->memory data path (cpu_and_mem.sv:234-259) is genuinely 32 bits wide today - FLD/FSD are two word-width phases in the SQ/LQ, verified in store_queue.sv:356/370 and load_queue.sv:2331 - so RV64 LD/SD/AMO*.D/LR.D need a real 64-bit tier: BRAM DATA_WIDTH, byte enables, cached_tier_adapter XLEN, and the MMIO read/write path all widen together, and phasing is not an acceptable fallback because a phased LD of mtime violates single-copy atomicity and AMO*.D cannot be split. The CLINT is the sharpest functional item: mtime/mtimecmp are 64-bit registers reachable only through 32-bit lo/hi word decodes, while RV64 Linux issues single 8-byte LD/SD against them. The nastiest silent hazards found: the FROST_XILINX_PRIMS FDRE loop hardcoded to 32 bits (hardware-only undriven upper half after widening), the sw.mem init-file format silently corrupting a 64-bit-widened BRAM via $readmemh token-width mismatch, cpu_tb's unused XLEN parameter with hardcoded [31:0] wildcard-connected taps, and bit-31 tier selects meeting RV64's sign-extended lui/auipc address constants. The recommended posture, consistent with the unchanged sub-4-GiB map, is to canonicalize 64-bit addresses to 32 bits at one documented point in the core, after which the cache hierarchy (frost_cache, hierarchy, arbiter, AXI bridge, behavioral DDR) needs essentially nothing - it is line-granular, ADDR_WIDTH-parameterized, and XLEN-free, with only the AXI bridge's hardcoded [31:0] addr_q as a latent trap if ADDR_WIDTH were ever raised. UARTs, FIFOs, and the RAM library are parameterized and unaffected, with the one caveat that tdp_bram_dc_byte_en couples its address-port and init-file format to DATA_WIDTH. Timing risk clusters on the widened BRAM write-data cascade (a previously closed WNS path) and the doubled MMIO/LQ read-data mux, not on the caches. frost.sv itself carries no PC/reset-vector logic and needs only parameter pass-through. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:1061`** [high] FROST_XILINX_PRIMS FDRE generate loop is hardcoded 'g_mmio_read_data < 32'. If mmio_read_data_reg widens to 64, bits [63:32] are silently undriven in the Vivado build only - sim (portable path, line 1074) would pass while hardware returns X/0 upper halves. + - *Action:* Bound the loop by $bits(mmio_read_data_reg) instead of literal 32. +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:707`** [high] Data BRAM instantiated with DATA_WIDTH(32) and INIT_FILE("sw.mem") (objcopy --verilog-data-width 4 tokens). If DATA_WIDTH is flipped to 64 for the widened tier, $readmemh loads each 4-byte token into a 64-bit row (upper half zero, depth halved) - a silently corrupt memory image. imem (line 657) must stay 32-bit-word organized for the predecode sideband, so imem and dmem would need different init formats from the same program image. + - *Action:* If dmem goes 64-bit: emit a second init file at --verilog-data-width 8 (or keep dmem as 2 interleaved 32-bit banks), and update tests/build plumbing that shares sw.mem. +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:542`** [high] Fetch tier select is program_counter[31] (registered at 542-546, transition detect at 557). On RV64, LUI/AUIPC/ADDIW-formed addresses are sign-extended, so a jump target built as 'li t0, 0x80000000' arrives as 0xFFFF_FFFF_8000_0000. If the widened PC is truncated to [31:0] at this seam, bit 31 still selects the cached tier correctly - but only under an explicit 'bits [63:32] ignored' policy. If instead high bits are preserved into wider compares elsewhere (fetch_provider tags), the same code would silently miss every tag compare. + - *Action:* Adopt and document a single canonicalization point: truncate/validate the 64-bit PC and data addresses to the 32-bit physical map once (in cpu_ooo or at this boundary), so all downstream [31]-bit tier selects and 32-bit tag compares remain sound. +- **`hw/rtl/cpu_and_mem/fetch_provider.sv:222`** [medium] fetch_high = fetch_addr[31] (also ask_q[31] at 294, i_pc[31] at 381, served_addr_q[31] in the SYNTHESIS-off assertion at 407): bit 31 doubles as the cached-tier discriminator. Same sign-extended-address exposure as cpu_and_mem line 542 - correct only under a canonicalized 32-bit physical address. + - *Action:* Covered by the single canonicalization policy; no local change if addresses are truncated before this module. +- **`hw/sim/cpu_tb.sv:22`** [high] cpu_tb declares 'parameter int unsigned XLEN = 32' but never uses it for widths: o_pc/o_data_mem_addr/o_data_mem_wr_data/i_data_mem_rd_data/o_mmio_load_addr/debug PCs are all literal [31:0] (29-37, 60, 74, 96-98) and byte enables [3:0] (35-36). The wildcard .* connection to cpu_ooo (203) will width-mismatch every widened port - Verilator errors, other tools silently truncate. + - *Action:* Rewrite the bench taps in terms of riscv_pkg::XLEN so flipping the package parameter propagates; also widen the tb data memory (DATA_WIDTH(32) at 175) and o_pc[31:2]+1 served-word math (128) in lockstep. +- **`hw/rtl/lib/cache/line_port_axi_bridge.sv:89`** [low] ADDR_WIDTH parameter exists but addr_q, o_axi_awaddr/araddr, and BASE_ADDR are hardcoded [31:0]/logic[31:0] (35, 53, 67, 89); 'addr_q <= i_req_addr - BASE_ADDR' (134) silently truncates if ADDR_WIDTH is ever raised above 32. + - *Action:* Either keep ADDR_WIDTH pinned at 32 by policy (and assert it in an initial check), or make addr_q/BASE_ADDR/AXI address widths follow ADDR_WIDTH. +- **`hw/rtl/lib/ram/tdp_bram_dc_byte_en.sv:36`** [medium] Byte-address port width is coupled to DATA_WIDTH ('input logic [DATA_WIDTH-1:0] i_port_a_byte_address') and $readmemh row format equals DATA_WIDTH. Widening the dmem instance to 64 changes the address port shape, ByteAddrBits (2->3), and the required init-file token width simultaneously - three coupled changes behind one parameter flip. + - *Action:* Decouple address-port width from DATA_WIDTH (or accept and re-verify all three effects together when widening the dmem instance); regenerate init files at the matching width. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:234`** [high] Entire core<->memory data tier is 32 bits: data_memory_address/write_data/read_data are [31:0], byte enables are [3:0] (lines 234-259). Verified in cpu_ooo: FSD is drained as two word-width phases (store_queue.sv:356/370 'Each phase is word-width') and FLD is a two-phase re-issue (load_queue.sv:2331). RV64 LD/SD/LR.D/SC.D/AMO*.D need a native 8-byte access class; splitting LD of mtime into two MMIO reads violates 64-bit single-copy atomicity, and AMO*.D cannot be phased. + - *Action:* Widen the data tier to 64 bits: data_memory_write_data/read_data/cached_write_data/cached_read_data -> [63:0], byte enables [3:0] -> [7:0] (lines 244, 249, 255), mmio_read_data_* -> 64. Alternatively keep 32+two-phase for BRAM only and add a 64-bit path for MMIO/cached, but a uniform widening is simpler. +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:1249`** [high] mtime/mtimecmp are 64-bit registers written exclusively through paired 32-bit lo/hi MMIO word decodes (writing_mtime_low/high, and the MtimecmpLow/High + ClintMtimecmpLo/Hi cases at 1262-1266) with 32-bit data_memory_write_data_registered. RV64 Linux CLINT/SBI code does a single 8-byte SD to mtimecmp and a single 8-byte LD from mtime; if SD is phased into two word writes there is a torn-mtimecmp window (lo updated against stale hi) that RV64 software, unlike RV32, does not guard against. + - *Action:* Add native 8-byte MMIO store/load decode for the CLINT registers (single-cycle full-width write of mtimecmp, single-copy-atomic read of mtime), while keeping the 32-bit lo/hi aliases for compatibility. +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:1008`** [high] mmio_read_data_comb is a [31:0] mux keyed on mmio_load_addr [31:0]; every arm returns at most 32 bits (mtime[31:0]/mtime[63:32] as separate addresses, lines 1017-1020, 1034-1038). A 64-bit LD from any MMIO address has no data path; result would be a truncated/garbage upper half. + - *Action:* Widen mmio_read_data_comb/reg and data_memory_or_peripheral_read_data (line 1085-1088) to 64 bits; define each register's 64-bit read view (CLINT mtime/mtimecmp full-width; byte-wide ns16550/UART regs zero-extended; decide FIFO semantics). +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:718`** [medium] Instruction-programming Port A writes the data BRAM with 32-bit words and [3:0] byte enables (i_instr_mem_we & {4{...}}). With a 64-bit dmem row, the JTAG/loader path needs address-LSB-steered lane placement into [7:0] byte enables; without it the loader corrupts every odd word. + - *Action:* Add a 32-bit-to-64-bit lane adapter on Port A (addr[2] selects upper/lower byte-enable nibble), or keep the loader 64-bit-aware. +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:746`** [medium] cached_tier_adapter instantiated with .XLEN(32) and 32-bit read/write data ports (lines 751-758). AMO*.D and 8-byte cached LD/SD require the adapter to select/merge an 8-byte lane within the 256-bit line instead of a 4-byte lane. + - *Action:* Flip to .XLEN(64), widen i_write_data/o_read_data and i_write_byte_en at this seam; adapter internals are another audit's scope but this instantiation is the contract. + +### Policy (needs a project decision) + +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:151`** [high] All MMIO window/register addresses are 32-bit 'int unsigned' localparams (151-185) compared with == against data_memory_address_registered (e.g. 340-344, 1095, 1186, 1235-1240, 1262-1268). If the address bus widens to 64, SV zero-extends the localparams, so any address with bits [63:32] set silently misses every decode (write dropped, read returns stale mmio_read_data_reg or BRAM garbage) instead of faulting. + - *Action:* Decide unmapped-high-bit behavior: mask addr[63:32] upstream (aliasing sign-extended addresses onto the 32-bit map - friendliest to RV64 code), or trap/complete-with-zero on nonzero high bits. Then either keep these decoders 32-bit or widen consistently. +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:1201`** [low] o_fifo0_wr_data/o_fifo1_wr_data pass the (to-be-widened) data_memory_write_data_registered into 32-bit FIFO ports; an SD to the FIFO address would silently drop the upper half. + - *Action:* Declare the FIFO MMIO registers 32-bit-access-only (take [31:0] explicitly) or widen the FIFOs; document in the MMIO map. +- **`hw/rtl/lib/cache/frost_cache.sv:142`** [low] Fully ADDR_WIDTH-parameterized and XLEN-free (line-granular). TagBits = ADDR_WIDTH - IndexBits - OffsetBits: raising ADDR_WIDTH to 64 would add 32 bits to every tag entry and lengthen the already timing-sensitive balanced tag compare (243-251) for zero benefit under a 32-bit physical map. + - *Action:* Keep ADDR_WIDTH=32 at every cache instantiation; canonicalize addresses above the hierarchy. +- **`hw/rtl/cpu_and_mem/cpu_and_mem.sv:45`** [low] CACHED_BASE/CACHED_SIZE_BYTES are 32-bit 'int unsigned' parameters threaded into cpu_ooo range checks. Fine at 32-bit physical, but any future >4GiB DDR window (X3 has more DRAM) forces these and every downstream ADDR_WIDTH to grow - worth capturing as an explicit Phase-1 non-goal. + - *Action:* Record 'physical map stays below 4 GiB' as a Phase-1 invariant in ROADMAP/docs so the 32-bit seams are a documented contract, not an accident. + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu_and_mem.sv:192` [low] PC/fetch plumbing hardwired 32-bit: program_counter (192), fetch_address (197), instruction_served_addr (204), served_last_word [29:0] = addr[31:2]+1 (205, 209, 646, 686), fuzz generate ask/served regs (432-435, 486). Mechanical widening iff the fetch seam carries 64-bit addresses; unchanged under a truncate-at-core policy. +- `hw/rtl/cpu_and_mem/cpu_and_mem.sv:346` [low] Debug taps cpu_debug_commit_pc/commit_2_pc [31:0] (346-348) feed hang_triage. Fine under sub-4-GiB map if PC is canonicalized; silent truncation of a widened o_debug_commit_pc otherwise. +- `hw/rtl/cpu_and_mem/fetch_provider.sv:106` [low] LineAddrBits = 32 - OffsetBits hardcodes a 32-bit fetch address space; i_pc/o_served_addr/o_line_req_addr are [31:0] (68, 81, 94), win_addr math uses [31:2] and +32'd4 (192-193, 258), and served_last_word is [29:0] (83). +- `hw/rtl/cpu_and_mem/hang_triage.sv:61` [low] All PC taps [31:0] (i_pc 61, commit PCs 63/65), kernel filter i_pc[31] (110), histogram bucket i_pc[21:16] (104), and the emit FSM prints exactly 8 hex nibbles per field (nib_idx at 229-233). Debug-only; adequate under the sub-4-GiB map, silently truncating if fed a widened PC. +- `hw/sim/cpu_tb.sv:131` [low] One-instruction-per-cycle feed model and the TbSlot2Blocker ecall trick (140-147) are XLEN-agnostic and remain valid for RV64 W-op directed tests; only widths change. i_mtime is already 64-bit (106). +- `hw/rtl/frost.sv:87` [low] Top-level ports are already width-appropriate for the unchanged sub-4-GiB physical map: instr-mem programming [31:0] (87-88), AXI addresses [31:0] (103, 117), CACHED_BASE 32'h8000_0000 (47). No PC/reset-vector logic lives in frost.sv (reset vector is in the core). Only pass-through of cpu_and_mem changes. +- `hw/rtl/lib/cache/axi_behavioral_memory.sv:87` [low] Word-granular 32-bit array with sw_ddr.mem in 4-byte format is untouched by XLEN: the port is 256-bit line-granular and addresses are region-relative 32-bit. RV64 programs load fine since the image is byte-exact. DDR_MODEL_LATENCY/JITTER hooks are width-irrelevant. +- `hw/rtl/lib/cache/frost_cache_hierarchy.sv:45` [low] Clean ADDR_WIDTH/LINE_BYTES parameterization end-to-end (ports 69-104, both L1s, arbiter, L2); no XLEN dependence, no width literals beyond the parameter defaults. +- `hw/rtl/lib/cache/line_port_arbiter.sv:39` [low] Fully parameterized pass-through arbiter; no XLEN or width assumptions. +- `hw/rtl/peripherals/uart_tx.sv:26` [low] Byte-wide, DATA_WIDTH-parameterized, no XLEN coupling (same for uart_rx.sv). + +## test infrastructure, CI, container, formal targets (config/harness code where XLEN is pinned outside RTL) + +The harness pins XLEN=32 in exactly four places that matter: the arch-test suite root (test_arch_compliance.py:48 rv32i_m), the riscv-tests suite/skip tables (test_riscv_tests.py:54,85), the torture signature layout (_GPR_WORDS/_TOTAL_WORDS at test_riscv_torture.py:217-218), and the CI matrices plus Linux lane (ci.yml:209,423-476,617-620); everything else (tests/Makefile, cocotb runner mechanics, frost.py, conftest, docker_entrypoint, check_linux_boot_regression, the .sby scripts) is genuinely XLEN-agnostic because ISA strings live in sw/apps Makefiles and widths elaborate from riscv_pkg.sv. The most valuable hazards: (1) reference-path collision - an rv64i_m test resolves to the committed RV32 golden signature because get_reference_path keys only on extension name + stem (test_arch_compliance.py:196); (2) skip-list silent expiry - ISA_SKIP_TESTS keyed by rv32 suite names stops applying on re-key, while the rv32ud 'move' skip must be inverted since fmv.d.x/fmv.x.d become legal on RV64; (3) the K-extension filter would silently drop packw coverage on rv64. Both signature extractors hardwire 8-hex-char lines, coupling the harness to a 4-byte dump granularity defined in sw-side macros. Hard-verified container facts: the pinned xPack gcc 15.2.0-1 in the frost image ships rv64imafdc_zicsr_zaamo_zalrsc/lp64d multilibs (rv64 builds work today), qemu-system-riscv64 is already installed, but Spike is absent from the image - all goldens came from an unpinned host Spike, which is the reproducibility gate on regenerating every arch/torture reference at rv64. To run ONE rv64 arch test today (./test_arch_compliance.py --test rv64i_m/I/src/add-01.S): FIRST the build breaks - sw/apps/arch_test/Makefile:25 pins ARCH=rv32imafdc.../ABI=ilp32 so the assembler rejects rv64 sources (path resolution via SUITE_DIR.parent would actually find the file); SECOND, if the build were fixed, the runner silently loads the RV32 golden for the same-named test and diffs against it; THIRD, the RTL is still XLEN=32 so the sim traps or mismatches, and only then do the extractor-granularity and reference-regeneration questions surface. Formal and Yosys lanes need no textual changes but their depth/timeout budgets (SBY_TASK_TIMEOUT_S=2400, xilinx 3600 s vs ~2750 s measured) are sized for the 32-bit design and must be re-measured. CI today runs ~71 simulation jobs across the suite matrices; the dual-vs-replace XLEN decision determines whether that roughly doubles. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`tests/test_arch_compliance.py:196`** [high] get_reference_path keys references only by extension dir name + test stem (ext_name = test_src.parent.parent.name, e.g. 'I', stem 'add-01'). rv64i_m/I/src/add-01.S has the SAME stem and extension name as the rv32 test, so pointing --test at an rv64 source today silently compares against the committed RV32 golden signature and reports a plausible-looking FAIL (or, for width-insensitive tests, a false PASS). No XLEN appears anywhere in the reference namespace (references/{ext}/{stem}.reference_output). + - *Action:* Namespace references per XLEN (references/rv64/{ext}/... or a suite-root-derived key that includes rv64i_m) and regenerate all goldens with an rv64 Spike; make run_single_test refuse a reference whose XLEN provenance does not match the build. +- **`tests/test_arch_compliance.py:81`** [medium] K-extension filter selects {pack, packh, brev8, zip, unzip}. ZIP/UNZIP are RV32-only (must decode illegal on RV64) and ZEXT.H changes encoding to PACKW on RV64; the rv64i_m/K suite instead carries packw tests that this prefix filter would exclude, so the rv64 K lane would silently run a reduced test set while looking green. + - *Action:* Re-derive the K filter for rv64 (add packw, drop zip/unzip) when SUITE_DIR flips; likewise re-audit EXTENSION_TEST_EXCLUDES (lines 89-96) against the rv64 C/B directories (different Zcb/clmul mixes). +- **`tests/test_riscv_tests.py:85`** [high] ISA_SKIP_TESTS / ISA_SKIP_TESTS_BRAM are keyed by rv32 suite names, so on an rv64 re-key every skip silently stops applying (dict lookup misses) and previously-excluded tests (ma_data, breakpoint, pmpaddr, csr, ma_addr, instret_overflow, fence_i-in-bram) run and fail. Conversely rv32ud's 'move' skip (line 90, 'Uses fmv.d.x/fmv.x.d (RV64-only)') must be REMOVED for rv64ud - the test becomes runnable and is exactly the new FMV.X.D/FMV.D.X coverage Phase 1 needs. + - *Action:* Re-key all skip dicts to the rv64 suite names in the same change that re-keys ISA_TEST_SUITES; delete the ud/move skip; re-validate each remaining skip's rationale at XLEN=64. + +### Design work (new logic or semantics) + +- **`tests/test_arch_compliance.py:312`** [medium] extract_signature accepts ONLY exactly-8-hex-char lines (len(stripped) == 8). This bakes the 4-byte signature granularity of the current custom RVMODEL_HALT UART dump into the harness. If the rv64 port's halt macro dumps 16-hex 64-bit words, every signature line is silently ignored and the test fails with the misleading 'No signature data in output'; if the macro keeps 4-byte words it works but only by convention that lives in another repo area (sw/apps/arch_test env macros). + - *Action:* Decide the rv64 signature granularity once (recommend keeping 4-byte/8-hex, matching riscv-arch-test's 4-byte RISCOF granularity) and assert it consistently in the halt macro, this extractor, test_riscv_torture.extract_signature, and the goldens. +- **`tests/test_riscv_torture.py:191`** [low] Torture extract_signature also hardwires 8-hex-char signature lines, same coupling as the arch-test extractor; must move in lockstep with whatever granularity the rv64 footer dump uses. + - *Action:* Apply the same one-granularity decision as test_arch_compliance.py:312. +- **`tests/test_run_cocotb.py:349`** [medium] Registry is XLEN-agnostic mechanically (it only names apps and forwards env), but the atomics entry's description 'RV32-A atomics to the cached DDR region (LR/SC, AMO)' (349) and the coverage comment citing rv32ua/rv32uc/rv32um gating (839) go stale, and no registry entries exist for RV64-only surface (LWU/LD/SD, W-ops, AMO*.D/LR.D/SC.D, FCVT.L/LU, 8-byte-alignment traps). New directed apps plus TEST_REGISTRY entries are needed or the fast cocotb lane has zero rv64-delta coverage. + - *Action:* Add rv64 directed-test apps and registry entries (with descriptions updated); fix the two stale doc strings. + +### Policy (needs a project decision) + +- **`.github/workflows/ci.yml:476`** [high] The Linux lane is rv32-pinned end to end: frost_nommu_rv32_defconfig (476, cache key 454, comment 423 'compiles a full rv32 uClibc cross toolchain'), and the QEMU reference boot uses qemu-system-riscv32 with -cpu rv32,mmu=off (617-620). An rv64 core cannot boot this image; the lane goes red or must be ported (rv64 nommu is actually the better-supported upstream Linux config). + - *Action:* Author frost_nommu_rv64_defconfig + rv64 buildroot toolchain config, switch QEMU job to qemu-system-riscv64 -cpu rv64,mmu=off (binary already in the image), update cache keys; or explicitly park the Linux lane during the transition. check_linux_boot_regression.py itself is XLEN-clean (mtimecmp already parsed as 64-bit, line 54). +- **`Dockerfile:150`** [high] Spike (riscv-isa-sim) is NOT built into the image anywhere (verified: no spike binary in the container). All golden references (arch-test via sw/apps/arch_test/generate_references.py using FROST_SPIKE/--isa=rv32imafdc_zicsr_zifencei_zba_zbb_zbs_zbkb_zicond, torture via generate_tests.py spike --isa=FROST_ISA) were generated on a host Spike and committed. The rv64 respin requires regenerating every reference with an rv64 ISA string, on a Spike whose version is currently unpinned and outside CI's reproducibility guarantee. + - *Action:* Decide: add a pinned riscv-isa-sim build stage to the Dockerfile (so reference regeneration is containerized and CI-reproducible) vs. keep host-Spike with a documented version pin. This gates all rv64 signature suites. +- **`tests/test_run_yosys.py:105`** [medium] Xilinx synthesis timeout defaults to 3600 s while the current UltraScale+ run already measures ~2750 s (comment at line 96). Widening the integer datapath, load/store paths, and CSRs to 64 bits plausibly pushes past 3600 s and turns the synthesis lane into timeout-flaky. + - *Action:* Budget new timeouts (FROST_YOSYS_XILINX_TIMEOUT_SEC default) after the first rv64 synthesis measurement; same review for the generic coarse 1800 s default. +- **`tests/test_run_formal.py:40`** [medium] SBY_TASK_TIMEOUT_S = 2400 is sized against RV32 solver runtimes, and the history in this very comment (ROB BMC depth cut 16 to 12 after a timing-closure change ballooned runtime) shows depth/timeout budgets are already tight. Widening PCs/addresses/CSRs 32 to 64 grows SMT bitvector state in csr_file, trap_unit, load_queue, store_queue, and lq_l0_cache targets (value carriers are already 64-bit, so ROB/RS/CDB grow less); expect re-tuning of depths (.sby files) and this timeout. The .sby scripts themselves contain no width literals - they elaborate riscv_pkg.sv, so the XLEN flip flows through automatically. + - *Action:* After the RTL flip, re-measure every bmc/cover task, re-tune depths in formal/*.sby and possibly SBY_TASK_TIMEOUT_S; no textual .sby edits required for correctness. +- **`.github/workflows/ci.yml:151`** [medium] CI cost/shape: sim lanes today are roughly 25 arch-test jobs (14 ext x 2 tiers - 3 excludes), 22 riscv-tests, 18 benchmarks, 2 torture, 4 cocotb, plus synthesis/formal/lint/python/3 Linux jobs (~80 total). A dual rv32+rv64 matrix roughly doubles the ~71 simulation jobs; the arch lane already needs timeout-minutes: 300 and excludes F/D ddr for slowness (167-173), and rv64 suites add W-op/conversion tests on a slower 64-bit Verilator model, so even the rv64-only swap inflates wall time. + - *Action:* Decide dual-XLEN CI vs rv64-only replacement before writing the matrix; if dual, shard further or demote one XLEN to nightly. + +### Mechanical (width/literal/comment hygiene) + +- `tests/test_arch_compliance.py:48` [medium] SUITE_DIR hardcodes riscv-test-suite/rv32i_m; discover_tests, extension validation (line 599), and the --test resolution examples (lines 25, 522, 542) all assume the rv32i_m tree. The rv64i_m tree in the same submodule is never enumerated. +- `tests/test_riscv_tests.py:54` [medium] ISA_TEST_SUITES is a hardcoded rv32* map (rv32ui...rv32uzbkb) and drives --all, pytest parametrization (line 421), and CI. The riscv-tests submodule's rv64ui/rv64um/... dirs exist but are unreachable via --all; --suites rv64ui only 'works' with a warning (line 603) and then breaks in the sw-side rv32 build. +- `tests/test_riscv_torture.py:218` [medium] _GPR_WORDS = 32 and _TOTAL_WORDS = 96 hardcode the RV32 signature layout (32 one-word GPRs + 32 doubles x 2 words), and compare_signatures slices the FP region as words [32:96]. With 64-bit GPRs the dump becomes 64 GPR words + 64 FP words = 128 (at 4-byte granularity); the length check at line 228 fails loudly, but the constants, the FP-slice arithmetic, and the GPR-skip rationale (lines 210-216) all need rework together with the regenerated torture tests/references. +- `tests/test_runner_helpers.py:93` [low] The fast regression fixtures encode the 8-hex format ('00000001'/'00000002' lines at 84-94) and reference stubs '00000000\n' (lines 54, 71), plus the rv32ui suite name at line 101. They will keep passing against stale rv32 assumptions and must be updated with the extractors so the guard tests actually guard the rv64 format. +- `.github/workflows/ci.yml:209` [medium] riscv-tests matrix pins suite: [rv32ui, rv32um, rv32ua, rv32uf, rv32ud, rv32uc, rv32mi, rv32uzba, rv32uzbb, rv32uzbs, rv32uzbkb]; benchmarks (240) and arch extensions (156) reuse names that survive, but every rv32* literal here must flip to rv64*. +- `Dockerfile:151` [low] The pinned xPack riscv-none-elf-gcc 15.2.0-1 is VERIFIED rv64-capable: --print-multi-lib in the current frost image lists rv64imafdc_zicsr_zaamo_zalrsc/lp64d (plus lp64/lp64f variants), so bare-metal rv64gc builds with newlib link out of the box; no toolchain change needed. B-extension march strings are accepted by the assembler and map onto the rv64imafdc multilib. +- `Dockerfile:161` [low] The Buildroot/QEMU layer comment block describes the Linux jobs as rv32 ('Buildroot compiles its own rv32 uClibc cross toolchain', 'qemu-system-riscv32'); qemu-system-misc already ships qemu-system-riscv64, so only the comments and the defconfig consumers go stale, not the package set. +- `tests/Makefile:377` [low] No XLEN pins found: file lists, -G parameter overrides (MEM_SIZE_BYTES/DDR_MODEL_BYTES etc.) and Verilator flags are width-agnostic; DDR_MODEL_BYTES=64 MiB and the sub-4-GiB map are unchanged by Phase 1. Only indirect effect is longer Verilator compile/sim times. + +## cocotb verification framework (verif/): encoders, reference models, monitors, testbench config, and cocotb_tests survey + +The verification framework is systematically RV32-shaped: verif/config.py declares XLEN=32 but nothing derives from it, and the real width truth lives in scattered literals (MASK32, 0xFFFFFFFC alignment masks, SHIFT_AMOUNT_MASK=0x1F, INT32 division constants) plus at least 16 private "XLEN = 32" copies inside tomasulo/ and cpu_ooo/ test files that hand-pack DUT struct bit layouts. The reference ALU (alu_model.py) is the densest hazard zone: a module-wide mask-to-32 decorator, 5-bit shamt truncation, 32-hardcoded rotates, 5-bit Zbs bit indexing, 4-byte-only orc_b/rev8/brev8, and to_signed32-based compares would all compile and silently produce wrong expected values at XLEN=64; LW's RV64 sign-extension semantic change and the missing W-op/LWU/LD/SD evaluators are the biggest model gaps. The FP model already handles NaN-boxing correctly (box32/unbox32, FLEN=64 carriers), but every FCVT.W*/FMV.X.W result returns a zero-extended 32-bit value where RV64 requires sign-extension into the 64-bit rd, and the entire FCVT.L*/FMV.X.D family is absent. The encoders need new OP-32/OP-IMM-32/LWU/LD/SD/AMO.D machinery, and make_i_shift_encoder's (sh & 0x1F)|(f7 << 5) layout silently corrupts funct7 for any 6-bit shamt. The compressed encoder has three RV64 booby traps: c.jal's encoding becomes C.ADDIW, and the C.FLW/C.FSW/C.FLWSP/C.FSWSP encodings become C.LD/C.SD/C.LDSP/C.SDSP with 8-scaled immediates — both silently execute different instructions than the model predicts; test_compressed.py drives them directly. CSR-side, op_tables.ZICNTR_CSRS and test_state.get_csr_value treat cycleh/timeh/instreth as readable, which become illegal CSRs on RV64, and mcause interrupt-bit checks pin bit 31 (loud in test_trap_unit.py, silent in test_real_program's is_irq instrumentation). Address handling truncates at 32 bits in several silent places (config alignment masks, instruction_generator MMIO check, generate_aligned_immediate), which interacts with the undecided high-address-bit policy. Monitors are mostly mechanical (RegisterFileMonitor's MASK32 compare fails loudly), while memory_model's write checker masks byte-enables to 4 bits and data to 32 — its fate depends on the data-bus-width decision. Migration cost estimate: a focused mechanical pass (config-driven masks + riscv_utils parametrization + monitor widening) is small; the design work concentrates in alu_model/op_tables/encoders (W-ops, shamt6, RVC recode) and the struct-packer synchronization across the tomasulo unit tests. The five highest-leverage files to make XLEN-parametric first: (1) verif/config.py (make every width/mask/division constant derive from XLEN), (2) verif/utils/riscv_utils.py (parametric to_signed/to_unsigned used by every model), (3) verif/models/alu_model.py (mirrors the RTL ALU; decorators are single choke points), (4) verif/encoders/op_tables.py with instruction_encode.py (shamt6 layout, W-op and RV64 load/store encoders, CSR legality), and (5) a new shared width module replacing the private XLEN=32 copies in cocotb_tests/tomasulo and cpu_ooo interface files (start from reservation_station/rs_interface.py and dispatch/dispatch_interface.py). + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`verif/config.py:70`** [high] MEMORY_WORD_ALIGN_MASK = 0xFFFFFFFC (and 0xFFFFFFFE at line 73) are 32-bit-wide alignment masks; ANDing a 64-bit address with them silently clears bits [63:32]. Used by memory_model.read_word/write_word and test_state.set_reservation. + - *Action:* Replace with ~0x3 / ~0x1 (or width-parametric masks) so upper address bits survive. +- **`verif/utils/riscv_utils.py:53`** [high] to_signed32/to_unsigned32/to_signed33 hardwire sign bit 31 and MASK32; they are the conversion primitives behind every signed compare, shift, min/max, branch, and division in the models. If riscv_pkg XLEN flips without these, all signed semantics on values with bit 63 set silently break. + - *Action:* Add parametric to_signed(val, width)/to_unsigned(val, width) (or to_signed64 family) and migrate every model call site. +- **`verif/models/alu_model.py:75`** [high] limit_shift_amount truncates shamt with & 0x1F; at XLEN=64 a shamt of 32-63 is silently reduced mod 32 and sll/srl/sra/rol/ror all return plausible-but-wrong values. + - *Action:* Mask to 6 bits for native ops; keep 5-bit masking only in new W-shift evaluators. +- **`verif/models/alu_model.py:170`** [high] lw() zero-extends the loaded 32-bit word; on RV64 LW sign-extends to 64 (semantic change to an existing op). lb/lh at lines 209/235 also '& MASK32' after sign_extend, killing 64-bit sign extension. LWU does not exist. + - *Action:* Sign-extend lw/lb/lh/lbu results to XLEN; add lwu; verify ld (line 183) stays as-is. +- **`verif/models/alu_model.py:398`** [high] bset/bclr/binv/bext mask the bit index with SHIFT_AMOUNT_MASK (5 bits); on RV64 bit positions 32-63 silently alias to 0-31. + - *Action:* Mask bit index to 6 bits at XLEN=64. +- **`verif/models/alu_model.py:503`** [high] rol/ror hardcode the complementary shift as (32 - shift_amount); at XLEN=64 rotation is silently wrong for every nonzero shamt. ROLW/RORW/RORIW do not exist. + - *Action:* Use (XLEN - shamt); add W-rotate evaluators. +- **`verif/models/alu_model.py:587`** [high] orc_b (range(4)), rev8 (explicit byte0-byte3, lines 594-604), and brev8 (range(4), line 652) iterate over exactly 4 bytes; at XLEN=64 the upper 32 bits are silently discarded. REV8's immediate encoding also differs on RV64. + - *Action:* Iterate XLEN/8 bytes; update rev8 encoder immediate (0b011010111000) in op_tables. +- **`verif/models/branch_model.py:45`** [high] blt/bge/bltu/bgeu compare via to_signed32/MASK32; with 64-bit register values the comparisons silently use only a 32-bit view. + - *Action:* Switch to XLEN-parametric signed/unsigned conversions. +- **`verif/models/fp_model.py:840`** [high] fcvt_w_s/fcvt_wu_s/fcvt_w_d/fcvt_wu_d return raw 32-bit values (result & MASK32, saturations like 0x7FFFFFFF/0xFFFFFFFF); on RV64 FCVT.W*/WU* results sign-extend into the 64-bit integer rd (e.g. WU max becomes 0xFFFFFFFF_FFFFFFFF). fmv_x_w (line 886) also returns & MASK32 but sign-extends on RV64. + - *Action:* Sign-extend all 32-bit int-conversion and FMV.X.W results to 64; same rule for future AMOW model reuse. +- **`verif/encoders/op_tables.py:323`** [high] make_i_shift_encoder packs (sh & 0x1F) | (f7 << 5); a 6-bit shamt's bit 5 lands where funct7's LSB sits, so on RV64 a shamt >= 32 silently encodes a different instruction (e.g. SRLI shamt=33 -> funct7 corrupted). Also blocks encoding legal RV64 shamts 32-63. + - *Action:* Rework to shamt6 layout ((sh & 0x3F) | (f6 << 6)) for native shifts/rori/bseti/bclri/binvi/bexti; separate 5-bit W-shift encoders where bit 25 set is illegal. +- **`verif/encoders/op_tables.py:644`** [high] C_JUMPS registers "c.jal" (enc_c_jal); on RV64 that 16-bit encoding IS C.ADDIW, so any test emitting it executes a different instruction while the model predicts a jump — silent divergence. + - *Action:* Remove c.jal from RV64 tables; add c.addiw/c.addw/c.subw entries. +- **`verif/encoders/compressed_encode.py:159`** [high] enc_c_flw/enc_c_fsw (159-212) and enc_c_flwsp/enc_c_fswsp (768-816) emit funct3=011/111 quadrant-0/2 encodings that on RV64 are C.LD/C.SD/C.LDSP/C.SDSP with immediates scaled by 8, not 4 — the same bytes mean different instructions with different immediate decode. test_compressed.py (lines 585-654) drives these directly. + - *Action:* Repurpose/rename as C.LD/C.SD encoders with 8-scaled immediates on RV64; rewrite the compressed FP-word directed tests (C.FLD/C.FSD forms are unchanged and can substitute). +- **`verif/models/memory_model.py:187`** [medium] driver_and_monitor masks the byte-enable with & 0xF and write data with & MASK32; if the RV64 data interface widens to 64 bits with 8 byte-enables, upper-lane stores are silently never checked and data compares truncate. + - *Action:* Widen mask/data handling to the chosen bus width (policy-dependent); also revisit the 4-byte-word DUT-RAM copy loop at lines 87-95 if testbench RAM organization changes. +- **`verif/utils/memory_utils.py:275`** [medium] generate_aligned_immediate computes effective_address as (base + imm) & 0xFFFFFFFF; with 64-bit random register bases the model wraps at 4 GiB while hardware computes a full 64-bit address — divergent addresses given the unmapped-high-bit policy is undecided. + - *Action:* Mask to XLEN and constrain random bases below 4 GiB per the physical-map policy decision. +- **`verif/cocotb_tests/instruction_generator.py:195`** [high] _is_mmio_address masks to 32 bits before comparing to MMIO_BASE_ADDR: a 64-bit base like 0x1_0000_0000 aliases to 0x0 and is judged safe RAM, and _effective_address (line 201) wraps at 32 bits — random memory ops could target addresses the DUT decodes differently. + - *Action:* Decide high-bit policy, then make address math XLEN-wide and constrain generated bases; also widen shamt generation (line 396, randint(0, SHIFT_AMOUNT_MASK)) to 6 bits for native shifts. +- **`verif/cocotb_tests/test_real_program.py:2435`** [low] Debug instrumentation classifies interrupts with (trap_cause & 0x8000_0000); on RV64 the interrupt bit is bit 63, so is_irq silently reads False for real interrupts, corrupting wedge/trap diagnostics (not pass/fail). + - *Action:* Test bit XLEN-1; audit the file's other 0x8000_0000-style probes and :08x formatting. + +### Design work (new logic or semantics) + +- **`verif/config.py:143`** [high] SHIFT_AMOUNT_BITS=5 / SHIFT_AMOUNT_MASK=0x1F feed alu_model.limit_shift_amount and instruction_generator shamt generation; on RV64 native shifts take 6-bit shamt while W-shifts keep 5. + - *Action:* Introduce SHIFT_AMOUNT_MASK_XLEN=0x3F for native ops and keep 0x1F for W-form ops; both must exist simultaneously. +- **`verif/config.py:283`** [medium] DIVISION_OVERFLOW_DIVIDEND=-0x80000000 and DIVISION_BY_ZERO_QUOTIENT=0xFFFFFFFF are INT32-shaped; RV64 DIV overflow is INT64_MIN/-1 and div-by-zero returns 2^64-1, while DIVW/REMW still need the 32-bit constants plus sign-extension. + - *Action:* Add 64-bit variants and keep the 32-bit ones for W-form division evaluators. +- **`verif/models/alu_model.py:252`** [high] mul/mulh/mulhsu/mulhu implement 32x32 semantics (>> 32, to_signed33); RV64 needs 64x64 MUL/MULH* (128-bit product, >> 64) plus a new MULW. + - *Action:* Widen to XLEN-parametric multiply-high with 2*XLEN products; add mulw evaluator with sign-extension. +- **`verif/models/alu_model.py:289`** [high] DivisionOperations uses to_signed32, hardcoded 0x80000000 overflow return (line 304), and 32-bit constants; DIVW/DIVUW/REMW/REMUW are absent. + - *Action:* Parameterize to XLEN with INT64_MIN/-1 special case; add the four W-division evaluators (32-bit compute, sign-extend result; div-by-zero returns all-ones/dividend at 64 bits). +- **`verif/models/alu_model.py:516`** [medium] clz/ctz/cpop scan exactly 32 bits and return 32 for zero input; CLZW/CTZW/CPOPW absent. + - *Action:* Widen to 64-bit scans returning 64; add W variants. +- **`verif/models/alu_model.py:625`** [medium] pack() packs 16-bit halves (RV32 PACK). On RV64, PACK packs 32-bit halves, ZEXT.H changes encoding to PACKW rd,rs1,x0, and PACKW is new; zip/unzip (lines 663-696) are RV32-only and must decode as illegal. + - *Action:* Rework pack for 64-bit, add packw, re-encode zext.h, delete/illegal-ify zip and unzip evaluators and their op_tables/I_UNARY entries (lines 519-520). +- **`verif/models/alu_model.py:704`** [medium] AMO evaluators are .W-only and cpu_model returns lw() (zero-extended) as AMO rd; on RV64 AMO*.W rd sign-extends to 64. LR.D/SC.D/AMO*.D evaluators absent; reservation granule in test_state (word-aligned, line 266) may become 8 bytes. + - *Action:* Sign-extend .W results, add .D evaluators/encoders, decide reservation granule. +- **`verif/models/fp_model.py:862`** [high] fcvt_s_w/fcvt_d_w test bit 0x80000000 of rs1_int; fed a 64-bit register value they misinterpret the operand. FCVT.L.S/LU.S/S.L/S.LU, FCVT.L.D/LU.D/D.L/D.LU, FMV.X.D, FMV.D.X models are absent entirely (D-path FMV currently only via fld/fsd). + - *Action:* Mask W-conversions to low 32 bits explicitly; add the eight L-form conversion models (single-precision results must NaN-box via box32) and FMV.X.D/FMV.D.X. +- **`verif/encoders/instruction_encode.py:60`** [high] Opcode enum lacks OP-IMM-32 (0x1B) and OP-32 (0x3B); no encoders exist for LWU/LD/SD, ADDIW/SLLIW/SRLIW/SRAIW, ADDW/SUBW/SLLW/SRLW/SRAW, MULW/DIV*W/REM*W, or the Zba/Zbb/Zbkb W-forms. + - *Action:* Add the two opcodes, W-op encoder factories, ld/sd/lwu load-store encoders (funct3 011/110), and register them in op_tables. +- **`verif/encoders/instruction_encode.py:420`** [medium] AMOType.encode hardcodes funct3=Funct3.WORD, so AMO*.D/LR.D/SC.D cannot be encoded. + - *Action:* Add a funct3 parameter (default WORD) and .D wrapper functions. +- **`verif/encoders/instruction_encode.py:628`** [medium] CSRAddress defines CYCLEH/TIMEH/INSTRETH (0xC80-0xC82); on RV64 these CSRs must raise illegal-instruction, and cycle/time/instret become single 64-bit CSRs. + - *Action:* Keep addresses for illegal-CSR trap tests but remove them from legal-read paths; update test_state.get_csr_value. +- **`verif/encoders/op_tables.py:495`** [high] ZICNTR_CSRS feeds random CSR-read generation with CYCLEH/TIMEH/INSTRETH, which are illegal CSRs on RV64 — random tests would inject illegal instructions expecting normal retirement. + - *Action:* Drop the H-counters from the random pool; add directed illegal-CSR trap coverage instead. +- **`verif/encoders/compressed_encode.py:382`** [low] enc_c_srli/enc_c_srai/enc_c_slli assert 1 <= shamt <= 31 and hardcode instruction bit 12 (shamt[5]) to 0; RV64 legalizes 6-bit compressed shamts. + - *Action:* Accept shamt 1-63 and emit shamt[5] into bit 12 (loud assert today, so not silent). +- **`verif/utils/memory_utils.py:122`** [medium] calculate_byte_mask_for_store knows only sb/sh/sw with 4-bit masks, and get_alignment_for_operation (line 182) raises on ld/sd/lwu — no 8-byte alignment class exists. + - *Action:* Add sd (and lwu/ld alignment=4/8) plus 8-lane mask support per the bus-width decision. +- **`verif/cocotb_tests/test_state.py:296`** [medium] get_csr_value models CYCLEH/TIMEH/INSTRETH as readable high halves and masks CYCLE/INSTRET to 32 bits; on RV64 counters are single 64-bit CSRs and the H-forms trap. mcounteren modeling (returns 0x7) stays 32-bit per spec. + - *Action:* Return full 64-bit counter values; move H-CSRs to illegal-instruction expectations. +- **`verif/cocotb_tests/test_directed_multicycle.py:457`** [low] FLD result modeled as low_word/high_word 32-bit pair mirroring the current 32-bit data bus; changes with the RV64 bus-width decision. + - *Action:* Update split-word modeling (and test_directed_atomics init patterns at line 314) once bus width is decided. + +### Policy (needs a project decision) + +- **`verif/cocotb_tests/tomasulo/reservation_station/rs_interface.py:33`** [high] Local 'XLEN = 32' constant plus hand-maintained packed-struct bit-offset packers (pack_rs_dispatch lines 105-174, unpack_rs_issue 208-270) mirror riscv_pkg struct layouts field-by-field. The same private XLEN=32/FLEN=64 copies exist in at least 15 more files (dispatch_interface.py:32, sq_interface.py:30, lq_interface.py:30, lq_model.py:25, sq_model.py:25, reorder_buffer_model.py:36, rat_model.py:38, fu_shims/fp_add_shim_interface.py:31, and cpu_ooo/* tests). If riscv_pkg XLEN flips and any one copy does not, packers drive misaligned bit vectors — some fields land in don't-care positions and pass silently wrong. + - *Action:* Centralize width constants in one module (or read DUT parameters at runtime) before flipping; then verify each packer against the actual RTL struct (e.g. whether imm stays 32-bit or becomes XLEN in rs_dispatch_t must match line 145). + +### Mechanical (width/literal/comment hygiene) + +- `verif/config.py:254` [high] XLEN: Final[int] = 32 is declared but nothing else in config derives from it; MASK32, SHIFT_AMOUNT_MASK, alignment masks, and division constants are all independent literals. +- `verif/models/alu_model.py:64` [high] mask_to_32_bits decorator wraps every arithmetic op result with & MASK32 — the reference-ALU-wide 32-bit truncation point. +- `verif/monitors/monitors.py:160` [medium] RegisterFileMonitor.compare masks expected values with MASK32 (and formats :08x); once the DUT regfile is 64-bit every sign-extended result mismatches. Loud failure, but it breaks every regfile-checked test on day one. +- `verif/cocotb_tests/test_helpers.py:367` [medium] initialize_registers seeds x1-x31 with randint(0, 2**32 - 1); on RV64 the upper 32 bits of every register start zero and random torture never exercises them (silent coverage hole, not a failure). +- `verif/cocotb_tests/cpu_model.py:420` [medium] JALR target computed as (rs1 + imm) & 0xFFFFFFFE & MASK32 — the clear-bit-0 mask doubles as a 32-bit truncation; all PC/link/store-address arithmetic in this file (lines 276, 379, 415-426, 516-560) is & MASK32. +- `verif/cocotb_tests/control/test_trap_unit.py:107` [low] Asserts o_trap_cause == 0x80000007 (also lines 135, 182): mcause interrupt bit is bit 31; on RV64 it moves to bit 63 (0x8000000000000007). Loud failure, purely mechanical. +- `verif/verification_types.py:27` [low] NewType docstrings pin Address/RegisterValue/ProgramCounter as 32-bit; documentation-only but part of the same change per repo doc policy. + +## software tree, Linux image pipeline, FPGA build (rv64 build-axis audit) + +The software/build axis is in decent shape for RV64: the C-app march/mabi flags live in exactly one place (sw/common/common.mk:91,44), the linker scripts carry no elf32 pinning and an unchanged sub-4GiB memory map, crt0's lw/sw loops are legal-if-slow on rv64, and the entire image/loader pipeline (objcopy verilog-data-width 4, xxd -g4, JTAG 32-bit AXI bursts, .mem/.txt word formats) is byte-stream-based and XLEN-independent. The hard problems are concentrated in four places. First, sw/lib/include/csr.h+trap.h: every CSR accessor is uint32_t, which on lp64 sign-extends pointer casts into mtvec/mepc writes (psABI keeps 32-bit values sign-extended in registers) and puts MCAUSE_INTERRUPT_BIT at bit 31 instead of 63 - silent misclassification of every interrupt; plus rv32-only cycleh/timeh/instreth accessors that must become single 64-bit reads. Second, the Linux pipeline is rv32 end-to-end: BR2_RISCV_32 defconfig, CONFIG_ARCH_RV32I kernel configs, an rv32 DTS/isa string generated by build_fpga_boot.py, a boot shim whose li 0x80000000 sign-extends on rv64, a post-image riscv32-*-gcc toolchain glob, kernel-byte-offset patches in patch_linux_image.py pinned to the exact rv32 build, and qemu-system-riscv32 in CI. Third, the freshly-added Zicntr counter-delta stress payload reads 0xc80-0xc82 by number and its SIGILL guard would silently report counters unavailable on rv64 while still printing the PASS token - a quiet coverage loss. Fourth, ISA-content apps (isa_test rev8/pack expectations, c_ext_test's c.jal, umode_test's cycleh gating matrix) and the FROST-local rv32 FreeRTOS port (4-byte sw/lw context frames, uint32_t port types) need semantic rewrites, and the rv32 riscv-tests/arch-test/torture suites need rv64 counterparts. Standalone-asm apps additionally hardcode ARCH/ABI in 9 Makefiles plus ld -m elf32lriscv. Minimal file set for an rv64 hello_world through cocotb: sw/common/common.mk (march/mabi) is the only strictly required sw-side edit (crt0/link.ld/uart/timer work as-is), plus the other-agent-owned generate_imem_predecode_init.py and cocotb harness; csr.h/limits.h fixes are needed the moment any test touches CSRs. Minimal set for an rv64 Buildroot boot: new rv64 defconfig + regenerated kernel base config/fragment, build_fpga_boot.py (isa strings/shim/li fix), post-image.sh glob, patch_linux_image.py re-derivation or gating, frost_stress.c counter path, sw/apps/linux_boot/Makefile defconfig name, and the CI qemu job - with README updates in the same change per project doc policy. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`sw/lib/include/csr.h:110`** [high] MCAUSE_INTERRUPT_BIT is (1U << 31); on RV64 the mcause interrupt bit is bit 63. Every `mcause & MCAUSE_INTERRUPT_BIT` check (trap handlers across apps/lib) silently classifies interrupts as exceptions. With csr_read also returning uint32_t the bit is doubly unreachable. + - *Action:* Make it (1UL << (__riscv_xlen - 1)) and widen csr accessors to unsigned long; audit every user of mcause +- **`sw/lib/include/csr.h:142`** [high] csr_read/csr_write/csr_set/csr_clear/csr_swap/csr_read_imm/csr_write_imm all use uint32_t (lines 142, 184, 196, 208). On RV64/lp64 the RISC-V psABI keeps 32-bit values SIGN-extended in registers, so csr_write(mtvec, (uint32_t)&handler) with a DDR-resident handler (0x8000_xxxx) writes 0xFFFFFFFF_8000xxxx into mtvec/mepc/mscratch. Reads truncate 64-bit CSR values (mcause, mtval, mepc). + - *Action:* Switch all accessor value types to unsigned long/uintptr_t; same for trap.h set_trap_handler (line 195) and disable/restore_interrupts (lines 113-127) +- **`linux/buildroot-external/board/frost/build_fpga_boot.py:177`** [high] Boot shim emits `li a1, 0x80000000` (DTB base) and `li t0, 0x80000000` (kernel entry). On an rv64 assembler li/lui sign-extends: a1/t0 become 0xFFFFFFFF_80000000. The kernel receives a sign-extended DTB pointer and the far-jump PC has bits [63:32] set - behavior depends entirely on the (undecided) high-address-bit policy of the core. + - *Action:* Rewrite the shim address materialization to produce zero-extended sub-4GiB addresses (or define/verify the core's high-bit-ignoring policy first) +- **`linux/buildroot-external/package/frost-stress/src/frost_stress.c:116`** [high] The just-added Zicntr counter-delta code reads cycleh/timeh/instreth by number (0xc80/0xc81/0xc82, lines 116-147). On RV64 these CSRs are illegal; the payload's existing SIGILL guard (lines 149-159) longjmps out and reports counters as 'unavailable' - the stress run still prints FROST_USERSPACE_STRESS_PASS, so the new counter coverage is lost SILENTLY on an rv64 boot. + - *Action:* Add an __riscv_xlen==64 path doing single 64-bit csrr of 0xc00/0xc01/0xc02, and make counter-unavailable a failure on FROST (keep the guard only for QEMU) +- **`sw/lib/include/limits.h:33`** [medium] LONG_MIN/LONG_MAX/ULONG_MAX hardcoded to 32-bit values (lines 33-34, 38). On lp64 `long` is 64-bit, so these constants are silently wrong for any range check or overflow guard using them. + - *Action:* Define from __LONG_MAX__ or gate on __riscv_xlen + +### Design work (new logic or semantics) + +- **`sw/common/link.ld:56`** [medium] __stack_reserve = 112K was measured with ilp32 (CoreMark-PRO parser high-water). lp64 doubles pointer/long footprints on stack frames, so the reserve and the RAM-fit ASSERT (line 230) may be undersized; likewise the 96K ROM budget - rv64 code is typically a few percent larger and lp64 data larger. + - *Action:* Re-measure stack high-water and ROM/RAM fit under lp64d; adjust reserve/regions or accept per-app MEM_CONFIG=ddr +- **`sw/lib/include/csr.h:229`** [medium] rdcycleh/rdtimeh/rdinstreth (lines 229-302) and the hi/lo/hi2 64-bit composition loops (rdcycle64/rdtime64/rdinstret64, lines 240-316) plus CSR_CYCLEH/TIMEH/INSTRETH defines (58-60) are rv32-only: the high-half CSRs MUST be illegal on RV64. Callers include isa_test.c:2789-2807 and umode_test. + - *Action:* On rv64, rd*64 becomes a single csrr; delete/gate the *h accessors; update dependent tests +- **`sw/apps/freertos_demo/port_frost_asm.S:26`** [medium] FreeRTOS uses a FROST-local rv32 port (no upstream portable/GCC/RISC-V dir): port_frost_asm.S saves 28 GPRs + mepc + mstatus in 4-byte slots (sw/lw, lines 40-86, portCONTEXT_SIZE comment line 26 '31 words = 124 bytes'); portmacro.h:37/77/79 pin portPOINTER_SIZE_TYPE/portSTACK_TYPE/portUBASE_TYPE to uint32_t. Silent stack-frame corruption if compiled lp64 unchanged. + - *Action:* Full port rewrite: sd/ld 8-byte slots, portCONTEXT_SIZE 256, uint64_t port types - or drop the FreeRTOS demo (decision) +- **`sw/apps/c_ext_test/c_ext_test.S:47`** [low] c.jal used throughout (47, 67-69, 139) - the encoding becomes C.ADDIW on RV64, so the test won't assemble under rv64 march and its coverage intent (RV32C recode points) inverts. + - *Action:* Write an rv64 variant covering C.ADDIW, C.SUBW/C.ADDW, C.LD/C.SD/C.LDSP/C.SDSP and 6-bit C shamts; retire the rv32-only cases +- **`sw/apps/isa_test/isa_test.c:3028`** [medium] Zbb/Zbkb tests bake rv32 semantics: rev8 expected values for 32-bit reversal (3028-3034; REV8 immediate also differs on rv64), pack/packh 32-bit results (3211-3224), and rdcycleh/rdtimeh/rdinstreth CSR tests (2789-2807). ZEXT.H encoding changes (PACKW-based) too. + - *Action:* Fork rv64 expected values, add W-op/LWU/LD/SD coverage, drop *h CSR tests, ensure ZIP/UNZIP (rv32-only) become illegal-instruction tests +- **`sw/apps/umode_test/main.c:172`** [medium] The mcounteren U-mode gating test (just landed) reads cycleh/timeh/instreth from U-mode (172-176, 199-201) and asserts CY-bit gating on the high-half CSRs. On RV64 those CSRs are unconditionally illegal - the test's pass criteria change semantically, not just mechanically. + - *Action:* Rewrite the gating matrix for rv64: cycle/time/instret only, plus assert 0xC80-0xC82 trap illegal regardless of mcounteren +- **`linux/buildroot-external/configs/frost_nommu_rv32_defconfig:21`** [medium] BR2_riscv + BR2_RISCV_32 (line 22) select the rv32 uClibc/bFLT internal toolchain; filename, provenance comment (derived from qemu_riscv32_nommu_virt_defconfig) and toolchain-default note (line 15) are all rv32. + - *Action:* New frost_nommu_rv64_defconfig with BR2_RISCV_64; re-derive from qemu_riscv64_nommu_virt_defconfig and re-validate the uClibc/elf2flt rv64-FLAT userspace path (including the vendored uclibc dl_pagesize patch) +- **`linux/buildroot-external/board/frost/linux-nommu-base.config:5`** [medium] CONFIG_ARCH_RV32I=y here and in linux-nommu-frost.config.fragment:69 (fragment prose at 4, 55, 63, 135 documents rv32). Kernel must become CONFIG_ARCH_RV64I; the whole base config was captured from an rv32 build and should be regenerated rather than sed-edited. + - *Action:* Regenerate base config from the rv64 nommu virt defconfig; rewrite fragment deltas and comments +- **`linux/buildroot-external/board/frost/build_fpga_boot.py:122`** [medium] Generated DTS hardcodes riscv,isa-base = "rv32i", riscv,isa = "rv32imafdc...", compatible "frost,nommu-rv32" and model string (102-103); SHIM_MARCH/SHIM_MABI defaults rv32i_zicsr/ilp32 (69-70, doc 46-47). #address-cells=1 (100) stays valid for the sub-4GiB map but should be a conscious choice. + - *Action:* Emit rv64 isa strings/compatible, default shim march rv64i_zicsr/lp64, fix the li sign-extension (separate finding), decide address-cells +- **`linux/buildroot-external/board/frost/patch_linux_image.py:94`** [medium] Kernel-image byte patches are pinned to the exact rv32 6.18.7 build: absolute addresses (PROC_GET_INODE_MODE_* 0x001071B2/0x00107220/0x0010718C, PROC_LOOKUP_* 0x0010BC82/0x0010BC7C), expected instruction bytes (lui a5,0x8 = b7 87 00 00), and cpu_relax offsets (0x1C/0x20). An rv64 kernel invalidates every offset; the byte-verify makes it fail loud, but all patch sites must be re-derived (or retired) for the rv64 image. The 4-byte word file format itself is XLEN-agnostic. + - *Action:* Re-derive every patch offset/byte sequence against the rv64 kernel, or gate the mutations off for rv64 bring-up +- **`sw/common/common.mk:296`** [low] imem predecode init flow: common.mk lines 296-313 and fpga/build/build.py:423-452 (via hello_world's GENERATE_IMEM_INIT=1) are the two callers of generate_imem_predecode_init.py (audited by another agent). The predecode content is XLEN/RVC-sensitive (C.JAL disappears, new W-ops); callers themselves need no change beyond consuming any new sideband files. + - *Action:* Coordinate: when the predecode script's outputs change for rv64, both call sites' file lists must track it + +### Policy (needs a project decision) + +- **`sw/lib/include/csr.h:85`** [low] Custom Tomasulo profiling CSR pair MPERFDATA/MPERFDATAH (0xFC0/0xFC1) is a split 32-bit high/low interface mirrored in RTL and tomasulo_profile.h consumers. + - *Action:* Decide whether custom perf CSRs become single 64-bit reads on rv64 (RTL + sw/lib/include/tomasulo_profile.h + tools change together) or keep the split interface +- **`fpga/load_software/file_to_ddr.tcl:15`** [low] Loader/image formats (sw.txt/sw_ddr.txt one 32-bit word per line; objcopy --verilog-data-width 4; xxd -e -g4; jtag_axi M_AXI_DATA_WIDTH 32 in build_step.tcl:340) are byte-stream/bus-side and XLEN-independent - they work unchanged for rv64 ELFs. No change needed; flagged so nobody 'fixes' them to 8-byte words unnecessarily. + - *Action:* Keep 32-bit word image formats (decision: do NOT widen); document that the format is memory-side, not XLEN-side +- **`sw/apps/riscv_tests/Makefile:17`** [medium] Suite runners target rv32 test sources (riscv-tests/isa/rv32ui, riscv-arch-test rv32i_m) per usage docs (riscv_tests:17, arch_test:17); riscv_torture generates rv32 programs. rv64 needs the rv64 suite directories and torture config - a test-matrix decision, not a flag flip. + - *Action:* Decide the rv64 test matrix: rv64ui/rv64um/... suites, arch-test rv64i_m, torture rv64 config; decide whether rv32 suites are retired or kept for a dual-XLEN build +- **`sw/lib/include/trap.h:216`** [medium] rdmtime/set_timer_cmp use 32-bit MMIO hi/lo halves (216-242) - CORRECT on rv64 too since the CLINT-compatible MMIO registers stay 32-bit; only the max-first mtimecmp ordering comment stays valid. No change needed (flagged to prevent accidental 64-bit MMIO 'fix' that the bus may not support). + - *Action:* Decide whether the CLINT alias gains 64-bit access on rv64 (Linux CLINT driver does sbi/mmio 32-bit pairs on rv32, single 64-bit reads on rv64 - check the kernel's rv64 clint timer path against the bus's 8-byte MMIO support) + +### Mechanical (width/literal/comment hygiene) + +- `sw/common/common.mk:91` [low] RISCV_FLAGS hardcodes -march=rv32imafdc_zicsr_zicntr_zifencei_zba_zbb_zbs_zicond_zbkb_zihintpause; MABI default ilp32d at line 44. This is the single central flag site for all C apps (good); comments at lines 48-65 document rv32. +- `sw/common/standalone_asm.mk:67` [low] LINK_FLAGS := -m elf32lriscv - ld emulation hardcoded; rv64 objects fail to link (loud, but must change). +- `sw/apps/branch_pred_test/Makefile:17` [medium] Hardcoded ARCH := rv32... / ABI := ilp32 is scattered across 9 standalone/suite app Makefiles: branch_pred_test:17, c_ext_test:17, ras_test:17, fpu_assembly_test:17, cf_ext_test:17, fetch_stall_repro:17, arch_test:25, riscv_tests:26, riscv_torture:26, coremark_pro:101, plus the doc template in sw/CONTRIBUTING.md:227. +- `sw/common/crt0.S:21` [low] Data-copy and BSS-zero loops use lw/sw with 4-byte strides. Functionally CORRECT on rv64 (word ops are legal; linker ALIGN(4) guarantees exact termination), just half-rate. Not a correctness change. +- `sw/common/link.ld:200` [low] __global_pointer$ = ADDR(.sdata)+0x800 and MMIO PROVIDE symbols are XLEN-agnostic; MEMORY origins/lengths (lines 41-47) unchanged under the sub-4GiB map; no OUTPUT_FORMAT/OUTPUT_ARCH pinning elf32 in either linker script - scripts work for both classes as written (only ALIGN(4) -> ALIGN(8) niceties). +- `sw/apps/linux_boot/Makefile:110` [low] Invokes Buildroot with frost_nommu_rv32_defconfig (also referenced in defconfig docs and linux/buildroot-external/README.md:93). +- `linux/buildroot-external/board/frost/post-image.sh:38` [low] Toolchain discovery globs ${HOST_DIR}/bin/riscv32-*-gcc; an rv64 Buildroot produces riscv64-* and the script exits (loud failure). +- `.github/workflows/ci.yml:617` [low] linux-boot-qemu job runs qemu-system-riscv32 with -cpu rv32,mmu=off (620). rv64 boot needs qemu-system-riscv64 / -cpu rv64,mmu=off (binary availability in the frost Docker image must be checked). +- `sw/README.md:627` [low] Docs assert ILP32D ABI, 32-bit overflow guidance (227), rv32 template in CONTRIBUTING.md:227-228/277, ISA support table 'RV32GCB'; linux/README.md:79/116 and buildroot-external/README.md (39, 63, 93, 102, 147, 199) document the rv32 pipeline. + +## PD (pre-decode) stage: slot-1 RVC decompression mux, early source extraction, and the predicted-taken redirect fast path (PC+imm_b), plus its cocotb unit test + +pd_stage.sv (read in full, 505 lines) is mostly XLEN-clean — instruction carriers are correctly fixed [31:0]/instr_t, source extraction is 5-bit, and the PD-to-ID registers take their widths from riscv_pkg structs — but it contains two textbook silent-misbehavior hazards, both confirmed: pd_imm_b_native (line 265) and pd_imm_b_compressed (line 275) build fixed 32-bit sign-extended B-type immediates ({19{bit31}} and {23{bit12}} replications) and assign them into [XLEN-1:0], so at XLEN=64 every negative branch offset zero-extends and the PC+imm adders (297-298) feed a garbage target into pd_redirect_target_r, corrupting the documented 4-5-cycle-saving predicted-taken redirect fast path for both native branches and C.BEQZ/C.BNEZ. Fixes are one-line replication-width changes ({{(XLEN-13){...}}} / {{(XLEN-9){...}}}), hence the file's mechanical-only verdict, but the failure mode is high-severity silent wrong-fetch. A third hazard is the module's XLEN=32 parameter default: the core passes riscv_pkg::XLEN through cpu_ooo (clean), but the cocotb bench elaborates pd_stage standalone on the default, which would silently truncate 64-bit struct PCs into 32-bit adders once the package flips. The RV64 RVC recode does NOT change PD's own logic: branch detection matches only the XLEN-invariant C.BEQZ/C.BNEZ encodings and never treats the former C.JAL slot as control flow; the recode work lives in the parameterless rvc_decompressor instance (line 71), which is only an instantiation-coupling concern here. The compressed immediate bit layout is likewise unchanged on RV64, so only the sign-extension width changes. On timing, the file carries two documented WNS post-mortems on exactly the cone being widened (the PC+imm carry chain and the PD->ID BTB-target register), so the 64-bit widening must preserve the existing structure (parallel adders, select after carry, registered redirect) and be re-checked in STA. The cocotb test needs its XLEN constant flipped to 64 (struct packing widths), the 0xFFFFFFFF target mask widened (after which the existing imm=-4 native test becomes the regression tripwire for hazard 1), and — the real gap — a new compressed-branch redirect test, since pd_imm_b_compressed currently has zero coverage and hazard 2 would otherwise ship undetected. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv:265`** [high] pd_imm_b_native is a fixed 32-bit concatenation ({19{effective_instr[31]}} + 13 payload bits) assigned to logic [XLEN-1:0]. At XLEN=64 the 32-bit expression ZERO-extends into the 64-bit immediate, so every negative native B-type offset becomes a huge positive immediate. The PC+imm redirect target (line 297) then sends fetch to PC+~0x0_FFFF_xxxx garbage whenever the direction predictor fires the PD fast-path redirect on a BTB-missing backward branch. Compiles clean; silent wrong-fetch at runtime (recovered only by EX misprediction machinery treating the redirect as a bogus prediction, i.e. wrong-path fetch storms, not a correctness proof). + - *Action:* Replace the fixed {19{...}} with {{(XLEN-13){i_from_if_to_pd.effective_instr[31]}}, ...} so the replication tracks XLEN; update the 'sign-extend bits [31:13]' comment. +- **`hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv:275`** [high] pd_imm_b_compressed has the identical bug: {23{raw_parcel[12]}} + 9 payload bits = fixed 32 bits into [XLEN-1:0]. Negative C.BEQZ/C.BNEZ offsets zero-extend at XLEN=64, corrupting the compressed-branch arm of pd_backward_target (line 298). This arm currently has ZERO cocotb coverage (see test finding), so it is the most likely path to survive to hardware. + - *Action:* Replace {23{...}} with {{(XLEN-9){i_from_if_to_pd.raw_parcel[12]}}, ...}; the C.BEQZ/C.BNEZ immediate bit layout itself is unchanged between RV32C and RV64C so only the replication width changes. +- **`hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv:41`** [medium] parameter int unsigned XLEN = 32 default. cpu_ooo.sv:500-502 passes .XLEN(XLEN) (itself defaulted to riscv_pkg::XLEN at cpu_ooo.sv:31), so the full-core build is clean-parametric. But the cocotb unit test elaborates pd_stage standalone with the DEFAULT, while the port structs (from_if_to_pd_t etc.) are riscv_pkg types whose fields widen with the package XLEN — flipping the package to 64 while the default stays 32 gives a bench where 64-bit PCs feed a 32-bit adder and a 32-bit o_pd_redirect_target port, silently truncating. + - *Action:* Change the default to riscv_pkg::XLEN (or make the testbench/Makefile pass -GXLEN=64) so standalone elaboration cannot diverge from the package structs. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv:71`** [low] The slot-1 rvc_decompressor instance is parameterless. The RV64 C-table recode (C.JAL slot becomes C.ADDIW, C.FLW/C.FSW/C.FLWSP/C.FSWSP become C.LD/C.SD/C.LDSP/C.SDSP, C.SUBW/C.ADDW become valid, C.SLLI/C.SRLI/C.SRAI bit12=1 becomes a legal 6-bit shamt) all lives inside that module; if it grows an RV64/XLEN parameter, this instantiation must pass it. Within pd_stage itself the recode is benign: pd_compressed_branch (lines 289-292) matches only quadrant-01 funct3=110/111 (C.BEQZ/C.BNEZ, unchanged encodings), and nothing here treats quadrant-01 funct3=001 (the former C.JAL) as control flow — the recode does NOT create a false or missed PD redirect. decomp_illegal semantics changes flow through lines 382-385 unchanged. + - *Action:* When rvc_decompressor is recoded for RV64, propagate its new parameter (if any) here; no PD-local branch-detection change needed. +- **`verif/cocotb_tests/pd_stage/test_pd_stage.py:636`** [medium] The only redirect test uses a native B-type branch (_pack_b). The compressed arm — pd_imm_b_compressed and the pd_compressed_branch select at pd_stage.sv:299 — has no test at all, so the second sign-extension hazard (pd_stage.sv:275) would ship undetected even after the native test is fixed for 64-bit. + - *Action:* Add a C.BEQZ/C.BNEZ negative-offset redirect test (quadrant 01, funct3 110/111, raw_parcel-driven since PD extracts the compressed immediate from raw_parcel, not effective_instr) asserting the full 64-bit target; also add one forward/positive-offset case for both arms. + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv:323` [low] Comment says registering the redirect eliminated the '32-bit adder + routing' cross-module path; with XLEN=64 the adder is 64-bit. Comment-only staleness, but this file's timing comments are load-bearing project documentation (per repo doc policy) and this specific cone has documented WNS history. +- `verif/cocotb_tests/pd_stage/test_pd_stage.py:26` [medium] XLEN = 32 constant drives the bit widths used to pack/unpack the from_if_to_pd_t and from_pd_to_id_t structs (program_counter, btb_predicted_target, ras_predicted_target fields at lines 48, 56, 58, 67, 77, 79). At XLEN=64 every field offset in both structs shifts, so all packing/unpacking misaligns (loud failure, but every test breaks until updated). +- `verif/cocotb_tests/pd_stage/test_pd_stage.py:669` [low] assert int(dut.o_pd_redirect_target.value) == (BASE_PC - 4) & 0xFFFFFFFF hardcodes a 32-bit mask on the redirect target. At XLEN=64 with the RTL zero-extension bug, the DUT would produce 0x17FFF_FFFC while (BASE_PC-4)&0xFFFFFFFF is 0x80000FFC — the masked comparison still fails, which is good, but the mask must widen to (1 << XLEN) - 1 (or be dropped) so the assertion checks the full 64-bit target and remains the regression tripwire for the sign-extension hazard. + +## cocotb unit tests: if_stage (aligner/decompressor/PC/branch-prediction), predecode mirrors, id_stage, ex_stage branch_jump_unit, cache + +All 21 assigned test files were read in full. The cache suite (frost_cache, line_port_arbiter, fence_speed) is genuinely width-neutral - line-port protocol, byte strobes, sub-4-GiB addresses - and needs no RV64 work. The PC-side if_stage unit tests (pc_controller, pc_increment_calculator, c_ext_state, control_flow_tracker, metadata tracker, direction predictor, return_address_stack, branch_prediction_controller) are width-neutral too, with pc_controller already using the len(dut.o_pc)-derived-mask pattern the whole suite should adopt. The dangerous concentration is in the C-extension recode surface: test_rvc_decompressor.py asserts C.SUBW/C.ADDW space and C.SLLI shamt[5] are illegal (both valid on RV64) and has zero C.LD/C.SD/C.LDSP/C.SDSP/C.ADDIW vectors, so a recode that deletes the failing asserts ships the new table untested. The single worst finding is test_ras_detector.py, whose C.JAL-is-a-call assertion passes against un-recoded RTL and would actively pin RAS-corrupting behavior (C.ADDIW pushing return addresses) - an inverted, silently-green test. test_instruction_aligner.py and test_if_stage.py hardcode two C.JAL expansion constants (0xD0DFF0EF, 0x108000EF) that become wrong ADDIW rows. The predecode mirror test_imem_predecode_line.py is structurally sound - both sides derive from sw/common/generate_imem_predecode_init.py, so recoding that generator first turns four test files into automatic drift catchers - but its directed parcel list and the fast-replica bench's deliberately independent second model must gain RV64 rows (C.ADDIW non-control, C.SUBW/ADDW, OP-32/OP-IMM-32 near-misses). test_branch_jump_unit.py is the clearest vacuous-pass hazard outside the C-table: every operand fits in 32 bits, so a comparator left at [31:0] after widening would pass the entire suite; 64-bit discriminating vectors are mandatory. test_id_stage.py needs the same treatment (no LD/LWU/SD/W-op vectors) plus mechanical tracking of its private XLEN/STORE_OP_WIDTH/struct tables and its fragile hardcoded instruction_operation ordinals. Private width-constant copies (XLEN=32, MASK32, 0xFFFFFFFF masks, the duplicated 18-bit sideband layout) appear in six files and should move to the centralized verif width module the main audit proposed. Finally, two tests encode the pc[31]-anchored region assumptions (served-window guard arming, fetch-provider DDR select) that break if RTL mechanically migrates to pc[63] - the bit-31 address-map policy decision gates them. + +### Hazards (compile-clean, silently wrong at XLEN=64) + +- **`verif/cocotb_tests/if_stage/branch_prediction/test_ras_detector.py:176`** [high] test_compressed_call_return_and_coroutine_classification asserts the funct3=001/quadrant-01 parcel built by _make_c_jal (line 75) classifies as a RAS CALL. On RV64 that encoding is C.ADDIW (not control flow, not a call). This assertion PASSES against un-recoded RTL and actively pins the wrong behavior: an un-updated ras_detector would push a bogus RAS entry on every C.ADDIW and this test would keep it green. + - *Action:* Flip the expectation to _assert_classification(dut) (no call/ret/coroutine) for the funct3=001 quadrant-01 parcel and add it as an explicit C.ADDIW-is-not-a-call regression; keep C.JALR/C.JR call/return vectors unchanged. +- **`verif/cocotb_tests/ex_stage/test_branch_jump_unit.py:23`** [high] MASK32/_u32 truncates every driven operand, target, and immediate to 32 bits (lines 43-45, 64-68), and all comparison vectors fit in 32 bits (e.g. 0xFFFFFFFF as -1 at lines 100-124). After widening, a comparator accidentally left at [31:0] or a sign-bit-31 index would pass every retained vector - complete vacuous-pass coverage hole for the upper 32 bits; meanwhile the 0xFFFFFFFF signed vectors change meaning (become +2^32-1) and fail confusingly. + - *Action:* Replace MASK32 with an XLEN-derived mask; rewrite -1 vectors as 0xFFFFFFFF_FFFFFFFF; add discriminating vectors with sign bit 63 (BLT/BGE) and upper-half-only differences (BEQ/BNE/BLTU/BGEU with a=1<<32, b=0); add JALR vectors with 64-bit rs1 and bit-0 clearing above 4 GiB. +- **`verif/cocotb_tests/if_stage/test_if_stage.py:333`** [high] The served-window guard the tests depend on 'only arms in the cached region (pc_reg[XLEN-1])' per the tracker docstring. A mechanical XLEN flip moves that to bit 63, which is never set for the sub-4-GiB map: the guard goes dead, and test_fetch_window_lead_parity_plus2_desync (line 1199, served_word_offset=1) plus the workqueue_init_early +2-desync protection silently lose their RTL backstop (the test will fail loudly, but only because the guard died - the fix is in RTL policy, not the test). + - *Action:* Pin the cached-region detect to the address-map bit (bit 31) or an explicit region-decode function in RTL; keep BASE_PC=0x80001000 vectors and assert the guard still arms there. +- **`verif/cocotb_tests/if_stage/branch_prediction/test_branch_predictor.py:124`** [medium] Slot-2 predecessor lookups mask base PCs with a hardcoded 0xFFFFFFFF (lines 124, 133, 409). At XLEN=64 with full-width BTB keys, (pc-2)&0xFFFFFFFF is the wrong predecessor for pc=0 (should borrow to 2^64-2); with compressed low-32 keys the tests pass while never exercising bits 63:32 of the key path. + - *Action:* Replace masks with an XLEN-derived mask; decide key-width policy first (see decisions), then either rewrite wrap vectors at the 2^64 boundary or document/assert the low-32 compression. +- **`verif/cocotb_tests/id_stage/test_id_stage.py:46`** [medium] Hardcoded instruction_operation enum ordinals (ADD=0, ADDI=10, JAL=21, JALR=22, BEQ=23, LW=31, SW=36, FMADD_S=111). The RV64 decoder adds ~30 ops (W-forms, LD/LWU/SD, FCVT.L*, Zba/Zbb W-forms); any insertion before existing members silently renumbers all of these, producing confusing wholesale failures (or, if the enum is reordered symmetrically, spurious passes). + - *Action:* Generate the ordinal table from riscv_pkg (or append-only discipline documented in the enum) and re-derive these constants in the same commit that grows instr_op_e. + +### Design work (new logic or semantics) + +- **`verif/cocotb_tests/if_stage/test_rvc_decompressor.py:444`** [high] test_quadrant1_alu_group_expands_and_rejects_rv64_only_ops asserts sub_raw|(1<<12) (the C.SUBW/C.ADDW encoding space) is illegal. Valid on RV64; after recode the assert fails, and if it is simply deleted, C.SUBW/C.ADDW expansion (funct7=0100000/0000000, OP-32 opcode 0b0111011) is untested. + - *Action:* Replace with positive expansion vectors: C.SUBW -> SUBW rd',rd',rs2' and C.ADDW -> ADDW, plus the still-reserved funct2=10/11 rows as illegal. +- **`verif/cocotb_tests/if_stage/test_rvc_decompressor.py:558`** [high] test_shift_and_lwsp_rd_zero_illegal_cases asserts C.SLLI with bit12=1 (shamt[5]=1) is illegal - RV32-only reservation. On RV64 it is a valid 6-bit shamt. Deleting the vector without replacement leaves shamt6 expansion untested (the decompressor must emit shamt[5] into instruction bit 25). + - *Action:* Convert to a positive vector: C.SLLI rd, shamt=32..63 expands to SLLI with imm[5]=1; add matching C.SRLI/C.SRAI bit12=1 vectors (quadrant-01 funct3=100). Keep the C.LWSP rd=x0 illegal case. +- **`verif/cocotb_tests/if_stage/test_rvc_decompressor.py:214`** [medium] test_all_rvc_source_hot_metadata_matches_decompressor exhaustively cross-checks all 49,152 parcels against generate_imem_predecode_init.rvc_source_hot. The recode (C.JAL->C.ADDIW, C.FLW/C.FSW->C.LD/C.SD int-register forms, C.FLWSP/C.FSWSP->C.LDSP/C.SDSP) changes source-register usage for whole encoding rows; RTL decompressor and the Python golden model must be recoded in lockstep or this test fails on every affected parcel. + - *Action:* Recode sw/common/generate_imem_predecode_init.py rvc_source_hot alongside the decompressor; this test then serves as the drift catcher - do not weaken it. +- **`verif/cocotb_tests/if_stage/test_rvc_decompressor.py:286`** [high] Quadrant-0/quadrant-2 coverage has only C.LW/C.SW/C.LWSP/C.SWSP vectors. The RV64 rows C.LD/C.SD (funct3=011/111, 8-scaled uimm[7:3]) and C.LDSP/C.SDSP, plus C.ADDIW (old C.JAL row, rd!=0 required; rd=0 reserved-illegal), have no expansion vectors - deleting RV32-only cases would leave the whole new table vacuously untested. + - *Action:* Add expansion vectors: C.LD->LD (funct3=011, imm helpers scaled by 8), C.SD->SD, C.LDSP->LD via x2, C.SDSP->SD via x2, C.ADDIW->ADDIW with rd=0 asserted illegal, and LWU absence (no compressed form). +- **`verif/cocotb_tests/if_stage/test_instruction_aligner.py:277`** [medium] test_high_parcel_selects_current_hi_and_next_lo_slot2 hardcodes the slot-2 effective expansion of parcel 0x3331 as 0xD0DFF0EF (C.JAL -> JAL x1). On RV64, 0x3331 is C.ADDIW and the expansion is an ADDIW encoding; the expected constant (and the driven rvc_source_hot_lo=3 at line 261) becomes wrong. + - *Action:* Recompute the expected expansion for the RV64 decompressor (ADDIW rd, rd, imm) or switch the vector to a still-stable parcel (e.g. C.J/C.LI); update the driven source-hot value to match the new expansion. +- **`verif/cocotb_tests/if_stage/test_if_stage.py:528`** [medium] COMPRESSED_HINT=0x2221 (line 34) is expected to expand to 0x108000EF (C.JAL -> JAL x1) at lines 529 and 1124. On RV64, 0x2221 is C.ADDIW x4 - both expected-effective constants and the paired-slot control-flow assumptions become wrong. + - *Action:* Recompute the two expected expansions as the ADDIW encoding (or substitute a truly width-stable hint parcel such as C.LI) and re-derive the source-hot value driven at line 508. +- **`verif/cocotb_tests/predecode/test_imem_predecode_line.py:170`** [high] The directed parcel list treats quadrant-01 funct3=001 as control flow ('C.JAL', lines 169-173). Expectations are derived from the generator, so the test stays green through the recode - it is the RTL-vs-Python drift catcher - but it gains no RV64-specific discrimination unless the directed list grows: no C.ADDIW-as-non-control case, no C.SUBW/C.ADDW row, no C.SLLI bit12=1, no C.LD/C.SD/C.LDSP/C.SDSP shapes, and no OP-32 (0b0111011)/OP-IMM-32 (0b0011011) native near-miss opcodes. + - *Action:* Prerequisite: recode sw/common/generate_imem_predecode_init.py (compressed_control drops funct3=001; rvc_source_hot rows for C.ADDIW/C.LD/C.SD/C.LDSP/C.SDSP). Then extend parcels[] and NEAR_MISS_OPCODES with the RV64 rows above so the exhaustive/random tests plus directed vectors pin the new table on both the RTL and Python sides. +- **`verif/cocotb_tests/predecode/test_imem_predecode_fast_replica.py:102`** [medium] _expected_compressed_control - the deliberately independent second model - classifies quadrant-01 funct3=001 (C.JAL) as control. On RV64 it must drop 0b001 from the control set. If updated by copy-paste from the generator instead of independently from the spec, the cross-check loses its value; if not updated, every assert at lines 271-283 fails once the generator recodes. + - *Action:* Independently re-derive _expected_compressed_control (and _expected_allows_slot2_after_hi) from the RV64 C-table: quadrant-01 control set becomes {101,110,111}; verify C.ADDIW rows now report allows-slot2. +- **`verif/cocotb_tests/id_stage/test_id_stage.py:555`** [high] Decode coverage is RV32-only: no vectors for LD/LWU/SD (mem-size-double int forms, is_load_unsigned for LWU, LW-now-sign-extends semantics), no ADDIW/SLLIW/W-ALU ops, no 6-bit shamt legality, no MULW/DIVW dispatch typing. After the decoder recode these paths ship untested unless new vectors are added - classic vacuous-pass gap. + - *Action:* Add slot-1/slot-2 vectors: LD (funct3=011, RS_MEM, double size), LWU (funct3=110, is_load_unsigned), SD (store_operation=STD), ADDIW/ADDW/SUBW/SLLIW (OP-IMM-32/OP-32 opcodes with correct instruction_operation), SLLI with shamt bit 25 set legal, SRAIW with bit 25 set illegal. + +### Mechanical (width/literal/comment hygiene) + +- `verif/cocotb_tests/if_stage/test_instruction_aligner.py:31` [medium] Private copy of the full 18-bit sideband layout (SB_* indices, SIDEBAND_WIDTH=18, lines 31-45) and the _sideband() derivation logic, duplicated again in test_if_stage.py lines 39-53. Any RV64 predecode sideband change (bit growth or re-derivation) must be edited in two hand-written copies; a missed copy mispacks the bus silently. +- `verif/cocotb_tests/if_stage/test_if_stage.py:26` [medium] Private XLEN=32 constant drives every packed-struct field width (FROM_EX_FIELDS, TRAP_CTRL_FIELDS, IF_TO_PD_FIELDS, lines 65-105) and the served-window word masks (lines 339-340, 381). If riscv_pkg XLEN flips and this constant does not, all struct packing/unpacking misaligns. +- `verif/cocotb_tests/if_stage/branch_prediction/test_branch_predictor.py:663` [medium] test_shifted_slot2_alt_lookup_is_exact_across_key_wraps uses the case (actual 0x00000000 keyed by base 0xFFFFFFFC), explicitly labelled 'full XLEN wrap'. On RV64 the U-4 predecessor of 0 is 0xFFFFFFFF_FFFFFFFC; the vector silently stops testing the borrow it was written for. Same issue for second_pc=0 at line 549 via the line-124 mask. +- `verif/cocotb_tests/predecode/test_fetch_provider.py:163` [low] o_served_last_word expectation masks with hardcoded 0x3FFF_FFFF (the RV32 XLEN-2=30-bit word tag). Numerically harmless for these sub-4-GiB addresses, but it is a frozen copy of an RTL width that becomes 62 bits; also the provider's high-region select (DDR_BASE bit 31, line 38) must not migrate to bit 63 in RTL or every window in this bench stops validating. +- `verif/cocotb_tests/id_stage/test_id_stage.py:26` [medium] Private XLEN=32/FLEN=64/STORE_OP_WIDTH=2/INSTR_OP_WIDTH=32 constants plus hand-copied struct tables (PD_TO_ID_FIELDS, ID_TO_EX_FIELDS with immediates hardcoded 32 at lines 119-123). RV64 widens PCs/targets/rs-data to 64, immediates to XLEN per the main audit, and adding SD forces store_op_e from 2 to 3 bits (STD member) - every one of these constants must track the RTL struct or packing silently misaligns. +- `verif/cocotb_tests/id_stage/test_id_stage.py:225` [low] _sign_extend masks results to 0xFFFFFFFF ('XLEN-masked' per its docstring). Once immediate/PC fields widen, expected values like _sign_extend(-16,12)=0xFFFFFFF0 mismatch the RTL's 64-bit sign extension 0xFFFFFFFF_FFFFFFF0. +- `verif/cocotb_tests/if_stage/test_pc_controller.py:139` [low] _assert_pending_predecessor_relation already derives its mask from len(dut.o_pc) - the pattern the rest of the suite should adopt; all PC vectors are sub-4-GiB and carry over unchanged. + +## FP div/mul FU shims + FPU arithmetic internals (gap-fill: fp_div_shim, fp_mul_shim, 8 fpu arith primitives) + +This gap-fill audit closes the question raised by the fp_add_shim findings: do fp_div_shim and fp_mul_shim share the same hazard family? They do not. Both shims were read in full (797 and 493 lines). fp_div_shim contains no reference to XLEN whatsoever — every datapath width is FLEN, TagW, or a flags width, and its 36/65-stage latency trackers are FP-pipeline properties. fp_mul_shim declares localparam XLEN = riscv_pkg::XLEN at line 58 but never uses it; it is dead code and the only finding of substance, a trivial mechanical cleanup. The specific fp_add_shim hazards were checked one-for-one: no fpu_convert_unit (or any XLEN-parameterized module) is instantiated by either shim, so there is no missing-.XLEN-override analog; the unbox32 helpers and {32'hFFFF_FFFF, result} NaN-boxing are keyed to FLEN, which stays 64, so they remain correct unchanged; there are no {(FLEN-XLEN){...}} replications; and no FP-to-integer results flow through these shims, so the RV64 sign-extension rule never applies here. The eight FPU arithmetic internals were verified rather than assumed FLEN-neutral: all derive every width from FP_WIDTH (or WIDTH/MANT_BITS/EXP_BITS parameters), and a full-body grep for [31:0], 32'h/32'd, {32{, [4:0] shamts, and stray 'd/'h literals found nothing integer-coupled — the only riscv_pkg references are fp_flags_t and rounding-mode enums. fp_lzc and fp_subnorm_shift, the shared primitives the FCVT.L rework will stretch, were read in full and are cleanly width-parametric; fp_convert already instantiates fp_lzc with .WIDTH(XLEN), so the 64-bit LZC comes for free, with only a timing watch on its linear-loop coding style at 64 bits. Net: zero hazards, zero design work, one dead localparam to delete in this subsystem; all real RV64 FP work concentrates in the convert path owned by the fp_add_shim audit. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_lzc.sv:38`** [low] fp_lzc is fully width-parametric ($clog2(WIDTH+1) count output) and fp_convert.sv:156 already instantiates it with .WIDTH(XLEN), so the FCVT.L/LU rework gets a 64-bit LZC for free. However the implementation is a linear priority loop with an increment-per-zero chain (o_lzc = o_lzc + 1 inside a WIDTH-iteration loop); synthesis typically restructures this, but at WIDTH=64 the int-to-fp normalization stage in fp_convert plausibly deepens. Correctness is fine; this is a timing watch item, not a bug. + - *Action:* Keep as-is for the widening; if the fp_convert s2 stage misses timing at 300 MHz with WIDTH=64, replace the linear loop with a recursive/binary LZC (drop-in, same ports). + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_mul_shim.sv:58` [low] localparam int unsigned XLEN = riscv_pkg::XLEN is declared but never used anywhere in the module (grep confirms line 58 is the only XLEN occurrence in the file). It is dead code, not a live hazard: flipping riscv_pkg::XLEN to 64 changes nothing here. The critic's suspicion that fp_mul_shim shares fp_add_shim's missing-.XLEN-override family is disproven — the only subunits instantiated (fpu_mult_unit line 206, fpu_fma_unit line 233) have exactly one parameter, FP_WIDTH_D=64 (FLEN-coupled), and no XLEN parameter to forget. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_div_shim.sv:57` [low] unbox32 (lines 57-59) checks value[FLEN-1:32] and the SP result NaN-boxing at lines 271 and 279 writes {32'hFFFF_FFFF, result_s} into FLEN-wide unit_result. Both are keyed to FLEN, which stays 64 in the RV64 plan, so they remain CORRECT unchanged — verified, not assumed. fp_mul_shim's identical unbox32 (lines 61-63) and its src extraction (lines 144-149) are likewise FLEN-only. Flagging so the RV64 change explicitly leaves these alone; the only way they break is if FLEN were ever parameterized differently. +- `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_subnorm_shift.sv:41` [low] fp_subnorm_shift is fully parametric over MANT_BITS/EXP_EXT_BITS (TotalBits = MANT_BITS+3, ShiftBits = $clog2(TotalBits+1), sticky loop bounded by TotalBits with a dynamic i < shift_amt compare). No hidden 24/48-bit assumptions; the convert path can reuse it unchanged. Note the planned ~117-bit FCVT shifter (ExtMantBits = MantBits + XLEN) lives in fp_convert.sv itself (line 79), not in this primitive — no change lands here. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_div_shim.sv:615` [low] o_fu_complete.value is driven from an FLEN-wide FIFO payload (sdp_dist_ram DATA_WIDTH(FLEN), line 508) into fu_complete_t.value, which is already FLEN=64 per the verified context; all tags/counters/latency trackers (DivSDepth=36, DivDDepth=65) are FP-pipeline properties independent of XLEN. Same for fp_mul_shim's fifo_value[FLEN-1:0] (line 184) and its o_fu_complete drive (line 445). The entire CDB-slot-5/6 payload path is XLEN-independent. No FCVT.L/LU, FMV.X.D or any FP-to-integer op routes through these shims (div/sqrt/mul/fma only), so none of the RV64 sign-extension semantics touch them. + +## Residual sweep: zero-width-replication empirical test, unclaimed lib RTL, documentation staleness, hardware-validation scripts + +The load-bearing empirical question is settled: {{(FLEN-XLEN){1'b0}}, x} with a zero replication count at XLEN=64 is accepted by every consumer of this RTL — CI's pinned Verilator 5.050 is clean even under -Wall, Yosys 0.64 read_verilog -sv is clean and a sat -verify pass proves the concat is exact pass-through (no bit loss), Verible lint/syntax are clean, Vivado 2025.2 xvlog is clean, and Vivado synth_design merely emits a benign per-site 'WARNING: [Synth 8-693] zero replication count - replication ignored' with correct netlist semantics. The ~15 sites flagged as hazards by other auditors should therefore be reclassified as cosmetics; the only reason to touch them is a warning-free Vivado log, and the rewrite is provably netlist-neutral. The unclaimed RTL is benign: cache_perf_pkg.sv is single-bit event flags (unaffected), sdp_dist_ram_2r.sv is fully width-parameterized with its sole instantiator passing explicit widths, and both cache test harnesses are line-granular with 32-bit AXI addressing that exactly matches line_port_axi_bridge's deliberately fixed [31:0] AXI ports — fine forever under the sub-4-GiB map, provided the cache-tier-keeps-ADDR_WIDTH=32 policy is recorded. The documentation sweep found the staleness concentrated in ISA strings and RV32-measured numbers: RV32GCB headlines (README.md:5/68/415, hw/rtl/README.md:4, tomasulo/README.md:5), instruction counts, rv32* test-suite names (README.md:303-306), the 977-CoreMark/3.26-per-MHz headline, the 17-stage-divider/4-stage-multiplier claims (fu_shims/README.md:47-48), the 'FSD on the 32-bit bus takes two phases' and word-granular forwarding taxonomy (store_queue/README.md:17-21/74-76), the SC 'word address' reservation-match claim (tomasulo_wrapper/README.md:81), and the mperfdata/mperfdatah 32-bit-halves protocol (perf/README.md:17-19/41-43). ROADMAP.md is already the RV64 plan and needs only phase-exit status plus the deferred rv32-fate decision. The two hardware-validation vehicles are in good shape for the X3 re-closure: both scripts judge boots purely by text tokens with no xlen/march/bus-width assumptions, linux_boot_soak.py's counter-delta gating ('counters=unavailable' → FAIL) remains valid because mcounteren stays a 32-bit CSR on RV64, and the only real landmine is hw_regression.py's BASELINE_SCORES, whose RV32-binary values under a 1% tolerance will spuriously fail (or slack-mask) the first rv64 runs unless reset to None and re-recorded at phase exit. + +### Design work (new logic or semantics) + +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/README.md:74`** [medium] 'FSD on the 32-bit bus takes two phases (low word at addr, high word at addr+4); the entry has a phase bit ...' (74-76) and the test-list line 244 'FSD two-phase'. On RV64, SD (and SC.D/AMO*.D stores) either join the two-phase drain or the bus widens to 64 bits — either way this text and the forwarding taxonomy at lines 17-21 ('FLD from an exact-address FSD (full 64-bit payload), or any byte/half/word load whose byte mask is a subset ... within one word (either word of a DOUBLE store counts as fully written)') describe RV32 semantics: LD/LWU as consumers and SD as a producer of 64-bit integer memory images are new cases the doc must cover. + - *Action:* After the SQ/data-bus design lands (SD reusing the FSD phase machinery vs a 64-bit port), rewrite the forwarding-coverage and drain-phasing sections in the same change. +- **`hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/README.md:81`** [medium] SC state-machine doc: 'sc_success ... requires the reservation to be valid and its address to match the SC's own word address'. On RV64 with LR.D/SC.D the reservation granule becomes at least 8 bytes and the match must be size-aware (an SC.W against an LR.D address, and vice versa, per the reservation-set rules). The word-address phrasing documents RV32-only behavior. + - *Action:* When sc_pending_unit/LQ reservation logic gains the D-granule, update this section (granule size, LR.D/SC.D pairing rules) in the same change. + +### Policy (needs a project decision) + +- **`hw/rtl/lib/cache/frost_cache_test_harness.sv:126`** [low] Harness internal AXI address wires are hardcoded logic [31:0] (line 126) and BASE_ADDR is parameter logic [31:0] = 32'h8000_0000 (line 37), while the upstream line-port address is ADDR_WIDTH-parameterized (default 32). This exactly matches line_port_axi_bridge.sv, whose o_axi_awaddr/o_axi_araddr are fixed [31:0] by design (lines 53/67). Everything else in the harness is LINE_BYTES-granular (line data/wstrb) with zero XLEN dependence; the cocotb cache benches drive it via -G generics. + - *Action:* Nothing to change IF the project decision is that the cache tier keeps 32-bit physical addresses (sub-4-GiB map) and the LSU/cached_tier_adapter truncates/checks bits [63:32] above this seam. If instead ADDR_WIDTH were ever raised past 32, the [31:0] AXI wires here and in the bridge silently truncate — so record the keep-ADDR_WIDTH=32 decision explicitly. +- **`hw/rtl/lib/cache/line_port_arbiter_test_harness.sv:97`** [low] Same shape as frost_cache_test_harness: ADDR_WIDTH parameter (default 32, line 28), BASE_ADDR parameter logic [31:0] (line 30), internal AXI address wires hardcoded [31:0] (line 97) matching the bridge's fixed 32-bit AXI address ports. Line-granular data paths carry no XLEN dependence. + - *Action:* Unchanged under the sub-4-GiB physical-map decision; covered by the same cache-tier ADDR_WIDTH=32 policy call as the sibling harness. +- **`hw/rtl/cpu_and_mem/cpu/cpu_ooo/perf/README.md:18`** [low] Custom perf CSR protocol documents mperfdata (0xFC0) = 'Selected counter, low 32 bits' and mperfdatah (0xFC1) = 'high 32 bits', with lines 41-43 promising 'the two halves are consistent without a hi/lo re-read loop'. On RV64 every CSR read returns 64 bits, so mperfdata could return the whole counter and mperfdatah becomes redundant (or is kept for sw compatibility). Whichever way csr_file goes, this README and tomasulo_profile.h software must move in the same change. + - *Action:* Decide: keep the hi/lo pair (sw-compatible, mperfdatah reads bits 63:32 zero-extended... or full) vs collapse to a single 64-bit mperfdata read; update README + csr_file + profiling sw together. +- **`ROADMAP.md:110`** [low] ROADMAP is already RV64-aware (Phase 1 IS the widening plan; lines 94-115 match this audit's scope) and needs no staleness fix now — but lines 110-111 explicitly defer 'whether the rv32 configuration remains a maintained build or is frozen ... decided at phase exit', which is the same dual-XLEN-vs-rv64-only decision that determines the final wording of every ISA-string doc fix above. + - *Action:* At phase exit: mark Phase 1 done, record the rv32-config decision, and phrase the README/ISA docs accordingly (RV64GCB vs configurable RV32/RV64). + +### Mechanical (width/literal/comment hygiene) + +- `hw/rtl/cpu_and_mem/cpu/tomasulo/register_alias_table/register_alias_table.sv:471` [low] EMPIRICAL RESULT settling the multi-auditor zero-width-replication question. A test module using the exact repo idiom {{(FLEN-XLEN){1'b0}}, x} with FLEN=XLEN=64 (assign form, always_comb form, and multi-member concat form) was run through every consumer toolchain. Verilator 5.050 (pinned CI image): lint-only CLEAN, and CLEAN under -Wall. Yosys 0.64 read_verilog -sv: CLEAN, and 'sat -verify -prove' proves o_a==i_x and o_b==i_x ('SAT proof finished - no model found: SUCCESS!') — semantics are exact pass-through, no bit loss. Verible lint + syntax: CLEAN. Vivado 2025.2 xvlog -sv: CLEAN. Vivado 2025.2 synth_design (part xcux35-vsva1365-3-e, the project's X3 part): ACCEPTS with one benign 'WARNING: [Synth 8-693] zero replication count - replication ignored' per site and a semantically correct netlist (verified via constant-propagation of the concat members). Conclusion: the ~15 sites flagged as 'per-tool lint hazard' by other auditors (this line plus register_alias_table.sv:475/487/491/537/541/553/557, reorder_buffer/reorder_buffer.sv:1008/1013, tomasulo_wrapper.sv:886/946/1419, int_muldiv_shim.sv:267/518, int_alu_shim.sv:139/158, fp_add_shim.sv:406/425, load_queue.sv:1732/1834/1844/1853/1875, sq_forwarding_unit.sv:397, cpu_ooo.sv:2308) are NOT blockers and NOT hazards — reclassify them all as cosmetic. All repo sites are concat-context with constant counts (the LRM-legal form), matching the tested pattern; cpu_ooo.sv:2418's {(XLEN-$bits(cause)){1'b0}} count stays nonzero at 64 and is not even zero-width. +- `README.md:5` [low] Headline claims 'RV32GCB (G = IMAFD) ... Machine + User (M/U) privilege modes'. Stale the moment XLEN flips. +- `README.md:13` [low] Performance headline '3.26 CoreMark/MHz (977 CoreMark at 300 MHz on UltraScale+)' is an RV32-binary measurement; the rv64 recompile plus 64-bit datapath respin will move it. fpga/hw_regression.py:121-122 explicitly cites this README figure as the source of its baseline. +- `README.md:68` [low] 'ISA: RV32GCB (G = IMAFD) ... 170+ instructions' and line 72 'RV32I | Base integer instruction set (37 instructions)'. Both the ISA string and both instruction counts change on RV64 (RV64I base has more instructions; W-ops/LWU/LD/SD/64-bit FCVT/AMO*.D/Zba-W/Zbb-W forms grow the total; ZIP/UNZIP drop). +- `README.md:305` [low] Test-matrix text is rv32-specific: line 303 'rv32ua/rv32uc riscv-tests', line 305 '126 self-checking tests ... across rv32ui, rv32um, rv32ua, rv32uf, rv32ud, rv32uc, rv32mi', line 306 '20 randomly generated RV32IMAFDC instruction sequences ... Spike'. Suite names, test counts, and the torture ISA string all change when the matrices mirror to rv64 (ROADMAP Phase 1 verification plan). +- `README.md:415` [low] Glossary entry 'RV32I | RISC-V 32-bit base integer instruction set' (and the G = IMAFD framing around it) is RV32-specific. +- `hw/rtl/README.md:4` [low] 'The current CPU is an out-of-order RV32GCB implementation ...'. Only RV32-specific claim found in this file; the memory map (all below 4 GiB), MMIO table, 64-bit fetch-window statements, and CACHED_BASE=32'h8000_0000 parameter table remain valid under the unchanged physical map. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/README.md:5` [low] 'existing ISA support (RV32IMACBFD + Zbkb + Zicond + Zicntr + Zifencei + Zihintpause)' — stale ISA string. Also line 59's load_queue table entry 'FP64 phasing' and line 60's store_queue entry 'FSD phasing' describe two-phase 64-bit accesses over the 32-bit data bus; on RV64 LD/SD join that behavior (or the bus widens), so the phrasing must follow the data-bus decision. +- `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/README.md:47` [low] 'the multiplier is 4-stage with up to 4 in-flight multiplies, the divider is 17-stage with up to 17 in-flight divisions' — these are the 32-bit unit latencies/occupancies. A 64x64 multiplier and 64-bit divider will have different depths (the radix-constant divider roughly doubles), changing both the stage counts and the credit-based occupancy figures documented here. The NaN-boxing statements (lines 34/44/74-75) remain correct on RV64 and newly cover FCVT.S.L/LU results. +- `fpga/hw_regression.py:124` [medium] BASELINE_SCORES pins RV32-binary hardware scores (x3: coremark 977.13, coremark_pro 111.14; genesys2: 433.65/39.0) with a default 1% drop tolerance. The rv64 recompile plus 64-bit datapath will shift these; left as-is the coremark/coremark_pro stages of the X3 timing-re-closure validation runs will fail spuriously (or, if scores rise then later regress within the stale-baseline slack, mask a real regression). The None-baseline mechanism (lines 63-65, 175-180) already exists for re-arming. Everything else in the file is XLEN-agnostic: pass/fail is pure text-token parsing (<>/<>/<>/ERROR/:fails=N, lines 192-211), the coremark score uses the mtime-derived 'Total 64-bit ticks' line (line 163) which is unchanged, and the linux stage checks only Buildroot login markers (lines 153-154). +- `fpga/linux_boot_soak.py:189` [low] UART scoring is fully XLEN-agnostic: PASS/FAIL tokens (FROST_USERSPACE_STRESS_PASS/_FAIL, lines 55-56), crash markers (58-66), and the Zicntr counter-delta check are pure string matches — the summary line is located by 'FROST_USERSPACE_STRESS:' + 'verdict=' (line 189) and counters are judged only via the substring 'counters=unavailable' (line 196). The mcounteren rationale in the docstring (lines 25-28) and comment (198-199) stays TRUE on RV64 (mcounteren remains a 32-bit CSR, reset 0x7). One cross-cutting dependency: on RV64 the userspace payload must read counters via full-width rdcycle/rdinstret only — any leftover rdcycleh/rdinstreth path becomes illegal and would trap; if the payload then degrades to counters=unavailable, this script correctly turns that into FAIL(counters-unavailable), so the detection net is already in place. No change to this file. + +--- + +# Appendix: per-file coverage + +Best coverage achieved across auditors (`full` = read line-by-line, +`targeted` = grep-driven with hot regions read, `skimmed` = structure +only). Verdict is the auditor's classification of the file's RV64 cost. + +| File | Coverage | Verdict | +|---|---|---| +| `.github/workflows/ci.yml (linux-boot-qemu job)` | targeted | mechanical-only | +| `.github/workflows/ci.yml` | full | design-work | +| `Dockerfile` | full | design-work | +| `README.md` | targeted | mechanical-only | +| `ROADMAP.md` | targeted | unaffected | +| `boards/README.md` | skimmed | unaffected | +| `docker_entrypoint.py` | full | unaffected | +| `formal/README.md` | full | unaffected | +| `formal/cdb_arbiter.sby` | skimmed | unaffected | +| `formal/csr_file.sby` | full | unaffected | +| `formal/fp_add_shim.sby` | skimmed | unaffected | +| `formal/fp_div_shim.sby` | skimmed | unaffected | +| `formal/fp_mul_shim.sby` | skimmed | unaffected | +| `formal/fu_cdb_adapter.sby` | skimmed | unaffected | +| `formal/fu_cdb_adapter_payload_no_refill.sby` | skimmed | unaffected | +| `formal/load_queue.sby` | skimmed | unaffected | +| `formal/lq_l0_cache.sby` | skimmed | unaffected | +| `formal/register_alias_table.sby` | skimmed | unaffected | +| `formal/reorder_buffer.sby` | full | unaffected | +| `formal/reservation_station.sby` | skimmed | unaffected | +| `formal/rs_issue2_selector.sby` | skimmed | unaffected | +| `formal/store_queue.sby` | skimmed | unaffected | +| `formal/tomasulo_wrapper.sby` | skimmed | unaffected | +| `formal/trap_unit.sby` | full | unaffected | +| `fpga/build/build.py` | targeted | unaffected | +| `fpga/build/build_step.tcl` | targeted | unaffected | +| `fpga/common/hw_defaults.py` | full | unaffected | +| `fpga/common/hw_target.py` | skimmed | unaffected | +| `fpga/hw_regression.py` | full | mechanical-only | +| `fpga/linux_boot_soak.py` | full | unaffected | +| `fpga/load_software/file_to_bram.tcl` | targeted | unaffected | +| `fpga/load_software/file_to_ddr.tcl` | targeted | unaffected | +| `fpga/load_software/load_software.py` | targeted | unaffected | +| `fpga/load_software/load_software.tcl` | targeted | unaffected | +| `hw/rtl/README.md` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/README.md` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/branch_recovery/branch_resolution.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/branch_recovery/early_misprediction_recovery.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/branch_recovery/misprediction_flush_controller.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/commit/commit_actions.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv` | targeted | design-work | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/frontend_control/frontend_validity_tracker.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/cached_tier_adapter.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/data_mem_request_router.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/perf/README.md` | targeted | design-work | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/perf/perf_counter_aggregator.sv` | targeted | unaffected | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/pipeline_control/ooo_pipeline_control.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/recovery/ex_comb_synthesizer.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/cpu_ooo/register_files/ooo_register_files.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/alu/divider.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/alu/multiplier.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/branch_jump_unit.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/dsp_tiled_multiplier_unsigned.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_adder.sv` | targeted | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_classify.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_classify_operand.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_compare.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert_sd.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_divider.sv` | targeted | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_fma.sv` | targeted | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_lzc.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_multiplier.sv` | targeted | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_operand_unpacker.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_result_assembler.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_sign_inject.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_sqrt.sv` | targeted | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_subnorm_shift.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_adder_unit.sv` | skimmed | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_classify_unit.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_compare_unit.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_convert_unit.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_div_sqrt_unit.sv` | skimmed | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_fma_unit.sv` | skimmed | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_mult_unit.sv` | skimmed | unaffected | +| `hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_sign_inject_unit.sv` | skimmed | unaffected | +| `hw/rtl/cpu_and_mem/cpu/id_stage/branch_target_precompute.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/id_stage/id_stage.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/id_stage/immediate_decoder.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/id_stage/instr_decoder.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/id_stage/instruction_type_decoder.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_prediction_controller.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/direction_predictor.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/prediction_metadata_tracker.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/ras_detector.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/return_address_stack.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/c_ext_state.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/instruction_aligner.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/rvc_decompressor.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/if_stage/control_flow_tracker.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/if_stage/pc_controller.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/if_stage/pc_increment_calculator.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/if_stage/pc_reg_precompute.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/README.md` | targeted | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/cdb_arbiter/cdb_arbiter.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/dispatch/README.md` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/dispatch/dispatch.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_cdb_adapter/fu_cdb_adapter.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/README.md` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_add_shim.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_div_shim.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/fp_mul_shim.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_alu_shim.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/fu_shims/int_muldiv_shim.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv` | targeted | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_unit.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_issue_selector.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_l0_cache.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/register_alias_table/register_alias_table.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/rob_serializer.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/reservation_station/reservation_station.sv` | targeted | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/reservation_station/rs_issue2_selector.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/README.md` | targeted | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/sq_forwarding_unit.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/store_queue.sv` | targeted | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/README.md` | targeted | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/atomics/sc_pending_unit.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/commit_bus/commit_bus_pipeline.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/dispatch_routing/dispatch_rs_router.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/perf/tomasulo_perf_counters.sv` | targeted | unaffected | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/store_addr/sq_early_addr_pipeline.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv` | targeted | design-work | +| `hw/rtl/cpu_and_mem/cpu/wb_stage/generic_regfile.sv` | full | clean-parametric | +| `hw/rtl/cpu_and_mem/cpu_and_mem.sv` | full | design-work | +| `hw/rtl/cpu_and_mem/fetch_provider.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/hang_triage.sv` | full | mechanical-only | +| `hw/rtl/cpu_and_mem/imem_predecode.sv` | full | unaffected | +| `hw/rtl/cpu_and_mem/imem_predecode_line.sv` | full | unaffected | +| `hw/rtl/frost.sv` | full | mechanical-only | +| `hw/rtl/lib/cache/axi_behavioral_memory.sv` | full | unaffected | +| `hw/rtl/lib/cache/cache_perf_pkg.sv` | full | unaffected | +| `hw/rtl/lib/cache/frost_cache.sv` | full | clean-parametric | +| `hw/rtl/lib/cache/frost_cache_hierarchy.sv` | full | clean-parametric | +| `hw/rtl/lib/cache/frost_cache_test_harness.sv` | full | unaffected | +| `hw/rtl/lib/cache/line_port_arbiter.sv` | full | clean-parametric | +| `hw/rtl/lib/cache/line_port_arbiter_test_harness.sv` | full | unaffected | +| `hw/rtl/lib/cache/line_port_axi_bridge.sv` | full | mechanical-only | +| `hw/rtl/lib/fifo/dc_fifo.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/fifo/sync_dist_ram_fifo.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/mwp_dist_ram.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/mwp_dist_ram_2r.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/mwp_dist_ram_ohread.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/sdp_block_ram.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/sdp_block_ram_dc.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/sdp_dist_ram.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/sdp_dist_ram_2r.sv` | full | clean-parametric | +| `hw/rtl/lib/ram/sdp_ram_byte_en.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/tdp_bram_dc.sv` | skimmed | clean-parametric | +| `hw/rtl/lib/ram/tdp_bram_dc_byte_en.sv` | skimmed | mechanical-only | +| `hw/rtl/peripherals/uart_rx.sv` | full | unaffected | +| `hw/rtl/peripherals/uart_tx.sv` | full | unaffected | +| `hw/sim/cpu_tb.sv` | full | design-work | +| `linux/README.md` | targeted | mechanical-only | +| `linux/build/Makefile` | full | unaffected | +| `linux/buildroot-external/README.md` | targeted | mechanical-only | +| `linux/buildroot-external/board/frost/build_fpga_boot.py` | full | design-work | +| `linux/buildroot-external/board/frost/linux-nommu-base.config` | targeted | mechanical-only | +| `linux/buildroot-external/board/frost/linux-nommu-frost.config.fragment` | targeted | mechanical-only | +| `linux/buildroot-external/board/frost/patch_linux_image.py` | targeted | design-work | +| `linux/buildroot-external/board/frost/post-image.sh` | full | mechanical-only | +| `linux/buildroot-external/configs/frost_nommu_rv32_defconfig` | full | mechanical-only | +| `linux/buildroot-external/external.mk` | full | unaffected | +| `linux/buildroot-external/package/frost-stress/frost-stress.mk` | full | unaffected | +| `linux/buildroot-external/package/frost-stress/src/frost_stress.c` | targeted | design-work | +| `scripts/frost.py` | targeted | unaffected | +| `sw/CONTRIBUTING.md` | targeted | mechanical-only | +| `sw/FreeRTOS-Kernel (port wiring only)` | skimmed | unaffected | +| `sw/README.md` | targeted | mechanical-only | +| `sw/apps/* Makefiles (9 with hardcoded ARCH/ABI: branch_pred_test, c_ext_test, ras_test, fpu_assembly_test, cf_ext_test, fetch_stall_repro, arch_test, riscv_tests, riscv_torture, coremark_pro; coremark MABI)` | targeted | mechanical-only | +| `sw/apps/c_ext_test/c_ext_test.S` | targeted | design-work | +| `sw/apps/compile_app.py, build_all_apps.py, software_registry.py` | skimmed | unaffected | +| `sw/apps/freertos_demo/ (Makefile, portmacro.h, port_frost.c, port_frost_asm.S)` | targeted | design-work | +| `sw/apps/hello_world/Makefile` | full | unaffected | +| `sw/apps/isa_test/isa_test.c` | targeted | design-work | +| `sw/apps/linux_boot/Makefile` | targeted | mechanical-only | +| `sw/apps/umode_test/main.c` | targeted | design-work | +| `sw/common/common.mk` | full | mechanical-only | +| `sw/common/crt0.S` | full | clean-parametric | +| `sw/common/crt0_ddr_boot.S` | full | clean-parametric | +| `sw/common/generate_imem_predecode_init.py` | full | design-work | +| `sw/common/link.ld` | full | mechanical-only | +| `sw/common/link_ddr.ld` | full | mechanical-only | +| `sw/common/standalone_asm.mk` | full | mechanical-only | +| `sw/lib/include/csr.h` | full | design-work | +| `sw/lib/include/limits.h` | targeted | mechanical-only | +| `sw/lib/include/mmio.h` | targeted | unaffected | +| `sw/lib/include/timer.h` | targeted | unaffected | +| `sw/lib/include/trap.h` | full | design-work | +| `sw/lib/src/memory.c` | targeted | clean-parametric | +| `sw/lib/src/sprintf.c` | targeted | clean-parametric | +| `sw/lib/src/string.c` | targeted | clean-parametric | +| `tests/Makefile` | full | unaffected | +| `tests/check_linux_boot_regression.py` | full | unaffected | +| `tests/conftest.py` | full | unaffected | +| `tests/test_arch_compliance.py` | full | design-work | +| `tests/test_riscv_tests.py` | full | design-work | +| `tests/test_riscv_torture.py` | full | design-work | +| `tests/test_run_cocotb.py` | targeted | mechanical-only | +| `tests/test_run_formal.py` | full | unaffected | +| `tests/test_run_yosys.py` | full | unaffected | +| `tests/test_runner_helpers.py` | full | mechanical-only | +| `verif/cocotb_tests/cache/test_fence_speed.py` | full | unaffected | +| `verif/cocotb_tests/cache/test_frost_cache.py` | full | unaffected | +| `verif/cocotb_tests/cache/test_line_port_arbiter.py` | full | unaffected | +| `verif/cocotb_tests/control/test_trap_unit.py` | targeted | mechanical-only | +| `verif/cocotb_tests/cpu_model.py` | targeted | design-work | +| `verif/cocotb_tests/cpu_ooo/** (commit, frontend, memory, perf, pipeline_control, recovery, register_files tests)` | skimmed | mechanical-only | +| `verif/cocotb_tests/ex_stage/test_branch_jump_unit.py` | full | design-work | +| `verif/cocotb_tests/id_stage/test_id_stage.py` | full | design-work | +| `verif/cocotb_tests/if_stage/**, pd_stage/**, id_stage/**, ex_stage/**, predecode/**, cache/**` | not-read | unknown | +| `verif/cocotb_tests/if_stage/branch_prediction/test_branch_prediction_controller.py` | full | unaffected | +| `verif/cocotb_tests/if_stage/branch_prediction/test_branch_predictor.py` | full | mechanical-only | +| `verif/cocotb_tests/if_stage/branch_prediction/test_direction_predictor.py` | full | unaffected | +| `verif/cocotb_tests/if_stage/branch_prediction/test_prediction_metadata_tracker.py` | full | unaffected | +| `verif/cocotb_tests/if_stage/branch_prediction/test_ras_detector.py` | full | design-work | +| `verif/cocotb_tests/if_stage/branch_prediction/test_return_address_stack.py` | full | unaffected | +| `verif/cocotb_tests/if_stage/test_c_ext_state.py` | full | unaffected | +| `verif/cocotb_tests/if_stage/test_control_flow_tracker.py` | full | unaffected | +| `verif/cocotb_tests/if_stage/test_if_stage.py` | full | design-work | +| `verif/cocotb_tests/if_stage/test_instruction_aligner.py` | full | design-work | +| `verif/cocotb_tests/if_stage/test_pc_controller.py` | full | clean-parametric | +| `verif/cocotb_tests/if_stage/test_pc_increment_calculator.py` | full | unaffected | +| `verif/cocotb_tests/if_stage/test_rvc_decompressor.py` | full | design-work | +| `verif/cocotb_tests/instruction_executor.py` | skimmed | mechanical-only | +| `verif/cocotb_tests/instruction_generator.py` | targeted | design-work | +| `verif/cocotb_tests/pd_stage/test_pd_stage.py` | full | design-work | +| `verif/cocotb_tests/predecode/test_fetch_provider.py` | full | mechanical-only | +| `verif/cocotb_tests/predecode/test_imem_predecode_fast_replica.py` | full | design-work | +| `verif/cocotb_tests/predecode/test_imem_predecode_line.py` | full | design-work | +| `verif/cocotb_tests/test_common.py` | skimmed | mechanical-only | +| `verif/cocotb_tests/test_compressed.py` | targeted | design-work | +| `verif/cocotb_tests/test_cpu.py` | targeted | mechanical-only | +| `verif/cocotb_tests/test_directed_atomics.py` | skimmed | mechanical-only | +| `verif/cocotb_tests/test_directed_multicycle.py` | skimmed | design-work | +| `verif/cocotb_tests/test_directed_traps.py` | targeted | mechanical-only | +| `verif/cocotb_tests/test_helpers.py` | targeted | mechanical-only | +| `verif/cocotb_tests/test_real_program.py` | targeted | mechanical-only | +| `verif/cocotb_tests/test_state.py` | full | design-work | +| `verif/cocotb_tests/tomasulo/ (dispatch, store_queue, load_queue, reorder_buffer, rat, fu_shims, cdb_arbiter, fu_cdb_adapter, tomasulo_wrapper interfaces/models)` | skimmed | mechanical-only | +| `verif/cocotb_tests/tomasulo/reservation_station/rs_interface.py` | targeted | mechanical-only | +| `verif/config.py` | full | design-work | +| `verif/encoders/__init__.py` | skimmed | unaffected | +| `verif/encoders/compressed_encode.py` | full | design-work | +| `verif/encoders/instruction_encode.py` | full | design-work | +| `verif/encoders/op_tables.py` | full | design-work | +| `verif/exceptions.py` | full | unaffected | +| `verif/models/__init__.py` | skimmed | unaffected | +| `verif/models/alu_model.py` | full | design-work | +| `verif/models/branch_model.py` | full | mechanical-only | +| `verif/models/fp_model.py` | full | design-work | +| `verif/models/memory_model.py` | full | design-work | +| `verif/monitors/__init__.py` | skimmed | unaffected | +| `verif/monitors/monitors.py` | full | mechanical-only | +| `verif/utils/__init__.py` | not-read | unknown | +| `verif/utils/instruction_logger.py` | full | mechanical-only | +| `verif/utils/memory_utils.py` | full | design-work | +| `verif/utils/riscv_utils.py` | full | design-work | +| `verif/utils/validation.py` | full | unaffected | +| `verif/verification_types.py` | full | mechanical-only | diff --git a/fpga/build/build.py b/fpga/build/build.py index 5e38afa8..31176777 100755 --- a/fpga/build/build.py +++ b/fpga/build/build.py @@ -417,6 +417,7 @@ def compile_hello_world(project_root: Path, output_dir: Path, clock_freq: int) - outputs = { "EXECUTABLE_ELF_FILE": output_dir / "sw.elf", "VERILOG_HEX_FILE": output_dir / "sw.mem", + "DWORD_HEX_FILE": output_dir / "sw64.mem", "RAW_BINARY_FILE": output_dir / "sw.bin", "VIVADO_BRAM_FILE": output_dir / "sw.txt", "DISASSEMBLY_FILE": output_dir / "sw.S", diff --git a/hw/rtl/README.md b/hw/rtl/README.md index 15a215f9..87d0f3dc 100644 --- a/hw/rtl/README.md +++ b/hw/rtl/README.md @@ -177,6 +177,15 @@ MMIO registers: | `0x4001_4000`/`4004` | CLINT MTIMECMP_LO/HI | SiFive CLINT alias of MTIMECMP | | `0x4001_BFF8`/`BFFC` | CLINT MTIME_LO/HI | SiFive CLINT alias of MTIME | +The MMIO bus rides the 64-bit data tier +([docs/rv64/m1_data_tier.md](../../docs/rv64/m1_data_tier.md)): registers +appear in their address-matching lanes of the aligned dword. The +dword-aligned CLINT pairs support native 64-bit access — an 8-byte load of +`mtime` (`0x4001_BFF8`) returns the whole counter single-copy-atomically, +and an 8-byte `mtimecmp` store lands atomically (the 32-bit lo/hi aliases +keep their word semantics). UART and FIFO registers are 32-bit-access-max: +a wider store writes only the addressed word lanes. + The hardware UART console is configured for 115200 baud, 8 data bits, no parity, and 1 stop bit (8N1). diff --git a/hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv b/hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv index 492a6236..81452c40 100644 --- a/hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv +++ b/hw/rtl/cpu_and_mem/cpu/control/trap_unit.sv @@ -77,7 +77,7 @@ * - ooo_pipeline_control.sv: Registers trap/MRET pulses for front-end flush */ module trap_unit #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, input logic i_rst, @@ -447,23 +447,30 @@ module trap_unit #( // Trap target: mtvec for trap entry, mepc for MRET // mtvec MODE (bits [1:0]): 0 = Direct (all traps go to BASE) // 1 = Vectored (interrupts go to BASE + 4*cause) + logic [XLEN-1:0] trap_target_selected; always_comb begin if (take_mret) begin - o_trap_target = i_mepc; + trap_target_selected = i_mepc; end else if (take_trap) begin // Check mtvec mode if (i_mtvec[1:0] == 2'b01 && interrupt_pending_eligible) begin // Vectored mode for interrupts: BASE + 4*cause_code // Use pre-computed small offset (6 bits) for faster timing than // extracting from full interrupt_cause which synthesis can't optimize - o_trap_target = {i_mtvec[XLEN-1:2], 2'b00} + {26'b0, vectored_offset}; + trap_target_selected = {i_mtvec[XLEN-1:2], 2'b00} + {26'b0, vectored_offset}; end else begin // Direct mode: all traps go to BASE (aligned to 4 bytes) - o_trap_target = {i_mtvec[XLEN-1:2], 2'b00}; + trap_target_selected = {i_mtvec[XLEN-1:2], 2'b00}; end end else begin - o_trap_target = '0; + trap_target_selected = '0; end + + // Canonicalize the redirect to the physical address space (identity at + // XLEN=32). mepc/mtvec themselves keep full-width storage in csr_file + // (WARL round-trip fidelity); only the fetch redirect derived from them + // is masked - plan decision D3. + o_trap_target = riscv_pkg::canonical_paddr(trap_target_selected); end // Trap entry information for CSR file diff --git a/hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv b/hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv index d025a462..649961f7 100644 --- a/hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv +++ b/hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.sv @@ -47,7 +47,10 @@ module cpu_ooo #( input logic [riscv_pkg::ImemFetchSidebandWidth-1:0] i_instr_sideband, input logic [1:0] i_instr_hi_rd_is_x2, // {next,current} high-parcel predicates input logic i_instr_bank_sel_r, // Fetch-word parity (for spanning select) - input logic [31:0] i_served_addr, // Selected served-window address tag + // Selected served-window address tag. XLEN-wide to match if_stage's + // consumer port; the cpu_and_mem driver stays a 32-bit physical address + // (zero-extended here at XLEN=64, canonical by construction). + input logic [XLEN-1:0] i_served_addr, input logic [XLEN-3:0] i_served_last_word, // Selected payload's registered S+1 word // Fetch window valid (see if_stage). Tie 1 for fixed 1-cycle providers. input logic i_instr_valid, @@ -66,31 +69,31 @@ module cpu_ooo #( input logic i_fence_i_sync_done, output logic o_fence_i_flush, // Data memory interface - input logic [XLEN-1:0] i_data_mem_rd_data, + input logic [riscv_pkg::MemDataBits-1:0] i_data_mem_rd_data, output logic [XLEN-1:0] o_data_mem_addr, - output logic [XLEN-1:0] o_data_mem_wr_data, - output logic [3:0] o_data_mem_per_byte_wr_en, + output logic [riscv_pkg::MemDataBits-1:0] o_data_mem_wr_data, + output logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_per_byte_wr_en, // BRAM-only byte-write-enable. Identical to o_data_mem_per_byte_wr_en // except MMIO-targeted stores are masked out at the SQ/AMO source using // their registered is_mmio flag. Breaks the issued_idx → WEA timing path // by keeping the address-range MMIO check out of the BRAM write-enable // combinational cone. Peripherals still consume the unmasked signal so // MMIO writes remain visible to UART/FIFO/timer logic. - output logic [3:0] o_data_mem_bram_byte_wr_en, + output logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_bram_byte_wr_en, output logic o_data_mem_read_enable, // Cached tier (high-address region). Tier-routed write/read requests // (already qualified by is_cached in the router) plus the handshake // completion inputs from the cached_tier_adapter. - output logic [3:0] o_data_mem_cached_byte_wr_en, + output logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_cached_byte_wr_en, // Cached-tier write data: SQ-store drain data, or the AMO new value on the // single cycle a cached AMO read-modify-write is launched to the adapter. // Driven by the router, which owns the SQ-vs-AMO cached-write mux. The mux // sits on the cached-only write-data path (not the wide BRAM write-data // cascade that was the old post-opt timing offender), and the AMO ALU cone // only reaches it through the rare, ROB-head-serialized cached AMO. - output logic [XLEN-1:0] o_data_mem_cached_wr_data, + output logic [riscv_pkg::MemDataBits-1:0] o_data_mem_cached_wr_data, output logic o_data_mem_cached_read_enable, - input logic [XLEN-1:0] i_cached_read_data, + input logic [riscv_pkg::MemDataBits-1:0] i_cached_read_data, input logic i_cached_read_valid, input logic i_cached_write_done, input logic i_cached_write_inflight, @@ -1034,8 +1037,9 @@ module cpu_ooo #( // Memory interfaces logic sq_mem_write_en; - logic [XLEN-1:0] sq_mem_write_addr, sq_mem_write_data; - logic [3:0] sq_mem_write_byte_en; + logic [XLEN-1:0] sq_mem_write_addr; + logic [riscv_pkg::MemDataBits-1:0] sq_mem_write_data; + logic [riscv_pkg::MemStrbBits-1:0] sq_mem_write_byte_en; logic sq_mem_write_is_mmio; // Registered cached-tier flag for the SQ write (parallels is_mmio). Used by // the router to steer the store's byte-write enables to the cached tier and @@ -1048,14 +1052,15 @@ module cpu_ooo #( logic lq_mem_addr_valid; logic [XLEN-1:0] lq_mem_read_addr; riscv_pkg::mem_size_e lq_mem_read_size; - logic [XLEN-1:0] lq_mem_read_data; + logic [riscv_pkg::MemDataBits-1:0] lq_mem_read_data; logic lq_mem_read_valid; logic lq_mem_request_valid; logic lq_mem_request_fire; // AMO memory interface logic amo_mem_write_en; - logic [XLEN-1:0] amo_mem_write_addr, amo_mem_write_data; + logic [XLEN-1:0] amo_mem_write_addr; + logic [riscv_pkg::MemDataBits-1:0] amo_mem_write_data; logic amo_mem_write_done; // RS issue (exposed but not externally driven — FU shims are inside wrapper) diff --git a/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/cached_tier_adapter.sv b/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/cached_tier_adapter.sv index 6b56c023..1c0febcd 100644 --- a/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/cached_tier_adapter.sv +++ b/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/cached_tier_adapter.sv @@ -15,13 +15,17 @@ */ /* - * cached_tier_adapter -- word<->line adapter between the data-memory request + * cached_tier_adapter -- beat<->line adapter between the data-memory request * router and the cache hierarchy (frost_cache_hierarchy upstream port). * + * The router side moves one MemDataBits (64-bit) beat per transaction with + * MemStrbBits byte strobes, carrying the aligned-dword view defined in + * docs/rv64/m1_data_tier.md; the line side is unchanged 256-bit lines. + * * Router-side protocol (handshake, variable-latency completion): * - i_read_req: 1-cycle pulse, an accepted cached-region load. The address * is on i_req_addr that cycle. Completion: o_read_valid pulse with - * o_read_data (the addressed word), any number of cycles later. + * o_read_data (the addressed beat), any number of cycles later. * - i_write_byte_en != 0: a cached-region store fired this cycle (addr/data * on i_req_addr/i_write_data). Completion: o_write_done pulse. * o_write_inflight stays high from the cycle AFTER the fire until the done @@ -29,10 +33,10 @@ * a cached store is pending (the fire cycle itself is covered by * sq_mem_write_en), preserving load-vs-store ordering on the port. * - * Word<->line conversion: a CPU read becomes a full-line read and the - * addressed word is muxed out of the 256-bit response. A CPU write becomes a - * line write with the word replicated across every lane and the 4 byte - * strobes shifted to the addressed lane (the cache merges on a miss). + * Beat<->line conversion: a CPU read becomes a full-line read and the + * addressed beat is muxed out of the 256-bit response. A CPU write becomes a + * line write with the beat replicated across every lane and the byte strobes + * shifted to the addressed lane (the cache merges on a miss). * * Serialization: at most one line request in flight. One pending-read slot + * one pending-write slot; when both are occupied the read is always the older @@ -41,28 +45,28 @@ * flight), so reads are served first. Invariants are assertion-checked in * simulation AND hardware-refused: a request arriving while its slot is * still pending is dropped (deterministic upstream stall) rather than - * absorbed, and the serving read's word select is snapshotted at launch, so + * absorbed, and the serving read's beat select is snapshotted at launch, so * an upstream gate regression can no longer silently corrupt the slot or - * return the wrong word as valid data. + * return the wrong beat as valid data. */ module cached_tier_adapter #( - parameter int unsigned XLEN = 32, + parameter int unsigned XLEN = riscv_pkg::XLEN, parameter int unsigned LINE_BYTES = 32 ) ( input logic i_clk, input logic i_rst, - // Router-facing request side. - input logic i_read_req, - input logic [XLEN-1:0] i_req_addr, - input logic [ 3:0] i_write_byte_en, - input logic [XLEN-1:0] i_write_data, + // Router-facing request side (one aligned beat per transaction). + input logic i_read_req, + input logic [ XLEN-1:0] i_req_addr, + input logic [riscv_pkg::MemStrbBits-1:0] i_write_byte_en, + input logic [riscv_pkg::MemDataBits-1:0] i_write_data, // Router-facing completion side. - output logic [XLEN-1:0] o_read_data, - output logic o_read_valid, - output logic o_write_done, - output logic o_write_inflight, + output logic [riscv_pkg::MemDataBits-1:0] o_read_data, + output logic o_read_valid, + output logic o_write_done, + output logic o_write_inflight, // Line port master (to the cache hierarchy). output logic o_line_req_valid, @@ -75,20 +79,22 @@ module cached_tier_adapter #( input logic [LINE_BYTES*8-1:0] i_line_resp_rdata ); - localparam int unsigned LineBits = LINE_BYTES * 8; - localparam int unsigned WordsPerLine = LINE_BYTES / (XLEN / 8); - localparam int unsigned WordSelBits = $clog2(WordsPerLine); + localparam int unsigned BeatBits = riscv_pkg::MemDataBits; + localparam int unsigned BeatStrbBits = riscv_pkg::MemStrbBits; + localparam int unsigned BeatOffBits = $clog2(BeatStrbBits); // addr bits below the beat index + localparam int unsigned BeatsPerLine = LINE_BYTES / BeatStrbBits; + localparam int unsigned BeatSelBits = $clog2(BeatsPerLine); localparam int unsigned OffsetBits = $clog2(LINE_BYTES); // ---- Pending request slots ------------------------------------------------- - logic pending_read_valid; - logic [XLEN-1:0] pending_read_addr; - logic pending_write_valid; - logic [XLEN-1:0] pending_write_addr; - logic [XLEN-1:0] pending_write_data; - logic [ 3:0] pending_write_byte_en; - - logic write_fire; + logic pending_read_valid; + logic [ XLEN-1:0] pending_read_addr; + logic pending_write_valid; + logic [ XLEN-1:0] pending_write_addr; + logic [ BeatBits-1:0] pending_write_data; + logic [BeatStrbBits-1:0] pending_write_byte_en; + + logic write_fire; assign write_fire = |i_write_byte_en; // ---- Issue FSM: one line transaction in flight ------------------------------ @@ -104,29 +110,30 @@ module cached_tier_adapter #( assign o_line_req_addr = issue_write ? {pending_write_addr[XLEN-1:OffsetBits], {OffsetBits{1'b0}}} : {pending_read_addr[XLEN-1:OffsetBits], {OffsetBits{1'b0}}}; - // Word replicated across every lane; the strobes select the addressed lanes. - assign o_line_req_wdata = {WordsPerLine{pending_write_data}}; + // Beat replicated across every lane; the strobes select the addressed lanes. + assign o_line_req_wdata = {BeatsPerLine{pending_write_data}}; always_comb begin o_line_req_wstrb = '0; - o_line_req_wstrb[pending_write_addr[OffsetBits-1:2]*4+:4] = pending_write_byte_en; + o_line_req_wstrb[pending_write_addr[OffsetBits-1:BeatOffBits]*BeatStrbBits+:BeatStrbBits] = + pending_write_byte_en; end logic line_req_fire; assign line_req_fire = o_line_req_valid && i_line_req_ready; - // Word select for the read response, captured from the pending read address. - logic [WordSelBits-1:0] read_word_sel; - assign read_word_sel = pending_read_addr[2+:WordSelBits]; + // Beat select for the read response, captured from the pending read address. + logic [BeatSelBits-1:0] read_beat_sel; + assign read_beat_sel = pending_read_addr[BeatOffBits+:BeatSelBits]; - // HARDENING: the serving transaction's word select, snapshotted at launch. + // HARDENING: the serving transaction's beat select, snapshotted at launch. // The upstream gates (LQ slow_outstanding, router write_port_busy) are the // real single-outstanding guarantee, but if they ever regress and a second // read overwrites pending_read_addr mid-flight, muxing the response through - // the LIVE address would silently return the WRONG WORD as valid data. + // the LIVE address would silently return the WRONG BEAT as valid data. // Snapshotting at line_req_fire makes the in-flight response immune; the // duplicate request itself is refused below (deterministic stall, caught by // the sim assertions) instead of corrupting state. - logic [WordSelBits-1:0] serving_word_sel_q; + logic [BeatSelBits-1:0] serving_beat_sel_q; always_ff @(posedge i_clk) begin if (i_rst) begin @@ -160,7 +167,7 @@ module cached_tier_adapter #( if (line_req_fire) begin busy_q <= 1'b1; serving_read_q <= issue_read; - serving_word_sel_q <= read_word_sel; + serving_beat_sel_q <= read_beat_sel; end // Retire on the line response. @@ -169,7 +176,7 @@ module cached_tier_adapter #( if (serving_read_q) begin pending_read_valid <= 1'b0; o_read_valid <= 1'b1; - o_read_data <= i_line_resp_rdata[serving_word_sel_q*XLEN+:XLEN]; + o_read_data <= i_line_resp_rdata[serving_beat_sel_q*BeatBits+:BeatBits]; end else begin pending_write_valid <= 1'b0; o_write_done <= 1'b1; diff --git a/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/data_mem_request_router.sv b/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/data_mem_request_router.sv index 885cfaff..2013305b 100644 --- a/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/data_mem_request_router.sv +++ b/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/data_mem_request_router.sv @@ -48,18 +48,18 @@ module data_mem_request_router #( input logic i_rst, // Store-queue write request (highest priority). - input logic i_sq_mem_write_en, - input logic [XLEN-1:0] i_sq_mem_write_addr, - input logic [XLEN-1:0] i_sq_mem_write_data, - input logic [ 3:0] i_sq_mem_write_byte_en, - input logic i_sq_mem_write_is_mmio, + input logic i_sq_mem_write_en, + input logic [ XLEN-1:0] i_sq_mem_write_addr, + input logic [riscv_pkg::MemDataBits-1:0] i_sq_mem_write_data, + input logic [riscv_pkg::MemStrbBits-1:0] i_sq_mem_write_byte_en, + input logic i_sq_mem_write_is_mmio, // Registered cached-tier flag for the SQ write (parallels is_mmio). - input logic i_sq_mem_write_is_cached, + input logic i_sq_mem_write_is_cached, // Atomic-unit write request. - input logic i_amo_mem_write_en, - input logic [XLEN-1:0] i_amo_mem_write_addr, - input logic [XLEN-1:0] i_amo_mem_write_data, + input logic i_amo_mem_write_en, + input logic [ XLEN-1:0] i_amo_mem_write_addr, + input logic [riscv_pkg::MemDataBits-1:0] i_amo_mem_write_data, // Load-queue read request. input logic i_lq_mem_read_en, @@ -68,52 +68,54 @@ module data_mem_request_router #( // External data memory read data (BRAM, combinational the cycle after a read // is accepted; the cpu_and_mem mux folds in registered MMIO read data). - input logic [XLEN-1:0] i_data_mem_rd_data, + input logic [riscv_pkg::MemDataBits-1:0] i_data_mem_rd_data, // Cached-tier completion (from cached_tier_adapter): handshake pulses with // variable latency, plus the write-inflight hold. - input logic [XLEN-1:0] i_cached_read_data, - input logic i_cached_read_valid, - input logic i_cached_write_done, - input logic i_cached_write_inflight, + input logic [riscv_pkg::MemDataBits-1:0] i_cached_read_data, + input logic i_cached_read_valid, + input logic i_cached_write_done, + input logic i_cached_write_inflight, // External data memory port. - output logic [XLEN-1:0] o_data_mem_addr, - output logic [XLEN-1:0] o_data_mem_wr_data, - output logic [ 3:0] o_data_mem_per_byte_wr_en, - output logic [ 3:0] o_data_mem_bram_byte_wr_en, - output logic o_data_mem_read_enable, + output logic [ XLEN-1:0] o_data_mem_addr, + output logic [riscv_pkg::MemDataBits-1:0] o_data_mem_wr_data, + output logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_per_byte_wr_en, + output logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_bram_byte_wr_en, + output logic o_data_mem_read_enable, // Cached-tier write/read requests (asserted only for cached-range accesses). - output logic [ 3:0] o_data_mem_cached_byte_wr_en, + output logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_cached_byte_wr_en, // Cached-tier write data. SQ-store drain data normally; the AMO new value // on the single cycle a cached AMO write is launched to the adapter. - output logic [XLEN-1:0] o_data_mem_cached_wr_data, - output logic o_data_mem_cached_read_enable, - output logic o_mmio_read_pulse, - output logic [XLEN-1:0] o_mmio_load_addr, - output logic o_mmio_load_valid, - output logic o_mmio_fifo0_read_pulse, - output logic o_mmio_fifo1_read_pulse, - output logic o_mmio_uart_rx_ready_pulse, + output logic [riscv_pkg::MemDataBits-1:0] o_data_mem_cached_wr_data, + output logic o_data_mem_cached_read_enable, + output logic o_mmio_read_pulse, + output logic [ XLEN-1:0] o_mmio_load_addr, + output logic o_mmio_load_valid, + output logic o_mmio_fifo0_read_pulse, + output logic o_mmio_fifo1_read_pulse, + output logic o_mmio_uart_rx_ready_pulse, // Status back to SQ / AMO / LQ. - output logic o_sq_mem_write_done, - output logic o_amo_mem_write_done, - output logic o_lq_mem_request_valid, - output logic [XLEN-1:0] o_lq_mem_read_data, - output logic o_lq_mem_read_valid + output logic o_sq_mem_write_done, + output logic o_amo_mem_write_done, + output logic o_lq_mem_request_valid, + output logic [riscv_pkg::MemDataBits-1:0] o_lq_mem_read_data, + output logic o_lq_mem_read_valid ); // --- Port aliases: keep the body close to the original extracted form. - logic sq_mem_write_en; - logic [XLEN-1:0] sq_mem_write_addr, sq_mem_write_data; - logic [3:0] sq_mem_write_byte_en; - logic sq_mem_write_is_mmio; - logic sq_mem_write_is_cached; - logic amo_mem_write_en; - logic [XLEN-1:0] amo_mem_write_addr, amo_mem_write_data; - logic lq_mem_read_en; - logic [XLEN-1:0] lq_mem_read_addr; - logic lq_mem_addr_valid; + logic sq_mem_write_en; + logic [ XLEN-1:0] sq_mem_write_addr; + logic [riscv_pkg::MemDataBits-1:0] sq_mem_write_data; + logic [riscv_pkg::MemStrbBits-1:0] sq_mem_write_byte_en; + logic sq_mem_write_is_mmio; + logic sq_mem_write_is_cached; + logic amo_mem_write_en; + logic [ XLEN-1:0] amo_mem_write_addr; + logic [riscv_pkg::MemDataBits-1:0] amo_mem_write_data; + logic lq_mem_read_en; + logic [ XLEN-1:0] lq_mem_read_addr; + logic lq_mem_addr_valid; assign sq_mem_write_en = i_sq_mem_write_en; assign sq_mem_write_addr = i_sq_mem_write_addr; assign sq_mem_write_data = i_sq_mem_write_data; @@ -127,27 +129,29 @@ module data_mem_request_router #( assign lq_mem_read_addr = i_lq_mem_read_addr; assign lq_mem_addr_valid = i_lq_mem_addr_valid; - // Router-internal state / nets. - localparam logic [XLEN-1:0] UartRxDataMmioAddr = MMIO_ADDR[XLEN-1:0] + XLEN'(32'h4); - localparam logic [XLEN-1:0] Fifo0MmioAddr = MMIO_ADDR[XLEN-1:0] + XLEN'(32'h8); - localparam logic [XLEN-1:0] Fifo1MmioAddr = MMIO_ADDR[XLEN-1:0] + XLEN'(32'hC); - - logic sq_write_done_fast; - logic write_port_busy; - logic amo_mem_write_done; - logic lq_mem_request_valid; - logic [XLEN-1:0] lq_mem_request_addr; - logic [XLEN-1:0] lq_mem_request_addr_eff; - logic [XLEN-1:0] lq_mem_read_data; - logic lq_mem_read_valid; - logic lq_mem_request_is_mmio; + // Router-internal state / nets. XLEN'() casts, not [XLEN-1:0] + // part-selects: MMIO_ADDR is a 32-bit int parameter, so a 64-bit + // part-select of it would be out of range. + localparam logic [XLEN-1:0] UartRxDataMmioAddr = XLEN'(MMIO_ADDR) + XLEN'(32'h4); + localparam logic [XLEN-1:0] Fifo0MmioAddr = XLEN'(MMIO_ADDR) + XLEN'(32'h8); + localparam logic [XLEN-1:0] Fifo1MmioAddr = XLEN'(MMIO_ADDR) + XLEN'(32'hC); + + logic sq_write_done_fast; + logic write_port_busy; + logic amo_mem_write_done; + logic lq_mem_request_valid; + logic [ XLEN-1:0] lq_mem_request_addr; + logic [ XLEN-1:0] lq_mem_request_addr_eff; + logic [riscv_pkg::MemDataBits-1:0] lq_mem_read_data; + logic lq_mem_read_valid; + logic lq_mem_request_is_mmio; // Effective queued-load address: held copy if a request is pending, else the // live LQ read address. assign lq_mem_request_addr_eff = lq_mem_request_valid ? lq_mem_request_addr : lq_mem_read_addr; assign lq_mem_request_is_mmio = - (lq_mem_request_addr_eff >= MMIO_ADDR[XLEN-1:0]) && - (lq_mem_request_addr_eff < (MMIO_ADDR[XLEN-1:0] + MMIO_SIZE_BYTES[XLEN-1:0])); + (lq_mem_request_addr_eff >= XLEN'(MMIO_ADDR)) && + (lq_mem_request_addr_eff < (XLEN'(MMIO_ADDR) + XLEN'(MMIO_SIZE_BYTES))); // AMO MMIO check: short cone from amo_entry_idx → lq_address_amo LUTRAM → // range comparison. AMOs on MMIO are undefined by spec but we preserve the @@ -156,8 +160,8 @@ module data_mem_request_router #( // amo_mem_write_addr never reaches the SQ-only path. logic amo_mem_write_is_mmio; assign amo_mem_write_is_mmio = - (amo_mem_write_addr >= MMIO_ADDR[XLEN-1:0]) && - (amo_mem_write_addr < (MMIO_ADDR[XLEN-1:0] + MMIO_SIZE_BYTES[XLEN-1:0])); + (amo_mem_write_addr >= XLEN'(MMIO_ADDR)) && + (amo_mem_write_addr < (XLEN'(MMIO_ADDR) + XLEN'(MMIO_SIZE_BYTES))); // ------------------------------------------------------------------------- // Cached-tier decode. @@ -178,13 +182,13 @@ module data_mem_request_router #( // dropped (undefined by spec; the BRAM-mask safety is preserved). logic lq_mem_request_is_cached; assign lq_mem_request_is_cached = - (lq_mem_request_addr_eff >= CACHED_BASE[XLEN-1:0]) && - (lq_mem_request_addr_eff < (CACHED_BASE[XLEN-1:0] + CACHED_SIZE_BYTES[XLEN-1:0])); + (lq_mem_request_addr_eff >= XLEN'(CACHED_BASE)) && + (lq_mem_request_addr_eff < (XLEN'(CACHED_BASE) + XLEN'(CACHED_SIZE_BYTES))); logic amo_mem_write_is_cached; assign amo_mem_write_is_cached = - (amo_mem_write_addr >= CACHED_BASE[XLEN-1:0]) && - (amo_mem_write_addr < (CACHED_BASE[XLEN-1:0] + CACHED_SIZE_BYTES[XLEN-1:0])); + (amo_mem_write_addr >= XLEN'(CACHED_BASE)) && + (amo_mem_write_addr < (XLEN'(CACHED_BASE) + XLEN'(CACHED_SIZE_BYTES))); // Cached AMO write handshake. The LQ holds i_amo_mem_write_en high for the // whole AMO write phase (until it sees o_amo_mem_write_done), but the @@ -233,7 +237,8 @@ module data_mem_request_router #( // writes must remain visible here so the registered shadow in cpu_and_mem // can dispatch them on the next cycle. o_data_mem_per_byte_wr_en = sq_mem_write_en ? sq_mem_write_byte_en : - amo_mem_write_en ? 4'b1111 : 4'b0000; + amo_mem_write_en ? + riscv_pkg::mem_strobe_for(2'b10, amo_mem_write_addr[2:0]) : '0; // BRAM-specific byte-write-enable: MMIO- AND cached-targeted stores are // pre-masked at the SQ/AMO source using registered tier flags. Keeping // these checks out of cpu_and_mem (where the old address-range test pulled @@ -245,7 +250,7 @@ module data_mem_request_router #( (sq_mem_write_en && !sq_mem_write_is_mmio && !sq_mem_write_is_cached) ? sq_mem_write_byte_en : (amo_mem_write_en && !amo_mem_write_is_mmio && !amo_mem_write_is_cached) ? - 4'b1111 : 4'b0000; + riscv_pkg::mem_strobe_for(2'b10, amo_mem_write_addr[2:0]) : '0; // Cached-tier byte-write-enable: a cached SQ store, or the single-cycle // launch pulse of a cached AMO write (word-width). The launch qualifier @@ -256,7 +261,8 @@ module data_mem_request_router #( // draining while a cached AMO write is in flight. o_data_mem_cached_byte_wr_en = (sq_mem_write_en && sq_mem_write_is_cached) ? sq_mem_write_byte_en : - amo_cached_write_launch ? 4'b1111 : 4'b0000; + amo_cached_write_launch ? + riscv_pkg::mem_strobe_for(2'b10, amo_mem_write_addr[2:0]) : '0; // Cached-tier write data: SQ-store drain data normally; the AMO new value // on the launch pulse. Off the BRAM WEA cone (separate cached-only port). diff --git a/hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv b/hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv index 7b7a1b60..bd6f9cbf 100644 --- a/hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv +++ b/hw/rtl/cpu_and_mem/cpu/csr/csr_file.sv @@ -65,7 +65,7 @@ - CSRRC/CSRRCI: Atomic read and clear bits */ module csr_file #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, input logic i_rst, diff --git a/hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv b/hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv index 4d1718bd..a5be8086 100644 --- a/hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv +++ b/hw/rtl/cpu_and_mem/cpu/ex_stage/alu/alu.sv @@ -34,7 +34,7 @@ * These use tree-based parallel structures for optimal timing. */ module alu #( - parameter int unsigned XLEN = 32, + parameter int unsigned XLEN = riscv_pkg::XLEN, parameter bit ENABLE_MULDIV = 1'b1 ) ( input logic i_clk, @@ -44,8 +44,8 @@ module alu #( input logic [XLEN-1:0] i_operand_a, // First operand (typically rs1 value) input logic [XLEN-1:0] i_operand_b, // Second operand (typically rs2 value or immediate) input logic [XLEN-1:0] i_program_counter, - input logic [31:0] i_immediate_u_type, // Upper immediate for LUI/AUIPC - input logic [31:0] i_immediate_i_type, // I-type immediate + input logic [XLEN-1:0] i_immediate_u_type, // Upper immediate for LUI/AUIPC + input logic [XLEN-1:0] i_immediate_i_type, // I-type immediate input logic i_is_multiply_operation, input logic i_is_divide_operation, input logic [XLEN-1:0] i_link_address, // Pre-computed link address (PC+2 or PC+4) @@ -230,9 +230,9 @@ module alu #( // Handle special cases per RISC-V spec if (i_operand_b == 0) o_result = riscv_pkg::NegativeOne; // Divide by zero: return -1 // Overflow: most negative number divided by -1 - else if ((i_operand_a == riscv_pkg::SignedInt32Min) && + else if ((i_operand_a == riscv_pkg::SignedIntMin) && (i_operand_b == riscv_pkg::NegativeOne)) - o_result = riscv_pkg::SignedInt32Min; // Return most negative number + o_result = riscv_pkg::SignedIntMin; // Return most negative number else o_result = divider_quotient_result; o_write_enable = divider_valid_output; end else o_write_enable = 1'b0; @@ -243,7 +243,7 @@ module alu #( divider_valid_input = ~divider_valid_input_registered; divider_is_signed_operation = 1'b0; if (i_operand_b == 0) - o_result = riscv_pkg::UnsignedInt32Max; // Divide by zero: return max unsigned + o_result = riscv_pkg::UnsignedIntMax; // Divide by zero: return max unsigned else o_result = divider_quotient_result; o_write_enable = divider_valid_output; end else o_write_enable = 1'b0; @@ -255,7 +255,7 @@ module alu #( divider_is_signed_operation = 1'b1; if (i_operand_b == 0) o_result = i_operand_a; // Remainder of divide by zero: return dividend - else if ((i_operand_a == riscv_pkg::SignedInt32Min) && + else if ((i_operand_a == riscv_pkg::SignedIntMin) && (i_operand_b == riscv_pkg::NegativeOne)) o_result = 32'h0000_0000; // Overflow case: remainder is 0 else o_result = divider_remainder_result; diff --git a/hw/rtl/cpu_and_mem/cpu/ex_stage/branch_jump_unit.sv b/hw/rtl/cpu_and_mem/cpu/ex_stage/branch_jump_unit.sv index 615c36f1..c84969cc 100644 --- a/hw/rtl/cpu_and_mem/cpu/ex_stage/branch_jump_unit.sv +++ b/hw/rtl/cpu_and_mem/cpu/ex_stage/branch_jump_unit.sv @@ -53,7 +53,7 @@ * - ex_comb_synthesizer.sv: Converts branch recovery into front-end redirects */ module branch_jump_unit #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( // Branch operation type (decoded from funct3) input riscv_pkg::branch_taken_op_e i_branch_operation, @@ -71,7 +71,7 @@ module branch_jump_unit #( input logic [XLEN-1:0] i_jal_target_precomputed, // PC + imm_j // JALR offset (I-type immediate, sign-extended) - input logic [31:0] i_immediate_i_type, + input logic [XLEN-1:0] i_immediate_i_type, // Outputs output logic o_branch_taken, // Branch/jump should be taken @@ -80,6 +80,7 @@ module branch_jump_unit #( // JALR target computed here (needs forwarded rs1 value) logic [XLEN-1:0] jalr_target; + logic [XLEN-1:0] target_selected; assign jalr_target = (i_operand_a + XLEN'(signed'(i_immediate_i_type))) & ~XLEN'(1); // Share comparators across branch types to reduce logic depth. @@ -111,10 +112,16 @@ module branch_jump_unit #( unique case ({ i_is_jump_and_link, i_is_jump_and_link_register }) - 2'b10: o_branch_target_address = i_jal_target_precomputed; // JAL: use pre-computed - 2'b01: o_branch_target_address = jalr_target; // JALR: computed here - default: o_branch_target_address = i_branch_target_precomputed; // Branch: use pre-computed + 2'b10: target_selected = i_jal_target_precomputed; // JAL: use pre-computed + 2'b01: target_selected = jalr_target; // JALR: computed here + default: target_selected = i_branch_target_precomputed; // Branch: use pre-computed endcase + + // Canonicalize to the physical address space (identity at XLEN=32; masks + // bits [63:32] at XLEN=64 so the resolved target, the misprediction + // compare against BTB/RAS-trained targets, and the redirect PC all live + // in the same sub-4-GiB space - plan decision D3). + o_branch_target_address = riscv_pkg::canonical_paddr(target_selected); end endmodule : branch_jump_unit diff --git a/hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv b/hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv index 8891089e..7f8f48f2 100644 --- a/hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv +++ b/hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fp_convert.sv @@ -41,7 +41,7 @@ - Inexact (NX): Result is not exact */ module fp_convert #( - parameter int unsigned XLEN = 32, + parameter int unsigned XLEN = riscv_pkg::XLEN, parameter int unsigned FP_WIDTH = 32 ) ( input logic i_clk, @@ -428,7 +428,12 @@ module fp_convert #( end endgenerate - assign move_int_result_s2_comb = fp_operand_reg[XLEN-1:0]; + // FMV.X.* move: at XLEN > FP_WIDTH (the S instance once XLEN=64) only + // FP_WIDTH operand bits exist, so slice the minimum and zero-extend to + // keep elaboration legal at either width. The RV64 FMV.X.W SIGN-extension + // semantic (and FMV.X.D/FMV.D.X) land with the Phase 1 conversion rework. + localparam int unsigned MoveIntWidth = (FP_WIDTH < XLEN) ? FP_WIDTH : XLEN; + assign move_int_result_s2_comb = XLEN'(fp_operand_reg[MoveIntWidth-1:0]); // ========================================================================= // Stage 3: FP->int rounding add (combinational from stage 3 regs) diff --git a/hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_convert_unit.sv b/hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_convert_unit.sv index adcdbe31..a5cefb90 100644 --- a/hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_convert_unit.sv +++ b/hw/rtl/cpu_and_mem/cpu/ex_stage/fpu/fpu_convert_unit.sv @@ -18,7 +18,7 @@ // Wraps fp_convert S + D + fp_convert_sd with a shared tracking FSM, // NaN-boxing, and dest reg capture. Handles both FP and integer results. module fpu_convert_unit #( - parameter int unsigned XLEN = 32, + parameter int unsigned XLEN = riscv_pkg::XLEN, parameter int unsigned FP_WIDTH_D = 64 ) ( input logic i_clk, diff --git a/hw/rtl/cpu_and_mem/cpu/id_stage/branch_target_precompute.sv b/hw/rtl/cpu_and_mem/cpu/id_stage/branch_target_precompute.sv index 57a23b6a..05edfa72 100644 --- a/hw/rtl/cpu_and_mem/cpu/id_stage/branch_target_precompute.sv +++ b/hw/rtl/cpu_and_mem/cpu/id_stage/branch_target_precompute.sv @@ -37,7 +37,7 @@ * EX stage comparison critical path. */ module branch_target_precompute #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( // PC and immediates for target computation input logic [XLEN-1:0] i_program_counter, diff --git a/hw/rtl/cpu_and_mem/cpu/id_stage/id_stage.sv b/hw/rtl/cpu_and_mem/cpu/id_stage/id_stage.sv index d58529b6..c53b280c 100644 --- a/hw/rtl/cpu_and_mem/cpu/id_stage/id_stage.sv +++ b/hw/rtl/cpu_and_mem/cpu/id_stage/id_stage.sv @@ -33,7 +33,7 @@ * └── branch_target_precompute - Pre-computed branch/jump targets and prediction verification */ module id_stage #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, input riscv_pkg::pipeline_ctrl_t i_pipeline_ctrl, diff --git a/hw/rtl/cpu_and_mem/cpu/id_stage/immediate_decoder.sv b/hw/rtl/cpu_and_mem/cpu/id_stage/immediate_decoder.sv index d7d1275e..a1b9f6c1 100644 --- a/hw/rtl/cpu_and_mem/cpu/id_stage/immediate_decoder.sv +++ b/hw/rtl/cpu_and_mem/cpu/id_stage/immediate_decoder.sv @@ -25,10 +25,13 @@ * - U-type: 20-bit upper immediate for LUI/AUIPC * - J-type: 21-bit signed immediate (x2) for JAL * - * All immediate values are sign-extended to 32 bits. + * All immediate values are sign-extended to XLEN bits (the replication + * counts below are XLEN-relative so the same source is correct for RV32 + * and RV64; U-type additionally replicates bit 31 on RV64 per the LUI/AUIPC + * sign-extension rule). */ module immediate_decoder #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input riscv_pkg::instr_t i_instruction, output logic [XLEN-1:0] o_immediate_i_type, @@ -41,20 +44,20 @@ module immediate_decoder #( // I-type: 12-bit immediate in bits [31:20] // Used by: loads, ALU-immediate, JALR assign o_immediate_i_type = { - {20{i_instruction.funct7[6]}}, i_instruction.funct7, i_instruction.source_reg_2 + {(XLEN - 12) {i_instruction.funct7[6]}}, i_instruction.funct7, i_instruction.source_reg_2 }; // S-type: 12-bit immediate split between bits [31:25] and [11:7] // Used by: stores assign o_immediate_s_type = { - {20{i_instruction.funct7[6]}}, i_instruction.funct7, i_instruction.dest_reg + {(XLEN - 12) {i_instruction.funct7[6]}}, i_instruction.funct7, i_instruction.dest_reg }; // B-type: 13-bit immediate (branch offset) scrambled in instruction // Bits: imm[12|10:5] in funct7, imm[4:1|11] in dest_reg, imm[0]=0 // Used by: conditional branches assign o_immediate_b_type = { - {19{i_instruction.funct7[6]}}, + {(XLEN - 13) {i_instruction.funct7[6]}}, i_instruction.funct7[6], i_instruction.dest_reg[0], i_instruction.funct7[5:0], @@ -62,15 +65,16 @@ module immediate_decoder #( 1'b0 }; - // U-type: 20-bit immediate in upper bits, lower 12 bits are zero - // Used by: LUI, AUIPC - assign o_immediate_u_type = {i_instruction[31:12], 12'h0}; + // U-type: 20-bit immediate in upper bits, lower 12 bits are zero. + // Bit 31 replicates (XLEN-31) times: once at RV32 (identity), 33 times at + // RV64 - LUI/AUIPC results are sign-extended from bit 31 per the spec. + assign o_immediate_u_type = {{(XLEN - 31) {i_instruction[31]}}, i_instruction[30:12], 12'h0}; // J-type: 21-bit jump offset scrambled in instruction // Bits: imm[20|10:1|11|19:12], imm[0]=0 // Used by: JAL assign o_immediate_j_type = { - {11{i_instruction[31]}}, + {(XLEN - 21) {i_instruction[31]}}, i_instruction[31], i_instruction[19:12], i_instruction[20], diff --git a/hw/rtl/cpu_and_mem/cpu/id_stage/instruction_type_decoder.sv b/hw/rtl/cpu_and_mem/cpu/id_stage/instruction_type_decoder.sv index 667cc0fb..53b6c5ad 100644 --- a/hw/rtl/cpu_and_mem/cpu/id_stage/instruction_type_decoder.sv +++ b/hw/rtl/cpu_and_mem/cpu/id_stage/instruction_type_decoder.sv @@ -31,7 +31,7 @@ * - JAL/JALR detection */ module instruction_type_decoder #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input riscv_pkg::instr_t i_instruction, input logic [XLEN-1:0] i_immediate_i_type, @@ -179,14 +179,14 @@ module instruction_type_decoder #( rd_is_link_reg && rs1_is_return_link && (i_instruction.dest_reg != i_instruction.source_reg_1) && - (i_immediate_i_type == 32'b0); + (i_immediate_i_type == '0); // Return: JALR with rs1 = x1, rd = x0, imm = 0 -- or the swap encoding. // The immediate for JALR is in I-type format: funct7[6:0] ++ source_reg_2[4:0] assign o_is_ras_return = (o_is_jalr && rs1_is_return_link && (i_instruction.dest_reg == 5'd0) && - (i_immediate_i_type == 32'b0)) || is_ras_coroutine; + (i_immediate_i_type == '0)) || is_ras_coroutine; // Call: JAL or JALR with rd in {x1, x5}. A coroutine already satisfies this // (its rd is a link register), so it needs no extra term here -- asserting diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv index 05c73ff9..427759ae 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/branch_predictor.sv @@ -70,7 +70,7 @@ * the new entry after it, including on back-to-back same-index updates. */ module branch_predictor #( - parameter int unsigned XLEN = 32, + parameter int unsigned XLEN = riscv_pkg::XLEN, parameter int unsigned BTB_INDEX_BITS = 8 // 256 entries ) ( input logic i_clk, diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/direction_predictor.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/direction_predictor.sv index 08f70bc1..f9c4b052 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/direction_predictor.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/direction_predictor.sv @@ -44,8 +44,8 @@ * (counters start weakly-NT). */ module direction_predictor #( - parameter int unsigned XLEN = 32, - parameter int unsigned BIM_BITS = 10 // bimodal index bits (1024 entries) + parameter int unsigned XLEN = riscv_pkg::XLEN, + parameter int unsigned BIM_BITS = 10 // bimodal index bits (1024 entries) ) ( input logic i_clk, input logic i_rst, diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/prediction_metadata_tracker.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/prediction_metadata_tracker.sv index 05df2788..74ce648f 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/prediction_metadata_tracker.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/prediction_metadata_tracker.sv @@ -34,7 +34,7 @@ * - prediction_to_pd: Final output based on instruction type */ module prediction_metadata_tracker #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, input logic i_reset, diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/c_ext_state.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/c_ext_state.sv index 784d9eea..65b069ea 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/c_ext_state.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/c_ext_state.sv @@ -31,7 +31,7 @@ holdoff machinery instead. */ module c_ext_state #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, @@ -53,11 +53,11 @@ module c_ext_state #( input logic i_prediction_from_buffer_holdoff, // Need buffered old-path word next cycle // Instruction data - input logic [31:0] i_effective_instr, // Current effective instruction word + input logic [ 31:0] i_effective_instr, // Current effective instruction word // BRAM word order doesn't match pc_reg (bank_sel_r ^ pc_reg[2]) - input logic i_fetch_word_swapped, - input logic [31:0] i_pc, // Current fetch PC - input logic [31:0] i_pc_reg, // Registered PC + input logic i_fetch_word_swapped, + input logic [XLEN-1:0] i_pc, // Current fetch PC + input logic [XLEN-1:0] i_pc_reg, // Registered PC // Instruction type detection (from instruction aligner) input logic i_is_compressed, // Current parcel is compressed diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/instruction_aligner.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/instruction_aligner.sv index 10564461..4441f4a1 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/instruction_aligner.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/c_extension/instruction_aligner.sv @@ -39,7 +39,7 @@ This module is purely combinational. */ module instruction_aligner #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( // 64-bit instruction fetch: {next_word[31:0], current_word[31:0]} input logic [63:0] i_instr, @@ -50,7 +50,7 @@ module instruction_aligner #( input logic i_instr_bank_sel_r, // Registered fetch-word parity (PC[2] from BRAM cycle) input logic [31:0] i_instr_buffer, // Buffered instruction word input logic [riscv_pkg::ImemSidebandWidth-1:0] i_instr_buffer_sideband, - input logic [31:0] i_pc_reg, // Registered PC + input logic [XLEN-1:0] i_pc_reg, // Registered PC // C-extension state input logic i_prev_was_compressed_at_lo, // Previous was compressed at lo diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/control_flow_tracker.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/control_flow_tracker.sv index 42f740ef..ff2799f8 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/control_flow_tracker.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/control_flow_tracker.sv @@ -41,7 +41,7 @@ * - instruction_aligner.sv: Uses holdoff to insert NOPs */ module control_flow_tracker #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, input logic i_reset, diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv index aa83553e..4ac39327 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/if_stage.sv @@ -83,7 +83,7 @@ * redirected through the i_from_ex_comb interface. */ module if_stage #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, input riscv_pkg::from_ex_comb_t i_from_ex_comb, @@ -878,11 +878,15 @@ module if_stage #( `endif logic window_cannot_serve_pc_reg; - // Gated to the cached region (pc_reg[XLEN-1], i.e. >= CACHED_BASE): the low BRAM + // Gated to the cached region (physical bit 31, i.e. >= CACHED_BASE): the low BRAM // fetch path is fixed 1-cycle/always-valid and never desyncs, and its served-addr // tracking is approximate -- firing there only causes spurious squashes. - assign window_cannot_serve_pc_reg = i_instr_valid && pc_reg[XLEN-1] && - !served_window_covers_pc_reg; + // The region test keys on the FIXED physical bit (riscv_pkg::CachedRegionBit), + // never [XLEN-1]: at XLEN=64 bit 63 is never set for sub-4-GiB PCs, which + // would silently kill this guard and resurrect the mid-instruction-byte + // pc_reg desync it exists to stop (the workqueue_init_early boot Oops). + assign window_cannot_serve_pc_reg = i_instr_valid && + pc_reg[riscv_pkg::CachedRegionBit] && !served_window_covers_pc_reg; // The existing (pre-served-window-guard) squash conditions. logic sel_nop_existing; diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/pc_controller.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/pc_controller.sv index 42f45d77..c489a018 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/pc_controller.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/pc_controller.sv @@ -71,7 +71,7 @@ predicted branches commit without a redirect here. */ module pc_controller #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, @@ -769,10 +769,16 @@ module pc_controller #( logic pc_update_en; assign pc_update_en = i_reset || trap_or_mret || i_fence_i_flush || !i_stall; + // The PC flops canonicalize to the physical address space (identity at + // XLEN=32; masks bits [63:32] at XLEN=64 - plan decision D3). Redirect + // producers (branch resolution, trap unit) mask their own outputs too; + // masking here makes the PC canonical by induction regardless of source, + // so every downstream PC register, BTB/RAS entry, and served-window + // compare carries structurally-zero upper bits. always_ff @(posedge i_clk) begin if (pc_update_en) begin - o_pc <= next_pc; - o_pc_reg <= next_pc_reg; + o_pc <= riscv_pkg::canonical_paddr(next_pc); + o_pc_reg <= riscv_pkg::canonical_paddr(next_pc_reg); end end diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/pc_increment_calculator.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/pc_increment_calculator.sv index 9c9a3e97..5f4cb061 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/pc_increment_calculator.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/pc_increment_calculator.sv @@ -46,7 +46,7 @@ - c_ext_state.sv: Provides spanning and compression state */ module pc_increment_calculator #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( // Current PC values (registered outputs from pc_controller) input logic [XLEN-1:0] i_pc, @@ -348,7 +348,7 @@ module pc_increment_calculator #( endmodule : pc_increment_calculator module pc_fetch_advance_mux #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic [XLEN-1:0] i_next_pc_plus_2, input logic [XLEN-1:0] i_next_pc_plus_4, @@ -388,7 +388,7 @@ module pc_fetch_advance_mux #( endmodule : pc_fetch_advance_mux module pc_reg_advance_mux #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic [XLEN-1:0] i_pc_reg_if_compressed, input logic [XLEN-1:0] i_pc_reg_if_32bit, diff --git a/hw/rtl/cpu_and_mem/cpu/if_stage/pc_reg_precompute.sv b/hw/rtl/cpu_and_mem/cpu/if_stage/pc_reg_precompute.sv index c6d8d2a4..5d78ca5d 100644 --- a/hw/rtl/cpu_and_mem/cpu/if_stage/pc_reg_precompute.sv +++ b/hw/rtl/cpu_and_mem/cpu/if_stage/pc_reg_precompute.sv @@ -33,7 +33,7 @@ */ (* keep_hierarchy = "yes" *) module pc_reg_precompute #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic [XLEN-1:0] i_pc_reg, diff --git a/hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv b/hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv index fbbed2e2..eb7af062 100644 --- a/hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv +++ b/hw/rtl/cpu_and_mem/cpu/pd_stage/pd_stage.sv @@ -38,7 +38,7 @@ and FENCE.I recovery. */ module pd_stage #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, input riscv_pkg::pipeline_ctrl_t i_pipeline_ctrl, @@ -263,7 +263,7 @@ module pd_stage #( logic [XLEN-1:0] pd_imm_b_native; assign pd_imm_b_native = { - {19{i_from_if_to_pd.effective_instr[31]}}, // sign-extend bits [31:13] + {(XLEN - 13) {i_from_if_to_pd.effective_instr[31]}}, // sign-extend bits [XLEN-1:13] i_from_if_to_pd.effective_instr[31], // imm[12] i_from_if_to_pd.effective_instr[7], // imm[11] i_from_if_to_pd.effective_instr[30:25], // imm[10:5] @@ -273,7 +273,7 @@ module pd_stage #( logic [XLEN-1:0] pd_imm_b_compressed; assign pd_imm_b_compressed = { - {23{i_from_if_to_pd.raw_parcel[12]}}, // sign-extend bits [31:9] + {(XLEN - 9) {i_from_if_to_pd.raw_parcel[12]}}, // sign-extend bits [XLEN-1:9] i_from_if_to_pd.raw_parcel[12], // imm[8] i_from_if_to_pd.raw_parcel[6:5], // imm[7:6] i_from_if_to_pd.raw_parcel[2], // imm[5] diff --git a/hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv b/hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv index 94f5450a..14dd29b5 100644 --- a/hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv +++ b/hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv @@ -730,18 +730,23 @@ package riscv_pkg; localparam int unsigned MieMtiBit = 7; // Machine Timer Interrupt localparam int unsigned MieMeiBit = 11; // Machine External Interrupt - // Exception cause codes (mcause values when interrupt bit = 0) - localparam bit [31:0] ExcIllegalInstr = 32'd2; - localparam bit [31:0] ExcBreakpoint = 32'd3; - localparam bit [31:0] ExcLoadAddrMisalign = 32'd4; - localparam bit [31:0] ExcStoreAddrMisalign = 32'd6; - localparam bit [31:0] ExcEcallUmode = 32'd8; - localparam bit [31:0] ExcEcallMmode = 32'd11; - - // Interrupt cause codes (mcause values when interrupt bit = 1) - localparam bit [31:0] IntMachineSoftware = 32'h8000_0003; - localparam bit [31:0] IntMachineTimer = 32'h8000_0007; - localparam bit [31:0] IntMachineExternal = 32'h8000_000B; + // Exception cause codes (mcause values when the interrupt bit is clear). + // XLEN-wide; the small synchronous cause codes zero-extend identically at + // either width. + localparam bit [XLEN-1:0] ExcIllegalInstr = XLEN'(2); + localparam bit [XLEN-1:0] ExcBreakpoint = XLEN'(3); + localparam bit [XLEN-1:0] ExcLoadAddrMisalign = XLEN'(4); + localparam bit [XLEN-1:0] ExcStoreAddrMisalign = XLEN'(6); + localparam bit [XLEN-1:0] ExcEcallUmode = XLEN'(8); + localparam bit [XLEN-1:0] ExcEcallMmode = XLEN'(11); + + // Interrupt cause codes (mcause values when the interrupt bit is set). + // The interrupt bit is the MSB of mcause - bit XLEN-1, NOT literally bit + // 31 - so these are built XLEN-wide by construction. Never compare them + // against 32-bit slices of a wider mcause. + localparam bit [XLEN-1:0] IntMachineSoftware = {1'b1, {(XLEN - 4) {1'b0}}, 3'd3}; + localparam bit [XLEN-1:0] IntMachineTimer = {1'b1, {(XLEN - 4) {1'b0}}, 3'd7}; + localparam bit [XLEN-1:0] IntMachineExternal = {1'b1, {(XLEN - 5) {1'b0}}, 4'd11}; // =========================================================================== // Section 4: Control Enumerations @@ -788,7 +793,58 @@ package riscv_pkg; localparam bit [31:0] NOP = 32'h0000_0013; // addi x0, x0, 0 + // XLEN is selected at build time: define FROST_RV64 for the RV64 build + // (ROADMAP Phase 1, docs/rv64/phase1_plan.md decision D1); the default + // remains the RV32GCB configuration. This localparam is the single source + // of truth for the core's width - module-level XLEN parameters default to + // it and exist only so unit benches can elaborate standalone. +`ifdef FROST_RV64 + localparam int unsigned XLEN = 64; +`else localparam int unsigned XLEN = 32; +`endif + + // Physical-map geometry. Phase 1 invariant: the entire physical map lives + // below 4 GiB (256 KiB low BRAM at 0, MMIO in the 01 quadrant at + // 0x4000_0000, 1 GiB cached DDR at 0x8000_0000). Region decodes therefore + // key on FIXED physical bit positions - bit 31 selects the cached region, + // addr[31:30]==01 is MMIO - never on XLEN-relative positions like + // [XLEN-1], which silently go dead at XLEN=64. Addresses are + // canonicalized to this space at their producers (PC redirects, trap + // targets, AGU outputs) per docs/rv64/phase1_plan.md decision D3, so + // bits [XLEN-1:32] of fetch/memory addresses are structurally zero and + // synthesis sweeps them from downstream storage and comparators. + localparam int unsigned PhysAddrBits = 32; + localparam int unsigned CachedRegionBit = 31; + + // Data-tier beat width (docs/rv64/m1_data_tier.md). Deliberately a + // separate constant from XLEN: the 64-bit single-beat data tier is + // implemented and proven while the core is still rv32. Every data-side + // bus carries the aligned dword at addr[31:3]; byte lane i is byte + // address {addr[31:3], i}. Sub-beat writes replicate their data across + // the beat and select lanes with the strobe; reads return the full + // beat and consumers extract by addr[2:0]. + localparam int unsigned MemDataBits = 64; + localparam int unsigned MemStrbBits = MemDataBits / 8; + + // 8-lane strobe for a sub-beat access at the given offset (see the + // contract above; DOUBLE covers the whole beat). + function automatic logic [MemStrbBits-1:0] mem_strobe_for(input logic [1:0] size_bits, + input logic [2:0] offset); + unique case (size_bits) + 2'b00: mem_strobe_for = MemStrbBits'(8'h01) << offset; // byte + 2'b01: mem_strobe_for = MemStrbBits'(8'h03) << {offset[2:1], 1'b0}; // half + 2'b10: mem_strobe_for = offset[2] ? 8'hF0 : 8'h0F; // word + default: mem_strobe_for = 8'hFF; // double + endcase + endfunction + + // Canonicalize an address to the physical space: identity at XLEN=32, + // zero-extends the low 32 bits at XLEN=64 (out-of-map high bits alias + // onto the map; a real access-fault path is deferred to Phase 3 PMA). + function automatic logic [XLEN-1:0] canonical_paddr(input logic [XLEN-1:0] addr); + canonical_paddr = XLEN'(addr[PhysAddrBits-1:0]); + endfunction // FP register width: 64-bit to support D extension (RV32D). localparam int unsigned FpWidth = 64; localparam int unsigned FpSingleWidth = 32; @@ -803,12 +859,13 @@ package riscv_pkg; localparam logic [PcAdvanceSelWidth-1:0] PcAdvancePlus6 = 2'd2; localparam logic [PcAdvanceSelWidth-1:0] PcAdvancePlus8 = 2'd3; - // Magic number constants for RISC-V 32-bit operations - // Used in ALU for special case handling (e.g., division overflow) - localparam bit [31:0] SignedInt32Min = 32'h8000_0000; // -2^31 (most negative) - localparam bit [31:0] SignedInt32Max = 32'h7FFF_FFFF; // 2^31 - 1 (most positive) - localparam bit [31:0] UnsignedInt32Max = 32'hFFFF_FFFF; // All ones (also -1 signed) - localparam bit [31:0] NegativeOne = 32'hFFFF_FFFF; // -1 in two's complement + // XLEN-wide arithmetic special-case constants, used for DIV/REM overflow + // and divide-by-zero handling. Explicit 32-bit variants for the RV64 + // W-instruction special cases (DIVW/REMW) arrive with RV64M in Phase 1. + localparam bit [XLEN-1:0] SignedIntMin = {1'b1, {(XLEN - 1) {1'b0}}}; // -2^(XLEN-1) + localparam bit [XLEN-1:0] SignedIntMax = {1'b0, {(XLEN - 1) {1'b1}}}; // 2^(XLEN-1) - 1 + localparam bit [XLEN-1:0] UnsignedIntMax = '1; // All ones + localparam bit [XLEN-1:0] NegativeOne = '1; // -1 in two's complement // =========================================================================== // Section 6: Pipeline Control @@ -923,12 +980,13 @@ package riscv_pkg; // Clocked signals passed from Instruction Decode (ID) stage to Execute (EX) stage typedef struct packed { logic [XLEN-1:0] program_counter; - // Immediate values decoded from instruction (different formats) - logic [31:0] immediate_i_type; // I-type: 12-bit sign-extended - logic [31:0] immediate_s_type; // S-type: for stores - logic [31:0] immediate_b_type; // B-type: for branches - logic [31:0] immediate_u_type; // U-type: upper 20 bits - logic [31:0] immediate_j_type; // J-type: for jumps + // Immediate values decoded from instruction (different formats), + // sign-extended to XLEN by immediate_decoder. + logic [XLEN-1:0] immediate_i_type; // I-type: 12-bit sign-extended + logic [XLEN-1:0] immediate_s_type; // S-type: for stores + logic [XLEN-1:0] immediate_b_type; // B-type: for branches + logic [XLEN-1:0] immediate_u_type; // U-type: upper 20 bits + logic [XLEN-1:0] immediate_j_type; // J-type: for jumps // Register file read data (read in ID stage using early source regs from PD) // This moves the regfile read out of the EX stage critical path logic [XLEN-1:0] source_reg_1_data; @@ -1391,7 +1449,7 @@ package riscv_pkg; // Mapping from riscv_pkg 32-bit constants to Reorder Buffer 5-bit cause: // exc_cause = riscv_pkg::Exc*[4:0] (low 5 bits) // Examples: ExcBreakpoint (3) -> 5'd3, ExcLoadAddrMisalign (4) -> 5'd4 - // The mcause CSR's interrupt bit (bit 31) is never set for Reorder Buffer-tracked exceptions. + // The mcause CSR's interrupt bit (bit XLEN-1) is never set for Reorder Buffer-tracked exceptions. // When committing an exception, the trap unit constructs the full mcause value. localparam int unsigned ExcCauseWidth = 5; @@ -1605,7 +1663,7 @@ package riscv_pkg; MEM_SIZE_BYTE = 2'b00, // 8-bit MEM_SIZE_HALF = 2'b01, // 16-bit MEM_SIZE_WORD = 2'b10, // 32-bit - MEM_SIZE_DOUBLE = 2'b11 // 64-bit (FLD/FSD only) + MEM_SIZE_DOUBLE = 2'b11 // 64-bit (FLD/FSD today; RV64 LD/SD/LR.D/SC.D/AMO*.D join in Phase 1) } mem_size_e; // --------------------------------------------------------------------------- diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/README.md b/hw/rtl/cpu_and_mem/cpu/tomasulo/README.md index bf3a7c52..220895c4 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/README.md +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/README.md @@ -56,8 +56,8 @@ is not a fully symmetric 2-issue execution engine — see | [`reorder_buffer/`](reorder_buffer/README.md) | In-order commit, precise exceptions, serializing instructions | | [`register_alias_table/`](register_alias_table/README.md) | INT + FP rename tables, branch checkpoints | | [`reservation_station/`](reservation_station/README.md) | Generic RS, instantiated 6× | -| [`load_queue/`](load_queue/README.md) | Loads, L0 cache, MMIO, FP64 phasing, LR/AMO | -| [`store_queue/`](store_queue/README.md) | Stores, store-to-load forwarding, FSD phasing | +| [`load_queue/`](load_queue/README.md) | Loads, L0 cache, MMIO, single-beat dwords, LR/AMO | +| [`store_queue/`](store_queue/README.md) | Stores, store-to-load forwarding, single-beat drains | | [`cdb_arbiter/`](cdb_arbiter/README.md) | 2-lane CDB priority arbiter | | [`fu_cdb_adapter/`](fu_cdb_adapter/README.md) | One-deep holding register per FU slot | | [`fu_shims/`](fu_shims/README.md) | Adapters from RS issue to the reused FUs | diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/README.md b/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/README.md index d14f07b1..4d2ddbc6 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/README.md +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/README.md @@ -14,9 +14,9 @@ issue. The LQ uses conservative disambiguation: a load can't issue to memory until every older store address is known. If a matching older store turns up that covers the load's bytes, the LQ pulls the data from the SQ via store-to-load forwarding and skips memory -entirely — the SQ supplies a memory-image word and a local -`load_unit` instance applies byte/half extraction and sign extension -for integer loads, mirroring the L0 hit path. Otherwise it +entirely — the SQ supplies the aligned-dword memory image and a local +`load_unit` instance applies word/half/byte extraction and sign +extension for integer loads, mirroring the L0 hit path. Otherwise it checks the L0 cache; on a hit, the result is available the same cycle, and on a miss it issues to main memory. Main memory is not uniform: low-BRAM loads return in one cycle, while loads to the @@ -64,12 +64,12 @@ MMIO loads are an additional case. Their reads can have side effects speculatively. The LQ pins MMIO loads to the ROB head — they only fire when their entry is the oldest in flight. -FP64 loads (FLD) on the 32-bit memory bus need two sequential -accesses, so the entry has a phase bit and the data field is split -lo/hi in the LUTRAM so each phase writes only its half. A registered -physical-generation pulse initializes a reused entry's resident phase bit. -A current address update supplies phase zero directly because it can only -match an entry before that entry's first memory phase. +Dword loads (FLD today; RV64 LD reuses the same size-keyed paths in +Phase 1 M3) complete in a single beat: the 64-bit data tier +([docs/rv64/m1_data_tier.md](../../../../../docs/rv64/m1_data_tier.md)) +returns the aligned dword at `addr[31:3]` and the entry's FLEN-wide +data slot captures it whole. The old 32-bit-bus two-phase FLD machinery +(per-entry phase bit, split lo/hi data halves, `+4` re-issue) is gone. The per-entry AMO opcode is compacted from the 32-bit `instr_op_e` to a 4-bit semantic code and stored in per-entry FFs. Accepted slot-1 and slot-2 @@ -88,14 +88,17 @@ path without changing AMO latency. ## L0 cache -The L0 is a 128-entry direct-mapped cache, filled on memory -response, implemented inside the LQ by [`lq_l0_cache.sv`](lq_l0_cache.sv). +The L0 is a 128-entry direct-mapped cache with dword-granule (aligned +8-byte) lines, filled one full beat per memory response, implemented +inside the LQ by [`lq_l0_cache.sv`](lq_l0_cache.sv). It's a hit-path optimization: loads check it in parallel with SQ -disambiguation, and a hit returns the result the same cycle. Stores -invalidate matching lines on commit (the SQ pulses an invalidate back -to the LQ), and AMO write-completion invalidates the AMO's line too; -both share the single invalidate port. That keeps the cache coherent -without needing a write-through path of its own. +disambiguation, and a hit returns the result the same cycle — every +load size is eligible, including FLD (the line carries the whole +dword). Stores invalidate their containing dword line on commit (the +SQ pulses an invalidate back to the LQ), and AMO write-completion +invalidates the AMO's line too; both share the single invalidate +port. That keeps the cache coherent without needing a write-through +path of its own. Three things the cache intentionally *doesn't* do: @@ -195,8 +198,8 @@ Two bypass paths shave a cycle each off the load critical latency: from the response / cache / forward data path instead of routing through `lq_data_valid` + a priority encoder. The entry frees and the CDB broadcast arms the same cycle. AMOs stay on the standard - path, as do two-phase memory FLDs — but a *forwarded* FLD bypasses, - since the SQ delivers its full 64-bit payload in a single probe. + path, as do DOUBLE-size memory responses; L0-hit and forwarded FLDs + bypass (both deliver the full 64-bit payload in a single probe). ## Back-to-back issue @@ -213,7 +216,7 @@ writes (they can't collide on the same port anymore). ## Issued-entry snapshot The response handler reads from a flat snapshot of the issued load's -attributes (addr / size / FP / LR / AMO / MMIO / sign_ext / fp64_phase / +attributes (addr / size / FP / LR / AMO / MMIO / sign_ext / rob_tag) captured at launch, not from the per-entry LUTRAMs indexed by `issued_idx`. Removing the `lq_*[issued_idx]` read path takes the LQ entry array out of the `data_memory` read-address cone. The AMO-only operation @@ -257,12 +260,14 @@ response side needs no extra queue read port. The address-update CAM matches against `rob_tag`, not the address itself; the resolved address is then written into the address LUTRAM. -The 64-bit load-result payload lives in its own distributed RAM split -lo/hi (to support FLD's two-phase fills), each half in a 2-write-port +The 64-bit load-result payload lives in one FLEN-wide 2-write-port LUTRAM: port 0 is reserved for memory response, port 1 handles cache -hits, SQ forwards, and AMO write-completion. The split lets a memory +hits, SQ forwards, and AMO write-completion. The two ports let a memory response for the previously-issued load and a cache hit on the -newly-captured load land in the same cycle without colliding. +newly-captured load land in the same cycle without colliding. DOUBLE +loads store the full beat; every other load stores its extracted (or, +for FLW, addressed-word) value zero-extended, with NaN-boxing applied +at CDB broadcast. Allocation metadata has separate slot-1 and slot-2 write paths. When both slots allocate loads, slot 1 takes the older free entry and slot 2 takes the next free @@ -296,7 +301,7 @@ still live alongside the decomposition. ## Verification Cocotb tests cover allocation including slot-2-only and paired slot-1/slot-2 -cases, address update, every load size, SQ forwarding, MMIO ordering, FLD -two-phase, FLW NaN-boxing, partial and full flush, AMO read-modify-write, +cases, address update, every load size, SQ forwarding, MMIO ordering, +single-beat FLD, FLW NaN-boxing, partial and full flush, AMO read-modify-write, LR/SC reservation, and constrained-random stress. Inline formal properties prove pointer invariants, issue prerequisites, MMIO ordering, and flush behavior. diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv b/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv index 394534a7..1b7d7789 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_queue.sv @@ -26,7 +26,8 @@ * - Parameterized depth (8 entries; LUTRAM payload, FF control state) * - CAM-style tag search for address update (all entries in parallel) * - Oldest-first priority scan for issue selection - * - Two-phase FLD support (64-bit double on 32-bit bus) + * - Single-beat dword loads on the 64-bit data tier (FLD today; RV64 LD + * reuses the same size-keyed paths in M3 — docs/rv64/m1_data_tier.md) * - Store-to-load forwarding via SQ disambiguation interface * - MMIO loads execute only at ROB head (non-speculative) * - Partial flush (age-based) and full flush support @@ -38,8 +39,8 @@ * FFs for CAM-style parallel tag search (matched on rob_tag), per-entry * invalidation, and oldest-first priority scan. * The load address (sdp_dist_ram) and load-result payload lq_data - * (mwp_dist_ram, split lo/hi for FLD partial writes, 2 write ports - * for primary + AMO overlap) live in distributed RAM. The AMO operation is + * (one FLEN-wide mwp_dist_ram, 2 write ports for primary + AMO overlap) + * live in distributed RAM. The AMO operation is * compacted to a 4-bit semantic code in per-entry FFs. Allocation writes are * staged for one cycle (well before an AMO can issue), keeping the late * dispatch/allocation cone local to a tiny request register. The selected @@ -140,13 +141,15 @@ module load_queue #( // ========================================================================= // Memory Interface (to data memory bus) // ========================================================================= - output logic o_mem_read_en, - output logic o_mem_addr_valid, - output logic [riscv_pkg::XLEN-1:0] o_mem_read_addr, - output riscv_pkg::mem_size_e o_mem_read_size, - input logic [riscv_pkg::XLEN-1:0] i_mem_read_data, - input logic i_mem_read_valid, - input logic i_mem_bus_busy, + output logic o_mem_read_en, + output logic o_mem_addr_valid, + output logic [ riscv_pkg::XLEN-1:0] o_mem_read_addr, + output riscv_pkg::mem_size_e o_mem_read_size, + // Aligned MemDataBits beat carrying the dword at addr[31:3] + // (docs/rv64/m1_data_tier.md); consumers extract by addr[2:0]. + input logic [riscv_pkg::MemDataBits-1:0] i_mem_read_data, + input logic i_mem_read_valid, + input logic i_mem_bus_busy, // ========================================================================= // CDB Result (to fu_cdb_adapter, FU_MEM slot) @@ -185,10 +188,12 @@ module load_queue #( // ========================================================================= // AMO Memory Write Interface // ========================================================================= - output logic o_amo_mem_write_en, - output logic [riscv_pkg::XLEN-1:0] o_amo_mem_write_addr, - output logic [riscv_pkg::XLEN-1:0] o_amo_mem_write_data, - input logic i_amo_mem_write_done, + output logic o_amo_mem_write_en, + output logic [ riscv_pkg::XLEN-1:0] o_amo_mem_write_addr, + // Word-sized AMO result replicated across the beat ({2{result}}); the + // router derives the word-lane strobes from o_amo_mem_write_addr[2]. + output logic [riscv_pkg::MemDataBits-1:0] o_amo_mem_write_data, + input logic i_amo_mem_write_done, // ========================================================================= // Flush @@ -340,8 +345,10 @@ module load_queue #( logic [XLEN-1:0] cached_base; logic [XLEN-1:0] cached_limit; begin - cached_base = CACHED_BASE[XLEN-1:0]; - cached_limit = CACHED_BASE[XLEN-1:0] + CACHED_SIZE_BYTES[XLEN-1:0]; + // XLEN'() casts, not [XLEN-1:0] part-selects: the parameters are + // 32-bit ints, so a 64-bit part-select would be out of range. + cached_base = XLEN'(CACHED_BASE); + cached_limit = XLEN'(CACHED_BASE) + XLEN'(CACHED_SIZE_BYTES); is_cached_addr = (addr >= cached_base) && (addr < cached_limit); end endfunction @@ -377,7 +384,6 @@ module load_queue #( logic [ DEPTH-1:0] lq_addr_valid; logic [ DEPTH-1:0] lq_sign_ext; logic [ DEPTH-1:0] lq_is_mmio; - logic [ DEPTH-1:0] lq_fp64_phase; logic [ DEPTH-1:0] lq_issued; logic [ DEPTH-1:0] lq_data_valid; logic [ DEPTH-1:0] lq_forwarded; @@ -464,52 +470,37 @@ module load_queue #( logic [ XLEN-1:0] amo_write_data_q; // =========================================================================== - // lq_data LUTRAM — split lo/hi for FLD partial-word writes + // lq_data LUTRAM — FLEN-wide single-beat payloads // =========================================================================== // lq_data payload is only read at issue_cdb_idx (CDB broadcast). // Writes come from two independent sources that can overlap: // Port 0 (mem resp): memory response (dedicated) // Port 1 (local): cache hit / SQ forward / AMO write completion - // Split into 32-bit lo and hi halves so FLD can write each phase - // independently without read-modify-write. + // Value semantics: DOUBLE loads store the full aligned beat; every other + // load stores its extracted-int result (or, for FLW, the addressed raw + // word) zero-extended into FLEN. NaN-boxing happens at CDB broadcast. // Forward declaration (used as LUTRAM read address) logic [IdxWidth-1:0] issue_cdb_idx; - logic [ XLEN-1:0] lq_data_lo_rd; // LUTRAM async read at issue_cdb_idx - logic [ XLEN-1:0] lq_data_hi_rd; + logic [ FLEN-1:0] lq_data_rd; // LUTRAM async read at issue_cdb_idx - // Write port signals (2 ports each for lo and hi) - logic [ 1:0] lq_data_lo_we; - logic [ 1:0] lq_data_hi_we; + // Write port signals + logic [ 1:0] lq_data_we; logic [ 1:0][IdxWidth-1:0] lq_data_wr_addr; - logic [ 1:0][ XLEN-1:0] lq_data_lo_wd; - logic [ 1:0][ XLEN-1:0] lq_data_hi_wd; + logic [ 1:0][ FLEN-1:0] lq_data_wd; mwp_dist_ram #( .ADDR_WIDTH(IdxWidth), - .DATA_WIDTH(XLEN), + .DATA_WIDTH(FLEN), .NUM_WRITE_PORTS(2) - ) u_lq_data_lo ( + ) u_lq_data ( .i_clk, - .i_write_enable (lq_data_lo_we), + .i_write_enable (lq_data_we), .i_write_address(lq_data_wr_addr), - .i_write_data (lq_data_lo_wd), + .i_write_data (lq_data_wd), .i_read_address (issue_cdb_idx), - .o_read_data (lq_data_lo_rd) - ); - - mwp_dist_ram #( - .ADDR_WIDTH(IdxWidth), - .DATA_WIDTH(XLEN), - .NUM_WRITE_PORTS(2) - ) u_lq_data_hi ( - .i_clk, - .i_write_enable (lq_data_hi_we), - .i_write_address(lq_data_wr_addr), - .i_write_data (lq_data_hi_wd), - .i_read_address (issue_cdb_idx), - .o_read_data (lq_data_hi_rd) + .o_read_data (lq_data_rd) ); // =========================================================================== @@ -563,7 +554,6 @@ module load_queue #( logic sq_check_is_fp_q; logic sq_check_sign_ext_q; logic sq_check_is_mmio_q; - logic sq_check_fp64_phase_q; logic sq_check_is_lr_q; logic sq_check_is_amo_q; logic sq_check_no_older_store_q; @@ -613,7 +603,6 @@ module load_queue #( // rather than a re-derived late-address decode. logic issued_is_cached; logic issued_sign_ext; - logic issued_fp64_phase; logic [ReorderBufferTagWidth-1:0] issued_rob_tag; amo_kind_e issued_amo_kind; logic [XLEN-1:0] issued_amo_rs2; @@ -702,9 +691,11 @@ module load_queue #( // AMO cache invalidation: invalidate L0 cache when AMO write completes logic amo_cache_inv; assign amo_cache_inv = (amo_state == AMO_WRITE_ACTIVE) && i_amo_mem_write_done; + // Dword granule: the response beat fills a full L0 dword line, so a store + // landing in EITHER word of the in-flight dword must suppress the fill. assign issued_cached_line_invalidate_now = mem_outstanding && issued_is_cached && i_cache_invalidate_valid && - (i_cache_invalidate_addr[XLEN-1:2] == issued_addr[XLEN-1:2]); + (i_cache_invalidate_addr[XLEN-1:3] == issued_addr[XLEN-1:3]); // =========================================================================== // Count, Full, Empty @@ -1351,7 +1342,7 @@ module load_queue #( logic lu_is_half; logic lu_is_unsigned; logic [XLEN-1:0] lu_addr; - logic [XLEN-1:0] lu_raw_data; + logic [riscv_pkg::MemDataBits-1:0] lu_raw_data; load_unit u_load_unit ( .i_is_load_byte (lu_is_byte), @@ -1365,17 +1356,16 @@ module load_queue #( // =========================================================================== // L0 Cache Instance // =========================================================================== - logic cache_lookup_hit; - logic [XLEN-1:0] cache_lookup_data; - logic cache_fill_response_valid; - logic cache_fill_valid; - logic [XLEN-1:0] cache_fill_addr; - logic [XLEN-1:0] cache_fill_data; + logic cache_lookup_hit; + logic [riscv_pkg::MemDataBits-1:0] cache_lookup_data; + logic cache_fill_response_valid; + logic cache_fill_valid; + logic [ XLEN-1:0] cache_fill_addr; + logic [riscv_pkg::MemDataBits-1:0] cache_fill_data; lq_l0_cache #( - .DEPTH (128), - .XLEN (XLEN), - .MMIO_ADDR(32'h4000_0000) + .DEPTH(128), + .XLEN (XLEN) ) u_l0_cache ( .i_clk (i_clk), .i_rst_n(i_rst_n), @@ -1425,10 +1415,9 @@ module load_queue #( // Cache-hit fast path signal: Phase B candidate hits L0 cache, SQ // disambiguation confirms no conflicting store, and the consumer is a - // cache-safe load. Integer byte/half/word loads can reuse the cached raw - // word through the local load_unit. FLW can also reuse the cached word - // directly. FLD remains on the memory path because it is a two-phase - // operation on the 32-bit data bus. The bus-busy gate is required even for + // cache-safe load. Integer byte/half/word loads extract from the cached + // beat through the local load_unit; FLW takes its addressed word and FLD + // the full dword line. The bus-busy gate is required even for // cache hits: an SQ/AMO write can own the port one cycle before its L0 // invalidation is visible, so a phase-2 hit in that window could be stale. assign cache_hit_fast_path = ENABLE_L0_FAST_PATH @@ -1438,17 +1427,9 @@ module load_queue #( && cache_lookup_hit && !sq_check_is_mmio_q && !sq_check_is_lr_q - && !sq_check_is_amo_q - && (!sq_check_is_fp_q || (sq_check_size_q == riscv_pkg::MEM_SIZE_WORD)); + && !sq_check_is_amo_q; - always_comb begin - stage_mem_issue_addr = sq_check_addr_q; - if (sq_check_is_fp_q && - (sq_check_size_q == riscv_pkg::MEM_SIZE_DOUBLE) && - sq_check_fp64_phase_q) begin - stage_mem_issue_addr = sq_check_addr_q + 32'd4; - end - end + assign stage_mem_issue_addr = sq_check_addr_q; // Gate stage_mem_issue on !i_flush_all too. See comment on // launch_mem_issue below for the full rationale. @@ -1532,9 +1513,9 @@ module load_queue #( lu_cache_is_unsigned = !sq_check_sign_ext_q; end - // SQ-forward extraction: i_sq_forward.data carries a memory-image word for - // non-DOUBLE loads (the fwd unit shifts sub-word store data to its byte - // lanes), so integer byte/half loads extract exactly like an L0 hit. The + // SQ-forward extraction: i_sq_forward.data carries the aligned-dword memory + // image at the load's dword (the fwd unit shifts store data to its byte + // lanes), so integer loads extract from it exactly like a memory beat. The // flags/address are shared with u_cache_load_unit — same staged load, and // the forward and cache-hit paths are mutually exclusive by construction. logic [XLEN-1:0] lu_fwd_out; @@ -1543,7 +1524,7 @@ module load_queue #( .i_is_load_halfword (lu_cache_is_half), .i_is_load_unsigned (lu_cache_is_unsigned), .i_data_memory_address (sq_check_addr_q), - .i_data_memory_read_data (i_sq_forward.data[XLEN-1:0]), + .i_data_memory_read_data (i_sq_forward.data), .o_data_loaded_from_memory(lu_fwd_out) ); @@ -1554,11 +1535,9 @@ module load_queue #( // sq_do_forward, lu_cache_out, lu_data_out, etc.) for readable tool output. always_comb begin - lq_data_lo_we = '0; - lq_data_hi_we = '0; + lq_data_we = '0; lq_data_wr_addr = '0; - lq_data_lo_wd = '0; - lq_data_hi_wd = '0; + lq_data_wd = '0; // --------------------------------------------------------------- // Port 0: dedicated to memory response. @@ -1571,24 +1550,15 @@ module load_queue #( lq_data_wr_addr[0] = issued_idx; if (issued_is_amo) begin // AMO read: don't write data yet (port 1 handles after AMO write) - end else if (issued_is_fp - && riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE - && !issued_fp64_phase) begin - // FLD phase 0: write lo only - lq_data_lo_we[0] = 1'b1; - lq_data_lo_wd[0] = lu_data_out; - end else if (issued_is_fp - && riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE - && issued_fp64_phase) begin - // FLD phase 1: write hi only - lq_data_hi_we[0] = 1'b1; - lq_data_hi_wd[0] = i_mem_read_data; + end else if (riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE) begin + // FLD (RV64 LD in M3): the full aligned beat in one write + lq_data_we[0] = 1'b1; + lq_data_wd[0] = i_mem_read_data; end else begin - // LR / Non-FLD: write lo, clear hi - lq_data_lo_we[0] = 1'b1; - lq_data_hi_we[0] = 1'b1; - lq_data_lo_wd[0] = lu_data_out; - lq_data_hi_wd[0] = '0; + // LR / FLW / INT: extracted result (FLW's word arm is its addressed + // raw word), zero-extended into FLEN + lq_data_we[0] = 1'b1; + lq_data_wd[0] = FLEN'(lu_data_out); end end @@ -1610,25 +1580,23 @@ module load_queue #( // --------------------------------------------------------------- if (i_rst_n && !i_flush_all) begin if (cache_hit_fast_path) begin - lq_data_lo_we[1] = 1'b1; - lq_data_hi_we[1] = 1'b1; + lq_data_we[1] = 1'b1; lq_data_wr_addr[1] = sq_check_idx; - lq_data_lo_wd[1] = sq_check_is_fp_q ? cache_lookup_data : lu_cache_out; - lq_data_hi_wd[1] = '0; + // FLD takes the full cached dword line; FLW/INT extract from it + // (FLW's word arm is its addressed raw word). + lq_data_wd[1] = (sq_check_size_q == riscv_pkg::MEM_SIZE_DOUBLE) + ? cache_lookup_data : FLEN'(lu_cache_out); end else if (sq_do_forward) begin - lq_data_lo_we[1] = 1'b1; - lq_data_hi_we[1] = 1'b1; + lq_data_we[1] = 1'b1; lq_data_wr_addr[1] = sq_check_idx; - // FP forwards (FLW word image / FLD full 64-bit) take the payload - // raw; integer loads extract byte/half + sign from the image word. - lq_data_lo_wd[1] = sq_check_is_fp_q ? i_sq_forward.data[XLEN-1:0] : lu_fwd_out; - lq_data_hi_wd[1] = i_sq_forward.data[FLEN-1:XLEN]; + // FLD takes the forwarded dword image raw; FLW/INT extract their + // addressed word/half/byte from the image beat. + lq_data_wd[1] = (sq_check_size_q == riscv_pkg::MEM_SIZE_DOUBLE) + ? i_sq_forward.data : FLEN'(lu_fwd_out); end else if (amo_state == AMO_WRITE_ACTIVE && i_amo_mem_write_done) begin - lq_data_lo_we[1] = 1'b1; - lq_data_hi_we[1] = 1'b1; + lq_data_we[1] = 1'b1; lq_data_wr_addr[1] = amo_entry_idx; - lq_data_lo_wd[1] = amo_old_value; - lq_data_hi_wd[1] = '0; + lq_data_wd[1] = FLEN'(amo_old_value); end end end @@ -1645,12 +1613,10 @@ module load_queue #( // Full-flush-cycle and already-pending stale responses remain ineligible. // MMIO/LR/AMO exclusions and the cached-tier store-invalidation guards below // are unchanged. - // issued_addr already encodes the FLD phase 1 +4 (it was captured from - // launch_mem_issue_addr, which applied the +4 inside stage_mem_issue_addr), - // so the fill address is just the snapshot directly. Critically, this path - // no longer goes through the lq_address_issued LUTRAM read or the +4 carry - // chain, which were the dominant prefix of the cone reaching the data - // memory's ADDRARDADDR pin via lq_l0_cache.lookup_fill_bypass. + // The fill address is the issued_addr snapshot directly. Critically, this + // path does not go through the lq_address_issued LUTRAM read, which was + // the dominant prefix of the cone reaching the data memory's ADDRARDADDR + // pin via lq_l0_cache.lookup_fill_bypass. assign cache_fill_response_valid = i_mem_read_valid && mem_outstanding && !i_flush_all && !drop_mem_response_pending && lq_valid[issued_idx]; assign cache_fill_valid = cache_fill_response_valid @@ -1682,7 +1648,11 @@ module load_queue #( if (amo_state == AMO_WRITE_ACTIVE) begin o_amo_mem_write_en = 1'b1; o_amo_mem_write_addr = amo_write_addr_q; - o_amo_mem_write_data = amo_write_data_q; + // Word result replicated across the beat; the router's word-lane + // strobes (from addr[2]) select the addressed half. Replicate the + // 32-bit word explicitly so an XLEN-wide result register (RV64) does + // not silently truncate — AMO*.W stays word-sized at any XLEN. + o_amo_mem_write_data = {(riscv_pkg::MemDataBits / 32) {amo_write_data_q[31:0]}}; end end @@ -1719,17 +1689,15 @@ module load_queue #( issue_cdb_result.valid = issue_cdb_found && !i_flush_en; issue_cdb_result.tag = lq_rob_tag[issue_cdb_idx]; - if (lq_is_fp[issue_cdb_idx]) begin - if (riscv_pkg::mem_size_e'(lq_size_issue_cdb_rd) == riscv_pkg::MEM_SIZE_DOUBLE) begin - // FLD: raw 64-bit data (lo + hi from LUTRAM) - issue_cdb_result.value = {lq_data_hi_rd, lq_data_lo_rd}; - end else begin - // FLW: NaN-box 32-bit to 64-bit - issue_cdb_result.value = {32'hFFFF_FFFF, lq_data_lo_rd}; - end + if (riscv_pkg::mem_size_e'(lq_size_issue_cdb_rd) == riscv_pkg::MEM_SIZE_DOUBLE) begin + // FLD (RV64 LD in M3): raw 64-bit beat from the LUTRAM + issue_cdb_result.value = lq_data_rd; + end else if (lq_is_fp[issue_cdb_idx]) begin + // FLW: NaN-box the stored 32-bit word + issue_cdb_result.value = {32'hFFFF_FFFF, lq_data_rd[31:0]}; end else begin - // INT load: zero-extend XLEN to FLEN - issue_cdb_result.value = {{(FLEN - XLEN) {1'b0}}, lq_data_lo_rd}; + // INT load: stored value is already zero-extended into FLEN + issue_cdb_result.value = lq_data_rd; end end @@ -1756,8 +1724,9 @@ module load_queue #( // otherwise idle. Drives cdb_stage directly from the response-side formatted // result, shaving one head-wait cycle per eligible load. Falls back to the // standard data_valid path when cdb_stage is busy or when an older entry is - // already firing through issue_cdb_fire. AMOs (need write phase) and FLDs - // (two-phase, phase-1 value needs LUTRAM lo read) stay on the standard path. + // already firing through issue_cdb_fire. AMOs (need write phase) and + // DOUBLE-size memory responses stay on the standard path (the L0-hit and + // SQ-forward bypasses below do carry DOUBLE payloads). logic resp_bypass_ok; logic resp_bypass_fire; logic cache_hit_bypass_fire; @@ -1770,7 +1739,7 @@ module load_queue #( assign resp_bypass_ok = accept_mem_response && !issued_is_amo && - !(issued_is_fp && (riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE)); + !(riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE); assign resp_bypass_fire = cdb_stage_slot_available && !issue_cdb_fire && resp_bypass_ok && !i_flush_en; @@ -1796,7 +1765,7 @@ module load_queue #( logic misalign_bypass_data_sel; assign resp_bypass_data_sel = i_mem_read_valid && mem_outstanding && !drop_mem_response_pending && lq_valid[issued_idx] && !issued_is_amo && - !(issued_is_fp && (riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE)); + !(riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE); assign misalign_bypass_data_sel = !resp_bypass_data_sel && sq_check_misaligned; // cache_hit_fast_path is already flush-gated at its own assign. @@ -1812,7 +1781,7 @@ module load_queue #( // !sq_no_older_store and can_forward, hence i_sq_forward.match; a cache hit // with older stores resident can only pass sq_can_issue via the !match // disjunct), and forwarded FLDs are eligible — the SQ delivers the full - // 64-bit payload in one probe, unlike the two-phase FLD memory path. + // 64-bit image in one probe. // !i_flush_en keeps a same-cycle partial flush of the staged load off the // CDB (falls back to the standard path, where the flush cleans the entry). logic fwd_bypass_fire; @@ -1824,39 +1793,43 @@ module load_queue #( fwd_bypass_fire; // Mirror issue_cdb_result formatting, but sourced from the response-side - // signals (lu_data_out / lu_cache_out / raw word) instead of the LUTRAM. + // signals (lu_data_out / lu_cache_out / image beat) instead of the LUTRAM. + // DOUBLE responses never reach this arm (resp_bypass_ok excludes them). always_comb begin if (issued_is_fp) begin - // FLW: NaN-box raw 32-bit word - resp_bypass_value = {32'hFFFF_FFFF, lu_data_out}; + // FLW: NaN-box the addressed raw word (lu_data_out's word arm) + resp_bypass_value = {32'hFFFF_FFFF, lu_data_out[31:0]}; end else begin // INT / LR: zero-extend byte/half/word extracted value - resp_bypass_value = {{(FLEN - XLEN) {1'b0}}, lu_data_out}; + resp_bypass_value = FLEN'(lu_data_out); end end always_comb begin - if (sq_check_is_fp_q) begin - // FLW from L0: NaN-box raw cache data (L0 fast path gates out FLD) - cache_hit_bypass_value = {32'hFFFF_FFFF, cache_lookup_data}; + if (sq_check_size_q == riscv_pkg::MEM_SIZE_DOUBLE) begin + // FLD from L0: the full cached dword line + cache_hit_bypass_value = cache_lookup_data; + end else if (sq_check_is_fp_q) begin + // FLW from L0: NaN-box the addressed word of the cached beat + cache_hit_bypass_value = {32'hFFFF_FFFF, lu_cache_out[31:0]}; end else begin // INT from L0: cache-path load_unit already did byte/half extract - cache_hit_bypass_value = {{(FLEN - XLEN) {1'b0}}, lu_cache_out}; + cache_hit_bypass_value = FLEN'(lu_cache_out); end end // Forward-bypass payload: mirrors the forward write-port formatting. logic [FLEN-1:0] fwd_bypass_value; always_comb begin - if (!sq_check_is_fp_q) begin - // INT: fwd-path load_unit already did byte/half extract + extension - fwd_bypass_value = {{(FLEN - XLEN) {1'b0}}, lu_fwd_out}; - end else if (sq_check_size_q == riscv_pkg::MEM_SIZE_DOUBLE) begin - // FLD from FSD: full 64-bit payload straight from the SQ + if (sq_check_size_q == riscv_pkg::MEM_SIZE_DOUBLE) begin + // FLD from FSD: full 64-bit image straight from the SQ fwd_bypass_value = i_sq_forward.data; + end else if (sq_check_is_fp_q) begin + // FLW: NaN-box the addressed word of the forwarded image + fwd_bypass_value = {32'hFFFF_FFFF, lu_fwd_out[31:0]}; end else begin - // FLW: NaN-box the forwarded memory-image word - fwd_bypass_value = {32'hFFFF_FFFF, i_sq_forward.data[XLEN-1:0]}; + // INT: fwd-path load_unit already did byte/half extract + extension + fwd_bypass_value = FLEN'(lu_fwd_out); end end @@ -2327,18 +2300,12 @@ module load_queue #( // response identity/result and registered write payload are captured // in the data-payload block below. amo_state <= AMO_WRITE_ACTIVE; - end else if (issued_is_fp && - riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE && - !issued_fp64_phase) begin - // FLD phase 0: re-issue for phase 1; - // lq_fp64_phase in no-reset block - lq_issued[issued_idx] <= 1'b0; // Re-issue for phase 1 end else begin - // Non-AMO, non-FLD-phase-0 (LR, FLW, INT load, FLD phase 1): - // the completion bypass may have captured this result directly - // into cdb_stage the same cycle via resp_bypass_fire. In that - // case skip the data_valid/LUTRAM write — free_entry_en releases - // the slot. LR still arms reservation_valid either way. + // Non-AMO (LR, FLW, FLD, INT load): the completion bypass may have + // captured this result directly into cdb_stage the same cycle via + // resp_bypass_fire. In that case skip the data_valid/LUTRAM + // write — free_entry_en releases the slot. LR still arms + // reservation_valid either way. if (issued_is_lr) reservation_valid <= 1'b1; if (!resp_bypass_fire) begin // Standard path: let the priority encoder pick next cycle. @@ -2423,19 +2390,6 @@ module load_queue #( lq_is_lr[i] <= i_alloc_2.is_lr; lq_is_amo[i] <= i_alloc_2.is_amo; end - - // The registered physical-generation pulse arrives one cycle after - // allocation and initializes the resident phase bit from local state. - // This keeps the live free-target search off the phase-bit D path. - if (dep_replaced_oh[i]) begin - lq_fp64_phase[i] <= 1'b0; - end - end - // FLD phase advance: set phase 1 after phase 0 memory response - if (accept_mem_response && issued_is_fp && - riscv_pkg::mem_size_e'(issued_size) == riscv_pkg::MEM_SIZE_DOUBLE && - !issued_fp64_phase) begin - lq_fp64_phase[issued_idx] <= 1'b1; end end @@ -2490,7 +2444,6 @@ module load_queue #( logic issue_mem_is_fp; logic issue_mem_sign_ext; logic issue_mem_is_mmio; - logic issue_mem_fp64_phase; logic issue_mem_is_lr; logic issue_mem_is_amo; (* keep = "true", max_fanout = 32 *) logic sq_check_payload_en; @@ -2501,7 +2454,6 @@ module load_queue #( logic sq_check_is_fp_next; logic sq_check_sign_ext_next; logic sq_check_is_mmio_next; - logic sq_check_fp64_phase_next; logic sq_check_is_lr_next; logic sq_check_is_amo_next; assign sq_check_payload_en = sq_check_capture || sq_check_replace; @@ -2517,10 +2469,6 @@ module load_queue #( : lq_sign_ext[issue_mem_stored_idx]; assign issue_mem_is_mmio = issue_mem_from_update ? i_addr_update.is_mmio : lq_is_mmio[issue_mem_stored_idx]; - // An address update only matches an entry without a resident address, so it - // is necessarily the first memory phase. The constant also keeps the - // resident phase read out of the current-update payload cone. - assign issue_mem_fp64_phase = issue_mem_from_update ? 1'b0 : lq_fp64_phase[issue_mem_stored_idx]; assign issue_mem_is_lr = issue_mem_from_update ? lq_is_lr[update_issue_payload_idx] : lq_is_lr[issue_mem_stored_idx]; assign issue_mem_is_amo = issue_mem_from_update ? lq_is_amo[update_issue_payload_idx] @@ -2536,7 +2484,6 @@ module load_queue #( assign sq_check_is_fp_next = issue_mem_is_fp; assign sq_check_sign_ext_next = issue_mem_sign_ext; assign sq_check_is_mmio_next = issue_mem_is_mmio; - assign sq_check_fp64_phase_next = issue_mem_fp64_phase; assign sq_check_is_lr_next = issue_mem_is_lr; assign sq_check_is_amo_next = issue_mem_is_amo; @@ -2629,16 +2576,6 @@ module load_queue #( .R (1'b0) ); - FDRE #( - .INIT(1'b0) - ) sq_check_fp64_phase_ff ( - .C (i_clk), - .CE(sq_check_payload_en), - .D (sq_check_fp64_phase_next), - .Q (sq_check_fp64_phase_q), - .R (1'b0) - ); - FDRE #( .INIT(1'b0) ) sq_check_is_lr_ff ( @@ -2661,17 +2598,16 @@ module load_queue #( `else always_ff @(posedge i_clk) begin if (sq_check_payload_en) begin - sq_check_idx <= sq_check_idx_next; - sq_check_rob_tag_q <= sq_check_rob_tag_next; - sq_check_addr_q <= sq_check_addr_next; - sq_check_addr_q_b <= sq_check_addr_next; - sq_check_size_q <= sq_check_size_next; - sq_check_is_fp_q <= sq_check_is_fp_next; - sq_check_sign_ext_q <= sq_check_sign_ext_next; - sq_check_is_mmio_q <= sq_check_is_mmio_next; - sq_check_fp64_phase_q <= sq_check_fp64_phase_next; - sq_check_is_lr_q <= sq_check_is_lr_next; - sq_check_is_amo_q <= sq_check_is_amo_next; + sq_check_idx <= sq_check_idx_next; + sq_check_rob_tag_q <= sq_check_rob_tag_next; + sq_check_addr_q <= sq_check_addr_next; + sq_check_addr_q_b <= sq_check_addr_next; + sq_check_size_q <= sq_check_size_next; + sq_check_is_fp_q <= sq_check_is_fp_next; + sq_check_sign_ext_q <= sq_check_sign_ext_next; + sq_check_is_mmio_q <= sq_check_is_mmio_next; + sq_check_is_lr_q <= sq_check_is_lr_next; + sq_check_is_amo_q <= sq_check_is_amo_next; end end `endif @@ -2683,8 +2619,7 @@ module load_queue #( // issued_idx → lq_*[issued_idx] → cache_fill_addr → lq_l0_cache lookup // cone that fed the data_memory ADDRARDADDR pin via lookup_fill_bypass. // The captured fields are stable for the lifetime of the outstanding - // load (allocation-time fields don't change once written; sq_check_*_q - // already encodes the active FLD phase at launch time). + // load (allocation-time fields don't change once written). always_ff @(posedge i_clk) begin if (!i_rst_n || i_flush_all) begin issued_cached_line_invalidated <= 1'b0; @@ -2697,17 +2632,16 @@ module load_queue #( always_ff @(posedge i_clk) begin if (o_mem_read_en) begin - issued_idx <= launch_mem_issue_idx; - issued_addr <= launch_mem_issue_addr; - issued_size <= launch_mem_issue_size; - issued_is_fp <= sq_check_is_fp_q; - issued_is_lr <= sq_check_is_lr_q; - issued_is_amo <= sq_check_is_amo_q; - issued_is_mmio <= sq_check_is_mmio_q; - issued_is_cached <= launching_is_cached; - issued_sign_ext <= sq_check_sign_ext_q; - issued_fp64_phase <= sq_check_fp64_phase_q; - issued_rob_tag <= sq_check_rob_tag_q; + issued_idx <= launch_mem_issue_idx; + issued_addr <= launch_mem_issue_addr; + issued_size <= launch_mem_issue_size; + issued_is_fp <= sq_check_is_fp_q; + issued_is_lr <= sq_check_is_lr_q; + issued_is_amo <= sq_check_is_amo_q; + issued_is_mmio <= sq_check_is_mmio_q; + issued_is_cached <= launching_is_cached; + issued_sign_ext <= sq_check_sign_ext_q; + issued_rob_tag <= sq_check_rob_tag_q; if (sq_check_is_amo_q) begin issued_amo_kind <= lq_amo_kind[launch_mem_issue_idx]; issued_amo_rs2 <= lq_amo_rs2_rd; @@ -2723,12 +2657,17 @@ module load_queue #( // block above installs the next owner's values. issued_addr likewise still // carries the response owner's exact launch address throughout this edge. // ----------------------------------------------------------------- + // AMOs are word-sized (RV64A widens this in M3): select the addressed word + // of the response beat by addr[2] before the ALU sees it. + logic [XLEN-1:0] amo_beat_word; + assign amo_beat_word = XLEN'(i_mem_read_data[issued_addr[2]*32+:32]); + always_ff @(posedge i_clk) begin if (accept_mem_response && issued_is_amo) begin - amo_old_value <= i_mem_read_data; + amo_old_value <= amo_beat_word; amo_entry_idx <= issued_idx; amo_write_addr_q <= issued_addr; - amo_write_data_q <= amo_compute(issued_amo_kind, i_mem_read_data, issued_amo_rs2); + amo_write_data_q <= amo_compute(issued_amo_kind, amo_beat_word, issued_amo_rs2); end end @@ -3100,7 +3039,7 @@ module load_queue #( p_partial_flush_response_fills_l0 : assert (cache_fill_valid); p_partial_flush_fill_not_accepted : assert (!accept_mem_response); p_partial_flush_fill_is_drained : assert (drop_mem_response_now); - p_partial_flush_fill_skips_lq_data_write : assert (!lq_data_lo_we[0] && !lq_data_hi_we[0]); + p_partial_flush_fill_skips_lq_data_write : assert (!lq_data_we[0]); end end diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_unit.sv b/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_unit.sv index 491a2280..3725a08d 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_unit.sv +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/load_unit.sv @@ -17,27 +17,31 @@ /* * Load Unit - Data extraction and sign/zero extension for RISC-V load instructions * - * This module processes raw 32-bit memory read data and extracts the appropriate - * byte(s) based on load instruction type and address alignment. It handles all - * RISC-V base integer load instructions: + * Consumes one aligned MemDataBits (64-bit) beat carrying the addressed dword + * (docs/rv64/m1_data_tier.md) and extracts the addressed byte / halfword / + * word for the integer load types: * - * LB - Load Byte (sign-extended to 32 bits) - * LBU - Load Byte Unsigned (zero-extended to 32 bits) - * LH - Load Halfword (sign-extended to 32 bits) - * LHU - Load Halfword Unsigned (zero-extended to 32 bits) - * LW - Load Word (full 32 bits, no extension) + * LB - Load Byte (sign-extended) + * LBU - Load Byte Unsigned (zero-extended) + * LH - Load Halfword (sign-extended) + * LHU - Load Halfword Unsigned (zero-extended) + * LW - Load Word (addr[2] selects the beat's word) + * + * FP doubles do not pass through this unit: the load queue consumes the full + * beat directly for FLD. (RV64 LD/LWU extension semantics land with the + * Phase 1 M3 decode work; at XLEN=32 the word arm is the full result.) * * Byte Selection Logic: - * - LB/LBU: addr[1:0] selects one of four bytes (0, 1, 2, or 3) - * - LH/LHU: addr[1] selects lower or upper halfword - * - LW: uses full 32-bit word + * - LB/LBU: addr[2:0] selects one of eight beat bytes + * - LH/LHU: addr[2:1] selects one of four beat halfwords + * - LW: addr[2] selects the low or high beat word * * Related Modules: * - load_queue.sv: Instantiates this unit for memory and L0-cache result paths - * - lq_l0_cache.sv: Provides cached words that this unit extracts/sign-extends + * - lq_l0_cache.sv: Provides cached beats that this unit extracts/sign-extends */ module load_unit #( - parameter int unsigned XLEN = 32 + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( // Load type flags (from instruction decode) input logic i_is_load_byte, // LB or LBU instruction @@ -45,83 +49,66 @@ module load_unit #( input logic i_is_load_unsigned, // LBU or LHU (zero-extend instead of sign-extend) // Memory interface - input logic [XLEN-1:0] i_data_memory_address, // Address for byte selection - input logic [XLEN-1:0] i_data_memory_read_data, // Raw 32-bit data from memory + input logic [XLEN-1:0] i_data_memory_address, // Address for lane selection + input logic [riscv_pkg::MemDataBits-1:0] i_data_memory_read_data, // Aligned beat // Output output logic [XLEN-1:0] o_data_loaded_from_memory // Extracted and extended result ); // =========================================================================== - // Byte/Halfword Extraction and Sign/Zero Extension + // Byte/Halfword/Word Extraction and Sign/Zero Extension // =========================================================================== // - // Memory word layout (little-endian): - // Byte: [ 3 | 2 | 1 | 0 ] - // Halfword: [ high | low ] - // Bit: [31:24|23:16|15:8 | 7:0] - // - // Address bits select position: - // addr[1:0] = 00 -> byte 0 (bits 7:0) - // addr[1:0] = 01 -> byte 1 (bits 15:8) - // addr[1:0] = 10 -> byte 2 (bits 23:16) - // addr[1:0] = 11 -> byte 3 (bits 31:24) - // addr[1] = 0 -> lower halfword (bits 15:0) - // addr[1] = 1 -> upper halfword (bits 31:16) + // Beat layout (little-endian): byte lane i is byte address {addr[31:3], i}. // - // TIMING OPTIMIZATION: Pre-compute sign-extended results for all byte/halfword - // positions in PARALLEL. The late-arriving address (from CARRY8 chain) only - // controls the final mux, not the sign extension logic. This breaks the path: - // address -> byte_select -> sign_bit_select -> sign_extension - // Into: - // address -> final_mux (short) - // data -> sign_extension (parallel, doesn't wait for address) + // TIMING OPTIMIZATION (preserved from the 32-bit version): pre-compute + // sign-extended results for every lane in PARALLEL. The late-arriving + // address (from a CARRY8 chain) only controls the final muxes, not the + // sign-extension logic. - // Extract individual bytes - logic [7:0] byte0, byte1, byte2, byte3; - assign byte0 = i_data_memory_read_data[7:0]; - assign byte1 = i_data_memory_read_data[15:8]; - assign byte2 = i_data_memory_read_data[23:16]; - assign byte3 = i_data_memory_read_data[31:24]; + localparam int unsigned BeatBytes = riscv_pkg::MemStrbBits; + localparam int unsigned BeatHalves = riscv_pkg::MemDataBits / 16; + localparam int unsigned BeatWords = riscv_pkg::MemDataBits / 32; - // Pre-compute sign-extended bytes (all 4 in parallel) - logic [XLEN-1:0] byte0_ext, byte1_ext, byte2_ext, byte3_ext; - assign byte0_ext = {{(XLEN - 8) {i_is_load_unsigned ? 1'b0 : byte0[7]}}, byte0}; - assign byte1_ext = {{(XLEN - 8) {i_is_load_unsigned ? 1'b0 : byte1[7]}}, byte1}; - assign byte2_ext = {{(XLEN - 8) {i_is_load_unsigned ? 1'b0 : byte2[7]}}, byte2}; - assign byte3_ext = {{(XLEN - 8) {i_is_load_unsigned ? 1'b0 : byte3[7]}}, byte3}; + // Pre-compute sign-extended bytes (all lanes in parallel) + logic [XLEN-1:0] byte_ext[BeatBytes]; + for (genvar b = 0; b < BeatBytes; b++) begin : gen_byte_ext + assign byte_ext[b] = { + {(XLEN - 8) {i_is_load_unsigned ? 1'b0 : i_data_memory_read_data[b*8+7]}}, + i_data_memory_read_data[b*8+:8] + }; + end - // Extract halfwords - logic [15:0] half_lo, half_hi; - assign half_lo = i_data_memory_read_data[15:0]; - assign half_hi = i_data_memory_read_data[31:16]; + // Pre-compute sign-extended halfwords (all lanes in parallel) + logic [XLEN-1:0] half_ext[BeatHalves]; + for (genvar h = 0; h < BeatHalves; h++) begin : gen_half_ext + assign half_ext[h] = { + {(XLEN - 16) {i_is_load_unsigned ? 1'b0 : i_data_memory_read_data[h*16+15]}}, + i_data_memory_read_data[h*16+:16] + }; + end - // Pre-compute sign-extended halfwords (both in parallel) - logic [XLEN-1:0] half_lo_ext, half_hi_ext; - assign half_lo_ext = {{(XLEN - 16) {i_is_load_unsigned ? 1'b0 : half_lo[15]}}, half_lo}; - assign half_hi_ext = {{(XLEN - 16) {i_is_load_unsigned ? 1'b0 : half_hi[15]}}, half_hi}; + // Word lanes (LW consumes the addressed word; extension semantics for RV64 + // LW/LWU arrive with the M3 decode work - at XLEN=32 this is the result). + logic [31:0] word_lane[BeatWords]; + for (genvar w = 0; w < BeatWords; w++) begin : gen_word_lane + assign word_lane[w] = i_data_memory_read_data[w*32+:32]; + end - // Final mux: address selects pre-computed result - // Late-arriving address only controls this final selection + // Final muxes: the late-arriving address selects pre-computed results. logic [XLEN-1:0] byte_result; logic [XLEN-1:0] halfword_result; + logic [XLEN-1:0] word_result; - always_comb begin - case (i_data_memory_address[1:0]) - 2'b00: byte_result = byte0_ext; - 2'b01: byte_result = byte1_ext; - 2'b10: byte_result = byte2_ext; - 2'b11: byte_result = byte3_ext; - default: byte_result = byte0_ext; - endcase - end - - assign halfword_result = i_data_memory_address[1] ? half_hi_ext : half_lo_ext; + assign byte_result = byte_ext[i_data_memory_address[2:0]]; + assign halfword_result = half_ext[i_data_memory_address[2:1]]; + assign word_result = XLEN'(word_lane[i_data_memory_address[2]]); // Type selection: is_load_byte and is_load_halfword are registered (early) assign o_data_loaded_from_memory = i_is_load_byte ? byte_result : i_is_load_halfword ? halfword_result : - i_data_memory_read_data; + word_result; endmodule : load_unit diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_l0_cache.sv b/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_l0_cache.sv index 27b9e333..7576346c 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_l0_cache.sv +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/load_queue/lq_l0_cache.sv @@ -18,40 +18,42 @@ * LQ L0 Data Cache * * Simplified OoO-compatible L0 data cache for the Load Queue. - * Direct-mapped, word-aligned entries with FF-based valid bits and - * LUTRAM-backed tag/data arrays. + * Direct-mapped, dword-granule (aligned 8-byte) lines with FF-based valid + * bits and LUTRAM-backed tag/data arrays (docs/rv64/m1_data_tier.md). * * The module uses simple address/data ports suitable for the LQ's OoO issue path. * * Features: - * - Combinational lookup (hit in same cycle as address) - * - Fill on memory response + * - Combinational lookup (hit in same cycle as address); the line carries + * the full aligned dword, consumers extract by addr[2:0] + * - Fill on memory response (one full beat per fill) * - MMIO addresses always miss (addr[31:30] == 2'b01 quadrant; DDR at * 0x8000_0000+ is cacheable) * - i_flush_all clears every valid bit, but the LQ ties it to 0: L0 * contents always reflect architectural memory, so lines stay hot * across pipeline flushes * - Per-address invalidation port (driven by SQ store-write launch and - * AMO completion) + * AMO completion); any store invalidates its containing dword line + * (conservative for sub-dword stores — same policy as the word-granule + * version, one granule coarser) */ module lq_l0_cache #( - parameter int unsigned DEPTH = 128, - parameter int unsigned XLEN = 32, - parameter int unsigned MMIO_ADDR = 32'h4000_0000 + parameter int unsigned DEPTH = 128, + parameter int unsigned XLEN = riscv_pkg::XLEN ) ( input logic i_clk, input logic i_rst_n, // Lookup (combinational read) - input logic [XLEN-1:0] i_lookup_addr, - output logic o_lookup_hit, - output logic [XLEN-1:0] o_lookup_data, // raw word at word-aligned addr + input logic [ XLEN-1:0] i_lookup_addr, + output logic o_lookup_hit, + output logic [riscv_pkg::MemDataBits-1:0] o_lookup_data, // aligned dword line // Fill (write on memory response) - input logic i_fill_valid, - input logic [XLEN-1:0] i_fill_addr, - input logic [XLEN-1:0] i_fill_data, + input logic i_fill_valid, + input logic [ XLEN-1:0] i_fill_addr, + input logic [riscv_pkg::MemDataBits-1:0] i_fill_data, // Invalidate valid bits on the clock edge. input logic i_invalidate_valid, @@ -71,41 +73,49 @@ module lq_l0_cache #( // Local Parameters // =========================================================================== localparam int unsigned IndexWidth = $clog2(DEPTH); - localparam int unsigned TagWidth = XLEN - 2 - IndexWidth; // word-aligned: skip bit[1:0] + // Tags cover the PHYSICAL address above the dword index: bits + // [31 : 3+IndexWidth]. The sub-4-GiB map makes bits above 31 dead weight + // in a compare that sits on the historically critical lookup-hit cone, so + // the tag stays 32-bit-relative at any XLEN (D3: producers canonicalize + // bits [XLEN-1:32] to zero before addresses reach the memory tier). + localparam int unsigned TagWidth = 32 - 3 - IndexWidth; // =========================================================================== // Storage // =========================================================================== - logic [ DEPTH-1:0] valid; - logic [ TagWidth-1:0] tag_lookup_rd; - logic [ TagWidth-1:0] tag_inv_rd; - logic [ XLEN-1:0] data_lookup_rd; + logic [DEPTH-1:0] valid; + logic [TagWidth-1:0] tag_lookup_rd; + logic [TagWidth-1:0] tag_inv_rd; + logic [riscv_pkg::MemDataBits-1:0] data_lookup_rd; // =========================================================================== // Address decomposition // =========================================================================== - // Word-aligned: addr[1:0] ignored, index = addr[2 +: IndexWidth], - // tag = addr[(2+IndexWidth) +: TagWidth] + // Dword-granule: addr[2:0] ignored, index = addr[3 +: IndexWidth], + // tag = addr[(3+IndexWidth) +: TagWidth] - wire [IndexWidth-1:0] lookup_index = i_lookup_addr[2+:IndexWidth]; - wire [ TagWidth-1:0] lookup_tag = i_lookup_addr[(2+IndexWidth)+:TagWidth]; + wire [IndexWidth-1:0] lookup_index = i_lookup_addr[3+:IndexWidth]; + wire [TagWidth-1:0] lookup_tag = i_lookup_addr[(3+IndexWidth)+:TagWidth]; // MMIO = the 01 address quadrant; the cached (DDR) region (10 quadrant) is // cacheable here just like the low BRAM range (stores invalidate; reset clears). - wire lookup_mmio = (i_lookup_addr[XLEN-1:XLEN-2] == 2'b01); + // Decoded at FIXED physical bits [31:30], never [XLEN-1:XLEN-2]: at XLEN=64 + // the relative form tests always-zero bits 63:62 and MMIO would silently + // become cacheable (stale L0 hits on device registers). + wire lookup_mmio = (i_lookup_addr[31:30] == 2'b01); - wire [IndexWidth-1:0] fill_index = i_fill_addr[2+:IndexWidth]; - wire [ TagWidth-1:0] fill_tag = i_fill_addr[(2+IndexWidth)+:TagWidth]; + wire [IndexWidth-1:0] fill_index = i_fill_addr[3+:IndexWidth]; + wire [TagWidth-1:0] fill_tag = i_fill_addr[(3+IndexWidth)+:TagWidth]; - wire [IndexWidth-1:0] inv_index = i_invalidate_addr[2+:IndexWidth]; - wire [ TagWidth-1:0] inv_tag = i_invalidate_addr[(2+IndexWidth)+:TagWidth]; + wire [IndexWidth-1:0] inv_index = i_invalidate_addr[3+:IndexWidth]; + wire [TagWidth-1:0] inv_tag = i_invalidate_addr[(3+IndexWidth)+:TagWidth]; - wire [IndexWidth-1:0] lookup_inv_index = i_lookup_invalidate_addr[2+:IndexWidth]; - wire [ TagWidth-1:0] lookup_inv_tag = i_lookup_invalidate_addr[(2+IndexWidth)+:TagWidth]; - logic invalidate_fill_entry; - logic invalidate_existing_entry; - logic lookup_hit_array; - logic lookup_fill_bypass; - logic lookup_invalidated; + wire [IndexWidth-1:0] lookup_inv_index = i_lookup_invalidate_addr[3+:IndexWidth]; + wire [TagWidth-1:0] lookup_inv_tag = i_lookup_invalidate_addr[(3+IndexWidth)+:TagWidth]; + logic invalidate_fill_entry; + logic invalidate_existing_entry; + logic lookup_hit_array; + logic lookup_fill_bypass; + logic lookup_invalidated; // Tags are written only on fill and read at two independent addresses // (lookup and invalidate), so duplicate the simple dual-port RAM once @@ -138,7 +148,7 @@ module lq_l0_cache #( // an ideal fit for a small LUTRAM rather than a bank of FFs. sdp_dist_ram #( .ADDR_WIDTH(IndexWidth), - .DATA_WIDTH(XLEN) + .DATA_WIDTH(riscv_pkg::MemDataBits) ) u_data_ram ( .i_clk, .i_write_enable (i_fill_valid), @@ -202,7 +212,7 @@ module lq_l0_cache #( // replaces a DIFFERENT tag in that direct-mapped slot. If the fill and // invalidate target the same tag, the invalidate must win; otherwise a // load response can reinsert stale data into the cache in the same cycle - // that a committed store is trying to invalidate that word. + // that a committed store is trying to invalidate that dword. if (invalidate_fill_entry || invalidate_existing_entry) begin valid[inv_index] <= 1'b0; end @@ -230,7 +240,7 @@ module lq_l0_cache #( // MMIO addresses never hit always_comb begin - if (i_rst_n && (i_lookup_addr[XLEN-1:XLEN-2] == 2'b01)) begin + if (i_rst_n && (i_lookup_addr[31:30] == 2'b01)) begin p_mmio_never_hits : assert (!o_lookup_hit); end end @@ -250,7 +260,7 @@ module lq_l0_cache #( end end - // Fill followed by lookup at same word-aligned address should hit. + // Fill followed by lookup at same dword-aligned address should hit. // Track a single fill address across one cycle for a cleaner assertion. reg [XLEN-1:0] f_fill_addr_q; reg f_fill_valid_q; @@ -267,16 +277,16 @@ module lq_l0_cache #( always @(posedge i_clk) begin if (f_past_valid && i_rst_n && f_fill_valid_q && !i_flush_all - && i_lookup_addr[XLEN-1:2] == f_fill_addr_q[XLEN-1:2] - && !(i_lookup_addr[XLEN-1:XLEN-2] == 2'b01) + && i_lookup_addr[XLEN-1:3] == f_fill_addr_q[XLEN-1:3] + && !(i_lookup_addr[31:30] == 2'b01) && !(i_lookup_invalidate_valid - && i_lookup_invalidate_addr[2+:IndexWidth] - == f_fill_addr_q[2+:IndexWidth]) + && i_lookup_invalidate_addr[3+:IndexWidth] + == f_fill_addr_q[3+:IndexWidth]) && !(i_fill_valid - && i_fill_addr[2+:IndexWidth] - == f_fill_addr_q[2+:IndexWidth] - && i_fill_addr[(2+IndexWidth)+:TagWidth] - != f_fill_addr_q[(2+IndexWidth)+:TagWidth])) begin + && i_fill_addr[3+:IndexWidth] + == f_fill_addr_q[3+:IndexWidth] + && i_fill_addr[(3+IndexWidth)+:TagWidth] + != f_fill_addr_q[(3+IndexWidth)+:TagWidth])) begin p_fill_then_hit : assert (o_lookup_hit); end end diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv b/hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv index a5e9222f..d7cd9924 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/reorder_buffer/reorder_buffer.sv @@ -2749,8 +2749,14 @@ module reorder_buffer #( end end - // Retire trace: log every committed instruction (for debugging) + // Retire trace: log every committed instruction (for debugging). + // Format strings are XLEN-selected so the rv32 trace stays byte-identical + // while rv64 prints full 16-digit PCs/values (a %08x slice would silently + // truncate the debug artifact rv64 bring-up leans on). integer retire_trace_fd; + localparam string RetireTraceValFmt = (riscv_pkg::XLEN == 32) ? + "%0t pc=%08x rd=x%0d val=%08x\n" : "%0t pc=%016x rd=x%0d val=%016x\n"; + localparam string RetireTracePcFmt = (riscv_pkg::XLEN == 32) ? "%0t pc=%08x\n" : "%0t pc=%016x\n"; initial begin retire_trace_fd = $fopen("retire_trace.log", "w"); end @@ -2759,13 +2765,13 @@ module reorder_buffer #( if (head_dest_valid && !head_dest_rf && head_dest_reg != 5'd0) $fwrite( retire_trace_fd, - "%0t pc=%08x rd=x%0d val=%08x\n", + RetireTraceValFmt, $time, head_pc, head_dest_reg, - head_value_eff[31:0] + head_value_eff[riscv_pkg::XLEN-1:0] ); - else $fwrite(retire_trace_fd, "%0t pc=%08x\n", $time, head_pc); + else $fwrite(retire_trace_fd, RetireTracePcFmt, $time, head_pc); end end diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/README.md b/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/README.md index cd991503..dc0c205e 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/README.md +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/README.md @@ -13,16 +13,19 @@ Two concerns dominate the design: forwarding and ordering. **Forwarding.** A younger load may need data from an older store that's still in the SQ. When the LQ asks the SQ to disambiguate a load address, the SQ scans all entries combinationally for a -matching older store. A load forwards when the newest conflicting -store fully covers its bytes: FLD from an exact-address FSD (full -64-bit payload), or any byte/half/word load whose byte mask is a -subset of the store's byte mask within one word (either word of a -DOUBLE store counts as fully written). For covered loads the SQ -delivers a *memory-image word* — sub-word store data shifted to its -memory byte lanes — and the LQ applies the load's own byte/half -extraction and sign extension. If the newest conflicting store does -not cover the load's bytes, or some older store address isn't known -yet, the SQ tells the LQ to wait. The scan is combinational but the +matching older store. The overlap model is dword-granule +([docs/rv64/m1_data_tier.md](../../../../../docs/rv64/m1_data_tier.md)): +no access crosses its aligned 8-byte beat, so two accesses conflict +exactly when they share a dword address and their 8-lane byte masks +intersect. A load forwards when the newest conflicting store's lane +mask covers every lane the load reads (FLD from an exact-dword FSD is +the all-lanes case). For covered loads the SQ delivers the +*aligned-dword memory image* — store data shifted to its beat byte +lanes — and the LQ applies the load's own word/half/byte extraction +and sign extension (FLD consumes the image whole). If the newest +conflicting store does not cover the load's bytes, or some older +store address isn't known yet, the SQ tells the LQ to wait. The scan +is combinational but the result (`match`, `can_forward`, `data`, and `o_sq_all_older_addrs_known`) is registered, so the LQ sees it one cycle after raising `i_sq_check_valid`; this breaks the MEM_RS → SQ scan → LQ → BRAM path. @@ -71,9 +74,11 @@ driven by a registered drain cursor (`drain_idx_q`) — the first entry in ring order that is valid and not yet launched (`!sq_sent`) — which fires when that entry is committed (by the ROB) and has its address and data ready. The cursor never skips program order: if the oldest -undrained entry isn't ready, nothing fires. FSD on the 32-bit bus -takes two phases (low word at addr, high word at addr+4); the entry -has a phase bit and isn't freed until both writes complete. +undrained entry isn't ready, nothing fires. Every store size drains +in a single beat on the 64-bit data tier — FSD writes its aligned +dword with an all-lanes strobe (the old two-phase lo/hi drain and its +phase bit are gone), and sub-dword stores replicate their data across +the beat with the strobe selecting the addressed lanes. ## Registered memory-write outputs @@ -86,23 +91,23 @@ register-bounded and cuts hundreds of ps of setup slack. ## Pipelined drain -Plain fast-tier stores (BRAM, non-MMIO, non-FSD) complete exactly one -cycle after their bus cycle — the router's `sq_write_done_fast` is the -write-enable delayed one cycle — so consecutive plain drains overlap: -a new launch is allowed while the previous write's done is still in -flight, sustaining one store per cycle through a committed backlog. -The bookkeeping: +Plain fast-tier stores (BRAM, non-MMIO — single-beat FSD included) +complete exactly one cycle after their bus cycle — the router's +`sq_write_done_fast` is the write-enable delayed one cycle — so +consecutive plain drains overlap: a new launch is allowed while the +previous write's done is still in flight, sustaining one store per +cycle through a committed backlog. The bookkeeping: - `sq_sent` is set at **launch** (fire cycle) for completing writes, so the drain cursor moves to the next entry immediately; the done side - only frees entries (`sq_valid` clear) and advances the FSD phase. + only frees entries (`sq_valid` clear). - A 2-bit in-flight counter plus a 2-deep in-order metadata FIFO (entry index + completes flag, popped one per done) replace the old single `write_outstanding` bit. Dones arrive in launch order on the single write port, so FIFO slot 0 is always the oldest in-flight write. If a done stalls, the occupancy bound in the launch gate self-throttles the drain instead of overflowing the FIFO. -- Cached / MMIO / FSD writes stay strictly single-outstanding +- Cached / MMIO writes stay strictly single-outstanding (`write_inflight_special`): they only launch through the legacy serial gate, and nothing else launches until their done. A multi-cycle cached write therefore back-pressures the drain @@ -241,7 +246,7 @@ fanning out from one source FF. ## Verification Cocotb tests cover allocation including 2-wide cases, address/data update, -every store size, FSD two-phase, store-to-load forwarding, MMIO bypass, +every store size, single-beat FSD, store-to-load forwarding, MMIO bypass, partial/full flush, SC discard, same-edge drain removal plus 2-wide allocation, overlapping flush/discard removal, back-to-back pipelined drains (per-cycle bus sampling in `drain_pipelined_writes`), and constrained random. Inline diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/sq_forwarding_unit.sv b/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/sq_forwarding_unit.sv index 6f389b24..e622c883 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/sq_forwarding_unit.sv +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/sq_forwarding_unit.sv @@ -22,12 +22,22 @@ // from the FF-based SQ fields, // * Block 2 - newest-conflicting-store priority select, // * Block 3 - register the result (break MEM_RS -> SQ scan -> LQ path). +// +// Overlap model (docs/rv64/m1_data_tier.md): dword granule. No access +// crosses an aligned 8-byte beat (misaligned accesses trap before reaching +// this CAM, matching the old word model's alignment assumption), so two +// accesses conflict exactly when they share a dword address AND their 8-lane +// byte masks intersect; a store can forward when its lane mask covers the +// load's. The forwarded payload is the aligned-dword memory image at the +// load's dword — store data shifted to its byte lanes — from which the LQ +// extracts by the load's own addr[2:0] (or consumes whole for FLD/LD). +// // Forwarding data arrives as a per-entry FF mirror from store_queue. The scan -// registers only the winning entry index plus its extraction metadata; the -// mirrored payload is selected after that boundary during the LQ consume -// cycle. This keeps the SQ address compare / winner tree off all 64 payload -// D-pins without adding a pipeline stage. The helper functions are duplicated -// from store_queue (pure combinational, already duplicated across modules by +// registers only the winning entry index plus its store offset; the mirrored +// payload is selected after that boundary during the LQ consume cycle. This +// keeps the SQ address compare / winner tree off all 64 payload D-pins +// without adding a pipeline stage. The helper functions are duplicated from +// store_queue (pure combinational, already duplicated across modules by // design). // ============================================================================= module sq_forwarding_unit #( @@ -91,11 +101,8 @@ module sq_forwarding_unit #( localparam int unsigned XLEN = riscv_pkg::XLEN; localparam int unsigned FLEN = riscv_pkg::FLEN; localparam int unsigned MemSizeWidth = 2; - localparam int unsigned WordAddrWidth = XLEN - 2; + localparam int unsigned DwordAddrWidth = XLEN - 3; localparam int unsigned IdxWidth = $clog2(DEPTH); - localparam logic [1:0] FwdExtractExact = 2'd0; - localparam logic [1:0] FwdExtractLoWord = 2'd1; - localparam logic [1:0] FwdExtractHiWord = 2'd2; typedef struct packed { logic valid; @@ -106,8 +113,7 @@ module sq_forwarding_unit #( logic [IdxWidth-1:0] age; logic can_forward; logic [IdxWidth-1:0] idx; - logic [1:0] extract_type; - logic [1:0] store_off; + logic [2:0] store_off; } fwd_winner_t; function automatic fwd_winner_t choose_newer_winner(input fwd_winner_t lhs, @@ -136,9 +142,13 @@ module sq_forwarding_unit #( end end - function automatic logic word_addr_eq(input logic [WordAddrWidth-1:0] lhs, - input logic [WordAddrWidth-1:0] rhs); - logic [WordAddrWidth-1:0] diff; + // Hand-tiled dword-address comparator: the XOR is reduced in 5-bit groups + // (last group 4 bits at XLEN=32) so Vivado maps each group into one LUT + // ahead of a shallow final NOR — same shape the word-granule version used + // on this documented-critical compare cone, one bit narrower. + function automatic logic dword_addr_eq(input logic [DwordAddrWidth-1:0] lhs, + input logic [DwordAddrWidth-1:0] rhs); + logic [DwordAddrWidth-1:0] diff; logic [5:0] group_has_diff; begin diff = lhs ^ rhs; @@ -147,84 +157,17 @@ module sq_forwarding_unit #( group_has_diff[2] = |diff[14:10]; group_has_diff[3] = |diff[19:15]; group_has_diff[4] = |diff[24:20]; - group_has_diff[5] = |diff[29:25]; - word_addr_eq = ~(|group_has_diff); + group_has_diff[5] = |diff[DwordAddrWidth-1:25]; + dword_addr_eq = ~(|group_has_diff); end endfunction - function automatic logic full_addr_eq(input logic [XLEN-1:0] lhs, input logic [XLEN-1:0] rhs); - logic [XLEN-1:0] diff; - logic [6:0] group_has_diff; + // Generate byte-enable mask from address offset and size (8-lane strobes + // on the aligned-dword beat; DOUBLE covers the whole beat). + function automatic logic [riscv_pkg::MemStrbBits-1:0] gen_byte_en( + input logic [2:0] addr_offset, input riscv_pkg::mem_size_e size); begin - diff = lhs ^ rhs; - group_has_diff[0] = |diff[4:0]; - group_has_diff[1] = |diff[9:5]; - group_has_diff[2] = |diff[14:10]; - group_has_diff[3] = |diff[19:15]; - group_has_diff[4] = |diff[24:20]; - group_has_diff[5] = |diff[29:25]; - group_has_diff[6] = |diff[31:30]; - full_addr_eq = ~(|group_has_diff); - end - endfunction - - function automatic logic [4:0] inc5(input logic [4:0] value, input logic carry_in); - logic carry1; - logic carry2; - logic carry3; - logic carry4; - begin - carry1 = carry_in & value[0]; - carry2 = carry1 & value[1]; - carry3 = carry2 & value[2]; - carry4 = carry3 & value[3]; - inc5 = value ^ {carry4, carry3, carry2, carry1, carry_in}; - end - endfunction - - function automatic logic word_addr_inc_eq(input logic [WordAddrWidth-1:0] base, - input logic [WordAddrWidth-1:0] target); - logic [5:0] group_all_ones; - logic [5:0] carry_in; - logic [5:0] group_has_diff; - begin - group_all_ones[0] = &base[4:0]; - group_all_ones[1] = &base[9:5]; - group_all_ones[2] = &base[14:10]; - group_all_ones[3] = &base[19:15]; - group_all_ones[4] = &base[24:20]; - group_all_ones[5] = &base[29:25]; - - carry_in[0] = 1'b1; - carry_in[1] = group_all_ones[0]; - carry_in[2] = group_all_ones[0] & group_all_ones[1]; - carry_in[3] = group_all_ones[0] & group_all_ones[1] & group_all_ones[2]; - carry_in[4] = group_all_ones[0] & group_all_ones[1] & group_all_ones[2] & group_all_ones[3]; - carry_in[5] = group_all_ones[0] & group_all_ones[1] & group_all_ones[2] & - group_all_ones[3] & group_all_ones[4]; - - group_has_diff[0] = |(target[4:0] ^ inc5(base[4:0], carry_in[0])); - group_has_diff[1] = |(target[9:5] ^ inc5(base[9:5], carry_in[1])); - group_has_diff[2] = |(target[14:10] ^ inc5(base[14:10], carry_in[2])); - group_has_diff[3] = |(target[19:15] ^ inc5(base[19:15], carry_in[3])); - group_has_diff[4] = |(target[24:20] ^ inc5(base[24:20], carry_in[4])); - group_has_diff[5] = |(target[29:25] ^ inc5(base[29:25], carry_in[5])); - - word_addr_inc_eq = ~(|group_has_diff); - end - endfunction - - // Generate byte-enable mask from address offset and size - function automatic logic [3:0] gen_byte_en(input logic [1:0] addr_offset, - input riscv_pkg::mem_size_e size); - begin - case (size) - riscv_pkg::MEM_SIZE_BYTE: gen_byte_en = 4'b0001 << addr_offset; - riscv_pkg::MEM_SIZE_HALF: gen_byte_en = addr_offset[1] ? 4'b1100 : 4'b0011; - riscv_pkg::MEM_SIZE_WORD: gen_byte_en = 4'b1111; - riscv_pkg::MEM_SIZE_DOUBLE: gen_byte_en = 4'b1111; // Each phase is word-width - default: gen_byte_en = 4'b0000; - endcase + gen_byte_en = riscv_pkg::mem_strobe_for(2'(size), addr_offset); end endfunction @@ -235,16 +178,14 @@ module sq_forwarding_unit #( logic fwd_found_match; logic fwd_can_fwd; logic [IdxWidth-1:0] fwd_match_idx; - logic [1:0] fwd_extract_type; - logic [1:0] fwd_winner_store_off; - logic [3:0] fwd_load_byte_mask; + logic [2:0] fwd_winner_store_off; + logic [riscv_pkg::MemStrbBits-1:0] fwd_load_byte_mask; logic [DEPTH-1:0] fwd_addr_unknown_mask; logic [DEPTH-1:0] fwd_conflict_mask; logic [DEPTH-1:0] fwd_can_forward_mask; logic [ReorderBufferTagWidth:0] fwd_load_age; logic [ReorderBufferTagWidth:0] fwd_entry_age[DEPTH]; logic [IdxWidth-1:0] fwd_entry_slot_age[DEPTH]; - logic [1:0] fwd_entry_extract_type[DEPTH]; `ifdef FORMAL // Old implementation's pre-register payload expression, retained only as // an equivalence oracle for the registered-metadata retime. @@ -267,7 +208,7 @@ module sq_forwarding_unit #( fwd_winner_t fwd_winner; `endif - assign fwd_load_byte_mask = gen_byte_en(i_sq_check_addr[1:0], i_sq_check_size); + assign fwd_load_byte_mask = gen_byte_en(i_sq_check_addr[2:0], i_sq_check_size); assign fwd_load_age = {1'b0, i_sq_check_rob_tag} - {1'b0, rob_head_tag_q}; // Block 1: per-entry forwarding qualification from FF-based fields only @@ -275,20 +216,16 @@ module sq_forwarding_unit #( // Select older stores by ROB age directly so the forwarding path does not // need a head-relative barrel rotation over sq_valid/sq_addr_valid. always_comb begin - logic same_word; - logic base_match; - logic double_hi_match; - logic load_double_hi; + logic same_dword; logic older_store; logic store_committed; - logic [3:0] store_byte_mask; - logic [3:0] load_byte_mask; + logic [riscv_pkg::MemStrbBits-1:0] store_byte_mask; + logic [riscv_pkg::MemStrbBits-1:0] load_byte_mask; logic [ReorderBufferTagWidth-1:0] entry_rob_tag; logic [XLEN-1:0] entry_address; riscv_pkg::mem_size_e entry_size; `ifdef FORMAL logic [FLEN-1:0] entry_data_reference; - logic [XLEN-1:0] entry_image_lo_reference; `endif // Port-split: entries 0..DEPTH/2-1 use i_sq_check_addr, entries // DEPTH/2..DEPTH-1 use i_sq_check_addr_b. Both values are identical @@ -298,16 +235,13 @@ module sq_forwarding_unit #( // from a single source FF, contributing ~0.2 ns route hops on the // -0.178 ns post-synth path (LQ → SQ CAM → output FF). logic [XLEN-1:0] sq_check_addr_for_entry; - logic [WordAddrWidth-1:0] sq_check_word_for_entry; + logic [DwordAddrWidth-1:0] sq_check_dword_for_entry; for (int unsigned i = 0; i < DEPTH; i++) begin - same_word = 1'b0; - base_match = 1'b0; - double_hi_match = 1'b0; - load_double_hi = 1'b0; + same_dword = 1'b0; older_store = 1'b0; store_committed = 1'b0; - store_byte_mask = 4'b0000; + store_byte_mask = '0; load_byte_mask = fwd_load_byte_mask; // (i < DEPTH/2) is constant per loop iteration after synth unroll — // the select collapses to a wire-pick of one of the two address ports. @@ -315,11 +249,10 @@ module sq_forwarding_unit #( entry_address = sq_address_flat[i*XLEN+:XLEN]; `ifdef FORMAL entry_data_reference = sq_data_fwd_flat[i*FLEN+:FLEN]; - entry_image_lo_reference = entry_data_reference[XLEN-1:0] << {entry_address[1:0], 3'b000}; `endif entry_size = riscv_pkg::mem_size_e'(sq_size_flat[i*MemSizeWidth+:MemSizeWidth]); sq_check_addr_for_entry = (i < (DEPTH / 2)) ? i_sq_check_addr : i_sq_check_addr_b; - sq_check_word_for_entry = sq_check_addr_for_entry[XLEN-1:2]; + sq_check_dword_for_entry = sq_check_addr_for_entry[XLEN-1:3]; fwd_entry_age[i] = {1'b0, entry_rob_tag} - {1'b0, rob_head_tag_q}; // Program-order rank for winner selection: ring distance from the SQ // head. DEPTH is a power of two, so the subtraction wraps naturally. @@ -327,7 +260,6 @@ module sq_forwarding_unit #( fwd_addr_unknown_mask[i] = 1'b0; fwd_conflict_mask[i] = 1'b0; fwd_can_forward_mask[i] = 1'b0; - fwd_entry_extract_type[i] = FwdExtractExact; `ifdef FORMAL fwd_entry_data_reference[i] = '0; `endif @@ -347,66 +279,30 @@ module sq_forwarding_unit #( fwd_addr_unknown_mask[i] = 1'b1; end - // Check for address overlap + // Check for address overlap. No access crosses its aligned dword, + // so overlap is exactly: same dword AND intersecting 8-lane masks + // (a DOUBLE's mask is 8'hFF, covering the whole beat). if (sq_addr_valid[i]) begin - same_word = word_addr_eq(entry_address[XLEN-1:2], sq_check_word_for_entry); - store_byte_mask = gen_byte_en(entry_address[1:0], entry_size); - - // Non-double accesses only conflict when their byte ranges overlap. - base_match = same_word && ((entry_size == riscv_pkg::MEM_SIZE_DOUBLE) || - (i_sq_check_size == riscv_pkg::MEM_SIZE_DOUBLE) || - (|(store_byte_mask & load_byte_mask))); + same_dword = dword_addr_eq(entry_address[XLEN-1:3], sq_check_dword_for_entry); + store_byte_mask = gen_byte_en(entry_address[2:0], entry_size); - // DOUBLE store: also overlaps at word addr+4 - double_hi_match = (entry_size == riscv_pkg::MEM_SIZE_DOUBLE) && - word_addr_inc_eq(entry_address[XLEN-1:2], sq_check_word_for_entry); - - // DOUBLE load: check if store is at the +4 word - load_double_hi = (i_sq_check_size == riscv_pkg::MEM_SIZE_DOUBLE) && - word_addr_inc_eq(sq_check_word_for_entry, entry_address[XLEN-1:2]); - - if (base_match || double_hi_match || load_double_hi) begin + if (same_dword && (|(store_byte_mask & load_byte_mask))) begin fwd_conflict_mask[i] = 1'b1; // Forwarding: only non-MMIO, non-SC stores with valid data. A // store-conditional may fail at drain time and write nothing, so // its data must never reach a younger load early. - if (sq_data_valid[i] && !sq_is_mmio[i] && !sq_is_sc[i]) begin - // Case 1: FLD from FSD, exact address (full 64-bit payload) - if (base_match && full_addr_eq( - entry_address, sq_check_addr_for_entry - ) && (entry_size == riscv_pkg::MEM_SIZE_DOUBLE) && - (i_sq_check_size == riscv_pkg::MEM_SIZE_DOUBLE)) begin - fwd_can_forward_mask[i] = 1'b1; - fwd_entry_extract_type[i] = FwdExtractExact; -`ifdef FORMAL - fwd_entry_data_reference[i] = entry_data_reference; -`endif - // Case 2: byte/half/word load whose bytes the store fully - // covers in its base word. store_byte_mask places sub-word - // store data at its memory byte lanes (a DOUBLE store's base - // word is fully written, mask 1111), so covered loads read - // the memory-image word Block 3 reconstructs. The LQ applies - // the load's own byte/half extraction and sign extension. - end else if (base_match && - (i_sq_check_size != riscv_pkg::MEM_SIZE_DOUBLE) && - ((store_byte_mask & load_byte_mask) == load_byte_mask)) begin - fwd_can_forward_mask[i] = 1'b1; - fwd_entry_extract_type[i] = FwdExtractLoWord; -`ifdef FORMAL - fwd_entry_data_reference[i] = {{(FLEN - XLEN) {1'b0}}, entry_image_lo_reference}; -`endif - // Case 3: byte/half/word load inside the fully-written high - // word of a DOUBLE store (addr+4) - end else if (double_hi_match && (i_sq_check_size != riscv_pkg::MEM_SIZE_DOUBLE)) begin - fwd_can_forward_mask[i] = 1'b1; - fwd_entry_extract_type[i] = FwdExtractHiWord; + // + // Covered-subset forward: the store's lanes cover every lane the + // load reads (exact-dword FLD-from-FSD is the FF ⊆ FF case). + // Block 3 reconstructs the dword memory image; the LQ applies + // the load's own extraction and sign/NaN handling. + if (sq_data_valid[i] && !sq_is_mmio[i] && !sq_is_sc[i] && + ((store_byte_mask & load_byte_mask) == load_byte_mask)) begin + fwd_can_forward_mask[i] = 1'b1; `ifdef FORMAL - fwd_entry_data_reference[i] = { - {(FLEN - XLEN) {1'b0}}, entry_data_reference[FLEN-1:XLEN] - }; + fwd_entry_data_reference[i] = entry_data_reference << {entry_address[2:0], 3'b000}; `endif - end end end end @@ -417,8 +313,8 @@ module sq_forwarding_unit #( assign fwd_all_older_known = ~(|fwd_addr_unknown_mask); assign fwd_found_match = |fwd_conflict_mask; - // Block 2: newest conflicting store wins for data/extract selection, ranked - // by SQ ring-slot distance from i_sq_head_idx (allocation = program order; + // Block 2: newest conflicting store wins for data selection, ranked by SQ + // ring-slot distance from i_sq_head_idx (allocation = program order; // ROB-tag age is wrap-ambiguous once committed entries outlive their tag). // The heavy address/age qualification is already parallelized above, so this // block only prioritizes 1-bit match results and their precomputed metadata. @@ -429,7 +325,7 @@ module sq_forwarding_unit #( // synthesized implementation below remains the timing-optimized tree. logic fwd_formal_winner_valid; logic [IdxWidth-1:0] fwd_formal_winner_age; - logic [1:0] fwd_formal_winner_store_off; + logic [2:0] fwd_formal_winner_store_off; logic [FLEN-1:0] fwd_selected_data_reference; always_comb begin @@ -438,7 +334,6 @@ module sq_forwarding_unit #( fwd_formal_winner_store_off = '0; fwd_can_fwd = 1'b0; fwd_match_idx = '0; - fwd_extract_type = FwdExtractExact; fwd_selected_data_reference = '0; for (int unsigned i = 0; i < DEPTH; i++) begin @@ -446,10 +341,9 @@ module sq_forwarding_unit #( (!fwd_formal_winner_valid || (fwd_entry_slot_age[i] >= fwd_formal_winner_age))) begin fwd_formal_winner_valid = 1'b1; fwd_formal_winner_age = fwd_entry_slot_age[i]; - fwd_formal_winner_store_off = sq_address_flat[i*XLEN+:2]; + fwd_formal_winner_store_off = sq_address_flat[i*XLEN+:3]; fwd_can_fwd = fwd_can_forward_mask[i]; fwd_match_idx = IdxWidth'(i); - fwd_extract_type = fwd_entry_extract_type[i]; fwd_selected_data_reference = fwd_entry_data_reference[i]; end end @@ -468,12 +362,11 @@ module sq_forwarding_unit #( end for (int unsigned i = 0; i < DEPTH; i++) begin - fwd_node[FwdTreeWidth+i].valid = fwd_conflict_mask[i]; - fwd_node[FwdTreeWidth+i].age = fwd_entry_slot_age[i]; - fwd_node[FwdTreeWidth+i].can_forward = fwd_can_forward_mask[i]; - fwd_node[FwdTreeWidth+i].idx = IdxWidth'(i); - fwd_node[FwdTreeWidth+i].extract_type = fwd_entry_extract_type[i]; - fwd_node[FwdTreeWidth+i].store_off = sq_address_flat[i*XLEN+:2]; + fwd_node[FwdTreeWidth+i].valid = fwd_conflict_mask[i]; + fwd_node[FwdTreeWidth+i].age = fwd_entry_slot_age[i]; + fwd_node[FwdTreeWidth+i].can_forward = fwd_can_forward_mask[i]; + fwd_node[FwdTreeWidth+i].idx = IdxWidth'(i); + fwd_node[FwdTreeWidth+i].store_off = sq_address_flat[i*XLEN+:3]; end // Descending order so both children are final before their parent. @@ -481,11 +374,10 @@ module sq_forwarding_unit #( fwd_node[n] = choose_newer_winner(fwd_node[2*n], fwd_node[(2*n)+1]); end - fwd_winner = fwd_node[1]; + fwd_winner = fwd_node[1]; - fwd_can_fwd = fwd_winner.valid && fwd_winner.can_forward; - fwd_match_idx = fwd_winner.idx; - fwd_extract_type = fwd_winner.extract_type; + fwd_can_fwd = fwd_winner.valid && fwd_winner.can_forward; + fwd_match_idx = fwd_winner.idx; end assign fwd_winner_store_off = fwd_winner.store_off; `endif @@ -529,36 +421,30 @@ module sq_forwarding_unit #( // during the existing LQ consume cycle. A forwardable entry's mirror cannot // be overwritten before the consumer edge: sq_data_we requires the old // sq_data_valid bit to be zero, while can_forward requires it to be one. - // Capturing extract_type and store_off here also makes a same-edge free, - // flush, or slot reuse unable to change the selected payload interpretation. + // Capturing store_off here also makes a same-edge free, flush, or slot + // reuse unable to change the selected payload interpretation. logic [IdxWidth-1:0] fwd_match_idx_q; - logic [1:0] fwd_extract_type_q; - logic [1:0] fwd_winner_store_off_q; + logic [2:0] fwd_winner_store_off_q; // These are payload metadata, meaningful only with registered can_forward, // so like the old payload register they need no reset value. always_ff @(posedge i_clk) begin if (i_sq_check_capture_valid) begin fwd_match_idx_q <= fwd_match_idx; - fwd_extract_type_q <= fwd_extract_type; fwd_winner_store_off_q <= fwd_winner_store_off; end end logic [FLEN-1:0] fwd_selected_raw_q; - logic [XLEN-1:0] fwd_image_lo_q; always_comb begin fwd_selected_raw_q = sq_data_fwd_flat[fwd_match_idx_q*FLEN+:FLEN]; - fwd_image_lo_q = fwd_selected_raw_q[XLEN-1:0] << {fwd_winner_store_off_q, 3'b000}; // Consumers qualify data with can_forward, so keep that control off the - // 64 payload bits and let the compact extract metadata select directly. - case (fwd_extract_type_q) - FwdExtractLoWord: o_sq_forward.data = {{(FLEN - XLEN) {1'b0}}, fwd_image_lo_q}; - FwdExtractHiWord: - o_sq_forward.data = {{(FLEN - XLEN) {1'b0}}, fwd_selected_raw_q[FLEN-1:XLEN]}; - default: o_sq_forward.data = fwd_selected_raw_q; - endcase + // 64 payload bits. The image places the store data at its byte lanes in + // the aligned dword; covered-subset forwarding guarantees the load only + // reads lanes the store actually wrote (an aligned dword store shifts by + // zero and passes through whole). + o_sq_forward.data = fwd_selected_raw_q << {fwd_winner_store_off_q, 3'b000}; end `ifdef FORMAL @@ -582,12 +468,8 @@ module sq_forwarding_unit #( always_ff @(posedge i_clk) begin if (i_rst_n) begin - cover_extract_exact : - cover (o_sq_forward.can_forward && fwd_extract_type_q == FwdExtractExact); - cover_extract_lo_word : - cover (o_sq_forward.can_forward && fwd_extract_type_q == FwdExtractLoWord); - cover_extract_hi_word : - cover (o_sq_forward.can_forward && fwd_extract_type_q == FwdExtractHiWord); + cover_forward_aligned : cover (o_sq_forward.can_forward && fwd_winner_store_off_q == 3'b000); + cover_forward_shifted : cover (o_sq_forward.can_forward && fwd_winner_store_off_q != 3'b000); cover_wrapped_winner : cover (i_sq_check_capture_valid && fwd_can_fwd && fwd_match_idx < i_sq_head_idx); cover_flush_cycle_capture : cover (i_flush_all && i_sq_check_capture_valid); diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/store_queue.sv b/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/store_queue.sv index 2d832361..b9ca4922 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/store_queue.sv +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/store_queue/store_queue.sv @@ -27,7 +27,8 @@ * - In-order drain: the drain-cursor entry writes to memory when committed + * ready (pipelined to ~1/cycle for plain fast-tier stores; head frees at done) * - Store-to-load forwarding: combinational scan for LQ disambiguation - * - Two-phase FSD support (64-bit double → two 32-bit writes) + * - Single-beat 64-bit drains at every size, doubles included + * (docs/rv64/m1_data_tier.md; the two-phase FSD machinery is retired) * - MMIO store handling (cache bypass on commit) * - Partial flush: only uncommitted entries younger than flush_tag * - Full flush support @@ -172,25 +173,30 @@ module store_queue #( // ========================================================================= // Memory Write Interface (to data memory bus) // ========================================================================= - output logic o_mem_write_en, - output logic [riscv_pkg::XLEN-1:0] o_mem_write_addr, - output logic [riscv_pkg::XLEN-1:0] o_mem_write_data, - output logic [ 3:0] o_mem_write_byte_en, + output logic o_mem_write_en, + output logic [ riscv_pkg::XLEN-1:0] o_mem_write_addr, + output logic [riscv_pkg::MemDataBits-1:0] o_mem_write_data, + output logic [riscv_pkg::MemStrbBits-1:0] o_mem_write_byte_en, // Registered MMIO flag for the current head entry. Consumers at the // top level use this to gate the BRAM byte-write-enable at the SQ source // rather than recomputing an address-range check combinationally on the // muxed data memory address (which drags the LQ issue cone onto WEA). - output logic o_mem_write_is_mmio, + output logic o_mem_write_is_mmio, // Registered cached-tier flag for the current head entry (parallels is_mmio). // The router steers the store's byte-write enables to the cached tier when set. - output logic o_mem_write_is_cached, - input logic i_mem_write_done, + output logic o_mem_write_is_cached, + input logic i_mem_write_done, // ========================================================================= // L0 Cache Invalidation (to LQ) // ========================================================================= output logic o_cache_invalidate_valid, output logic [riscv_pkg::XLEN-1:0] o_cache_invalidate_addr, + // The launching write covers its full aligned dword (FSD; RV64 SD in + // M3). The wrapper's word-granule reservation snoop widens its compare + // to the dword for these, preserving the coverage the two-phase FSD + // drain used to deliver as two word-granule pulses. + output logic o_cache_invalidate_is_dword, // ========================================================================= // ROB Head Tag (for age comparisons) @@ -233,7 +239,6 @@ module store_queue #( localparam int unsigned IdxWidth = $clog2(DEPTH); localparam int unsigned PtrWidth = IdxWidth + 1; // Extra MSB for full/empty localparam int unsigned CountWidth = $clog2(DEPTH + 1); - localparam int unsigned WordAddrWidth = XLEN - 2; localparam int unsigned MemSizeWidth = 2; // =========================================================================== @@ -267,107 +272,27 @@ module store_queue #( end endfunction - function automatic logic word_addr_eq(input logic [WordAddrWidth-1:0] lhs, - input logic [WordAddrWidth-1:0] rhs); - logic [WordAddrWidth-1:0] diff; - logic [5:0] group_has_diff; - begin - diff = lhs ^ rhs; - group_has_diff[0] = |diff[4:0]; - group_has_diff[1] = |diff[9:5]; - group_has_diff[2] = |diff[14:10]; - group_has_diff[3] = |diff[19:15]; - group_has_diff[4] = |diff[24:20]; - group_has_diff[5] = |diff[29:25]; - word_addr_eq = ~(|group_has_diff); - end - endfunction - - function automatic logic full_addr_eq(input logic [XLEN-1:0] lhs, input logic [XLEN-1:0] rhs); - logic [XLEN-1:0] diff; - logic [6:0] group_has_diff; - begin - diff = lhs ^ rhs; - group_has_diff[0] = |diff[4:0]; - group_has_diff[1] = |diff[9:5]; - group_has_diff[2] = |diff[14:10]; - group_has_diff[3] = |diff[19:15]; - group_has_diff[4] = |diff[24:20]; - group_has_diff[5] = |diff[29:25]; - group_has_diff[6] = |diff[31:30]; - full_addr_eq = ~(|group_has_diff); - end - endfunction - - function automatic logic [4:0] inc5(input logic [4:0] value, input logic carry_in); - logic carry1; - logic carry2; - logic carry3; - logic carry4; - begin - carry1 = carry_in & value[0]; - carry2 = carry1 & value[1]; - carry3 = carry2 & value[2]; - carry4 = carry3 & value[3]; - inc5 = value ^ {carry4, carry3, carry2, carry1, carry_in}; - end - endfunction - - function automatic logic word_addr_inc_eq(input logic [WordAddrWidth-1:0] base, - input logic [WordAddrWidth-1:0] target); - logic [5:0] group_all_ones; - logic [5:0] carry_in; - logic [5:0] group_has_diff; - begin - group_all_ones[0] = &base[4:0]; - group_all_ones[1] = &base[9:5]; - group_all_ones[2] = &base[14:10]; - group_all_ones[3] = &base[19:15]; - group_all_ones[4] = &base[24:20]; - group_all_ones[5] = &base[29:25]; - - carry_in[0] = 1'b1; - carry_in[1] = group_all_ones[0]; - carry_in[2] = group_all_ones[0] & group_all_ones[1]; - carry_in[3] = group_all_ones[0] & group_all_ones[1] & group_all_ones[2]; - carry_in[4] = group_all_ones[0] & group_all_ones[1] & group_all_ones[2] & group_all_ones[3]; - carry_in[5] = group_all_ones[0] & group_all_ones[1] & group_all_ones[2] & - group_all_ones[3] & group_all_ones[4]; - - group_has_diff[0] = |(target[4:0] ^ inc5(base[4:0], carry_in[0])); - group_has_diff[1] = |(target[9:5] ^ inc5(base[9:5], carry_in[1])); - group_has_diff[2] = |(target[14:10] ^ inc5(base[14:10], carry_in[2])); - group_has_diff[3] = |(target[19:15] ^ inc5(base[19:15], carry_in[3])); - group_has_diff[4] = |(target[24:20] ^ inc5(base[24:20], carry_in[4])); - group_has_diff[5] = |(target[29:25] ^ inc5(base[29:25], carry_in[5])); - - word_addr_inc_eq = ~(|group_has_diff); - end - endfunction - // Generate byte-enable mask from address offset and size - function automatic logic [3:0] gen_byte_en(input logic [1:0] addr_offset, - input riscv_pkg::mem_size_e size); + function automatic logic [riscv_pkg::MemStrbBits-1:0] gen_byte_en( + input logic [2:0] addr_offset, input riscv_pkg::mem_size_e size); begin - case (size) - riscv_pkg::MEM_SIZE_BYTE: gen_byte_en = 4'b0001 << addr_offset; - riscv_pkg::MEM_SIZE_HALF: gen_byte_en = addr_offset[1] ? 4'b1100 : 4'b0011; - riscv_pkg::MEM_SIZE_WORD: gen_byte_en = 4'b1111; - riscv_pkg::MEM_SIZE_DOUBLE: gen_byte_en = 4'b1111; // Each phase is word-width - default: gen_byte_en = 4'b0000; - endcase + // 8-lane strobes on the aligned-dword beat (docs/rv64/m1_data_tier.md); + // the mem_size_e encoding matches the helper's 2-bit size argument. + gen_byte_en = riscv_pkg::mem_strobe_for(2'(size), addr_offset); end endfunction - // Generate write data with correct byte-lane positioning - function automatic logic [XLEN-1:0] gen_write_data( - input logic [FLEN-1:0] data, input riscv_pkg::mem_size_e size, input logic fp64_phase); + // Generate write data with correct byte-lane positioning: sub-beat sizes + // replicate across the beat (the strobes select the addressed lanes), + // doubles pass through single-beat. + function automatic logic [riscv_pkg::MemDataBits-1:0] gen_write_data( + input logic [FLEN-1:0] data, input riscv_pkg::mem_size_e size); begin case (size) - riscv_pkg::MEM_SIZE_BYTE: gen_write_data = {4{data[7:0]}}; - riscv_pkg::MEM_SIZE_HALF: gen_write_data = {2{data[15:0]}}; - riscv_pkg::MEM_SIZE_WORD: gen_write_data = data[31:0]; - riscv_pkg::MEM_SIZE_DOUBLE: gen_write_data = fp64_phase ? data[63:32] : data[31:0]; + riscv_pkg::MEM_SIZE_BYTE: gen_write_data = {8{data[7:0]}}; + riscv_pkg::MEM_SIZE_HALF: gen_write_data = {4{data[15:0]}}; + riscv_pkg::MEM_SIZE_WORD: gen_write_data = {2{data[31:0]}}; + riscv_pkg::MEM_SIZE_DOUBLE: gen_write_data = data[63:0]; default: gen_write_data = '0; endcase end @@ -388,7 +313,6 @@ module store_queue #( logic [ DEPTH-1:0] sq_addr_valid; logic [ DEPTH-1:0] sq_data_valid; logic [ DEPTH-1:0] sq_is_mmio; - logic [ DEPTH-1:0] sq_fp64_phase; logic [ DEPTH-1:0] sq_committed; logic [ DEPTH-1:0] sq_sent; logic [ DEPTH-1:0] sq_is_sc; @@ -488,13 +412,13 @@ module store_queue #( logic slot2_alloc_en; logic [ IdxWidth-1:0] slot2_alloc_idx; - // Memory write tracking. Plain fast-tier drains (BRAM, non-MMIO, non-FSD) - // are pipelined: up to two writes may be in flight (one on the bus, one - // awaiting its 1-cycle done), tracked by write_inflight_cnt plus a 2-deep - // in-order metadata FIFO (entry index + completes flag, popped one per - // done). Cached / MMIO / FSD writes stay strictly single-outstanding - // (write_inflight_special): the cached adapter is single-request and FSD - // needs its per-entry two-phase bookkeeping. + // Memory write tracking. Plain fast-tier drains (BRAM, non-MMIO — + // single-beat FSD included) are pipelined: up to two writes may be in + // flight (one on the bus, one awaiting its 1-cycle done), tracked by + // write_inflight_cnt plus a 2-deep in-order metadata FIFO (entry index + + // completes flag, popped one per done). Cached / MMIO writes stay + // strictly single-outstanding (write_inflight_special): the cached + // adapter is single-request and MMIO dispatch is serialized. logic [ 1:0] write_inflight_cnt; logic write_inflight_special; logic [ IdxWidth-1:0] write_fifo_idx0; @@ -728,64 +652,63 @@ module store_queue #( // Memory Write Logic (combinational) // =========================================================================== // The drain-cursor entry writes to memory when committed, addr_valid, - // data_valid. FSD uses two phases. + // data_valid. Every size drains in a single beat (FSD included). // // TIMING: The write interface is registered to break the head_ptr → // drain_ready → o_mem_write_en combinational cone that was the critical // path (-1.059 ns WNS). drain_ready feeds the combinational next-state // of a pipeline register; the actual o_mem_write_en output is a flop. // - // DRAIN PIPELINING: plain fast-tier stores (BRAM, non-MMIO, non-FSD) - // complete exactly one cycle after their bus cycle (the router's - // sq_write_done_fast is the write-enable delayed one cycle), so + // DRAIN PIPELINING: plain fast-tier stores (BRAM, non-MMIO — single-beat + // FSD included) complete exactly one cycle after their bus cycle (the + // router's sq_write_done_fast is the write-enable delayed one cycle), so // consecutive plain drains overlap: a new launch is allowed while the // previous write's done is still in flight, bounded to two in-flight by - // the metadata FIFO. Cached / MMIO / FSD writes keep the strict + // the metadata FIFO. Cached / MMIO writes keep the strict // one-at-a-time gate (write_inflight_cnt == 0 && !o_mem_write_en). assign drain_ready = sq_valid[drain_idx_q] && sq_committed[drain_idx_q] && sq_addr_valid[drain_idx_q] && sq_data_valid[drain_idx_q] && !sq_sent[drain_idx_q]; - logic [riscv_pkg::XLEN-1:0] mem_write_addr_next; - logic [riscv_pkg::XLEN-1:0] mem_write_data_next; - logic [ 3:0] mem_write_byte_en_next; - logic mem_write_is_mmio_next; - logic mem_write_is_cached_next; - logic mem_write_launch_serial_next; - logic mem_write_launch_pipelined_next; - logic mem_write_addr_cached_for_plain_next; + logic [ riscv_pkg::XLEN-1:0] mem_write_addr_next; + logic [riscv_pkg::MemDataBits-1:0] mem_write_data_next; + logic [riscv_pkg::MemStrbBits-1:0] mem_write_byte_en_next; + logic mem_write_is_mmio_next; + logic mem_write_is_cached_next; + logic mem_write_launch_serial_next; + logic mem_write_launch_pipelined_next; + logic mem_write_addr_cached_for_plain_next; always_comb begin - // FSD phase 1: write upper word at addr+4 - if (sq_size[drain_idx_q] == riscv_pkg::MEM_SIZE_DOUBLE && sq_fp64_phase[drain_idx_q]) begin - mem_write_addr_next = sq_address[drain_idx_q] + 32'd4; - end else begin - mem_write_addr_next = sq_address[drain_idx_q]; - end + // Single-beat drains at every size (docs/rv64/m1_data_tier.md): doubles + // are one 64-bit write, so no phase legs and no +4 second beat. + mem_write_addr_next = sq_address[drain_idx_q]; - mem_write_data_next = gen_write_data( - sq_data_drain_rd, riscv_pkg::mem_size_e'(sq_size[drain_idx_q]), sq_fp64_phase[drain_idx_q]); + mem_write_data_next = + gen_write_data(sq_data_drain_rd, riscv_pkg::mem_size_e'(sq_size[drain_idx_q])); mem_write_byte_en_next = - gen_byte_en(mem_write_addr_next[1:0], riscv_pkg::mem_size_e'(sq_size[drain_idx_q])); + gen_byte_en(mem_write_addr_next[2:0], riscv_pkg::mem_size_e'(sq_size[drain_idx_q])); mem_write_is_mmio_next = sq_is_mmio[drain_idx_q]; // cached-tier decode of the actual write address. Registered below into // o_mem_write_is_cached (parallel to is_mmio), so the comparator stays in // the addr->register cone and never reaches the BRAM WEA pin. + // XLEN'() casts, not [XLEN-1:0] part-selects: the parameters are 32-bit + // ints, so a 64-bit part-select would be out of range. mem_write_is_cached_next = - (mem_write_addr_next >= CACHED_BASE[riscv_pkg::XLEN-1:0]) && - (mem_write_addr_next < (CACHED_BASE[riscv_pkg::XLEN-1:0] + - CACHED_SIZE_BYTES[riscv_pkg::XLEN-1:0])); + (mem_write_addr_next >= XLEN'(CACHED_BASE)) && + (mem_write_addr_next < (XLEN'(CACHED_BASE) + XLEN'(CACHED_SIZE_BYTES))); end assign mem_write_addr_cached_for_plain_next = - (sq_address[drain_idx_q] >= CACHED_BASE[riscv_pkg::XLEN-1:0]) && - (sq_address[drain_idx_q] < (CACHED_BASE[riscv_pkg::XLEN-1:0] + - CACHED_SIZE_BYTES[riscv_pkg::XLEN-1:0])); - assign mem_write_completes_next = !(sq_size[drain_idx_q] == riscv_pkg::MEM_SIZE_DOUBLE && - !sq_fp64_phase[drain_idx_q]); - assign mem_write_plain_fast_next = (sq_size[drain_idx_q] != riscv_pkg::MEM_SIZE_DOUBLE) && - !sq_is_mmio[drain_idx_q] && + (sq_address[drain_idx_q] >= XLEN'(CACHED_BASE)) && + (sq_address[drain_idx_q] < (XLEN'(CACHED_BASE) + XLEN'(CACHED_SIZE_BYTES))); + // Single-beat doubles: every launch completes its entry, and DOUBLE joins + // the pipelined plain fast-tier drain (the old two-phase FSD flew alone). + // The write-FIFO completes plumbing is kept constant-true rather than + // excised; synthesis sweeps it. + assign mem_write_completes_next = 1'b1; + assign mem_write_plain_fast_next = !sq_is_mmio[drain_idx_q] && !mem_write_addr_cached_for_plain_next; // Launch gate: legacy serial arm for any write type, plus the pipelined @@ -842,10 +765,14 @@ module store_queue #( // CoreMark), and routing the cached-flight signal into the LQ's busy // instead pushed the L0-hit/CDB cone past timing. Both outputs come // straight from SQ output registers, adding no new logic levels anywhere. - // FSD fires one invalidate per phase (addr, then addr+4). MMIO stores - // also pulse harmlessly (the L0 never caches MMIO). + // A single-beat FSD covers its whole dword with one pulse (the LQ's L0 is + // dword-granule, and the wrapper reservation snoop widens on is_dword). + // MMIO stores also pulse harmlessly (the L0 never caches MMIO). assign o_cache_invalidate_valid = o_mem_write_en; assign o_cache_invalidate_addr = o_mem_write_addr; + // Full-beat strobe == dword-covering write; comes straight from the + // registered drain strobe, adding no new logic ahead of the output. + assign o_cache_invalidate_is_dword = (o_mem_write_byte_en == {riscv_pkg::MemStrbBits{1'b1}}); // =========================================================================== // Allocation (pure ring tail) @@ -1126,7 +1053,7 @@ module store_queue #( // and their done pulse; the metadata FIFO below carries entry index // and completes flag per in-flight write. sq_sent is set at LAUNCH // (fire cycle) for completing writes so the drain cursor can move on - // immediately; the done side only frees entries / advances FSD phase. + // immediately; the done side only frees entries. write_inflight_cnt <= write_inflight_cnt + (o_mem_write_en ? 2'd1 : 2'd0) - ((i_mem_write_done && (write_inflight_cnt != 2'd0)) ? 2'd1 : 2'd0); @@ -1341,10 +1268,9 @@ module store_queue #( // Allocation: write per-entry data for new entry at tail // ----------------------------------------------------------------- if (slot1_alloc_en) begin - sq_rob_tag[alloc_target[IdxWidth-1:0]] <= i_alloc.rob_tag; - sq_size[alloc_target[IdxWidth-1:0]] <= i_alloc.size; - sq_fp64_phase[alloc_target[IdxWidth-1:0]] <= 1'b0; - sq_is_sc[alloc_target[IdxWidth-1:0]] <= i_alloc.is_sc; + sq_rob_tag[alloc_target[IdxWidth-1:0]] <= i_alloc.rob_tag; + sq_size[alloc_target[IdxWidth-1:0]] <= i_alloc.size; + sq_is_sc[alloc_target[IdxWidth-1:0]] <= i_alloc.is_sc; if (i_alloc.addr_valid) begin sq_address[alloc_target[IdxWidth-1:0]] <= i_alloc.address; sq_is_mmio[alloc_target[IdxWidth-1:0]] <= i_alloc.is_mmio; @@ -1353,10 +1279,9 @@ module store_queue #( // Slot-2 alloc — fires when a slot-2 store allocates this cycle. if (slot2_alloc_en) begin - sq_rob_tag[slot2_alloc_idx] <= i_alloc_2.rob_tag; - sq_size[slot2_alloc_idx] <= i_alloc_2.size; - sq_fp64_phase[slot2_alloc_idx] <= 1'b0; - sq_is_sc[slot2_alloc_idx] <= i_alloc_2.is_sc; + sq_rob_tag[slot2_alloc_idx] <= i_alloc_2.rob_tag; + sq_size[slot2_alloc_idx] <= i_alloc_2.size; + sq_is_sc[slot2_alloc_idx] <= i_alloc_2.is_sc; if (i_alloc_2.addr_valid) begin sq_address[slot2_alloc_idx] <= i_alloc_2.address; sq_is_mmio[slot2_alloc_idx] <= i_alloc_2.is_mmio; @@ -1399,15 +1324,6 @@ module store_queue #( end end - // ----------------------------------------------------------------- - // Memory Write Completion: FSD phase advance (data only) - // ----------------------------------------------------------------- - if (i_mem_write_done && (write_inflight_cnt != 2'd0)) begin - if (!write_completes_entry) begin - sq_fp64_phase[write_entry_idx] <= 1'b1; - end - end - end // =========================================================================== @@ -1651,7 +1567,7 @@ module store_queue #( end // In-flight discipline: never more than the 2-deep metadata FIFO can - // hold, and a special (cached / MMIO / FSD) write flies alone. + // hold, and a special (cached / MMIO) write flies alone. always_comb begin if (i_rst_n) begin p_inflight_bound : assert (write_inflight_cnt <= 2'd2); @@ -1759,8 +1675,6 @@ module store_queue #( // Committed entry survives partial flush cover_committed_survives : cover (i_flush_en && |(sq_valid & sq_committed)); - // FSD two-phase memory write - cover_fsd_phase1 : cover (o_mem_write_en && sq_fp64_phase[mem_write_entry_idx_stg]); // Pipelined drain: two plain fast-tier writes in flight at once. cover_pipelined_drain : cover (o_mem_write_en && (write_inflight_cnt != 2'd0)); // Exercise the event counter's widest update: two accepted stores while diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/store_addr/sq_early_addr_pipeline.sv b/hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/store_addr/sq_early_addr_pipeline.sv index 01a19af1..d61e7529 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/store_addr/sq_early_addr_pipeline.sv +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/store_addr/sq_early_addr_pipeline.sv @@ -418,27 +418,38 @@ module sq_early_addr_pipeline ( end end - // Adder now runs on registered inputs — off the dispatch critical path + // Adder now runs on registered inputs — off the dispatch critical path. + // All six store-AGU adder outputs are canonicalized to the physical + // address space (identity at XLEN=32 - plan decision D3). logic [riscv_pkg::XLEN-1:0] sq_early_effective_addr; logic [riscv_pkg::XLEN-1:0] sq_early_repair_effective_addr; - assign sq_early_effective_addr = sq_early_addr_base_q + sq_early_addr_imm_q; - assign sq_early_repair_effective_addr = sq_early_addr_repair_base + sq_early_addr_repair_imm_q; + assign sq_early_effective_addr = riscv_pkg::canonical_paddr( + sq_early_addr_base_q + sq_early_addr_imm_q + ); + assign sq_early_repair_effective_addr = riscv_pkg::canonical_paddr( + sq_early_addr_repair_base + sq_early_addr_repair_imm_q + ); // Slot-2 adder logic [riscv_pkg::XLEN-1:0] sq_early_effective_addr_2; logic [riscv_pkg::XLEN-1:0] sq_early_repair_effective_addr_2; - assign sq_early_effective_addr_2 = sq_early_addr_base_2_q + sq_early_addr_imm_2_q; - assign sq_early_repair_effective_addr_2 = sq_early_addr_repair_base_2 + - sq_early_addr_repair_imm_2_q; + assign sq_early_effective_addr_2 = riscv_pkg::canonical_paddr( + sq_early_addr_base_2_q + sq_early_addr_imm_2_q + ); + assign sq_early_repair_effective_addr_2 = riscv_pkg::canonical_paddr( + sq_early_addr_repair_base_2 + sq_early_addr_repair_imm_2_q + ); // Held-candidate adders: run on the latched repaired base (registered), so // the drain path stays off the CDB/bypass comb cone. logic [riscv_pkg::XLEN-1:0] sq_early_hold_effective_addr; logic [riscv_pkg::XLEN-1:0] sq_early_hold_effective_addr_2; - assign sq_early_hold_effective_addr = sq_early_addr_repair_base_hold_q + - sq_early_addr_repair_imm_q; - assign sq_early_hold_effective_addr_2 = sq_early_addr_repair_base_hold_2_q + - sq_early_addr_repair_imm_2_q; + assign sq_early_hold_effective_addr = riscv_pkg::canonical_paddr( + sq_early_addr_repair_base_hold_q + sq_early_addr_repair_imm_q + ); + assign sq_early_hold_effective_addr_2 = riscv_pkg::canonical_paddr( + sq_early_addr_repair_base_hold_2_q + sq_early_addr_repair_imm_2_q + ); // Port arbitration: a fresh (ready-base) update is single-cycle perishable // and always wins; a just-matched candidate emits combinationally only on diff --git a/hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv b/hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv index 29e444d5..d6fd678a 100644 --- a/hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv +++ b/hw/rtl/cpu_and_mem/cpu/tomasulo/tomasulo_wrapper/tomasulo_wrapper.sv @@ -437,23 +437,23 @@ module tomasulo_wrapper #( // ========================================================================= // Store Queue: Memory Write Interface // ========================================================================= - output logic o_sq_mem_write_en, - output logic [riscv_pkg::XLEN-1:0] o_sq_mem_write_addr, - output logic [riscv_pkg::XLEN-1:0] o_sq_mem_write_data, - output logic [ 3:0] o_sq_mem_write_byte_en, - output logic o_sq_mem_write_is_mmio, - output logic o_sq_mem_write_is_cached, - input logic i_sq_mem_write_done, + output logic o_sq_mem_write_en, + output logic [ riscv_pkg::XLEN-1:0] o_sq_mem_write_addr, + output logic [riscv_pkg::MemDataBits-1:0] o_sq_mem_write_data, + output logic [riscv_pkg::MemStrbBits-1:0] o_sq_mem_write_byte_en, + output logic o_sq_mem_write_is_mmio, + output logic o_sq_mem_write_is_cached, + input logic i_sq_mem_write_done, // ========================================================================= // Load Queue: Memory Interface // ========================================================================= - output logic o_lq_mem_read_en, - output logic o_lq_mem_addr_valid, - output logic [riscv_pkg::XLEN-1:0] o_lq_mem_read_addr, - output riscv_pkg::mem_size_e o_lq_mem_read_size, - input logic [riscv_pkg::XLEN-1:0] i_lq_mem_read_data, - input logic i_lq_mem_read_valid, + output logic o_lq_mem_read_en, + output logic o_lq_mem_addr_valid, + output logic [ riscv_pkg::XLEN-1:0] o_lq_mem_read_addr, + output riscv_pkg::mem_size_e o_lq_mem_read_size, + input logic [riscv_pkg::MemDataBits-1:0] i_lq_mem_read_data, + input logic i_lq_mem_read_valid, // ========================================================================= // Load Queue: Status @@ -474,10 +474,10 @@ module tomasulo_wrapper #( // ========================================================================= // AMO Memory Write Interface (from LQ) // ========================================================================= - output logic o_amo_mem_write_en, - output logic [riscv_pkg::XLEN-1:0] o_amo_mem_write_addr, - output logic [riscv_pkg::XLEN-1:0] o_amo_mem_write_data, - input logic i_amo_mem_write_done, + output logic o_amo_mem_write_en, + output logic [ riscv_pkg::XLEN-1:0] o_amo_mem_write_addr, + output logic [riscv_pkg::MemDataBits-1:0] o_amo_mem_write_data, + input logic i_amo_mem_write_done, // ========================================================================= // Profiling Snapshot Interface @@ -1282,6 +1282,7 @@ module tomasulo_wrapper #( logic sq_cache_invalidate_valid; logic [riscv_pkg::XLEN-1:0] sq_cache_invalidate_addr; + logic sq_cache_invalidate_is_dword; // =========================================================================== // Atomics Wiring (LR/SC/AMO support) @@ -1299,11 +1300,18 @@ module tomasulo_wrapper #( logic sc_clear_reservation; assign sc_clear_reservation = commit_bus_q_valid && commit_q_is_sc; - // Reservation snoop invalidation: SQ write to reservation address + // Reservation snoop invalidation: SQ write to reservation address. The + // reservation granule stays word-sized; a dword-covering store (FSD single + // beat) snoops BOTH its words — the same coverage the two-phase FSD drain + // delivered as two word-granule pulses. logic reservation_snoop_invalidate; assign reservation_snoop_invalidate = sq_cache_invalidate_valid && lq_reservation_valid && - (sq_cache_invalidate_addr[riscv_pkg::XLEN-1:2] == lq_reservation_addr[riscv_pkg::XLEN-1:2]); + (sq_cache_invalidate_is_dword + ? (sq_cache_invalidate_addr[riscv_pkg::XLEN-1:3] == + lq_reservation_addr[riscv_pkg::XLEN-1:3]) + : (sq_cache_invalidate_addr[riscv_pkg::XLEN-1:2] == + lq_reservation_addr[riscv_pkg::XLEN-1:2])); // SC discard: failed SC invalidates its SQ entry // Uses pipelined commit bus to break ROB → SQ critical path. @@ -2866,7 +2874,12 @@ module tomasulo_wrapper #( // Load Queue: Address Update from MEM_RS Issue // =========================================================================== logic [riscv_pkg::XLEN-1:0] lq_effective_addr; - assign lq_effective_addr = o_mem_rs_issue.src1_value[riscv_pkg::XLEN-1:0] + o_mem_rs_issue.imm; + // AGU output is canonicalized to the physical address space (identity at + // XLEN=32; masks bits [63:32] at XLEN=64 - plan decision D3) so region + // decodes, CAM compares, and mtval capture all see sub-4-GiB addresses. + assign lq_effective_addr = riscv_pkg::canonical_paddr( + o_mem_rs_issue.src1_value[riscv_pkg::XLEN-1:0] + o_mem_rs_issue.imm + ); // MMIO detection: the 01 address quadrant [0x4000_0000, 0x8000_0000). // The cached (DDR) region is the 10 quadrant [0x8000_0000, 0xC000_0000) @@ -3111,8 +3124,11 @@ module tomasulo_wrapper #( // =========================================================================== // Store Queue: Address + Data Update from MEM_RS Issue // =========================================================================== - // Effective address: base (src1) + immediate (declared above near SC pending) - assign sq_effective_addr = o_mem_rs_issue.src1_value[riscv_pkg::XLEN-1:0] + o_mem_rs_issue.imm; + // Effective address: base (src1) + immediate (declared above near SC pending). + // Canonicalized like the LQ AGU output above (plan decision D3). + assign sq_effective_addr = riscv_pkg::canonical_paddr( + o_mem_rs_issue.src1_value[riscv_pkg::XLEN-1:0] + o_mem_rs_issue.imm + ); logic sq_addr_is_mmio; // MMIO quadrant test; see lq_addr_is_mmio above. @@ -3215,7 +3231,8 @@ module tomasulo_wrapper #( // L0 cache invalidation (to LQ) .o_cache_invalidate_valid(sq_cache_invalidate_valid), - .o_cache_invalidate_addr (sq_cache_invalidate_addr), + .o_cache_invalidate_addr(sq_cache_invalidate_addr), + .o_cache_invalidate_is_dword(sq_cache_invalidate_is_dword), // SC discard (pipelined — uses commit_bus_q) .i_sc_discard (sc_discard), diff --git a/hw/rtl/cpu_and_mem/cpu_and_mem.f b/hw/rtl/cpu_and_mem/cpu_and_mem.f index 95cabf80..1ba4d952 100644 --- a/hw/rtl/cpu_and_mem/cpu_and_mem.f +++ b/hw/rtl/cpu_and_mem/cpu_and_mem.f @@ -7,15 +7,18 @@ # Cache hierarchy (L1/L2 line caches + AXI bridge + behavioral main memory) -f $(ROOT)/hw/rtl/lib/cache/cache.f -# Word<->line adapter between the request router and the cache hierarchy -$(ROOT)/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/cached_tier_adapter.sv - # Pipeline utilities (stall capture registers) $(ROOT)/hw/rtl/lib/stall_capture_reg.sv # RISC-V OOO CPU core (Tomasulo out-of-order with all submodules) -f $(ROOT)/hw/rtl/cpu_and_mem/cpu/cpu_ooo/cpu_ooo.f +# Word<->line adapter between the request router and the cache hierarchy. +# Listed after the CPU core: its XLEN parameter default references +# riscv_pkg::XLEN (read in via cpu_ooo.f), and Yosys resolves package +# references in parameter defaults only if the package is parsed first. +$(ROOT)/hw/rtl/cpu_and_mem/cpu/cpu_ooo/memory_if/cached_tier_adapter.sv + # Instruction memory with predecode sideband $(ROOT)/hw/rtl/cpu_and_mem/imem_predecode.sv diff --git a/hw/rtl/cpu_and_mem/cpu_and_mem.sv b/hw/rtl/cpu_and_mem/cpu_and_mem.sv index 8a56154e..fcb87a96 100644 --- a/hw/rtl/cpu_and_mem/cpu_and_mem.sv +++ b/hw/rtl/cpu_and_mem/cpu_and_mem.sv @@ -142,6 +142,8 @@ module cpu_and_mem #( localparam int unsigned MemByteAddrWidth = $clog2(MEM_SIZE_BYTES); // (MEM_SIZE_BYTES/(4 bytes per word)) words; e.g. 256 KiB -> 64k words = 16 word address bits localparam int unsigned MemWordAddrWidth = MemByteAddrWidth - 2; + // Data-memory rows are MemDataBits dwords (docs/rv64/m1_data_tier.md). + localparam int unsigned MemDwordAddrWidth = MemByteAddrWidth - 3; // Memory-mapped I/O addresses for peripherals // IMPORTANT: If these addresses are changed, they must also be updated in: @@ -231,30 +233,31 @@ module cpu_and_mem #( logic [31:0] iup_req_wstrb; logic iup_resp_valid; logic [255:0] iup_resp_rdata; - logic [31:0] data_memory_address, data_memory_write_data, data_memory_write_data_registered; - logic [31:0] data_memory_or_peripheral_read_data; // Muxed from RAM or MMIO - logic [31:0] mmio_read_data_comb; - logic [31:0] mmio_read_data_reg; + logic [31:0] data_memory_address; + logic [riscv_pkg::MemDataBits-1:0] data_memory_write_data, data_memory_write_data_registered; + logic [riscv_pkg::MemDataBits-1:0] data_memory_or_peripheral_read_data; // From RAM or MMIO + logic [riscv_pkg::MemDataBits-1:0] mmio_read_data_comb; + logic [riscv_pkg::MemDataBits-1:0] mmio_read_data_reg; logic mmio_read_data_valid; logic [31:0] mmio_load_addr; logic mmio_load_valid; logic mmio_read_capture; - logic [31:0] data_memory_read_data; // From RAM only + logic [riscv_pkg::MemDataBits-1:0] data_memory_read_data; // From RAM only logic [31:0] data_memory_address_registered; // Delayed for read data alignment - logic [3:0] data_memory_byte_write_enable; + logic [riscv_pkg::MemStrbBits-1:0] data_memory_byte_write_enable; // MMIO-pre-masked copy routed straight to the BRAM WEA pins. Generated in // cpu_ooo using the SQ/AMO-side registered is_mmio flags so the BRAM // write-enable no longer depends on the late combinational // data_memory_address-range test. - logic [3:0] data_memory_bram_byte_write_enable; + logic [riscv_pkg::MemStrbBits-1:0] data_memory_bram_byte_write_enable; logic data_memory_read_enable; // Cached tier (high-address region). The router drives these tier-routed // requests (already qualified by is_cached); the cached_tier_adapter // completes them with handshake pulses. The BRAM keeps reading/writing the // low range unchanged. - logic [3:0] data_memory_cached_byte_write_enable; + logic [riscv_pkg::MemStrbBits-1:0] data_memory_cached_byte_write_enable; logic data_memory_cached_read_enable; - logic [31:0] data_memory_cached_read_data; + logic [riscv_pkg::MemDataBits-1:0] data_memory_cached_read_data; logic data_memory_cached_read_valid; logic data_memory_cached_write_done; logic data_memory_cached_write_inflight; @@ -270,16 +273,16 @@ module cpu_and_mem #( // cycle a cached AMO read-modify-write launches (the router muxes the two). // Kept separate from data_memory_write_data so the cached write path stays // off the wide BRAM write-data cascade. - logic [31:0] data_memory_cached_write_data; - logic mmio_read_pulse; - logic mmio_fifo0_read_pulse; - logic mmio_fifo1_read_pulse; - logic mmio_uart_rx_ready_pulse; + logic [riscv_pkg::MemDataBits-1:0] data_memory_cached_write_data; + logic mmio_read_pulse; + logic mmio_fifo0_read_pulse; + logic mmio_fifo1_read_pulse; + logic mmio_uart_rx_ready_pulse; // Timer registers (CLINT-style) - logic [63:0] mtime; // Machine time counter - logic [63:0] mtimecmp; // Machine timer compare register - logic msip; // Machine software interrupt pending + logic [ 63:0] mtime; // Machine time counter + logic [ 63:0] mtimecmp; // Machine timer compare register + logic msip; // Machine software interrupt pending // ns16550a UART face register file (8-bit). DLAB = ns_lcr[7]. logic [7:0] ns_dll, ns_dlm, ns_ier, ns_fcr, ns_lcr, ns_mcr, ns_scr; @@ -701,24 +704,37 @@ module cpu_and_mem #( end `endif - // Memory 1: Data memory + // Memory 1: Data memory - one MemDataBits-wide byte-enabled BRAM carrying + // the aligned-dword bus view (docs/rv64/m1_data_tier.md). Rows are dwords, + // so the row address width drops by one; init comes from sw64.mem (64-bit + // $readmemh tokens paired from sw.mem by the build - sw.mem itself stays + // the 32-bit-word format every loader and imem consumes). // Port A: Instruction programming (div4 clock, write only - fan out) // Port B: Data access (main clock, loads/stores from CPU) + + // Port A keeps its 32-bit programming face: the word write is steered into + // the addressed half of the dword row (addr[2] selects the strobe nibble), + // so the JTAG/loader flow stays untouched. + logic [riscv_pkg::MemStrbBits-1:0] instr_mem_dword_we; + assign instr_mem_dword_we = + i_instr_mem_addr[2] ? {i_instr_mem_we & {4{i_instr_mem_en}}, 4'b0000} : + {4'b0000, i_instr_mem_we & {4{i_instr_mem_en}}}; + tdp_bram_dc_byte_en #( - .DATA_WIDTH(32), - .ADDR_WIDTH(MemWordAddrWidth), + .DATA_WIDTH(riscv_pkg::MemDataBits), + .ADDR_WIDTH(MemDwordAddrWidth), .USE_INIT_FILE(1'b1), - .INIT_FILE("sw.mem") // Software initialization file + .INIT_FILE("sw64.mem") // Software initialization file (dword tokens) ) data_memory ( .i_port_a_clk(i_clk_div4), .i_port_b_clk(i_clk), // Port A: Instruction programming (div4 clock, write only) - .i_port_a_byte_address(i_instr_mem_addr), - .i_port_a_write_data(i_instr_mem_wrdata), - .i_port_a_byte_write_enable(i_instr_mem_we & {4{i_instr_mem_en}}), + .i_port_a_byte_address(riscv_pkg::MemDataBits'(i_instr_mem_addr)), + .i_port_a_write_data({2{i_instr_mem_wrdata}}), + .i_port_a_byte_write_enable(instr_mem_dword_we), .o_port_a_read_data( /* unused - write only */), // Port B: Data memory for loads and stores - .i_port_b_byte_address(data_memory_address), + .i_port_b_byte_address(riscv_pkg::MemDataBits'(data_memory_address)), .i_port_b_write_data(data_memory_write_data), .i_port_b_byte_write_enable(data_memory_bram_byte_write_enable), .o_port_b_read_data(data_memory_read_data) @@ -987,8 +1003,8 @@ module cpu_and_mem #( end // Pipeline registers for memory access signals (accounts for RAM read latency) - logic [3:0] data_memory_byte_write_enable_registered; - logic data_memory_read_enable_registered; + logic [riscv_pkg::MemStrbBits-1:0] data_memory_byte_write_enable_registered; + logic data_memory_read_enable_registered; always_ff @(posedge i_clk) begin data_memory_address_registered <= data_memory_address; data_memory_read_enable_registered <= i_rst ? 1'b0 : data_memory_read_enable; @@ -1001,41 +1017,62 @@ module cpu_and_mem #( // signal directly drives the high-fanout MMIO read-data capture enables. assign mmio_read_capture = mmio_read_pulse; - // MMIO read data selection (combinational, captured on mmio_read_pulse) + // MMIO read data selection (combinational, captured on mmio_read_pulse). + // + // The bus carries the ALIGNED-DWORD view (docs/rv64/m1_data_tier.md): each + // case arm is a dword address composing {word at +4, word at +0}, so a + // 32-bit load extracts its word by addr[2] downstream and the legacy hi/lo + // aliases fall out of the same arm - ClintMtimeHi is simply the upper lane + // of ClintMtimeLo's dword. The 64-bit CLINT registers read single-copy + // atomically as one beat. Word-decoded destructive side effects (UART RX + // consume, FIFO pops) are pulsed per word address in the request router, + // so a neighboring value appearing in the other lane consumes nothing. always_comb begin + logic [31:0] ns_thr_rbr_word; + logic [31:0] ns_ier_dlm_word; + ns_thr_rbr_word = ns_lcr[7] ? {24'b0, ns_dll} : {24'b0, i_uart_rx_data}; + ns_ier_dlm_word = ns_lcr[7] ? {24'b0, ns_dlm} : {24'b0, ns_ier}; + mmio_read_data_comb = '0; // Use MA-stage address captured from CPU for MMIO reads - unique case (mmio_load_addr) - // UART RX data - returns received byte in lower 8 bits (reading consumes byte) - UartRxDataMmioAddr: mmio_read_data_comb = {24'b0, i_uart_rx_data}; - // UART RX status - bit 0 indicates data available (non-destructive read) - UartRxStatusMmioAddr: mmio_read_data_comb = {31'b0, i_uart_rx_valid}; - // UART TX status - bit 0 indicates the TX FIFO can accept at least one byte. - UartTxStatusMmioAddr: mmio_read_data_comb = {31'b0, i_uart_tx_ready}; - Fifo0MmioAddr: mmio_read_data_comb = i_fifo0_rd_data; - Fifo1MmioAddr: mmio_read_data_comb = i_fifo1_rd_data; - MtimeLowMmioAddr: mmio_read_data_comb = mtime[31:0]; - MtimeHighMmioAddr: mmio_read_data_comb = mtime[63:32]; - MtimecmpLowMmioAddr: mmio_read_data_comb = mtimecmp[31:0]; - MtimecmpHighMmioAddr: mmio_read_data_comb = mtimecmp[63:32]; - MsipMmioAddr: mmio_read_data_comb = {31'b0, msip}; + unique case ({ + mmio_load_addr[31:3], 3'b000 + }) + // {UART RX data (read consumes byte, via the router's word pulse), + // UART TX (write-only, reads 0)} + {UartMmioAddr[31:3], 3'b000} : mmio_read_data_comb = {{24'b0, i_uart_rx_data}, 32'b0}; + // {FIFO1, FIFO0} + {Fifo0MmioAddr[31:3], 3'b000} : mmio_read_data_comb = {i_fifo1_rd_data, i_fifo0_rd_data}; + {MtimeLowMmioAddr[31:3], 3'b000} : mmio_read_data_comb = mtime; + {MtimecmpLowMmioAddr[31:3], 3'b000} : mmio_read_data_comb = mtimecmp; + // {UART RX status (bit 0: data available), msip} + { + MsipMmioAddr[31:3], 3'b000 + } : + mmio_read_data_comb = {{31'b0, i_uart_rx_valid}, {31'b0, msip}}; + // {-, UART TX status (bit 0: can accept byte)} + { + UartTxStatusMmioAddr[31:3], 3'b000 + } : + mmio_read_data_comb = {32'b0, {31'b0, i_uart_tx_ready}}; // ns16550a UART face (aliases native UART TX/RX). DLAB selects DLL/DLM. - Ns16550ThrRbr: mmio_read_data_comb = ns_lcr[7] ? {24'b0, ns_dll} : {24'b0, i_uart_rx_data}; - Ns16550IerDlm: mmio_read_data_comb = ns_lcr[7] ? {24'b0, ns_dlm} : {24'b0, ns_ier}; - Ns16550IirFcr: mmio_read_data_comb = {24'b0, ns_iir}; - Ns16550Lcr: mmio_read_data_comb = {24'b0, ns_lcr}; - Ns16550Mcr: mmio_read_data_comb = {24'b0, ns_mcr}; + {Ns16550ThrRbr[31:3], 3'b000} : mmio_read_data_comb = {ns_ier_dlm_word, ns_thr_rbr_word}; + {Ns16550IirFcr[31:3], 3'b000} : mmio_read_data_comb = {{24'b0, ns_lcr}, {24'b0, ns_iir}}; // LSR: TEMT|THRE from TX-ready (bits 6,5); DR from RX-valid (bit 0). - Ns16550Lsr: - mmio_read_data_comb = {24'b0, 1'b0, i_uart_tx_ready, i_uart_tx_ready, 4'b0, i_uart_rx_valid}; - Ns16550Msr: mmio_read_data_comb = {24'b0, 8'hB0}; // DCD|DSR|CTS asserted - Ns16550Scr: mmio_read_data_comb = {24'b0, ns_scr}; + { + Ns16550Mcr[31:3], 3'b000 + } : + mmio_read_data_comb = { + {24'b0, 1'b0, i_uart_tx_ready, i_uart_tx_ready, 4'b0, i_uart_rx_valid}, {24'b0, ns_mcr} + }; + { + Ns16550Msr[31:3], 3'b000 + } : + mmio_read_data_comb = {{24'b0, ns_scr}, {24'b0, 8'hB0}}; // scratch | DCD|DSR|CTS // SiFive CLINT alias (same registers as the native timer block). - ClintMsip: mmio_read_data_comb = {31'b0, msip}; - ClintMtimecmpLo: mmio_read_data_comb = mtimecmp[31:0]; - ClintMtimecmpHi: mmio_read_data_comb = mtimecmp[63:32]; - ClintMtimeLo: mmio_read_data_comb = mtime[31:0]; - ClintMtimeHi: mmio_read_data_comb = mtime[63:32]; + {ClintMsip[31:3], 3'b000} : mmio_read_data_comb = {32'b0, {31'b0, msip}}; + {ClintMtimecmpLo[31:3], 3'b000} : mmio_read_data_comb = mtimecmp; + {ClintMtimeLo[31:3], 3'b000} : mmio_read_data_comb = mtime; default: ; endcase end @@ -1058,7 +1095,7 @@ module cpu_and_mem #( // Xilinx-specific timing steering: make the MMIO data capture flops explicit // so Vivado cannot encode zero-valued read cases as synchronous reset pins. for ( - genvar g_mmio_read_data = 0; g_mmio_read_data < 32; g_mmio_read_data++ + genvar g_mmio_read_data = 0; g_mmio_read_data < $bits(mmio_read_data_reg); g_mmio_read_data++ ) begin : gen_mmio_read_data_ff FDRE #( .INIT(1'b0) @@ -1198,10 +1235,12 @@ module cpu_and_mem #( end // FIFO write logic - write to FIFOs when CPU writes to FIFO MMIO addresses - assign o_fifo0_wr_data = data_memory_write_data_registered; + // FIFO registers are 32-bit-access-max (m1_data_tier.md): the addressed + // word is identical in both lanes for sub-dword stores (replication). + assign o_fifo0_wr_data = data_memory_write_data_registered[31:0]; assign o_fifo0_wr_en = |data_memory_byte_write_enable_registered && data_memory_address_registered == Fifo0MmioAddr; - assign o_fifo1_wr_data = data_memory_write_data_registered; + assign o_fifo1_wr_data = data_memory_write_data_registered[31:0]; assign o_fifo1_wr_en = |data_memory_byte_write_enable_registered && data_memory_address_registered == Fifo1MmioAddr; @@ -1231,13 +1270,27 @@ module cpu_and_mem #( // SystemVerilog partial assignments (mtime[31:0] <= ...) only override those bits, // leaving other bits to take the value from the full assignment (mtime <= mtime + N). // This would cause the non-written half to increment during a write, which is wrong. + // Dword-decoded + lane-strobed (the lo/hi word aliases live in one dword; + // see the CLINT comment below): a 64-bit store updates both halves in the + // same cycle. + logic write_hits_mtime; + assign write_hits_mtime = + ({data_memory_address_registered[31:3], 3'b000} == {MtimeLowMmioAddr[31:3], 3'b000}) || + ({data_memory_address_registered[31:3], 3'b000} == {ClintMtimeLo[31:3], 3'b000}); logic writing_mtime_low, writing_mtime_high; - assign writing_mtime_low = |data_memory_byte_write_enable_registered && - ((data_memory_address_registered == MtimeLowMmioAddr) || - (data_memory_address_registered == ClintMtimeLo)); - assign writing_mtime_high = |data_memory_byte_write_enable_registered && - ((data_memory_address_registered == MtimeHighMmioAddr) || - (data_memory_address_registered == ClintMtimeHi)); + assign writing_mtime_low = write_hits_mtime && |data_memory_byte_write_enable_registered[3:0]; + assign writing_mtime_high = write_hits_mtime && |data_memory_byte_write_enable_registered[7:4]; + + // CLINT 64-bit registers are dword-decoded and lane-strobed + // (docs/rv64/m1_data_tier.md): a 32-bit lo/hi alias write carries its word + // replicated across the beat with only its lane's strobes set, so per-lane + // updates reproduce the legacy lo/hi semantics exactly, while a 64-bit + // store sets all eight strobes and lands single-copy atomically (what RV64 + // Linux does to mtimecmp; a 64-bit mtime read is likewise one beat). + logic write_hits_mtimecmp; + assign write_hits_mtimecmp = + ({data_memory_address_registered[31:3], 3'b000} == {MtimecmpLowMmioAddr[31:3], 3'b000}) || + ({data_memory_address_registered[31:3], 3'b000} == {ClintMtimecmpLo[31:3], 3'b000}); always_ff @(posedge i_clk) begin if (i_rst) begin @@ -1246,28 +1299,30 @@ module cpu_and_mem #( msip <= 1'b0; end else begin // mtime update: either write from CPU or increment (not both) - if (writing_mtime_low) begin - mtime[31:0] <= data_memory_write_data_registered; - // High bits: don't increment, just hold value - end else if (writing_mtime_high) begin - mtime[63:32] <= data_memory_write_data_registered; - // Low bits: don't increment, just hold value + if (writing_mtime_low || writing_mtime_high) begin + if (writing_mtime_low) mtime[31:0] <= data_memory_write_data_registered[31:0]; + if (writing_mtime_high) mtime[63:32] <= data_memory_write_data_registered[63:32]; + // Unwritten lanes: don't increment, just hold value end else begin // Normal operation: increment mtime (speedup factor for simulation) mtime <= mtime + 64'(SIM_TIMER_SPEEDUP); end - // mtimecmp and msip writes - if (|data_memory_byte_write_enable_registered) begin - unique case (data_memory_address_registered) - // mtimecmp controls timer interrupt threshold - MtimecmpLowMmioAddr, ClintMtimecmpLo: mtimecmp[31:0] <= data_memory_write_data_registered; - MtimecmpHighMmioAddr, ClintMtimecmpHi: - mtimecmp[63:32] <= data_memory_write_data_registered; - // msip controls software interrupt (only bit 0 is writable) - MsipMmioAddr, ClintMsip: msip <= data_memory_write_data_registered[0]; - default: ; - endcase + // mtimecmp lane-strobed writes (see the dword-decode comment above) + if (write_hits_mtimecmp) begin + if (|data_memory_byte_write_enable_registered[3:0]) + mtimecmp[31:0] <= data_memory_write_data_registered[31:0]; + if (|data_memory_byte_write_enable_registered[7:4]) + mtimecmp[63:32] <= data_memory_write_data_registered[63:32]; + end + + // msip controls software interrupt (only bit 0 is writable). Word + // register: stays word-decoded (the CLINT alias block is 32-bit-access + // for msip; Linux writes it with sw). + if (|data_memory_byte_write_enable_registered && + ((data_memory_address_registered == MsipMmioAddr) || + (data_memory_address_registered == ClintMsip))) begin + msip <= data_memory_write_data_registered[0]; end end end diff --git a/hw/rtl/lib/ram/tdp_bram_dc_byte_en.sv b/hw/rtl/lib/ram/tdp_bram_dc_byte_en.sv index f3f09fb4..3ccff48a 100644 --- a/hw/rtl/lib/ram/tdp_bram_dc_byte_en.sv +++ b/hw/rtl/lib/ram/tdp_bram_dc_byte_en.sv @@ -29,7 +29,9 @@ module tdp_bram_dc_byte_en #( parameter int unsigned DATA_WIDTH = 32, // Data width in bits (must be multiple of 8) parameter int unsigned ADDR_WIDTH = 14, // Word address width (memory depth = 2^ADDR_WIDTH) parameter bit USE_INIT_FILE = 1'b1, - parameter bit [47:0] INIT_FILE = "sw.mem" // Optional hex file for initialization + // Optional hex file for initialization (up to 8 characters, e.g. the + // dword-token "sw64.mem" the 64-bit data tier loads from) + parameter bit [63:0] INIT_FILE = "sw.mem" ) ( // Port A input logic i_port_a_clk, @@ -68,7 +70,7 @@ module tdp_bram_dc_byte_en #( if (USE_INIT_FILE) $readmemh(INIT_FILE, memory); // Initialize with non-zero pattern to catch bugs where code assumes zero-init else - for (int i = 0; i < MemDepthInWords; ++i) memory[i] = i; + for (int i = 0; i < MemDepthInWords; ++i) memory[i] = DATA_WIDTH'(i); // Address conversion from byte-addressing to word-addressing // Lower bits are byte offset within word, remaining bits are word address diff --git a/hw/sim/cpu_tb.sv b/hw/sim/cpu_tb.sv index f738d686..1009ae2c 100644 --- a/hw/sim/cpu_tb.sv +++ b/hw/sim/cpu_tb.sv @@ -19,21 +19,22 @@ module cpu_tb import riscv_pkg::*; #( - parameter int unsigned XLEN = 32, + parameter int unsigned XLEN = riscv_pkg::XLEN, parameter int unsigned MEM_SIZE_BYTES = 2 ** 16 ) ( input logic i_clk, input logic i_rst, // Instruction memory interface - output logic [31:0] o_pc, // Program counter for instruction fetch + output logic [riscv_pkg::XLEN-1:0] o_pc, // Program counter for instruction fetch input logic [31:0] instruction_from_testbench, - // Data memory interface - output logic [31:0] o_data_mem_addr, - output logic [31:0] o_data_mem_wr_data, - output logic [3:0] o_data_mem_per_byte_wr_en, - output logic [3:0] o_data_mem_bram_byte_wr_en, + // Data memory interface (aligned MemDataBits beats with 8-lane strobes; + // docs/rv64/m1_data_tier.md) + output logic [riscv_pkg::XLEN-1:0] o_data_mem_addr, + output logic [riscv_pkg::MemDataBits-1:0] o_data_mem_wr_data, + output logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_per_byte_wr_en, + output logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_bram_byte_wr_en, output logic o_data_mem_read_enable, // Control signals @@ -55,9 +56,9 @@ module cpu_tb logic [1:0] i_instr_hi_rd_is_x2; // {next,current} high-parcel predicates logic i_instr_bank_sel_r; // Fetch-word parity (pc_reg[2]) for the window logic i_instr_valid; // Fetch window valid (tie 1: fixed 1-cycle provider) - logic [31:0] i_served_addr; // Selected BRAM window address tag - logic [29:0] i_served_last_word; // Registered second-word tag for that payload - logic [31:0] i_data_mem_rd_data; // Data memory read data to CPU + logic [riscv_pkg::XLEN-1:0] i_served_addr; // Selected BRAM window address tag + logic [riscv_pkg::XLEN-3:0] i_served_last_word; // Registered second-word tag for that payload + logic [riscv_pkg::MemDataBits-1:0] i_data_mem_rd_data; // Data memory read data to CPU logic pipeline_stall_from_cpu; // Stall signal monitoring (registered, 1-cycle delay) logic pipeline_stall_comb; // Stall signal (combinational, immediate) logic reset_to_cpu; // Reset signal monitoring @@ -65,13 +66,14 @@ module cpu_tb // Registered 1-cycle fetch state (mimics block-RAM instruction memory latency) logic [31:0] tb_cur_word; // current fetch word presented to the CPU logic tb_bank_sel_q; // parity (PC[2]) of the fetched address - logic [31:0] tb_served_addr_q; // address whose window is presented (o_pc, 1 cycle back) - logic [29:0] tb_served_last_word_q; // second word of that registered window + // Address whose window is presented (o_pc, one cycle back). + logic [riscv_pkg::XLEN-1:0] tb_served_addr_q; + logic [riscv_pkg::XLEN-3:0] tb_served_last_word_q; // second word of that registered window // Ports below are unused by this instruction-feed testbench but must exist as // local signals so the wildcard (.*) connection to cpu_ooo resolves. logic o_mmio_read_pulse; - logic [31:0] o_mmio_load_addr; + logic [riscv_pkg::XLEN-1:0] o_mmio_load_addr; logic o_mmio_load_valid; logic o_mmio_fifo0_read_pulse; logic o_mmio_fifo1_read_pulse; @@ -84,18 +86,18 @@ module cpu_tb logic o_fence_i_flush; // Cached (high-address) tier request outputs + response inputs (tied idle: // the directed programs touch only the low BRAM range, never CACHED_BASE). - logic [3:0] o_data_mem_cached_byte_wr_en; - logic [31:0] o_data_mem_cached_wr_data; + logic [riscv_pkg::MemStrbBits-1:0] o_data_mem_cached_byte_wr_en; + logic [riscv_pkg::MemDataBits-1:0] o_data_mem_cached_wr_data; logic o_data_mem_cached_read_enable; - logic [31:0] i_cached_read_data; + logic [riscv_pkg::MemDataBits-1:0] i_cached_read_data; logic i_cached_read_valid; logic i_cached_write_done; logic i_cached_write_inflight; cache_perf_pkg::cache_perf_events_t i_cache_perf_events; // Debug taps (read from cocotb via device_under_test.*; also exposed here). logic [5:0] o_debug_irq_status; - logic [31:0] o_debug_commit_pc; - logic [31:0] o_debug_commit_2_pc; + logic [riscv_pkg::XLEN-1:0] o_debug_commit_pc; + logic [riscv_pkg::XLEN-1:0] o_debug_commit_2_pc; logic [1:0] o_debug_commit_valid; // Interrupt and timer signals for CPU (controllable from testbench) @@ -125,7 +127,7 @@ module cpu_tb tb_cur_word <= instruction_from_testbench; tb_bank_sel_q <= o_pc[2]; // parity of the fetched address tb_served_addr_q <= o_pc; // served-window tag: the address fetched last cycle - tb_served_last_word_q <= o_pc[31:2] + 1'b1; + tb_served_last_word_q <= o_pc[riscv_pkg::XLEN-1:2] + 1'b1; end // 64-bit fetch window {next_word, current_word}. The testbench feeds @@ -168,12 +170,13 @@ module cpu_tb // Memory addressing parameters localparam int unsigned MemByteAddrWidth = $clog2(MEM_SIZE_BYTES); - localparam int unsigned MemWordAddrWidth = MemByteAddrWidth - 2; + localparam int unsigned MemDwordAddrWidth = MemByteAddrWidth - 3; - // Data memory (dual-port RAM, only port B used for data access) + // Data memory (dual-port RAM, only port B used for data access): one + // MemDataBits-wide byte-enabled BRAM, mirroring the production dmem tier. tdp_bram_dc_byte_en #( - .DATA_WIDTH(32), - .ADDR_WIDTH(MemWordAddrWidth), + .DATA_WIDTH(riscv_pkg::MemDataBits), + .ADDR_WIDTH(MemDwordAddrWidth), .USE_INIT_FILE(1'b0) // Don't load from file in testbench ) data_memory_for_simulation ( // Both ports use same clock (single clock domain operation) @@ -186,7 +189,7 @@ module cpu_tb .o_port_a_read_data( /*not connected*/), // Port B: CPU data memory access. Use the BRAM-specific byte-write-enable // so the testbench mirrors the production MMIO-pre-mask behavior. - .i_port_b_byte_address(o_data_mem_addr), + .i_port_b_byte_address(riscv_pkg::MemDataBits'(o_data_mem_addr)), .i_port_b_write_data(o_data_mem_wr_data), .i_port_b_byte_write_enable(o_data_mem_bram_byte_wr_en), .o_port_b_read_data(i_data_mem_rd_data) diff --git a/linux/README.md b/linux/README.md index d8420ba6..7b40cbc1 100644 --- a/linux/README.md +++ b/linux/README.md @@ -66,7 +66,12 @@ cpio at `+8 MiB + 64 KiB` (`0x8081_0000`, bounds passed via M-mode only, no PLIC. The DT wires the CLINT to the hart's `cpu-intc` for machine software (cause 3) and machine timer (cause 7) interrupts; `CONFIG_RISCV_TIMER` drives clocksource/clockevents directly from -`mtime`/`mtimecmp` (no SBI calls). `timebase-frequency` equals the CPU clock +`mtime`/`mtimecmp` (no SBI calls). The dword-aligned CLINT registers +support native 64-bit access on the 64-bit data tier: an 8-byte load of +`mtime` is single-copy atomic (rv32 code may still use the classic +hi/lo/hi word loop; RV64 code reads it in one `ld` with no tearing +exposure), and an 8-byte `mtimecmp` store lands atomically. +`timebase-frequency` equals the CPU clock — `mtime` increments every core cycle, no divider (simulation builds may scale it via the `SIM_TIMER_SPEEDUP` parameter) — and is stamped into the DTB by the packer from `FPGA_CPU_CLK_FREQ` (133.33 MHz Genesys2 default, 300 MHz diff --git a/sw/README.md b/sw/README.md index 65c493ae..23663a9c 100644 --- a/sw/README.md +++ b/sw/README.md @@ -448,15 +448,16 @@ includes it. Each skip is printed with its reason. ./sw/apps/clean_all_apps.py ``` -This removes all build artifacts (sw.elf, sw.mem, sw.bin, sw.txt, sw.S, the -cached-region `sw_ddr.mem`/`sw_ddr.txt`/`sw_ddr.bin` images, and any split-bank -`sw_imem_*.mem` files) from every application directory. +This removes all build artifacts (sw.elf, sw.mem, sw64.mem, sw.bin, sw.txt, +sw.S, the cached-region `sw_ddr.mem`/`sw_ddr.txt`/`sw_ddr.bin` images, and any +split-bank `sw_imem_*.mem` files) from every application directory. ### Build Outputs Compilation produces: - `sw.elf` — ELF executable with debug symbols -- `sw.mem` — Verilog hex format for `$readmemh` (low BRAM image) +- `sw.mem` — Verilog hex format for `$readmemh` (low BRAM image, 32-bit words) +- `sw64.mem` — dword-paired copy of `sw.mem` for the 64-bit data BRAM's `$readmemh` (docs/rv64/m1_data_tier.md) - `sw.bin` — raw binary (low BRAM image) - `sw.txt` — BRAM initialization for Vivado - `sw_ddr.mem` — cached-region (DDR) image for `$readmemh`, region-relative (offset 0 = `0x8000_0000`); a single zero word when the program puts nothing in the cached region diff --git a/sw/apps/arch_test/Makefile b/sw/apps/arch_test/Makefile index 87341fc6..f1c5e6bc 100644 --- a/sw/apps/arch_test/Makefile +++ b/sw/apps/arch_test/Makefile @@ -69,12 +69,13 @@ endif # Output files - same naming as common.mk EXECUTABLE_ELF_FILE = sw.elf VERILOG_HEX_FILE = sw.mem +DWORD_HEX_FILE = sw64.mem DDR_VERILOG_HEX_FILE = sw_ddr.mem RAW_BINARY_FILE = sw.bin DISASSEMBLY_FILE = sw.S # Build targets -all: $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(DISASSEMBLY_FILE) +all: $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(DISASSEMBLY_FILE) # Compile defines the test declares in its RVTEST_CASE strings (riscof's # `def NAME=True` clauses): TEST_CASE_1 gates the body; tests that need @@ -100,6 +101,12 @@ $(VERILOG_HEX_FILE): $(EXECUTABLE_ELF_FILE) $(OBJCOPY) -O verilog --verilog-data-width 4 -R .comment -R .note.gnu.build-id \ $(foreach s,$(DDR_SECTIONS),-R $(s)) $< $@ + +# Dword-paired image for the 64-bit data BRAM ($readmemh rows are dwords; +# docs/rv64/m1_data_tier.md). Every loader keeps the 32-bit-word formats. +$(DWORD_HEX_FILE): $(VERILOG_HEX_FILE) ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py '$<' '$@' + # DDR image, region-relative addresses (the behavioral DDR / JTAG loader # rebase it at 0x8000_0000). With no DDR sections (bram config) emit an empty # image so the harness symlink and the probed $readmemh both stay happy. @@ -121,7 +128,7 @@ $(DISASSEMBLY_FILE): $(EXECUTABLE_ELF_FILE) $(OBJDUMP) -d $< > $@ clean: - rm -f $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) \ + rm -f $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(DDR_VERILOG_HEX_FILE) \ $(RAW_BINARY_FILE) $(DISASSEMBLY_FILE) .PHONY: all clean diff --git a/sw/apps/coremark_pro/Makefile b/sw/apps/coremark_pro/Makefile index 7614cb93..4b8d1fd3 100644 --- a/sw/apps/coremark_pro/Makefile +++ b/sw/apps/coremark_pro/Makefile @@ -570,6 +570,7 @@ BUILD_MAKEFILES := $(MAKEFILE_LIST) # --- Output files ---------------------------------------------------------- EXECUTABLE_ELF_FILE := sw.elf VERILOG_HEX_FILE := sw.mem +DWORD_HEX_FILE := sw64.mem RAW_BINARY_FILE := sw.bin VIVADO_BRAM_FILE := sw.txt DDR_HEX_FILE := sw_ddr.mem @@ -585,8 +586,8 @@ shell_quote = '$(subst ','"'"',$(1))' # --- Build targets --------------------------------------------------------- .DEFAULT_GOAL := all -all: $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) \ - $(DDR_HEX_FILE) $(DDR_TXT_FILE) $(DISASSEMBLY_FILE) +all: $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(RAW_BINARY_FILE) \ + $(VIVADO_BRAM_FILE) $(DDR_HEX_FILE) $(DDR_TXT_FILE) $(DISASSEMBLY_FILE) -include $(DEPENDENCY_FILES) @@ -659,6 +660,11 @@ $(VERILOG_HEX_FILE): $(EXECUTABLE_ELF_FILE) $(OBJCOPY) -O verilog --verilog-data-width 4 -R .comment -R .note.gnu.build-id \ $(addprefix -R ,$(DDR_SPLIT_SECTIONS)) $< $@ +# Dword-paired image for the 64-bit data BRAM ($readmemh rows are dwords; +# docs/rv64/m1_data_tier.md). Every loader keeps the 32-bit-word formats. +$(DWORD_HEX_FILE): $(VERILOG_HEX_FILE) ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py '$<' '$@' + $(RAW_BINARY_FILE): $(EXECUTABLE_ELF_FILE) $(OBJCOPY) -O binary -R .comment -R .note.gnu.build-id \ $(addprefix -R ,$(DDR_SPLIT_SECTIONS)) $< $@ @@ -702,7 +708,8 @@ size: $(EXECUTABLE_ELF_FILE) $(SIZE) $< clean: - $(RM) *.o $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) \ + $(RM) *.o $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) \ + $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) \ $(DDR_HEX_FILE) $(DDR_TXT_FILE) sw_ddr.bin $(DISASSEMBLY_FILE) \ .frost-deps-*.o $(BUILD_CONFIG_FILE) diff --git a/sw/apps/linux_boot/Makefile b/sw/apps/linux_boot/Makefile index 78066526..13aa4003 100644 --- a/sw/apps/linux_boot/Makefile +++ b/sw/apps/linux_boot/Makefile @@ -80,22 +80,34 @@ endif # removes packer scratch files. ifeq ($(FROST_LINUX_PREBUILT),1) -all: +# sw64.mem is a pure re-encoding of the staged sw.mem (dword rows for the +# 64-bit data BRAM; docs/rv64/m1_data_tier.md), so deriving it locally does +# not contradict "staged images are authoritative". +all: sw64.mem @test -f sw.mem -a -f sw_ddr.mem || { \ echo "ERROR: FROST_LINUX_PREBUILT=1 but sw.mem / sw_ddr.mem are not staged here."; \ exit 1; } @echo "linux_boot: FROST_LINUX_PREBUILT=1 -> using staged images as-is" +sw64.mem: sw.mem ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py '$<' '$@' + clean: - rm -f shim.elf shim.bin frost_boot_shim.S \ + rm -f sw64.mem shim.elf shim.bin frost_boot_shim.S \ frost-nommu-fpga.dts frost-nommu-fpga.dtb distclean: clean else # --- Normal self-build mode --------------------------------------------- -# sw_ddr.mem's recipe emits all four images at once, so it is the canonical goal. -all: sw_ddr.mem +# sw_ddr.mem's recipe emits all four images at once, so it is the canonical +# goal; sw64.mem is derived from the packed sw.mem afterwards. +all: sw_ddr.mem sw64.mem + +# Dword-paired copy of the boot shim image for the 64-bit data BRAM's +# $readmemh (docs/rv64/m1_data_tier.md). +sw64.mem: sw.mem ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py '$<' '$@' # Stage 1: kernel Image + rootfs via Buildroot. Skipped when already built. # Grouped target (&:): one Buildroot invocation produces both files (a plain @@ -124,7 +136,7 @@ sw_ddr.mem sw.mem sw_ddr.txt sw.txt: $(KIMAGE) $(INITRD_IMG) $(BOARD_DIR)/build_ # Buildroot kernel/rootfs (Stage 1) survive, so a re-pack for another board is # fast. This is what load_software.py runs before every load. clean: - rm -f sw.mem sw.txt sw_ddr.mem sw_ddr.txt \ + rm -f sw.mem sw64.mem sw.txt sw_ddr.mem sw_ddr.txt \ shim.elf shim.bin frost_boot_shim.S \ frost-nommu-fpga.dts frost-nommu-fpga.dtb diff --git a/sw/apps/riscv_tests/Makefile b/sw/apps/riscv_tests/Makefile index 757543dc..a5167eb0 100644 --- a/sw/apps/riscv_tests/Makefile +++ b/sw/apps/riscv_tests/Makefile @@ -63,12 +63,13 @@ endif # Output files EXECUTABLE_ELF_FILE = sw.elf VERILOG_HEX_FILE = sw.mem +DWORD_HEX_FILE = sw64.mem DDR_VERILOG_HEX_FILE = sw_ddr.mem RAW_BINARY_FILE = sw.bin DISASSEMBLY_FILE = sw.S # Build targets -all: $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(DISASSEMBLY_FILE) +all: $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(DISASSEMBLY_FILE) # Compile and link .S test file via gcc (handles #include). # ISA tests provide their own _start; the ddr config additionally links a ROM @@ -86,6 +87,12 @@ $(VERILOG_HEX_FILE): $(EXECUTABLE_ELF_FILE) $(OBJCOPY) -O verilog --verilog-data-width 4 -R .comment -R .note.gnu.build-id \ $(foreach s,$(DDR_SECTIONS),-R $(s)) $< $@ + +# Dword-paired image for the 64-bit data BRAM ($readmemh rows are dwords; +# docs/rv64/m1_data_tier.md). Every loader keeps the 32-bit-word formats. +$(DWORD_HEX_FILE): $(VERILOG_HEX_FILE) ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py '$<' '$@' + # DDR image, region-relative addresses (the behavioral DDR rebases at # 0x8000_0000). With no DDR sections (bram config) emit an empty image so the # harness symlink and the sim's $readmemh both stay happy. @@ -107,7 +114,7 @@ $(DISASSEMBLY_FILE): $(EXECUTABLE_ELF_FILE) $(OBJDUMP) -d $< > $@ clean: - rm -f $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) \ + rm -f $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(DDR_VERILOG_HEX_FILE) \ $(RAW_BINARY_FILE) $(DISASSEMBLY_FILE) .PHONY: all clean diff --git a/sw/apps/riscv_tests/Makefile.bench b/sw/apps/riscv_tests/Makefile.bench index 32641b33..bf845a0f 100644 --- a/sw/apps/riscv_tests/Makefile.bench +++ b/sw/apps/riscv_tests/Makefile.bench @@ -60,6 +60,7 @@ LDFLAGS = -static -nostdlib -nostartfiles -lm -lgcc -T $(LINKER_SCRIPT) # Output files EXECUTABLE_ELF_FILE = sw.elf VERILOG_HEX_FILE = sw.mem +DWORD_HEX_FILE = sw64.mem DDR_VERILOG_HEX_FILE = sw_ddr.mem RAW_BINARY_FILE = sw.bin DISASSEMBLY_FILE = sw.S @@ -81,7 +82,7 @@ BENCH_C_SRC = $(wildcard $(BENCH_DIR)/$(BENCH)/*.c) BENCH_S_SRC = $(wildcard $(BENCH_DIR)/$(BENCH)/*.S) # Build targets -all: $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(DISASSEMBLY_FILE) +all: $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(DISASSEMBLY_FILE) # Frost library objects: compiled with the lib's own include path in isolation. frostlib_%.o: $(LIB_DIR)/src/%.c @@ -103,6 +104,12 @@ $(VERILOG_HEX_FILE): $(EXECUTABLE_ELF_FILE) $(OBJCOPY) -O verilog --verilog-data-width 4 -R .comment -R .note.gnu.build-id \ $(foreach s,$(DDR_SECTIONS),-R $(s)) $< $@ + +# Dword-paired image for the 64-bit data BRAM ($readmemh rows are dwords; +# docs/rv64/m1_data_tier.md). Every loader keeps the 32-bit-word formats. +$(DWORD_HEX_FILE): $(VERILOG_HEX_FILE) ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py '$<' '$@' + # DDR image, region-relative addresses. With no DDR sections (bram config) emit # an empty image so the harness symlink and the sim's $readmemh stay happy. $(DDR_VERILOG_HEX_FILE): $(EXECUTABLE_ELF_FILE) @@ -121,6 +128,6 @@ $(DISASSEMBLY_FILE): $(EXECUTABLE_ELF_FILE) $(OBJDUMP) -d $< > $@ clean: - rm -f $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(RAW_BINARY_FILE) $(DISASSEMBLY_FILE) $(LIB_OBJ) + rm -f $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(RAW_BINARY_FILE) $(DISASSEMBLY_FILE) $(LIB_OBJ) .PHONY: all clean diff --git a/sw/apps/riscv_torture/Makefile b/sw/apps/riscv_torture/Makefile index 84b71ba3..7b53217c 100644 --- a/sw/apps/riscv_torture/Makefile +++ b/sw/apps/riscv_torture/Makefile @@ -61,12 +61,13 @@ endif # Output files EXECUTABLE_ELF_FILE = sw.elf VERILOG_HEX_FILE = sw.mem +DWORD_HEX_FILE = sw64.mem DDR_VERILOG_HEX_FILE = sw_ddr.mem RAW_BINARY_FILE = sw.bin DISASSEMBLY_FILE = sw.S # Build targets -all: $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(DISASSEMBLY_FILE) +all: $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(DDR_VERILOG_HEX_FILE) $(DISASSEMBLY_FILE) # Compile the adapted torture test (header + test + footer as single .S file). # The ddr config also assembles the ROM boot stub ($(BOOT_STUB)); the bram @@ -84,6 +85,12 @@ $(VERILOG_HEX_FILE): $(EXECUTABLE_ELF_FILE) $(OBJCOPY) -O verilog --verilog-data-width 4 -R .comment -R .note.gnu.build-id \ $(foreach s,$(DDR_SECTIONS),-R $(s)) $< $@ + +# Dword-paired image for the 64-bit data BRAM ($readmemh rows are dwords; +# docs/rv64/m1_data_tier.md). Every loader keeps the 32-bit-word formats. +$(DWORD_HEX_FILE): $(VERILOG_HEX_FILE) ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py '$<' '$@' + # DDR image, region-relative addresses (the behavioral DDR rebases at # 0x8000_0000). With no DDR sections (bram config) emit an empty image so the # harness symlink and the sim's $readmemh both stay happy. @@ -103,7 +110,7 @@ $(DISASSEMBLY_FILE): $(EXECUTABLE_ELF_FILE) $(OBJDUMP) -d $< > $@ clean: - rm -f $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) \ + rm -f $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(DDR_VERILOG_HEX_FILE) \ $(RAW_BINARY_FILE) $(DISASSEMBLY_FILE) .PHONY: all clean diff --git a/sw/common/common.mk b/sw/common/common.mk index 163337ae..042e27b7 100644 --- a/sw/common/common.mk +++ b/sw/common/common.mk @@ -149,6 +149,7 @@ EXTRA_ASM_SRC ?= # Output file names EXECUTABLE_ELF_FILE := sw.elf # ELF executable with debug info VERILOG_HEX_FILE := sw.mem # Verilog hex format for $readmemh +DWORD_HEX_FILE := sw64.mem # Dword-paired copy for the 64-bit data BRAM RAW_BINARY_FILE := sw.bin # Raw binary (no ELF headers) VIVADO_BRAM_FILE := sw.txt # BRAM initialization format for Vivado DDR_HEX_FILE := sw_ddr.mem # Cached-region (DDR) image, region-relative @@ -198,7 +199,7 @@ shell_quote = '$(subst ','"'"',$(1))' # -MP keeps a removed header from making the old dependency fragment unparseable. # Build targets -all: $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) $(DDR_HEX_FILE) \ +all: $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) $(DDR_HEX_FILE) \ $(DDR_TXT_FILE) $(DISASSEMBLY_FILE) $(IMEM_INIT_TARGETS) # Keep `all` as the default goal even after the generated fragment exists (its @@ -245,6 +246,11 @@ $(VERILOG_HEX_FILE): $(EXECUTABLE_ELF_FILE) $(OBJCOPY) -O verilog --verilog-data-width 4 -R .comment -R .note.gnu.build-id \ $(addprefix -R ,$(DDR_SPLIT_SECTIONS)) $< $@ +# Dword-paired image for the 64-bit data BRAM ($readmemh rows are dwords; +# docs/rv64/m1_data_tier.md). Every loader keeps the 32-bit-word formats. +$(DWORD_HEX_FILE): $(VERILOG_HEX_FILE) ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py $< $@ + # Generate raw binary file (stripped of ELF headers and metadata; cached-region # sections excluded so the binary spans only the low BRAM image) $(RAW_BINARY_FILE): $(EXECUTABLE_ELF_FILE) @@ -319,7 +325,7 @@ size: $(EXECUTABLE_ELF_FILE) # Clean all build artifacts clean: - $(RM) $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) $(DDR_HEX_FILE) \ + $(RM) $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) $(DDR_HEX_FILE) \ $(DDR_TXT_FILE) sw_ddr.bin $(DISASSEMBLY_FILE) $(BUILD_CONFIG_FILE) $(DEPENDENCY_FILE) \ $(IMEM_EVEN_COLD_INIT_FILE) $(IMEM_ODD_COLD_INIT_FILE) \ $(IMEM_EVEN_FRONTEND_HOT_INIT_FILE) $(IMEM_ODD_FRONTEND_HOT_INIT_FILE) \ diff --git a/sw/common/make_dword_mem.py b/sw/common/make_dword_mem.py new file mode 100644 index 00000000..0fe25179 --- /dev/null +++ b/sw/common/make_dword_mem.py @@ -0,0 +1,77 @@ +#!/usr/bin/env python3 +# Copyright 2026 Two Sigma Open Source, LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Pair a 32-bit-word verilog-hex image into 64-bit-dword rows. + +The data memory BRAM is one MemDataBits(=64)-wide byte-enabled RAM +(docs/rv64/m1_data_tier.md), so its ``$readmemh`` init file needs one +64-bit token per dword row. Every other consumer of the software image +(imem with its per-word predecode sideband, the JTAG loaders, sw.txt) +keeps the 32-bit-word ``sw.mem`` format, so this script derives the +dword file rather than changing the objcopy flow. + +Input: objcopy ``-O verilog --verilog-data-width 4`` output - ``@ADDR`` +records in word units followed by 8-hex-digit little-endian word tokens. +Output: the same format at dword granularity (``@ADDR`` in dword units, +16-hex-digit tokens, low word in bits [31:0]). + +Halves are merged through a sparse map so adjacent sections that share a +dword row compose correctly; a half never written by the input is zero - +matching both Verilator's 2-state zero-init and hardware BRAM defaults. +""" + +import sys +from pathlib import Path + + +def convert(text: str) -> str: + """Convert 32-bit-word verilog-hex text to 64-bit-dword rows.""" + rows: dict[int, list[int]] = {} + word_addr = 0 + for token in text.split(): + if token.startswith("@"): + word_addr = int(token[1:], 16) + continue + value = int(token, 16) + row = rows.setdefault(word_addr // 2, [0, 0]) + row[word_addr % 2] = value + word_addr += 1 + + out: list[str] = [] + prev_row = None + for row_addr in sorted(rows): + if prev_row is None or row_addr != prev_row + 1: + out.append(f"@{row_addr:08X}") + lo, hi = rows[row_addr] + out.append(f"{(hi << 32) | lo:016X}") + prev_row = row_addr + return "\n".join(out) + "\n" if out else "" + + +def main() -> int: + """CLI entry point: make_dword_mem.py .""" + if len(sys.argv) != 3: + print( + "usage: make_dword_mem.py ", + file=sys.stderr, + ) + return 2 + src, dst = Path(sys.argv[1]), Path(sys.argv[2]) + dst.write_text(convert(src.read_text())) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/sw/common/standalone_asm.mk b/sw/common/standalone_asm.mk index 9b06c6f8..06fe96d0 100644 --- a/sw/common/standalone_asm.mk +++ b/sw/common/standalone_asm.mk @@ -55,6 +55,7 @@ endif EXECUTABLE_ELF_FILE := sw.elf VERILOG_HEX_FILE := sw.mem +DWORD_HEX_FILE := sw64.mem DDR_VERILOG_HEX_FILE := sw_ddr.mem RAW_BINARY_FILE := sw.bin VIVADO_BRAM_FILE := sw.txt @@ -74,7 +75,8 @@ BUILD_MAKEFILES := $(MAKEFILE_LIST) EFFECTIVE_BUILD_CONFIG = MEM_CONFIG=$(MEM_CONFIG)|ARCH=$(ARCH)|ABI=$(ABI)|AS=$(AS)|LD=$(LD)|CC=$(CC)|OBJCOPY=$(OBJCOPY)|OBJDUMP=$(OBJDUMP)|ASM_FLAGS=$(ASM_FLAGS)|BOOT_CFLAGS=$(BOOT_CFLAGS)|LINK_FLAGS=$(LINK_FLAGS)|LINKER_SCRIPT=$(LINKER_SCRIPT)|BOOT_STUB_OBJ=$(BOOT_STUB_OBJ)|DDR_SECTIONS=$(DDR_SECTIONS)|ASM_SRC=$(ASM_SRC) shell_quote = '$(subst ','"'"',$(1))' -all: $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) \ +all: $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) \ + $(DDR_VERILOG_HEX_FILE) \ $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) $(DISASSEMBLY_FILE) .PHONY: FORCE @@ -117,6 +119,11 @@ $(VERILOG_HEX_FILE): $(EXECUTABLE_ELF_FILE) $(OBJCOPY) -O verilog --verilog-data-width 4 -R .comment -R .note.gnu.build-id \ $(addprefix -R ,$(DDR_SECTIONS)) '$<' '$@' +# Dword-paired image for the 64-bit data BRAM ($readmemh rows are dwords; +# docs/rv64/m1_data_tier.md). Every loader keeps the 32-bit-word formats. +$(DWORD_HEX_FILE): $(VERILOG_HEX_FILE) ../../common/make_dword_mem.py + python3 ../../common/make_dword_mem.py '$<' '$@' + # Generate the cached-region image atomically. An empty selected-section set is # legitimate in the BRAM tier and becomes one zero word for unconditional # $readmemh consumers; any objcopy error is fatal and leaves an old target @@ -141,7 +148,8 @@ $(DISASSEMBLY_FILE): $(EXECUTABLE_ELF_FILE) $(OBJDUMP) -d '$<' > '$@' clean: - $(RM) $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DDR_VERILOG_HEX_FILE) \ + $(RM) $(EXECUTABLE_ELF_FILE) $(VERILOG_HEX_FILE) $(DWORD_HEX_FILE) \ + $(DDR_VERILOG_HEX_FILE) \ $(RAW_BINARY_FILE) $(VIVADO_BRAM_FILE) $(DISASSEMBLY_FILE) \ $(ASSEMBLY_OBJECT_FILE) $(DDR_BOOT_STUB_OBJ) $(BUILD_CONFIG_FILE) diff --git a/tests/Makefile b/tests/Makefile index a85a73ed..c7d6a267 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -207,6 +207,7 @@ VERILOG_SOURCES := else ifeq ($(TOPLEVEL),direction_predictor) # IF-stage branch direction predictor unit test COMPILE_ARGS := \ + $(ROOT)/hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv \ $(ROOT)/hw/rtl/lib/ram/sdp_dist_ram.sv \ $(ROOT)/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/direction_predictor.sv VERILOG_SOURCES := @@ -224,11 +225,13 @@ VERILOG_SOURCES := else ifeq ($(TOPLEVEL),prediction_metadata_tracker) # IF-stage prediction metadata tracker unit test COMPILE_ARGS := \ + $(ROOT)/hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv \ $(ROOT)/hw/rtl/cpu_and_mem/cpu/if_stage/branch_prediction/prediction_metadata_tracker.sv VERILOG_SOURCES := else ifeq ($(TOPLEVEL),control_flow_tracker) # IF-stage control-flow holdoff tracker unit test COMPILE_ARGS := \ + $(ROOT)/hw/rtl/cpu_and_mem/cpu/riscv_pkg.sv \ $(ROOT)/hw/rtl/cpu_and_mem/cpu/if_stage/control_flow_tracker.sv VERILOG_SOURCES := else ifeq ($(TOPLEVEL),pc_increment_calculator) @@ -372,6 +375,13 @@ ifneq ($(strip $(FROST_VERILATOR_EXTRA_ARGS)),) EXTRA_ARGS += $(FROST_VERILATOR_EXTRA_ARGS) endif +# RV64 build (ROADMAP Phase 1): FROST_RV64=1 defines FROST_RV64 for the RTL +# (riscv_pkg selects XLEN=64) and is read by verif/config.py so the Python +# side flips in lockstep (docs/rv64/phase1_plan.md decision D1). +ifeq ($(FROST_RV64),1) + EXTRA_ARGS += -DFROST_RV64 +endif + # Low-BRAM size: matches hardware (256 KiB, frost.sv MEM_SIZE_BYTES default). # Only for toplevels that have the MEM_SIZE_BYTES parameter. SIM_MEM_SIZE_BYTES ?= 262144 diff --git a/tests/test_arch_compliance.py b/tests/test_arch_compliance.py index 8e147c60..c5bb2794 100755 --- a/tests/test_arch_compliance.py +++ b/tests/test_arch_compliance.py @@ -257,7 +257,7 @@ def run_simulation() -> subprocess.CompletedProcess[str] | None: subprocess.run(["make", "clean"], check=False, env=env) # Set up sw.mem / sw_ddr.mem symlinks pointing to our compiled test - for mem_name in ("sw.mem", "sw_ddr.mem"): + for mem_name in ("sw.mem", "sw64.mem", "sw_ddr.mem"): mem_path = Path(mem_name) if mem_path.exists() or mem_path.is_symlink(): mem_path.unlink() @@ -287,7 +287,7 @@ def run_simulation() -> subprocess.CompletedProcess[str] | None: except subprocess.TimeoutExpired: return None finally: - for mem_name in ("sw.mem", "sw_ddr.mem"): + for mem_name in ("sw.mem", "sw64.mem", "sw_ddr.mem"): mem_path = Path(mem_name) if mem_path.exists() or mem_path.is_symlink(): mem_path.unlink() diff --git a/tests/test_riscv_tests.py b/tests/test_riscv_tests.py index 3e6c684d..d21d4cfb 100755 --- a/tests/test_riscv_tests.py +++ b/tests/test_riscv_tests.py @@ -236,11 +236,11 @@ def run_simulation( if needs_clean: subprocess.run(["make", "clean"], check=False) - sw_mem_path = Path("sw.mem") - if sw_mem_path.exists() or sw_mem_path.is_symlink(): - sw_mem_path.unlink() - sw_mem_target = RISCV_TESTS_APP_DIR / "sw.mem" - sw_mem_path.symlink_to(sw_mem_target) + for mem_name in ("sw.mem", "sw64.mem"): + mem_path = Path(mem_name) + if mem_path.exists() or mem_path.is_symlink(): + mem_path.unlink() + mem_path.symlink_to(RISCV_TESTS_APP_DIR / mem_name) # The ddr config splits the test into the DDR image; the sim preloads # the behavioral DDR from sw_ddr.mem (empty for the bram config). @@ -273,7 +273,7 @@ def run_simulation( except subprocess.TimeoutExpired: return None finally: - for mem_name in ("sw.mem", "sw_ddr.mem"): + for mem_name in ("sw.mem", "sw64.mem", "sw_ddr.mem"): mem_path = Path(mem_name) if mem_path.exists() or mem_path.is_symlink(): mem_path.unlink() diff --git a/tests/test_riscv_torture.py b/tests/test_riscv_torture.py index e4e50c07..95b832ab 100755 --- a/tests/test_riscv_torture.py +++ b/tests/test_riscv_torture.py @@ -133,11 +133,11 @@ def run_simulation(simulator: str) -> subprocess.CompletedProcess[str] | None: if needs_clean: subprocess.run(["make", "clean"], check=False) - sw_mem_path = Path("sw.mem") - if sw_mem_path.exists() or sw_mem_path.is_symlink(): - sw_mem_path.unlink() - sw_mem_target = TORTURE_APP_DIR / "sw.mem" - sw_mem_path.symlink_to(sw_mem_target) + for mem_name in ("sw.mem", "sw64.mem"): + mem_path = Path(mem_name) + if mem_path.exists() or mem_path.is_symlink(): + mem_path.unlink() + mem_path.symlink_to(TORTURE_APP_DIR / mem_name) # The ddr config splits the test into the DDR image; the sim preloads # the behavioral DDR from sw_ddr.mem (empty for the bram config). @@ -170,7 +170,7 @@ def run_simulation(simulator: str) -> subprocess.CompletedProcess[str] | None: except subprocess.TimeoutExpired: return None finally: - for mem_name in ("sw.mem", "sw_ddr.mem"): + for mem_name in ("sw.mem", "sw64.mem", "sw_ddr.mem"): mem_path = Path(mem_name) if mem_path.exists() or mem_path.is_symlink(): mem_path.unlink() diff --git a/tests/test_run_cocotb.py b/tests/test_run_cocotb.py index b7d4e9c7..aafba787 100755 --- a/tests/test_run_cocotb.py +++ b/tests/test_run_cocotb.py @@ -1001,7 +1001,17 @@ def _ensure_symlink(link: Path, target: str) -> None: link, and an unconditional unlink+recreate opens a window where a sibling's $readmemh sees no file. A lost creation race against a sibling pointing at the same target is accepted as success. + + The target must exist: a dangling link makes the RTL's $readmemh + fail quietly and the affected memory reads as zeros (a missing + sw64.mem left the fpu_assembly_test data BRAM empty — every load + returned 0 and the program silently fell through to its done spin). """ + if not Path(target).exists(): + raise FileNotFoundError( + f"program memory image '{target}' does not exist; " + "the app build should have produced it" + ) try: if link.is_symlink() and os.readlink(link) == target: return @@ -1217,6 +1227,10 @@ def run_simulation( program_memory_file = self._get_program_memory_file() if program_memory_file: self._ensure_symlink(Path("sw.mem"), program_memory_file) + self._ensure_symlink( + Path("sw64.mem"), + program_memory_file.replace("sw.mem", "sw64.mem"), + ) self._ensure_symlink( Path("sw_ddr.mem"), program_memory_file.replace("sw.mem", "sw_ddr.mem"), diff --git a/tests/test_run_yosys.py b/tests/test_run_yosys.py index 20d7643c..5c812b9d 100755 --- a/tests/test_run_yosys.py +++ b/tests/test_run_yosys.py @@ -178,17 +178,23 @@ def __init__(self, filelist_key: str = "frost") -> None: self.setup_sw_mem() def setup_sw_mem(self) -> None: - """Compile hello_world and set up sw.mem symlink for synthesis.""" - # Compile hello_world to ensure sw.mem exists + """Compile hello_world and set up sw.mem/sw64.mem symlinks for synthesis. + + The imem BRAM $readmemh's sw.mem (32-bit words) and the 64-bit data + BRAM $readmemh's sw64.mem (dword tokens; docs/rv64/m1_data_tier.md) — + both produced by the hello_world build. + """ + # Compile hello_world to ensure sw.mem/sw64.mem exist if not _compile_hello_world(self.root_dir): raise RuntimeError("Failed to compile hello_world for synthesis") - sw_mem_target = self.root_dir / "sw" / "apps" / "hello_world" / "sw.mem" - sw_mem_link = self.test_dir / "sw.mem" + for mem_name in ("sw.mem", "sw64.mem"): + mem_target = self.root_dir / "sw" / "apps" / "hello_world" / mem_name + mem_link = self.test_dir / mem_name - if sw_mem_link.exists() or sw_mem_link.is_symlink(): - sw_mem_link.unlink() - sw_mem_link.symlink_to(sw_mem_target) + if mem_link.exists() or mem_link.is_symlink(): + mem_link.unlink() + mem_link.symlink_to(mem_target) def parse_filelist(self, filelist_path: Path) -> list[str]: """Parse a filelist file and return deduplicated list of Verilog files. @@ -255,9 +261,14 @@ def run_synthesis( # assertions, etc.) is excluded during synthesis. # -DFROST_XILINX_PRIMS: Enable Xilinx primitive instantiations only # for synth_xilinx targets; generic/ASIC synthesis stays agnostic. + # -DFROST_RV64: RV64 build (ROADMAP Phase 1, plan D1) - riscv_pkg + # selects XLEN=64; driven by the same environment variable the + # cocotb Makefile and verif/config.py key on. defines = "-DSYNTHESIS" if synth_command.startswith("synth_xilinx"): defines += " -DFROST_XILINX_PRIMS" + if os.environ.get("FROST_RV64") == "1": + defines += " -DFROST_RV64" # Build Yosys script yosys_script = [] diff --git a/verif/cocotb_tests/cpu_model.py b/verif/cocotb_tests/cpu_model.py index 4c58fbc4..84a61585 100644 --- a/verif/cocotb_tests/cpu_model.py +++ b/verif/cocotb_tests/cpu_model.py @@ -43,7 +43,7 @@ """ import cocotb -from config import MASK32, MEMORY_WORD_ALIGN_MASK +from config import MASK32, MEMORY_WORD_ALIGN_MASK, MEMORY_DWORD_ALIGN_MASK from encoders.op_tables import ( R_ALU, I_ALU, @@ -80,7 +80,8 @@ from models.alu_model import lw from utils.memory_utils import ( calculate_byte_mask_for_store, - get_byte_offset, + get_beat_byte_offset, + replicate_store_data_for_beat, ) from cocotb_tests.test_state import TestState @@ -444,15 +445,15 @@ def model_memory_write( queues and the memory model. Memory Write Encoding: - RISC-V stores write data to a 32-bit word-aligned memory interface. - For sub-word stores (SB, SH), the data must be shifted to the correct - byte position within the word. + Stores write aligned 64-bit beats with 8-lane byte strobes + (docs/rv64/m1_data_tier.md). Sub-beat data is replicated across + the beat and the strobe selects the addressed lanes. Example: SB x5, 2(x1) where x1=0x1001, x5=0xAB - Address = 0x1001 + 2 = 0x1003 - - Byte offset = 3 (address & 0x3) - - Write data = 0xAB << 24 = 0xAB000000 - - Write mask = 0b1000 (byte 3 only) + - Beat offset = 3 (address & 0x7) + - Write data = 0xABABABABABABABAB (byte replicated) + - Write mask = 0b00001000 (lane 3 only) Args: state: Test state with register values and expected queues @@ -477,9 +478,11 @@ def model_memory_write( cocotb.log.info( f"op sc.w SUCCESS: writing data {write_data} to address {write_address}" ) - # Update expected queues + # Update expected queues (word data rides the beat replicated) state.memory_write_address_expected_queue.append(write_address) - state.memory_write_data_expected_queue.append(write_data) + state.memory_write_data_expected_queue.append( + replicate_store_data_for_beat("sw", write_data) + ) # Update memory model mem_model.write_word(write_address, write_data) return @@ -503,9 +506,13 @@ def model_memory_write( f"old={old_value}, rs2={state.register_file_previous[source_register_2]}, " f"new={new_value}" ) - # Update expected queues + # Update expected queues. AMO writes are word-sized: the LQ + # replicates the result across the beat and the router's strobes + # select the addressed word lanes. state.memory_write_address_expected_queue.append(write_address) - state.memory_write_data_expected_queue.append(new_value) + state.memory_write_data_expected_queue.append( + replicate_store_data_for_beat("sw", new_value) + ) # Update memory model mem_model.write_word(write_address, new_value) return @@ -519,34 +526,29 @@ def model_memory_write( # Get data from FP register file fp_value = state.fp_register_file_previous[source_register_2] if operation == "fsd": - # RTL writes FSD as two 32-bit stores: low word then high word. - low_word = fp_value & MASK32 - high_word = (fp_value >> 32) & MASK32 + # Single-beat FSD: one 64-bit write covering the aligned dword + # (docs/rv64/m1_data_tier.md — the two-phase drain is gone). cocotb.log.info( f"op {operation} storing fp_rs2_val 0x{fp_value:016X} " f"to address 0x{write_address:08X}" ) - # Queue low word then high word (little-endian) state.memory_write_address_expected_queue.append(write_address) - state.memory_write_data_expected_queue.append(low_word) - state.memory_write_address_expected_queue.append( - (write_address + 4) & MASK32 + state.memory_write_data_expected_queue.append( + replicate_store_data_for_beat("fsd", fp_value) ) - state.memory_write_data_expected_queue.append(high_word) # Update memory model - mem_model.write_word(write_address & MEMORY_WORD_ALIGN_MASK, low_word) - mem_model.write_word( - (write_address + 4) & MEMORY_WORD_ALIGN_MASK, high_word - ) + mem_model.write_dword(write_address & MEMORY_DWORD_ALIGN_MASK, fp_value) else: write_data = fp_value & MASK32 cocotb.log.info( f"op {operation} with fp_rs2_val 0x{write_data:08X} " f"storing to address 0x{write_address:08X}" ) - # Update expected queues + # Update expected queues (word data rides the beat replicated) state.memory_write_address_expected_queue.append(write_address) - state.memory_write_data_expected_queue.append(write_data) + state.memory_write_data_expected_queue.append( + replicate_store_data_for_beat("fsw", write_data) + ) # Update memory model (word-aligned store) mem_model.write_word(write_address & MEMORY_WORD_ALIGN_MASK, write_data) return @@ -559,38 +561,34 @@ def model_memory_write( state.register_file_previous[source_register_1] + immediate ) & MASK32 - # Get byte position within 32-bit word (0-3) - byte_offset = get_byte_offset(write_address) + # Get byte position within the data-tier beat (0-7) + beat_offset = get_beat_byte_offset(write_address) # Get value to store from source register source_register_2_value = ( state.register_file_previous[source_register_2] & MASK32 ) - # Calculate write mask (which bytes in word to update) - write_mask = calculate_byte_mask_for_store(operation, byte_offset) - - # Calculate write data, shifting to correct byte lanes for sub-word stores - # For SB/SH: shift left so data aligns with byte position - # For SW: no shift needed, use full word - if operation in ("sb", "sh"): - write_data = (source_register_2_value << (8 * byte_offset)) & MASK32 - else: # sw - write_data = source_register_2_value & MASK32 + # Beat contract (docs/rv64/m1_data_tier.md): data is replicated + # across the 64-bit beat and the 8-lane strobe selects the lanes. + write_mask = calculate_byte_mask_for_store(operation, beat_offset) + write_data = replicate_store_data_for_beat(operation, source_register_2_value) cocotb.log.info( - f"op {operation} with rs2_val {source_register_2_value} storing data value of " - f"{write_data} to address {write_address} with wr_mask {write_mask}" + f"op {operation} with rs2_val {source_register_2_value} storing beat " + f"0x{write_data:016X} to address 0x{write_address:08X} " + f"with wr_mask 0b{write_mask:08b}" ) # Update expected queues state.memory_write_address_expected_queue.append(write_address) state.memory_write_data_expected_queue.append(write_data) - # Update memory model - write_address_word = write_address & MEMORY_WORD_ALIGN_MASK - for i in range(4): + # Update memory model: strobed lanes of the beat land at the + # containing aligned dword + write_address_beat = write_address & MEMORY_DWORD_ALIGN_MASK + for i in range(8): if (write_mask >> i) & 1: mem_model.write_byte( - write_address_word + i, (write_data >> (8 * i)) & 0xFF + write_address_beat + i, (write_data >> (8 * i)) & 0xFF ) diff --git a/verif/cocotb_tests/cpu_ooo/frontend/test_frontend_validity_tracker.py b/verif/cocotb_tests/cpu_ooo/frontend/test_frontend_validity_tracker.py index 3ec0fd8c..0dbc3aa1 100644 --- a/verif/cocotb_tests/cpu_ooo/frontend/test_frontend_validity_tracker.py +++ b/verif/cocotb_tests/cpu_ooo/frontend/test_frontend_validity_tracker.py @@ -20,11 +20,10 @@ import cocotb from cocotb.clock import Clock from cocotb.triggers import FallingEdge, RisingEdge, Timer +from config import FLEN, XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 -FLEN = 64 INSTR_OP_WIDTH = 32 BRANCH_OP_WIDTH = 3 STORE_OP_WIDTH = 2 diff --git a/verif/cocotb_tests/cpu_ooo/memory/test_data_mem_request_router.py b/verif/cocotb_tests/cpu_ooo/memory/test_data_mem_request_router.py index 028607f2..aa190b77 100644 --- a/verif/cocotb_tests/cpu_ooo/memory/test_data_mem_request_router.py +++ b/verif/cocotb_tests/cpu_ooo/memory/test_data_mem_request_router.py @@ -277,7 +277,9 @@ async def test_amo_write_bram_and_priority(dut: Any) -> None: dut.i_amo_mem_write_data.value = 0x77 await _settle() assert int(dut.o_amo_mem_write_done.value) == 1 - assert int(dut.o_data_mem_bram_byte_wr_en.value) == 0b1111 + # Word-lane strobe on the 64-bit beat: addr[2]=0 selects the low lanes + # (docs/rv64/m1_data_tier.md). + assert int(dut.o_data_mem_bram_byte_wr_en.value) == 0x0F # SQ arrives: AMO must defer. dut.i_sq_mem_write_en.value = 1 dut.i_sq_mem_write_addr.value = FAST_ADDR @@ -310,7 +312,9 @@ async def test_amo_cached_write_handshake(dut: Any) -> None: assert ( int(dut.o_data_mem_bram_byte_wr_en.value) == 0 ), "cached AMO must not hit BRAM" - assert int(dut.o_data_mem_cached_byte_wr_en.value) == 0b1111 + # Word-lane strobe on the 64-bit beat: CACHED_ADDR has addr[2]=1, so the + # AMO word occupies the high lanes (docs/rv64/m1_data_tier.md). + assert int(dut.o_data_mem_cached_byte_wr_en.value) == 0xF0 assert int(dut.o_data_mem_cached_wr_data.value) == 0xCAFEF00D assert int(dut.o_amo_mem_write_done.value) == 0, "no fast done for a cached AMO" await _advance_cycle(dut) diff --git a/verif/cocotb_tests/cpu_ooo/perf/test_perf_counter_aggregator.py b/verif/cocotb_tests/cpu_ooo/perf/test_perf_counter_aggregator.py index 5cbcd671..97581b91 100644 --- a/verif/cocotb_tests/cpu_ooo/perf/test_perf_counter_aggregator.py +++ b/verif/cocotb_tests/cpu_ooo/perf/test_perf_counter_aggregator.py @@ -20,11 +20,10 @@ import cocotb from cocotb.clock import Clock from cocotb.triggers import FallingEdge, RisingEdge, Timer +from config import FLEN, XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 -FLEN = 64 ROB_TAG_WIDTH = 5 CHECKPOINT_ID_WIDTH = 3 REG_ADDR_WIDTH = 5 diff --git a/verif/cocotb_tests/cpu_ooo/pipeline_control/test_ooo_pipeline_control.py b/verif/cocotb_tests/cpu_ooo/pipeline_control/test_ooo_pipeline_control.py index 4f73914e..76977126 100644 --- a/verif/cocotb_tests/cpu_ooo/pipeline_control/test_ooo_pipeline_control.py +++ b/verif/cocotb_tests/cpu_ooo/pipeline_control/test_ooo_pipeline_control.py @@ -20,11 +20,10 @@ import cocotb from cocotb.clock import Clock from cocotb.triggers import FallingEdge, RisingEdge, Timer +from config import FLEN, XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 -FLEN = 64 ROB_TAG_WIDTH = 5 CHECKPOINT_ID_WIDTH = 3 REG_ADDR_WIDTH = 5 diff --git a/verif/cocotb_tests/cpu_ooo/recovery/test_branch_resolution.py b/verif/cocotb_tests/cpu_ooo/recovery/test_branch_resolution.py index c2f5c209..086483a6 100644 --- a/verif/cocotb_tests/cpu_ooo/recovery/test_branch_resolution.py +++ b/verif/cocotb_tests/cpu_ooo/recovery/test_branch_resolution.py @@ -19,10 +19,9 @@ import cocotb from cocotb.triggers import Timer +from config import FLEN, XLEN -XLEN = 32 -FLEN = 64 INSTR_OP_WIDTH = 32 ROB_TAG_WIDTH = 5 CHECKPOINT_ID_WIDTH = 3 diff --git a/verif/cocotb_tests/cpu_ooo/recovery/test_early_misprediction_recovery.py b/verif/cocotb_tests/cpu_ooo/recovery/test_early_misprediction_recovery.py index dd0e2990..8b9c2053 100644 --- a/verif/cocotb_tests/cpu_ooo/recovery/test_early_misprediction_recovery.py +++ b/verif/cocotb_tests/cpu_ooo/recovery/test_early_misprediction_recovery.py @@ -20,11 +20,10 @@ import cocotb from cocotb.clock import Clock from cocotb.triggers import FallingEdge, RisingEdge, Timer +from config import FLEN, XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 -FLEN = 64 INSTR_OP_WIDTH = 32 ROB_TAG_WIDTH = 5 CHECKPOINT_ID_WIDTH = 3 diff --git a/verif/cocotb_tests/cpu_ooo/recovery/test_ex_comb_synthesizer.py b/verif/cocotb_tests/cpu_ooo/recovery/test_ex_comb_synthesizer.py index b063a5ee..4164f967 100644 --- a/verif/cocotb_tests/cpu_ooo/recovery/test_ex_comb_synthesizer.py +++ b/verif/cocotb_tests/cpu_ooo/recovery/test_ex_comb_synthesizer.py @@ -19,9 +19,9 @@ import cocotb from cocotb.triggers import Timer +from config import XLEN -XLEN = 32 ROB_TAG_WIDTH = 5 CHECKPOINT_ID_WIDTH = 3 RAS_PTR_BITS = 3 diff --git a/verif/cocotb_tests/cpu_ooo/recovery/test_misprediction_flush_controller.py b/verif/cocotb_tests/cpu_ooo/recovery/test_misprediction_flush_controller.py index 18433706..6a464319 100644 --- a/verif/cocotb_tests/cpu_ooo/recovery/test_misprediction_flush_controller.py +++ b/verif/cocotb_tests/cpu_ooo/recovery/test_misprediction_flush_controller.py @@ -22,10 +22,10 @@ from cocotb.triggers import FallingEdge, RisingEdge, Timer from cocotb_tests.tomasulo.reorder_buffer.reorder_buffer_interface import COMMIT_FIELDS +from config import XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 ROB_TAG_WIDTH = 5 CHECKPOINT_ID_WIDTH = 3 NUM_CHECKPOINTS = 8 diff --git a/verif/cocotb_tests/cpu_ooo/register_files/test_ooo_register_files.py b/verif/cocotb_tests/cpu_ooo/register_files/test_ooo_register_files.py index d0b7a66d..489e945d 100644 --- a/verif/cocotb_tests/cpu_ooo/register_files/test_ooo_register_files.py +++ b/verif/cocotb_tests/cpu_ooo/register_files/test_ooo_register_files.py @@ -20,11 +20,10 @@ import cocotb from cocotb.clock import Clock from cocotb.triggers import RisingEdge, Timer +from config import FLEN, XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 -FLEN = 64 INSTR_OP_WIDTH = 32 BRANCH_OP_WIDTH = 3 STORE_OP_WIDTH = 2 diff --git a/verif/cocotb_tests/id_stage/test_id_stage.py b/verif/cocotb_tests/id_stage/test_id_stage.py index 42c31164..d5066989 100644 --- a/verif/cocotb_tests/id_stage/test_id_stage.py +++ b/verif/cocotb_tests/id_stage/test_id_stage.py @@ -20,11 +20,10 @@ import cocotb from cocotb.clock import Clock from cocotb.triggers import FallingEdge, RisingEdge, Timer +from config import FLEN, XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 -FLEN = 64 RAS_PTR_BITS = 3 BP_DIR_IDX_BITS = 10 INSTR_OP_WIDTH = 32 diff --git a/verif/cocotb_tests/if_stage/test_if_stage.py b/verif/cocotb_tests/if_stage/test_if_stage.py index a4ffb65d..5502a83a 100644 --- a/verif/cocotb_tests/if_stage/test_if_stage.py +++ b/verif/cocotb_tests/if_stage/test_if_stage.py @@ -20,10 +20,10 @@ import cocotb from cocotb.clock import Clock from cocotb.triggers import FallingEdge, RisingEdge, Timer +from config import XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 RAS_PTR_BITS = 3 BP_DIR_IDX_BITS = 10 NOP_INSTR = 0x00000013 diff --git a/verif/cocotb_tests/pd_stage/test_pd_stage.py b/verif/cocotb_tests/pd_stage/test_pd_stage.py index 2271c64e..9d363fc7 100644 --- a/verif/cocotb_tests/pd_stage/test_pd_stage.py +++ b/verif/cocotb_tests/pd_stage/test_pd_stage.py @@ -20,10 +20,10 @@ import cocotb from cocotb.clock import Clock from cocotb.triggers import FallingEdge, RisingEdge, Timer +from config import XLEN CLOCK_PERIOD_NS = 10 -XLEN = 32 RAS_PTR_BITS = 3 BP_DIR_IDX_BITS = 10 diff --git a/verif/cocotb_tests/test_directed_atomics.py b/verif/cocotb_tests/test_directed_atomics.py index 2e766488..7eab838f 100644 --- a/verif/cocotb_tests/test_directed_atomics.py +++ b/verif/cocotb_tests/test_directed_atomics.py @@ -69,6 +69,7 @@ from config import MASK32 from models.memory_model import MemoryModel +from utils.memory_utils import replicate_store_data_for_beat from cocotb_tests.test_helpers import DUTInterface from cocotb_tests.test_state import TestState from cocotb_tests.test_common import ( @@ -166,10 +167,12 @@ async def execute_lr_sc_instruction( writeback_value = 0 if success else 1 if success: - # Model memory write + # Model memory write (word data rides the beat replicated) write_data = state.register_file_previous[rs2] state.memory_write_address_expected_queue.append(address) - state.memory_write_data_expected_queue.append(write_data) + state.memory_write_data_expected_queue.append( + replicate_store_data_for_beat("sw", write_data) + ) mem_model.write_word(address, write_data) cocotb.log.info( f"SC.W x{rd}, x{rs2}, (x{rs1}): addr=0x{address:08X}, " @@ -240,9 +243,11 @@ async def execute_store( address = (state.register_file_previous[rs1] + imm) & MASK32 write_data = state.register_file_previous[rs2] & MASK32 - # Queue expected memory write + # Queue expected memory write (word data rides the beat replicated) state.memory_write_address_expected_queue.append(address) - state.memory_write_data_expected_queue.append(write_data) + state.memory_write_data_expected_queue.append( + replicate_store_data_for_beat("sw", write_data) + ) # Update software memory model mem_model.write_word(address, write_data) @@ -375,17 +380,18 @@ async def run_directed_lr_sc_test(dut: Any, config: TestConfig | None = None) -> cocotb.log.info("=== Waiting for stores to complete ===") await wait_for_memory_writes(dut_if, state, "init stores to reach memory") - # Debug: Check what's in the DUT's memory after stores - word_addr_1 = test_address_1 >> 2 # Convert byte address to word address - word_addr_2 = test_address_2 >> 2 + # Debug: Check what's in the DUT's memory after stores (the simulation + # data BRAM stores 64-bit dword rows; the helper extracts the word lane) + from models.memory_model import peek_dut_memory_word + try: - mem_val_1 = int(dut.data_memory_for_simulation.memory[word_addr_1].value) - mem_val_2 = int(dut.data_memory_for_simulation.memory[word_addr_2].value) + mem_val_1 = peek_dut_memory_word(dut, test_address_1) + mem_val_2 = peek_dut_memory_word(dut, test_address_2) cocotb.log.info( - f"DEBUG: DUT memory[{word_addr_1}] (addr 0x{test_address_1:08X}) = 0x{mem_val_1:08X}" + f"DEBUG: DUT memory word at 0x{test_address_1:08X} = 0x{mem_val_1:08X}" ) cocotb.log.info( - f"DEBUG: DUT memory[{word_addr_2}] (addr 0x{test_address_2:08X}) = 0x{mem_val_2:08X}" + f"DEBUG: DUT memory word at 0x{test_address_2:08X} = 0x{mem_val_2:08X}" ) except Exception as e: cocotb.log.warning(f"DEBUG: Could not read DUT memory: {e}") diff --git a/verif/cocotb_tests/test_directed_multicycle.py b/verif/cocotb_tests/test_directed_multicycle.py index 4b9f03b6..b9f55e11 100644 --- a/verif/cocotb_tests/test_directed_multicycle.py +++ b/verif/cocotb_tests/test_directed_multicycle.py @@ -452,20 +452,17 @@ async def test_fld_faddd_load_use_hazard(dut: Any) -> None: from models.fp_model import fadd_d # Initialize memory with a known double at an 8-byte aligned address + # (the simulation data BRAM stores aligned 64-bit dword rows, so the + # double lands in a single row). + from models.memory_model import poke_dut_memory_dword + base_addr = 0x100 load_bits = 0x3FF0000000000000 # 1.0 double - low_word = load_bits & MASK32 - high_word = (load_bits >> 32) & MASK32 - word_index = base_addr >> 2 - dut.data_memory_for_simulation.memory[word_index].value = low_word - dut.data_memory_for_simulation.memory[word_index + 1].value = high_word - mem_model.write_word(base_addr, low_word) - mem_model.write_word(base_addr + 4, high_word) + poke_dut_memory_dword(dut, base_addr, load_bits) + mem_model.write_dword(base_addr, load_bits) - cocotb.log.info( - f"Memory init @0x{base_addr:08X}: low=0x{low_word:08X}, high=0x{high_word:08X}" - ) + cocotb.log.info(f"Memory init @0x{base_addr:08X}: 0x{load_bits:016X}") # x1 = base_addr (first instruction after warmup) instr = enc_addi(1, 0, base_addr) @@ -543,8 +540,10 @@ async def test_lh_bext_load_use_hazard(dut: Any) -> None: value_b = 0x00000000 bit_index = 1 - dut.data_memory_for_simulation.memory[addr_a >> 2].value = value_a - dut.data_memory_for_simulation.memory[addr_b >> 2].value = value_b + from models.memory_model import poke_dut_memory_word + + poke_dut_memory_word(dut, addr_a, value_a) + poke_dut_memory_word(dut, addr_b, value_b) mem_model.write_word(addr_a, value_a) mem_model.write_word(addr_b, value_b) diff --git a/verif/cocotb_tests/test_directed_traps.py b/verif/cocotb_tests/test_directed_traps.py index cce7f2e5..e0f11b37 100644 --- a/verif/cocotb_tests/test_directed_traps.py +++ b/verif/cocotb_tests/test_directed_traps.py @@ -1403,7 +1403,6 @@ async def run_directed_interrupt_commit_race_test( post_trap = 8 # cycles to keep observing after o_trap_taken fire_lo, fire_hi = 0, 40 mem_base = 0x400 # byte base of the load region (x4); BRAM, non-cached - word_base = mem_base >> 2 enc_addi = I_ALU["addi"][0] enc_slli = I_ALU["slli"][0] @@ -1480,11 +1479,17 @@ async def setup_phase() -> int: await feed(nop) # Preload the load region with this generation's expected values (the # data BRAM persists across reset, so refresh it every iteration). + # Whole dword rows per deposit: word-granule RMW pokes to the same + # row within one delta would lose the first word (queued deposits). if mode == "load": - for i in range(n_stream): - dut.data_memory_for_simulation.memory[ - word_base + i - ].value = expected_val(i, gen) + from models.memory_model import poke_dut_memory_dword + + for i in range(0, n_stream, 2): + low_word = expected_val(i, gen) + high_word = expected_val(i + 1, gen) if i + 1 < n_stream else 0 + poke_dut_memory_dword( + dut, mem_base + 4 * i, (high_word << 32) | low_word + ) # Construct CSR operands (no deposits needed): x1=mtvec(0x1000), # x2=mie.MTIE(0x80), x3=mstatus.MIE(0x08), x4=load base. await feed(enc_addi(1, 0, 1)) # x1 = 1 diff --git a/verif/cocotb_tests/tomasulo/dispatch/dispatch_interface.py b/verif/cocotb_tests/tomasulo/dispatch/dispatch_interface.py index e5aba349..c7488bba 100644 --- a/verif/cocotb_tests/tomasulo/dispatch/dispatch_interface.py +++ b/verif/cocotb_tests/tomasulo/dispatch/dispatch_interface.py @@ -24,13 +24,12 @@ from typing import Any from cocotb.triggers import RisingEdge, FallingEdge +from config import FLEN, XLEN # ============================================================================= # Width constants from riscv_pkg # ============================================================================= ROB_TAG_WIDTH = 5 -XLEN = 32 -FLEN = 64 REG_ADDR_WIDTH = 5 CHECKPOINT_ID_WIDTH = 3 RAS_PTR_BITS = 3 diff --git a/verif/cocotb_tests/tomasulo/fu_shims/fp_add_shim_interface.py b/verif/cocotb_tests/tomasulo/fu_shims/fp_add_shim_interface.py index 34d51a53..513a8409 100644 --- a/verif/cocotb_tests/tomasulo/fu_shims/fp_add_shim_interface.py +++ b/verif/cocotb_tests/tomasulo/fu_shims/fp_add_shim_interface.py @@ -23,13 +23,12 @@ from typing import Any from cocotb.triggers import FallingEdge, RisingEdge +from config import FLEN, XLEN # ============================================================================= # Width constants from riscv_pkg # ============================================================================= ROB_TAG_WIDTH = 5 -XLEN = 32 -FLEN = 64 MASK_TAG = (1 << ROB_TAG_WIDTH) - 1 # 0x1F MASK32 = (1 << XLEN) - 1 diff --git a/verif/cocotb_tests/tomasulo/load_queue/lq_interface.py b/verif/cocotb_tests/tomasulo/load_queue/lq_interface.py index 14a468b3..d512428a 100644 --- a/verif/cocotb_tests/tomasulo/load_queue/lq_interface.py +++ b/verif/cocotb_tests/tomasulo/load_queue/lq_interface.py @@ -24,11 +24,10 @@ from cocotb.triggers import FallingEdge, RisingEdge from .lq_model import FuComplete +from config import FLEN, XLEN # Width constants from riscv_pkg ROB_TAG_WIDTH = 5 -XLEN = 32 -FLEN = 64 MASK_TAG = (1 << ROB_TAG_WIDTH) - 1 MASK32 = (1 << XLEN) - 1 @@ -345,9 +344,20 @@ def read_sq_check(self) -> dict: # Memory Interface # ========================================================================= - def drive_mem_response(self, data: int) -> None: - """Drive memory read response.""" - self.dut.i_mem_read_data.value = data & MASK32 + def drive_mem_response(self, data: int, *, dword: bool = False) -> None: + """Drive a memory read response beat. + + The data tier returns aligned 64-bit beats (docs/rv64/m1_data_tier.md). + For a full-beat (FLD) response pass ``dword=True`` with the 64-bit + value. Otherwise ``data`` is a 32-bit word: it is replicated into + both word lanes so the response is correct at either ``addr[2]``, + mirroring how word data is positioned on the store side. + """ + if dword: + self.dut.i_mem_read_data.value = data & MASK64 + else: + word = data & MASK32 + self.dut.i_mem_read_data.value = (word << 32) | word self.dut.i_mem_read_valid.value = 1 def clear_mem_response(self) -> None: diff --git a/verif/cocotb_tests/tomasulo/load_queue/lq_model.py b/verif/cocotb_tests/tomasulo/load_queue/lq_model.py index 2dd0aa03..5d78b1eb 100644 --- a/verif/cocotb_tests/tomasulo/load_queue/lq_model.py +++ b/verif/cocotb_tests/tomasulo/load_queue/lq_model.py @@ -19,11 +19,10 @@ """ from dataclasses import dataclass +from config import FLEN, XLEN # Width constants from riscv_pkg ROB_TAG_WIDTH = 5 -XLEN = 32 -FLEN = 64 LQ_DEPTH = 8 MASK_TAG = (1 << ROB_TAG_WIDTH) - 1 @@ -49,7 +48,6 @@ class LQEntry: size: int = MEM_SIZE_WORD sign_ext: bool = False is_mmio: bool = False - fp64_phase: int = 0 issued: bool = False data_valid: bool = False data: int = 0 @@ -98,18 +96,24 @@ def sign_extend_half(val: int, unsigned: bool) -> int: def load_unit_model(size: int, sign_ext: bool, address: int, raw_data: int) -> int: - """Model the load_unit: extract byte/half and sign extend.""" - raw_data = raw_data & MASK32 + """Model the load_unit: extract from the 64-bit beat and sign extend. + + The data tier returns the aligned dword at addr[31:3]; the load unit + selects the addressed byte/half/word by addr[2:0] + (docs/rv64/m1_data_tier.md). + """ + raw_data = raw_data & MASK64 if size == MEM_SIZE_BYTE: - byte_sel = address & 0x3 + byte_sel = address & 0x7 byte_val = (raw_data >> (byte_sel * 8)) & 0xFF return sign_extend_byte(byte_val, not sign_ext) & MASK32 elif size == MEM_SIZE_HALF: - half_sel = (address >> 1) & 0x1 + half_sel = (address >> 1) & 0x3 half_val = (raw_data >> (half_sel * 16)) & 0xFFFF return sign_extend_half(half_val, not sign_ext) & MASK32 else: - return raw_data & MASK32 + word_sel = (address >> 2) & 0x1 + return (raw_data >> (word_sel * 32)) & MASK32 def is_younger(entry_tag: int, flush_tag: int, head: int) -> bool: @@ -212,7 +216,6 @@ def alloc( e.size = size e.sign_ext = sign_ext e.is_mmio = False - e.fp64_phase = 0 e.issued = False e.data_valid = False e.data = 0 @@ -313,11 +316,10 @@ def cache_hit_complete(self) -> None: e = self.entries[mem_idx] - # Mirror load_queue.sv cache_hit_fast_path gating. + # Mirror load_queue.sv cache_hit_fast_path gating (every size is + # L0-eligible on the dword-line cache, including FLD). if e.is_mmio: return - if e.is_fp and e.size == MEM_SIZE_DOUBLE: - return if e.is_lr or e.is_amo: return @@ -335,27 +337,24 @@ def issue_to_memory( if not can_issue: return None - addr = e.address - if e.is_fp and e.size == MEM_SIZE_DOUBLE and e.fp64_phase: - addr = (addr + 4) & MASK32 - e.issued = True self.mem_outstanding = True self.issued_idx = mem_idx - return {"addr": addr, "size": e.size} + return {"addr": e.address, "size": e.size} def mem_response(self, data: int) -> None: - """Handle memory response.""" + """Handle a memory response beat (aligned 64-bit dword).""" if not self.mem_outstanding: return idx = self.issued_idx e = self.entries[idx] - data = data & MASK32 + data = data & MASK64 if e.is_amo: - # AMO: latch old value, start write phase - self.amo_old_value = data + # AMO: latch the addressed word as old value, start write phase + word_sel = (e.address >> 2) & 0x1 + self.amo_old_value = (data >> (word_sel * 32)) & MASK32 self.amo_entry_idx = idx self.amo_state = 1 # WRITE_ACTIVE self.mem_outstanding = False @@ -367,20 +366,13 @@ def mem_response(self, data: int) -> None: self.mem_outstanding = False self.reservation_valid = True self.reservation_addr = e.address - elif e.is_fp and e.size == MEM_SIZE_DOUBLE and not e.fp64_phase: - # FLD phase 0: store low word through load unit, advance to phase 1 - processed = load_unit_model(MEM_SIZE_WORD, False, e.address, data) - e.data = (e.data & ~MASK32) | (processed & MASK32) - e.fp64_phase = 1 - e.issued = False - self.mem_outstanding = False - elif e.is_fp and e.size == MEM_SIZE_DOUBLE and e.fp64_phase: - # FLD phase 1: store high word raw - e.data = (e.data & MASK32) | ((data & MASK32) << 32) + elif e.size == MEM_SIZE_DOUBLE: + # FLD (RV64 LD in M3): the full beat in one response + e.data = data e.data_valid = True self.mem_outstanding = False else: - # Single-phase: run through load unit + # Sub-beat: run through the load unit (word/half/byte extract) processed = load_unit_model(e.size, e.sign_ext, e.address, data) e.data = processed & MASK64 e.data_valid = True diff --git a/verif/cocotb_tests/tomasulo/load_queue/test_load_queue.py b/verif/cocotb_tests/tomasulo/load_queue/test_load_queue.py index f7bea445..88956c51 100644 --- a/verif/cocotb_tests/tomasulo/load_queue/test_load_queue.py +++ b/verif/cocotb_tests/tomasulo/load_queue/test_load_queue.py @@ -15,9 +15,13 @@ """Unit tests for the Load Queue. Tests cover reset, allocation, address update, full load flows (LW, LB, LBU, -LH, LHU), SQ forwarding, SQ disambiguation stall, MMIO ordering, FLD two-phase -and slot reuse, FLW NaN-boxing, flush, AMO dependency ordering, CDB -back-pressure, and constrained random. +LH, LHU), SQ forwarding, SQ disambiguation stall, MMIO ordering, single-beat +FLD, FLW NaN-boxing, flush, AMO dependency ordering, CDB back-pressure, and +constrained random. + +Bus contract (docs/rv64/m1_data_tier.md): memory responses are aligned +64-bit beats; the LQ extracts by addr[2:0]. drive_mem_response replicates a +word across both beat lanes (correct at either addr[2]) unless dword=True. """ import random @@ -41,6 +45,14 @@ CLOCK_PERIOD_NS = 10 LQ_DEPTH = 8 + + +def wbeat(word: int) -> int: + """Word write data replicated across the 64-bit beat ({2{word}}).""" + word &= MASK32 + return (word << 32) | word + + AMO_RESCUE_THRESHOLD = 16384 @@ -152,9 +164,12 @@ async def complete_prepared_amo( assert ( amo_write["addr"] == address ), f"{description}: expected write address 0x{address:08x}, got 0x{amo_write['addr']:08x}" - assert amo_write["data"] == (expected_write & MASK32), ( - f"{description}: expected write data 0x{expected_write & MASK32:08x}, " - f"got 0x{amo_write['data']:08x}" + # AMO write data rides the beat replicated ({2{result}}); the router's + # word strobes select the addressed half. + expected_beat = ((expected_write & MASK32) << 32) | (expected_write & MASK32) + assert amo_write["data"] == expected_beat, ( + f"{description}: expected write beat 0x{expected_beat:016x}, " + f"got 0x{amo_write['data']:016x}" ) dut_if.drive_amo_mem_write_done(True) @@ -334,13 +349,16 @@ async def test_alloc_slot1_slot2_pair_completes_in_order(dut: Any) -> None: assert dut_if.count == 2, f"Expected two allocated entries, got {dut_if.count}" + # Distinct dwords: with dword-granule L0 lines, a same-dword pair would + # let the second load hit the line filled by the first response instead + # of exercising the ordered memory-issue path this test locks. dut_if.drive_addr_update(rob_tag=10, address=0x1100) model.addr_update(10, 0x1100) await dut_if.step() dut_if.clear_addr_update() - dut_if.drive_addr_update(rob_tag=11, address=0x1104) - model.addr_update(11, 0x1104) + dut_if.drive_addr_update(rob_tag=11, address=0x1108) + model.addr_update(11, 0x1108) await dut_if.step() dut_if.clear_addr_update() @@ -606,47 +624,33 @@ async def test_mmio_load(dut: Any) -> None: # ============================================================================ -# Test 14: FLD two-phase +# Test 14: FLD single beat # ============================================================================ @cocotb.test() -async def test_fld_two_phase(dut: Any) -> None: - """FLD: two memory reads (addr, addr+4), 64-bit CDB broadcast.""" +async def test_fld_single_beat(dut: Any) -> None: + """FLD: one memory read returning the full beat, 64-bit CDB broadcast.""" dut_if, model = await setup(dut) await alloc_and_addr( dut_if, model, rob_tag=14, address=0x6000, is_fp=True, size=MEM_SIZE_DOUBLE ) - # Phase 0: memory read at addr + # Single issue: memory read at addr dut_if.drive_sq_all_older_known(True) dut_if.drive_sq_forward(match=False, can_forward=False) mem_req = await wait_for_mem_request(dut_if) - assert mem_req["en"], "Phase 0 should issue" + assert mem_req["en"], "FLD should issue" assert ( mem_req["addr"] == 0x6000 - ), f"Phase 0 addr should be 0x6000, got 0x{mem_req['addr']:x}" + ), f"FLD addr should be 0x6000, got 0x{mem_req['addr']:x}" await dut_if.step() - # Phase 0 response: low word - dut_if.drive_mem_response(0xAAAA_BBBB) - model.mem_response(0xAAAA_BBBB) - await dut_if.step() - dut_if.clear_mem_response() - - # Phase 1: should re-issue at addr+4 - mem_req = await wait_for_mem_request(dut_if) - assert mem_req["en"], "Phase 1 should issue" - assert ( - mem_req["addr"] == 0x6004 - ), f"Phase 1 addr should be 0x6004, got 0x{mem_req['addr']:x}" - - await dut_if.step() - - # Phase 1 response: high word - dut_if.drive_mem_response(0xCCCC_DDDD) - model.mem_response(0xCCCC_DDDD) + # One response carries the whole aligned dword + fld_beat = 0xCCCC_DDDD_AAAA_BBBB + dut_if.drive_mem_response(fld_beat, dword=True) + model.mem_response(fld_beat) await dut_if.step() dut_if.clear_mem_response() @@ -654,84 +658,12 @@ async def test_fld_two_phase(dut: Any) -> None: result = await wait_for_fu_complete(dut_if) assert result.valid, "CDB should be valid after FLD" assert result.tag == 14 - expected = (0xCCCC_DDDD << 32) | 0xAAAA_BBBB assert ( - result.value == expected - ), f"Expected 0x{expected:016x}, got 0x{result.value:016x}" + result.value == fld_beat + ), f"Expected 0x{fld_beat:016x}, got 0x{result.value:016x}" await accept_fu_complete(dut_if) -# ============================================================================ -# Test 14a: FLD phase initialization on physical-slot reuse -# ============================================================================ -@cocotb.test() -async def test_fld_phase_clears_on_physical_slot_reuse(dut: Any) -> None: - """A reused FLD slot starts at phase 0 on its earliest address update.""" - dut_if, _ = await setup(dut) - - # Complete an FLD in physical entry 0, leaving its unreset payload phase bit - # at phase 1 after the entry itself is freed. - dut_if.drive_alloc(rob_tag=0, is_fp=True, size=MEM_SIZE_DOUBLE) - await dut_if.step() - dut_if.clear_alloc() - dut_if.drive_addr_update(rob_tag=0, address=0x6000) - await dut_if.step() - dut_if.clear_addr_update() - dut_if.drive_sq_all_older_known(True) - dut_if.drive_sq_forward(match=False, can_forward=False) - - mem_req = await wait_for_mem_request(dut_if) - assert mem_req["en"] and mem_req["addr"] == 0x6000 - await dut_if.step() - dut_if.drive_mem_response(0x1111_2222) - await dut_if.step() - dut_if.clear_mem_response() - - mem_req = await wait_for_mem_request(dut_if) - assert mem_req["en"] and mem_req["addr"] == 0x6004 - await dut_if.step() - dut_if.drive_mem_response(0x3333_4444) - await dut_if.step() - dut_if.clear_mem_response() - result = await wait_for_fu_complete(dut_if) - assert result.valid and result.tag == 0 - await accept_fu_complete(dut_if) - assert dut_if.empty - - # Occupy entries 1..7 so the next allocation must reuse physical entry 0. - for rob_tag in range(1, LQ_DEPTH): - dut_if.drive_alloc(rob_tag=rob_tag, size=MEM_SIZE_WORD) - await dut_if.step() - dut_if.clear_alloc() - assert dut_if.count == LQ_DEPTH - 1 - - dut_if.drive_alloc(rob_tag=LQ_DEPTH, is_fp=True, size=MEM_SIZE_DOUBLE) - await dut_if.step() - dut_if.clear_alloc() - assert dut_if.full - - # Present the production MEM-RS look-ahead, then block SQ-check capture on - # the address-update edge. The later stored-entry scan must therefore read - # the resident phase bit cleared by the physical-generation pulse, rather - # than getting phase zero from the current-update bypass. - new_address = 0x7000 - dut_if.drive_pre_issue(rob_tag=LQ_DEPTH) - await dut_if.step() - dut_if.clear_pre_issue() - dut_if.drive_mem_bus_busy(True) - dut_if.drive_addr_update(rob_tag=LQ_DEPTH, address=new_address) - await dut_if.step() - dut_if.clear_addr_update() - dut_if.drive_mem_bus_busy(False) - - mem_req = await wait_for_mem_request(dut_if, max_cycles=8) - assert mem_req["en"], "Reused FLD did not issue" - assert mem_req["addr"] == new_address, ( - f"Reused FLD started at stale phase-1 address 0x{mem_req['addr']:x}, " - f"expected 0x{new_address:x}" - ) - - # ============================================================================ # Test 15: FLW NaN-boxing # ============================================================================ @@ -1734,43 +1666,28 @@ async def test_cache_mmio_bypass(dut: Any) -> None: # ============================================================================ @cocotb.test() async def test_fld_cache_fill_both_words(dut: Any) -> None: - """FLD fills both L0 words; later LW loads complete correctly in either mode. - - Regression test: before the fix, FLD phase 1 filled the cache at the base - address instead of addr+4, poisoning the entry for the base address. - """ + """FLD fills its dword L0 line; later LW loads hit either word of it.""" dut_if, model = await setup(dut) base_addr = 0x2000 low_word = 0xAAAA_BBBB high_word = 0xCCCC_DDDD + fld_beat = (high_word << 32) | low_word - # -- FLD at base_addr: two-phase memory completion -- + # -- FLD at base_addr: single-beat memory completion fills the line -- await alloc_and_addr( dut_if, model, rob_tag=1, address=base_addr, is_fp=True, size=MEM_SIZE_DOUBLE ) - # Phase 0: memory read at base_addr dut_if.drive_sq_all_older_known(True) dut_if.drive_sq_forward(match=False, can_forward=False) mem_req = await wait_for_mem_request(dut_if) - assert mem_req["en"], "FLD phase 0 should issue" + assert mem_req["en"], "FLD should issue" assert mem_req["addr"] == base_addr await dut_if.step() - dut_if.drive_mem_response(low_word) - model.mem_response(low_word) - await dut_if.step() - dut_if.clear_mem_response() - - # Phase 1: memory read at base_addr + 4 - mem_req = await wait_for_mem_request(dut_if) - assert mem_req["en"], "FLD phase 1 should issue" - assert mem_req["addr"] == base_addr + 4 - await dut_if.step() - - dut_if.drive_mem_response(high_word) - model.mem_response(high_word) + dut_if.drive_mem_response(fld_beat, dword=True) + model.mem_response(fld_beat) await dut_if.step() dut_if.clear_mem_response() @@ -1794,7 +1711,7 @@ async def test_fld_cache_fill_both_words(dut: Any) -> None: assert result.tag == 2 assert result.value == low_word, ( f"LW at base_addr: expected 0x{low_word:08x}, got 0x{result.value:08x} " - "(cache poisoned by FLD phase 1?)" + "(dword L0 line served the wrong word?)" ) # -- LW at base_addr + 4: should hit L0 cache with high_word -- @@ -2501,7 +2418,7 @@ async def test_amo_swap(dut: Any) -> None: amo_write = dut_if.read_amo_mem_write() assert amo_write["en"], "AMO write should be active" assert amo_write["addr"] == 0x7000, f"AMO write addr: {amo_write['addr']:#x}" - assert amo_write["data"] == rs2_val, f"AMOSWAP write: {amo_write['data']:#x}" + assert amo_write["data"] == wbeat(rs2_val), f"AMOSWAP write: {amo_write['data']:#x}" # Acknowledge AMO write dut_if.drive_amo_mem_write_done(True) @@ -2564,8 +2481,8 @@ async def test_amo_add(dut: Any) -> None: assert amo_write["en"], "AMO write should be active" expected_write = (old_val + rs2_val) & MASK32 assert ( - amo_write["data"] == expected_write - ), f"AMOADD should write {expected_write}, got {amo_write['data']}" + amo_write["data"] == wbeat(expected_write) + ), f"AMOADD should write beat {wbeat(expected_write):#x}, got {amo_write['data']:#x}" # Acknowledge dut_if.drive_amo_mem_write_done(True) @@ -2629,11 +2546,13 @@ async def test_slot2_only_amo_uses_compact_staged_kind(dut: Any) -> None: amo_write = dut_if.read_amo_mem_write() assert amo_write["en"], "Slot-2-only AMO write should be active" assert amo_write["addr"] == 0x8080 - assert amo_write["data"] == old_val + rs2_val, ( - f"Expected compact AMOADD result {old_val + rs2_val}, " - f"got {amo_write['data']}" + assert amo_write["data"] == wbeat(old_val + rs2_val), ( + f"Expected compact AMOADD result beat {wbeat(old_val + rs2_val):#x}, " + f"got {amo_write['data']:#x}" ) - assert amo_write["data"] != rs2_val, "AMO operation unexpectedly decoded as AMOSWAP" + assert amo_write["data"] != wbeat( + rs2_val + ), "AMO operation unexpectedly decoded as AMOSWAP" dut_if.drive_amo_mem_write_done(True) model.amo_write_done() diff --git a/verif/cocotb_tests/tomasulo/register_alias_table/rat_model.py b/verif/cocotb_tests/tomasulo/register_alias_table/rat_model.py index 0cd21c9c..cfee63d4 100644 --- a/verif/cocotb_tests/tomasulo/register_alias_table/rat_model.py +++ b/verif/cocotb_tests/tomasulo/register_alias_table/rat_model.py @@ -35,8 +35,6 @@ ROB_TAG_WIDTH = 5 NUM_CHECKPOINTS = 8 CHECKPOINT_ID_WIDTH = 3 -XLEN = 32 -FLEN = 64 RAS_PTR_BITS = 3 MASK32 = (1 << 32) - 1 diff --git a/verif/cocotb_tests/tomasulo/reorder_buffer/reorder_buffer_model.py b/verif/cocotb_tests/tomasulo/reorder_buffer/reorder_buffer_model.py index 3b3a7c5b..53b9573d 100644 --- a/verif/cocotb_tests/tomasulo/reorder_buffer/reorder_buffer_model.py +++ b/verif/cocotb_tests/tomasulo/reorder_buffer/reorder_buffer_model.py @@ -33,8 +33,6 @@ # Match RTL parameters REORDER_BUFFER_DEPTH = 32 -XLEN = 32 -FLEN = 64 MASK32 = (1 << 32) - 1 MASK64 = (1 << 64) - 1 diff --git a/verif/cocotb_tests/tomasulo/reservation_station/rs_interface.py b/verif/cocotb_tests/tomasulo/reservation_station/rs_interface.py index b0c73e9c..2735bc41 100644 --- a/verif/cocotb_tests/tomasulo/reservation_station/rs_interface.py +++ b/verif/cocotb_tests/tomasulo/reservation_station/rs_interface.py @@ -27,11 +27,10 @@ from typing import Any from cocotb.triggers import RisingEdge, FallingEdge +from config import FLEN, XLEN # Width constants from riscv_pkg ROB_TAG_WIDTH = 5 -XLEN = 32 -FLEN = 64 MASK_TAG = (1 << ROB_TAG_WIDTH) - 1 # 0x1F MASK32 = (1 << XLEN) - 1 diff --git a/verif/cocotb_tests/tomasulo/store_queue/sq_interface.py b/verif/cocotb_tests/tomasulo/store_queue/sq_interface.py index 8acf2eb0..87f93498 100644 --- a/verif/cocotb_tests/tomasulo/store_queue/sq_interface.py +++ b/verif/cocotb_tests/tomasulo/store_queue/sq_interface.py @@ -24,11 +24,10 @@ from cocotb.triggers import FallingEdge, RisingEdge from .sq_model import ForwardResult, MemWriteReq +from config import FLEN, XLEN # Width constants from riscv_pkg ROB_TAG_WIDTH = 5 -XLEN = 32 -FLEN = 64 MASK_TAG = (1 << ROB_TAG_WIDTH) - 1 MASK32 = (1 << XLEN) - 1 @@ -386,6 +385,7 @@ def read_cache_invalidate(self) -> dict: return { "valid": bool(self.dut.o_cache_invalidate_valid.value), "addr": int(self.dut.o_cache_invalidate_addr.value), + "is_dword": bool(self.dut.o_cache_invalidate_is_dword.value), } # ========================================================================= diff --git a/verif/cocotb_tests/tomasulo/store_queue/sq_model.py b/verif/cocotb_tests/tomasulo/store_queue/sq_model.py index 2d8ff83e..6045fdfa 100644 --- a/verif/cocotb_tests/tomasulo/store_queue/sq_model.py +++ b/verif/cocotb_tests/tomasulo/store_queue/sq_model.py @@ -14,16 +14,16 @@ """Golden model for the Store Queue. -Mirrors the RTL circular buffer, entry state machine, commit-ordered -memory writes, and FSD two-phase writes. +Mirrors the RTL circular buffer, entry state machine, and commit-ordered +single-beat memory writes (every size drains in one 64-bit beat; +docs/rv64/m1_data_tier.md). """ from dataclasses import dataclass +from config import FLEN, XLEN # Width constants from riscv_pkg ROB_TAG_WIDTH = 5 -XLEN = 32 -FLEN = 64 SQ_DEPTH = 8 MASK_TAG = (1 << ROB_TAG_WIDTH) - 1 @@ -50,7 +50,6 @@ class SQEntry: data: int = 0 size: int = MEM_SIZE_WORD is_mmio: bool = False - fp64_phase: int = 0 committed: bool = False sent: bool = False is_sc: bool = False @@ -158,7 +157,6 @@ def alloc( e.data = 0 e.size = size e.is_mmio = is_mmio - e.fp64_phase = 0 e.committed = False e.sent = False e.is_sc = is_sc @@ -203,19 +201,13 @@ def mem_write_initiate(self) -> None: self.write_outstanding = True def mem_write_done(self) -> None: - """Handle memory write completion.""" + """Handle memory write completion (single-beat: every done frees).""" if not self.write_outstanding: return e = self.entries[self.head_idx] - if e.size == MEM_SIZE_DOUBLE and not e.fp64_phase: - # FSD phase 0 → advance to phase 1 - e.fp64_phase = 1 - self.write_outstanding = False - else: - # Complete: free entry - e.valid = False - e.sent = True - self.write_outstanding = False + e.valid = False + e.sent = True + self.write_outstanding = False def advance_head(self) -> None: """Advance head pointer past freed entries (collapse to tail when empty).""" diff --git a/verif/cocotb_tests/tomasulo/store_queue/test_store_queue.py b/verif/cocotb_tests/tomasulo/store_queue/test_store_queue.py index 2d50306e..bfad077a 100644 --- a/verif/cocotb_tests/tomasulo/store_queue/test_store_queue.py +++ b/verif/cocotb_tests/tomasulo/store_queue/test_store_queue.py @@ -15,9 +15,14 @@ """Unit tests for the Store Queue. Tests cover reset, allocation, address/data update, commit + memory write -(SW/SH/SB), FSD two-phase commit, FSW, store-to-load forwarding, forwarding -stall, registered forwarding-metadata stability, MMIO stores, partial/full -flush, live-count event overlap, and constrained random. +(SW/SH/SB), single-beat FSD commit, FSW, store-to-load forwarding, +forwarding stall, registered forwarding-metadata stability, MMIO stores, +partial/full flush, live-count event overlap, and constrained random. + +Bus contract (docs/rv64/m1_data_tier.md): drains carry aligned 64-bit +beats with 8-lane strobes; sub-beat store data is replicated across the +beat and the strobe selects the addressed lanes. Forwarding delivers the +aligned-dword memory image (store data shifted to its beat byte lanes). """ import random @@ -42,6 +47,24 @@ SQ_DEPTH = 8 +def wbeat(word: int) -> int: + """Word store data replicated across the 64-bit beat ({2{word}}).""" + word &= 0xFFFF_FFFF + return (word << 32) | word + + +def hbeat(half: int) -> int: + """Halfword store data replicated across the beat ({4{half}}).""" + half &= 0xFFFF + return half | half << 16 | half << 32 | half << 48 + + +def bbeat(byte: int) -> int: + """Byte store data replicated across the beat ({8{byte}}).""" + byte &= 0xFF + return int.from_bytes(bytes([byte]) * 8, "little") + + async def setup(dut: Any) -> tuple[SQInterface, SQModel]: """Start clock, reset DUT, and return interface and model.""" cocotb.start_soon(Clock(dut.i_clk, CLOCK_PERIOD_NS, unit="ns").start()) @@ -241,7 +264,7 @@ async def test_alloc_with_initial_address(dut: Any) -> None: write_req = await commit_and_write(dut_if, model, rob_tag=6) assert write_req.addr == 0x1040 - assert write_req.data == 0x11223344 + assert write_req.data == wbeat(0x11223344) # ============================================================================ @@ -272,7 +295,7 @@ async def test_slot2_only_alloc_with_initial_address(dut: Any) -> None: write_req = await commit_and_write(dut_if, model, rob_tag=9) assert write_req.addr == 0x1090 - assert write_req.data == 0x55667788 + assert write_req.data == wbeat(0x55667788) assert dut_if.empty, "SQ should be empty after slot-2-only store drains" @@ -325,9 +348,9 @@ async def test_slot1_slot2_dual_alloc_early_addr_and_widen_commit(dut: Any) -> N writes = await drain_pipelined_writes(dut_if, model, 2) assert len(writes) == 2, f"Expected 2 drain writes, got {len(writes)}" assert writes[0].addr == 0x2040 - assert writes[0].data == 0xAAAA_0004 + assert writes[0].data == wbeat(0xAAAA_0004) assert writes[1].addr == 0x2044 - assert writes[1].data == 0xBBBB_0005 + assert writes[1].data == wbeat(0xBBBB_0005) assert dut_if.empty, "SQ should be empty after both widened stores drain" @@ -522,12 +545,12 @@ async def test_simple_sw(dut: Any) -> None: write_req = await commit_and_write(dut_if, model, rob_tag=7) assert write_req.addr == 0x2000, f"Expected addr=0x2000, got 0x{write_req.addr:x}" + assert write_req.data == wbeat( + 0xCAFEBABE + ), f"Expected replicated 0xCAFEBABE beat, got 0x{write_req.data:x}" assert ( - write_req.data == 0xCAFEBABE - ), f"Expected data=0xCAFEBABE, got 0x{write_req.data:x}" - assert ( - write_req.byte_en == 0xF - ), f"Expected byte_en=0xF for SW, got 0x{write_req.byte_en:x}" + write_req.byte_en == 0x0F + ), f"Expected byte_en=0x0F for SW at addr[2]=0, got 0x{write_req.byte_en:x}" assert dut_if.empty, "SQ should be empty after write completes" @@ -546,12 +569,12 @@ async def test_sh_lower(dut: Any) -> None: assert write_req.addr == 0x1000 assert ( - write_req.byte_en == 0x3 - ), f"Expected byte_en=0x3, got 0x{write_req.byte_en:x}" - # Data is replicated: {data[15:0], data[15:0]} - assert ( - write_req.data == 0x12341234 - ), f"Expected 0x12341234, got 0x{write_req.data:x}" + write_req.byte_en == 0x03 + ), f"Expected byte_en=0x03, got 0x{write_req.byte_en:x}" + # Data is replicated across the beat: {4{data[15:0]}} + assert write_req.data == hbeat( + 0x1234 + ), f"Expected replicated 0x1234 beat, got 0x{write_req.data:x}" # ============================================================================ @@ -569,11 +592,11 @@ async def test_sh_upper(dut: Any) -> None: assert write_req.addr == 0x1002 assert ( - write_req.byte_en == 0xC - ), f"Expected byte_en=0xC, got 0x{write_req.byte_en:x}" - assert ( - write_req.data == 0xABCDABCD - ), f"Expected 0xABCDABCD, got 0x{write_req.data:x}" + write_req.byte_en == 0x0C + ), f"Expected byte_en=0x0C, got 0x{write_req.byte_en:x}" + assert write_req.data == hbeat( + 0xABCD + ), f"Expected replicated 0xABCD beat, got 0x{write_req.data:x}" # ============================================================================ @@ -591,11 +614,11 @@ async def test_sb(dut: Any) -> None: assert write_req.addr == 0x1001 assert ( - write_req.byte_en == 0x2 - ), f"Expected byte_en=0x2, got 0x{write_req.byte_en:x}" - assert ( - write_req.data == 0x42424242 - ), f"Expected 0x42424242, got 0x{write_req.data:x}" + write_req.byte_en == 0x02 + ), f"Expected byte_en=0x02, got 0x{write_req.byte_en:x}" + assert write_req.data == bbeat( + 0x42 + ), f"Expected replicated 0x42 beat, got 0x{write_req.data:x}" # ============================================================================ @@ -618,16 +641,18 @@ async def test_fsw(dut: Any) -> None: write_req = await commit_and_write(dut_if, model, rob_tag=4) assert write_req.addr == 0x3000 - assert write_req.data == 0x40490FDB, f"Expected FP data, got 0x{write_req.data:x}" - assert write_req.byte_en == 0xF + assert write_req.data == wbeat( + 0x40490FDB + ), f"Expected replicated FP word beat, got 0x{write_req.data:x}" + assert write_req.byte_en == 0x0F # ============================================================================ -# Test 10: FSD two-phase commit +# Test 10: FSD single-beat commit # ============================================================================ @cocotb.test() -async def test_fsd_two_phase(dut: Any) -> None: - """FSD: phase 0 writes low word, phase 1 writes high word at addr+4.""" +async def test_fsd_single_beat(dut: Any) -> None: + """FSD: one 64-bit write covering the aligned dword with all lanes.""" dut_if, model = await setup(dut) fp64_data = 0x400921FB54442D18 # pi @@ -647,32 +672,14 @@ async def test_fsd_two_phase(dut: Any) -> None: await dut_if.step() dut_if.clear_commit() - # Phase 0: low word at addr + # Single beat: full dword at addr with an all-lanes strobe write_req = await wait_for_mem_write(dut_if) - assert write_req.en, "Phase 0 write expected" + assert write_req.en, "FSD write expected" assert ( write_req.addr == 0x4000 - ), f"Phase 0 addr should be 0x4000, got 0x{write_req.addr:x}" - assert write_req.data == (fp64_data & MASK32), "Phase 0 data mismatch" - assert write_req.byte_en == 0xF - - model.mem_write_initiate() - await dut_if.step() - dut_if.drive_mem_write_done() - model.mem_write_done() - await dut_if.step() - dut_if.clear_mem_write_done() - assert dut_if.count == model.count == 1, "FSD phase 0 must not remove the SQ entry" - assert not dut_if.empty, "FSD must remain live between its two write phases" - - # Phase 1: high word at addr+4 - write_req = await wait_for_mem_write(dut_if) - assert write_req.en, "Phase 1 write expected" - assert ( - write_req.addr == 0x4004 - ), f"Phase 1 addr should be 0x4004, got 0x{write_req.addr:x}" - assert write_req.data == ((fp64_data >> 32) & MASK32), "Phase 1 data mismatch" - assert write_req.byte_en == 0xF + ), f"FSD addr should be 0x4000, got 0x{write_req.addr:x}" + assert write_req.data == fp64_data, "FSD beat data mismatch" + assert write_req.byte_en == 0xFF, "FSD must strobe all 8 lanes" model.mem_write_initiate() await dut_if.step() @@ -1052,8 +1059,9 @@ async def test_forward_metadata_survives_flush_capture_edge(dut: Any) -> None: assert ( fwd.match and fwd.can_forward ), "Flush-edge probe should be captured coherently" - assert fwd.data == 0xDEAD_BEEF, ( - "Captured high-word extraction must survive the flush edge: " + assert fwd.data == store_data, ( + "Captured image reconstruction must survive the flush edge " + "(the LQ extracts the probe's word from the dword image): " f"got 0x{fwd.data:x}" ) assert dut_if.empty, "Full flush should clear all architectural SQ entries" @@ -1183,9 +1191,9 @@ async def test_in_order_write(dut: Any) -> None: assert ( writes[i].addr == addr ), f"Write {i}: expected addr 0x{addr:x}, got 0x{writes[i].addr:x}" - assert ( - writes[i].data == data - ), f"Write {i}: expected data 0x{data:x}, got 0x{writes[i].data:x}" + assert writes[i].data == wbeat( + data + ), f"Write {i}: expected beat 0x{wbeat(data):x}, got 0x{writes[i].data:x}" assert dut_if.empty, "SQ should be empty after all writes" @@ -1244,7 +1252,7 @@ async def test_no_write_without_data(dut: Any) -> None: # Now write should happen write_req = await wait_for_mem_write(dut_if) assert write_req.en, "Write should fire once data arrives" - assert write_req.data == 0xBEEF + assert write_req.data == wbeat(0xBEEF) # ============================================================================ @@ -1358,7 +1366,7 @@ async def test_forward_same_cycle_commit_after_head_advance(dut: Any) -> None: # ============================================================================ @cocotb.test() async def test_forward_fsd_overlap_plus4(dut: Any) -> None: - """FSD at addr A, FLW at addr A+4 → match + forward high word.""" + """FSD at addr A, FLW at addr A+4 → match + forward the dword image.""" dut_if, model = await setup(dut) fp64_data = 0x1234567890ABCDEF @@ -1377,12 +1385,12 @@ async def test_forward_fsd_overlap_plus4(dut: Any) -> None: await dut_if.step() # Wait for registered SQ forwarding output fwd = dut_if.read_sq_forward() - assert fwd.match, "DOUBLE store overlaps at +4" - assert fwd.can_forward, "FLW at FSD+4 should forward high word" - expected_hi = (fp64_data >> 32) & MASK32 - assert ( - fwd.data == expected_hi - ), f"Expected high word 0x{expected_hi:08x}, got 0x{fwd.data:x}" + assert fwd.match, "DOUBLE store overlaps the load's dword" + assert fwd.can_forward, "FSD covers a word load anywhere in its dword" + assert fwd.data == fp64_data, ( + "The SQ delivers the aligned-dword image (the LQ extracts the " + f"addressed word by addr[2]): got 0x{fwd.data:x}" + ) dut_if.clear_sq_check() @@ -1439,11 +1447,16 @@ async def test_non_mmio_forwards_over_mmio(dut: Any) -> None: # ============================================================================ -# Test 29: FSD phase-2 cache invalidation at addr+4 +# Test 29: FSD single-launch cache invalidation covers its dword # ============================================================================ @cocotb.test() -async def test_fsd_phase2_cache_invalidation(dut: Any) -> None: - """Each FSD phase's write LAUNCH invalidates its own word (base, addr+4).""" +async def test_fsd_cache_invalidation_single_beat(dut: Any) -> None: + """A single-beat FSD launch fires ONE invalidate flagged dword-covering. + + The L0 is dword-granule (one line covers both words), and the wrapper's + reservation snoop widens its compare on is_dword — one pulse preserves + the coverage the old two-phase drain delivered as two word pulses. + """ dut_if, model = await setup(dut) fp64_data = 0x400921FB54442D18 # pi @@ -1464,30 +1477,15 @@ async def test_fsd_phase2_cache_invalidation(dut: Any) -> None: await dut_if.step() dut_if.clear_commit() - # Phase 0: low word at base addr — invalidate fires with the launch. + # Single launch — invalidate fires with it, flagged dword-covering. write_req = await wait_for_mem_write(dut_if) - assert write_req.en, "Phase 0 write expected" + assert write_req.en, "FSD write expected" inv = dut_if.read_cache_invalidate() - assert inv["valid"], "Phase 0 cache invalidation expected at launch" + assert inv["valid"], "Cache invalidation expected at launch" assert ( inv["addr"] == base_addr - ), f"Phase 0 should invalidate at base 0x{base_addr:x}, got 0x{inv['addr']:x}" - - model.mem_write_initiate() - await dut_if.step() - dut_if.drive_mem_write_done() - model.mem_write_done() - await dut_if.step() - dut_if.clear_mem_write_done() - - # Phase 1: high word at addr+4 — its own launch-cycle invalidate. - write_req = await wait_for_mem_write(dut_if) - assert write_req.en, "Phase 1 write expected" - inv = dut_if.read_cache_invalidate() - assert inv["valid"], "Phase 1 cache invalidation expected at launch" - assert ( - inv["addr"] == base_addr + 4 - ), f"Phase 1 should invalidate at 0x{base_addr + 4:x}, got 0x{inv['addr']:x}" + ), f"Should invalidate at base 0x{base_addr:x}, got 0x{inv['addr']:x}" + assert inv["is_dword"], "FSD invalidate must be flagged dword-covering" model.mem_write_initiate() await dut_if.step() @@ -1524,10 +1522,10 @@ async def test_forward_flw_at_fsd_base(dut: Any) -> None: fwd = dut_if.read_sq_forward() assert fwd.match, "FLW at FSD base should match" assert fwd.can_forward, "FLW at FSD base should forward" - expected_lo = fp64_data & MASK32 - assert ( - fwd.data == expected_lo - ), f"Expected low word 0x{expected_lo:08x}, got 0x{fwd.data:x}" + assert fwd.data == fp64_data, ( + "The SQ delivers the aligned-dword image (the LQ extracts the low " + f"word by addr[2]=0): got 0x{fwd.data:x}" + ) dut_if.clear_sq_check() @@ -1557,10 +1555,10 @@ async def test_forward_flw_at_fsd_plus4(dut: Any) -> None: fwd = dut_if.read_sq_forward() assert fwd.match, "FLW at FSD+4 should match" assert fwd.can_forward, "FLW at FSD+4 should forward" - expected_hi = (fp64_data >> 32) & MASK32 - assert ( - fwd.data == expected_hi - ), f"Expected high word 0x{expected_hi:08x}, got 0x{fwd.data:x}" + assert fwd.data == fp64_data, ( + "The SQ delivers the aligned-dword image (the LQ extracts the high " + f"word by addr[2]=1): got 0x{fwd.data:x}" + ) dut_if.clear_sq_check() @@ -1589,8 +1587,10 @@ async def test_forward_lb_at_fsd_base(dut: Any) -> None: fwd = dut_if.read_sq_forward() assert fwd.match, "LB at FSD base should match" - assert fwd.can_forward, "FSD base word is fully written — byte load forwards" - assert fwd.data == 0xCAFEBABE, f"Expected low-word image, got 0x{fwd.data:x}" + assert fwd.can_forward, "FSD covers a byte load anywhere in its dword" + assert ( + fwd.data == fp64_data + ), f"Expected the full dword image (LQ extracts byte 0), got 0x{fwd.data:x}" dut_if.clear_sq_check() @@ -1855,19 +1855,23 @@ async def test_forward_lh_from_fsd_both_words(dut: Any) -> None: fwd = dut_if.read_sq_forward() assert fwd.match, "LH at FSD base should match" - assert fwd.can_forward, "FSD low word is fully written — half load forwards" - assert fwd.data == 0xCAFEBABE, f"Expected low-word image, got 0x{fwd.data:x}" + assert fwd.can_forward, "FSD covers a half load anywhere in its dword" + assert ( + fwd.data == fp64_data + ), f"Expected the full dword image (LQ extracts half 0), got 0x{fwd.data:x}" dut_if.clear_sq_check() await dut_if.step() - # LH at FSD addr+6: forwards the high-word image + # LH at FSD addr+6: the same dword image, LQ extracts lanes 6-7 dut_if.drive_sq_check(addr=0x7006, rob_tag=5, size=MEM_SIZE_HALF) await dut_if.step() fwd = dut_if.read_sq_forward() assert fwd.match, "LH in FSD high word should match" - assert fwd.can_forward, "FSD high word is fully written — half load forwards" - assert fwd.data == 0xDEADBEEF, f"Expected high-word image, got 0x{fwd.data:x}" + assert fwd.can_forward, "FSD covers a half load anywhere in its dword" + assert ( + fwd.data == fp64_data + ), f"Expected the full dword image (LQ extracts half 3), got 0x{fwd.data:x}" dut_if.clear_sq_check() diff --git a/verif/cocotb_tests/tomasulo/tomasulo_wrapper/test_tomasulo_wrapper.py b/verif/cocotb_tests/tomasulo/tomasulo_wrapper/test_tomasulo_wrapper.py index b9b54ff1..ad36c255 100644 --- a/verif/cocotb_tests/tomasulo/tomasulo_wrapper/test_tomasulo_wrapper.py +++ b/verif/cocotb_tests/tomasulo/tomasulo_wrapper/test_tomasulo_wrapper.py @@ -237,6 +237,17 @@ def log_random_seed() -> int: return seed +def wbeat(word: int) -> int: + """Word write data replicated across the 64-bit beat ({2{word}}). + + The data tier positions sub-beat store data by replication with the + 8-lane strobe selecting the addressed lanes (docs/rv64/m1_data_tier.md), + so drain/AMO write-data checks compare against the replicated beat. + """ + word &= 0xFFFF_FFFF + return (word << 32) | word + + async def setup_test(dut: Any) -> tuple[TomasuloInterface, TomasuloModel]: """Initialize clock, interface, model and reset DUT.""" clock = Clock(dut.i_clk, 10, unit="ns") @@ -3224,7 +3235,7 @@ async def test_lq_sq_forward_through_wrapper(dut: Any) -> None: await dut_if.step() assert sq_write["en"], "SQ should drain the committed store" assert sq_write["addr"] == expected_addr - assert sq_write["data"] == forward_data + assert sq_write["data"] == wbeat(forward_data) await dut_if.step() dut_if.drive_sq_mem_write_done() @@ -3453,7 +3464,7 @@ async def bus_monitor() -> None: await dut_if.step() w = dut_if.read_sq_mem_write() assert ( - w["en"] and w["addr"] == addr_y and w["data"] == data_y0 + w["en"] and w["addr"] == addr_y and w["data"] == wbeat(data_y0) ), f"S0 drain mismatch: {w}" await dut_if.step() dut_if.drive_sq_mem_write_done() @@ -3739,7 +3750,7 @@ async def bus_monitor() -> None: await dut_if.step() w = dut_if.read_sq_mem_write() assert ( - w["en"] and w["addr"] == expect_addr and w["data"] == expect_data + w["en"] and w["addr"] == expect_addr and w["data"] == wbeat(expect_data) ), f"post-flush drain mismatch: {w} != ({expect_addr:#x}, {expect_data:#x})" await dut_if.step() dut_if.drive_sq_mem_write_done() @@ -3764,11 +3775,11 @@ async def bus_monitor() -> None: # particular nothing for a flushed tag after the flush. # --------------------------------------------------------------------- assert sq_writes == [ - (addr_y, data_y0), - (addr_y, data_y1), - (addr_x, data_fresh), + (addr_y, wbeat(data_y0)), + (addr_y, wbeat(data_y1)), + (addr_x, wbeat(data_fresh)), ], f"unexpected SQ write history: {[(hex(a), hex(d)) for a, d in sq_writes]}" - assert all(d != data_stale for _, d in sq_writes), "dead store data escaped" + assert all(d != wbeat(data_stale) for _, d in sq_writes), "dead store data escaped" assert lq_reads == [], f"no load should touch memory in this test: {lq_reads}" assert cdb_log == [ (tag_p2, data_stale), @@ -4802,9 +4813,9 @@ async def test_amo_swap_integration(dut: Any) -> None: amo_write = dut_if.read_amo_mem_write() assert amo_write["en"], "AMO should request memory write" assert amo_write["addr"] == addr - assert ( - amo_write["data"] == rs2_val - ), f"AMOSWAP should write rs2={rs2_val:#x}, got {amo_write['data']:#x}" + assert amo_write["data"] == wbeat( + rs2_val + ), f"AMOSWAP should write rs2 beat {wbeat(rs2_val):#x}, got {amo_write['data']:#x}" # Acknowledge AMO write → old value goes to CDB dut_if.drive_amo_mem_write_done() @@ -5589,8 +5600,9 @@ async def _run_amo_test( amo_write = dut_if.read_amo_mem_write() assert amo_write["en"], f"{op_name} should request memory write" assert amo_write["addr"] == addr - assert amo_write["data"] == (expected_write & 0xFFFF_FFFF), ( - f"{op_name} write: expected {expected_write:#x}, " f"got {amo_write['data']:#x}" + assert amo_write["data"] == wbeat(expected_write), ( + f"{op_name} write: expected beat {wbeat(expected_write):#x}, " + f"got {amo_write['data']:#x}" ) # Acknowledge AMO write → old value goes to CDB diff --git a/verif/cocotb_tests/tomasulo/tomasulo_wrapper/tomasulo_interface.py b/verif/cocotb_tests/tomasulo/tomasulo_wrapper/tomasulo_interface.py index 3ed2bb3d..0223a34c 100644 --- a/verif/cocotb_tests/tomasulo/tomasulo_wrapper/tomasulo_interface.py +++ b/verif/cocotb_tests/tomasulo/tomasulo_wrapper/tomasulo_interface.py @@ -1125,9 +1125,19 @@ def sq_count(self) -> int: # Load Queue: Memory Interface # ========================================================================= - def drive_lq_mem_response(self, data: int) -> None: - """Drive LQ memory response (data + valid).""" - self.dut.i_lq_mem_read_data.value = data & MASK32 + def drive_lq_mem_response(self, data: int, *, dword: bool = False) -> None: + """Drive an LQ memory response beat (data + valid). + + The data tier returns aligned 64-bit beats (docs/rv64/m1_data_tier.md). + A 32-bit word is replicated into both lanes so the response is correct + at either addr[2]; pass ``dword=True`` with a full 64-bit value for + FLD-style beats. + """ + if dword: + self.dut.i_lq_mem_read_data.value = data & MASK64 + else: + word = data & MASK32 + self.dut.i_lq_mem_read_data.value = (word << 32) | word self.dut.i_lq_mem_read_valid.value = 1 def clear_lq_mem_response(self) -> None: diff --git a/verif/config.py b/verif/config.py index e3ccea6b..333b5d67 100644 --- a/verif/config.py +++ b/verif/config.py @@ -51,6 +51,8 @@ DEFAULT_MIN_COVERAGE_COUNT, DEFAULT_CLOCK_PERIOD_NS, DEFAULT_RESET_CYCLES) """ +import os + from dataclasses import dataclass from typing import Final @@ -73,12 +75,30 @@ MEMORY_HALFWORD_ALIGN_MASK: Final[int] = 0xFFFFFFFE """Mask for halfword-aligning addresses (clear bottom bit, 32-bit safe).""" -MEMORY_BYTE_OFFSET_MASK: Final[int] = 0x3 -"""Mask to extract byte offset within a word (bits [1:0]).""" - MEMORY_SIZE_WORDS: Final[int] = 2**14 """Size of memory in words (16K words = 64KB for 16-bit address space).""" +# ---------------------------------------------------------------------------- +# Data-tier beat contract (docs/rv64/m1_data_tier.md): every data-side bus +# carries the aligned dword at addr[31:3] with 8 byte-lane strobes; store +# data is replicated across the beat and the strobes select the lanes. +# ---------------------------------------------------------------------------- + +MEM_DATA_BITS: Final[int] = 64 +"""Data-tier beat width in bits (mirrors riscv_pkg::MemDataBits).""" + +MEM_STRB_BITS: Final[int] = MEM_DATA_BITS // 8 +"""Byte-lane strobe count per beat (mirrors riscv_pkg::MemStrbBits).""" + +MEMORY_DWORD_ALIGN_MASK: Final[int] = 0xFFFFFFF8 +"""Mask for dword-aligning addresses (clear bottom 3 bits, 32-bit safe).""" + +MEMORY_BEAT_OFFSET_MASK: Final[int] = 0x7 +"""Mask to extract the byte offset within a beat (bits [2:0]).""" + +MEMORY_SIZE_DWORDS: Final[int] = MEMORY_SIZE_WORDS // 2 +"""Size of memory in dword rows (the simulation data BRAM's row count).""" + MMIO_BASE_ADDR: Final[int] = 0x40000000 """Base address of MMIO peripheral range (UART, CLINT timer, etc.).""" @@ -251,8 +271,19 @@ class DUTSignalPaths: # RISC-V ISA Constants # ============================================================================ -XLEN: Final[int] = 32 -"""RISC-V XLEN parameter (32 for RV32).""" +XLEN: Final[int] = 64 if os.environ.get("FROST_RV64") == "1" else 32 +"""RISC-V XLEN parameter. + +Single source of truth for the verification side, in lockstep with the RTL: +riscv_pkg derives its XLEN localparam from the FROST_RV64 build define, and +this constant derives from the FROST_RV64 environment variable the build +plumbing exports alongside it (docs/rv64/phase1_plan.md decision D1). Every +cocotb interface/model imports XLEN/FLEN from here rather than keeping a +private copy. +""" + +FLEN: Final[int] = 64 +"""FP register width (FLEN): 64 for the D extension at either XLEN.""" NOP_INSTRUCTION: Final[int] = 0x00000013 """32-bit NOP encoding (addi x0, x0, 0).""" diff --git a/verif/models/memory_model.py b/verif/models/memory_model.py index c69b1688..3c7dcc09 100644 --- a/verif/models/memory_model.py +++ b/verif/models/memory_model.py @@ -49,12 +49,69 @@ from typing import Any from config import ( MASK32, + MASK64, + MEM_STRB_BITS, MEMORY_ADDRESS_MASK, MEMORY_WORD_ALIGN_MASK, - MEMORY_SIZE_WORDS, + MEMORY_DWORD_ALIGN_MASK, + MEMORY_SIZE_DWORDS, ) +def poke_dut_memory_word(device_under_test: Any, byte_address: int, value: int) -> None: + """Deposit one 32-bit word into the DUT's dword-row simulation data BRAM. + + The data BRAM stores aligned 64-bit rows (docs/rv64/m1_data_tier.md), so + a word deposit is a read-modify-write of the addressed row's word lane. + + CAUTION: cocotb ``.value`` writes are queued deposits — the read half of + a second RMW to the SAME row within one scheduler delta sees the + pre-deposit value and its write clobbers the first word. To write both + words of a row in one delta, use poke_dut_memory_dword instead. + + Args: + device_under_test: CoCoTB DUT handle with data_memory_for_simulation + byte_address: Word-aligned byte address to poke + value: 32-bit value to deposit + """ + row = byte_address >> 3 + shift = 32 if byte_address & 0x4 else 0 + row_handle = device_under_test.data_memory_for_simulation.memory[row] + current = int(row_handle.value) + row_handle.value = (current & ~(MASK32 << shift)) | ((value & MASK32) << shift) + + +def peek_dut_memory_word(device_under_test: Any, byte_address: int) -> int: + """Read one 32-bit word from the DUT's dword-row simulation data BRAM. + + Args: + device_under_test: CoCoTB DUT handle with data_memory_for_simulation + byte_address: Word-aligned byte address to read + + Returns: + The 32-bit word at that address + """ + row = byte_address >> 3 + shift = 32 if byte_address & 0x4 else 0 + row_value = int(device_under_test.data_memory_for_simulation.memory[row].value) + return (row_value >> shift) & MASK32 + + +def poke_dut_memory_dword( + device_under_test: Any, byte_address: int, value: int +) -> None: + """Deposit one aligned 64-bit dword row into the DUT's simulation data BRAM. + + Args: + device_under_test: CoCoTB DUT handle with data_memory_for_simulation + byte_address: Dword-aligned byte address to poke + value: 64-bit value to deposit + """ + device_under_test.data_memory_for_simulation.memory[byte_address >> 3].value = ( + value & MASK64 + ) + + class MemoryModel: """Software model of data memory for CPU verification. @@ -83,14 +140,13 @@ def __init__(self, device_under_test: Any) -> None: self.read_address: int = 0 # Address for pending load operation self.ram_bytes: dict[int, int] = {} # Byte-addressable memory dictionary - # Initialize testbench RAM to match DUT RAM contents - for word_index in range(MEMORY_SIZE_WORDS): - self.write_word( - word_index * 4, + # Initialize testbench RAM to match DUT RAM contents (the simulation + # data BRAM stores aligned 64-bit dword rows). + for row_index in range(MEMORY_SIZE_DWORDS): + self.write_dword( + row_index * 8, int( - device_under_test.data_memory_for_simulation.memory[ - word_index - ].value + device_under_test.data_memory_for_simulation.memory[row_index].value ), ) @@ -146,6 +202,31 @@ def write_word(self, address: int, value: int = 0) -> None: self.write_byte(aligned_address + 2, (value >> 16) & 0xFF) self.write_byte(aligned_address + 3, (value >> 24) & 0xFF) + def read_dword(self, address: int) -> int: + """Read a full aligned 64-bit dword from memory (little-endian). + + Args: + address: Byte address (will be aligned to 8-byte boundary) + + Returns: + 64-bit dword value assembled from 8 bytes + """ + aligned_address = address & MEMORY_DWORD_ALIGN_MASK + return self.read_word(aligned_address) | ( + self.read_word(aligned_address + 4) << 32 + ) + + def write_dword(self, address: int, value: int = 0) -> None: + """Write a full aligned 64-bit dword to memory (little-endian). + + Args: + address: Byte address (will be aligned to 8-byte boundary) + value: 64-bit dword value to write + """ + aligned_address = address & MEMORY_DWORD_ALIGN_MASK + self.write_word(aligned_address, value & MASK32) + self.write_word(aligned_address + 4, (value >> 32) & MASK32) + async def driver_and_monitor( self, write_data_expected_queue: list[int], @@ -183,12 +264,16 @@ async def driver_and_monitor( await FallingEdge(self.dut.i_clk) await RisingEdge(self.dut.i_clk) - # Check if DUT is performing a write (non-zero byte enable mask) - wr_mask = int(self.dut.o_data_mem_per_byte_wr_en.value) & 0xF + # Check if DUT is performing a write (non-zero 8-lane strobe) + wr_mask = int(self.dut.o_data_mem_per_byte_wr_en.value) & ( + (1 << MEM_STRB_BITS) - 1 + ) if wr_mask: - # Read write address and data from DUT outputs + # Read write address and beat data from DUT outputs. Store + # data rides the beat replicated (bus contract), so the full + # 64-bit compare is lane-independent. wr_addr = int(self.dut.o_data_mem_addr.value) & MASK32 - wr_data = int(self.dut.o_data_mem_wr_data.value) & MASK32 + wr_data = int(self.dut.o_data_mem_wr_data.value) & MASK64 # Verify against expected values from software model if write_address_expected_queue: @@ -205,13 +290,13 @@ async def driver_and_monitor( # Verify write data matches expected assert wr_data == exp_data, ( f"Memory-write data mismatch at 0x{wr_addr:08X}: " - f"got 0x{wr_data:08X}, expected 0x{exp_data:08X}, " + f"got 0x{wr_data:016X}, expected 0x{exp_data:016X}, " f"RANDOM_SEED {cocotb.RANDOM_SEED}" ) else: # No write was expected - this is an error raise AssertionError( f"Unexpected memory write: addr 0x{wr_addr:08X}, " - f"data 0x{wr_data:08X}, mask 0b{wr_mask:04b}, " + f"data 0x{wr_data:016X}, mask 0b{wr_mask:08b}, " f"RANDOM_SEED {cocotb.RANDOM_SEED}" ) diff --git a/verif/utils/memory_utils.py b/verif/utils/memory_utils.py index fe8ed05b..2d699800 100644 --- a/verif/utils/memory_utils.py +++ b/verif/utils/memory_utils.py @@ -27,7 +27,9 @@ BYTE_ALIGNMENT, HALFWORD_ALIGNMENT, WORD_ALIGNMENT, - MEMORY_BYTE_OFFSET_MASK, + MASK32, + MASK64, + MEMORY_BEAT_OFFSET_MASK, ) from exceptions import AlignmentError @@ -101,79 +103,117 @@ def ensure_aligned(address: int, alignment: int, operation: str) -> int: return address -def get_byte_offset(address: int) -> int: - """Get byte offset within word (bits [1:0] of address). +def get_beat_byte_offset(address: int) -> int: + """Get byte offset within the data-tier beat (bits [2:0] of address). Args: address: Memory address Returns: - Byte offset (0-3) within the containing word + Byte offset (0-7) within the containing aligned dword Examples: - >>> get_byte_offset(0x1000) + >>> get_beat_byte_offset(0x1000) 0 - >>> get_byte_offset(0x1003) - 3 + >>> get_beat_byte_offset(0x1007) + 7 """ - return address & MEMORY_BYTE_OFFSET_MASK + return address & MEMORY_BEAT_OFFSET_MASK -def calculate_byte_mask_for_store(operation: str, byte_offset: int) -> int: - """Calculate byte-enable mask for store operations. +def calculate_byte_mask_for_store(operation: str, beat_offset: int) -> int: + """Calculate the 8-lane byte strobe for a store on the data-tier beat. - RISC-V stores write to a 32-bit word-aligned memory interface. - The byte mask indicates which bytes within that word should be updated. - Each bit corresponds to one byte: bit 0 = byte 0, bit 1 = byte 1, etc. - - Memory Layout (little-endian): - Byte: 3 2 1 0 - Bits: [31:24] [23:16] [15:8] [7:0] - Mask: 0b1000 0b0100 0b0010 0b0001 + The data tier carries aligned 64-bit beats with one strobe bit per byte + lane (docs/rv64/m1_data_tier.md; mirrors riscv_pkg::mem_strobe_for). + Bit i of the mask selects byte address {addr[31:3], i}. Store Types: - SB (store byte): Write 1 byte -> mask has 1 bit set - SH (store halfword): Write 2 bytes -> mask has 2 consecutive bits set - SW (store word): Write 4 bytes -> mask = 0b1111 + SB (store byte): 1 lane -> 0x01 << offset + SH (store halfword): 2 lanes -> 0x03 << (offset & ~1) + SW/FSW (word): 4 lanes -> 0x0F or 0xF0 by addr[2] + FSD (double): all 8 lanes -> 0xFF Args: - operation: Store operation ("sb", "sh", or "sw") - byte_offset: Byte offset within word (0-3), from address[1:0] + operation: Store operation ("sb", "sh", "sw", "fsw", or "fsd") + beat_offset: Byte offset within the beat (0-7), from address[2:0] Returns: - 4-bit mask indicating which bytes to write + 8-bit strobe indicating which beat lanes to write Raises: ValueError: If operation is not a valid store instruction Examples: - >>> calculate_byte_mask_for_store("sb", 0) # Store to byte 0 -> 0b0001 + >>> calculate_byte_mask_for_store("sb", 0) # Lane 0 -> 0x01 1 - >>> calculate_byte_mask_for_store("sb", 3) # Store to byte 3 -> 0b1000 - 8 - >>> calculate_byte_mask_for_store("sh", 0) # Halfword bytes 0-1 -> 0b0011 - 3 - >>> calculate_byte_mask_for_store("sh", 2) # Halfword bytes 2-3 -> 0b1100 - 12 - >>> calculate_byte_mask_for_store("sw", 0) # Full word -> 0b1111 + >>> calculate_byte_mask_for_store("sb", 7) # Lane 7 -> 0x80 + 128 + >>> calculate_byte_mask_for_store("sh", 6) # Lanes 6-7 -> 0xC0 + 192 + >>> calculate_byte_mask_for_store("sw", 0) # Low word -> 0x0F 15 + >>> calculate_byte_mask_for_store("sw", 4) # High word -> 0xF0 + 240 + >>> calculate_byte_mask_for_store("fsd", 0) # Full beat -> 0xFF + 255 """ if operation == "sb": - # Store byte: Write single byte at position specified by byte_offset - # Offset 0 -> 0b0001, offset 1 -> 0b0010, offset 2 -> 0b0100, offset 3 -> 0b1000 - return 1 << byte_offset + return 1 << beat_offset elif operation == "sh": - # Store halfword: Write 2 consecutive bytes - # Halfwords must be 2-byte aligned, so offset is 0, 1, 2, or 3 - # But actual halfword addresses are offset 0 or 2 (aligned) - # Offset 0 or 1 -> bytes 0,1 (0b0011) - # Offset 2 or 3 -> bytes 2,3 (0b1100) - return 0b1100 if byte_offset > 1 else 0b0011 - - elif operation == "sw": - # Store word: Write all 4 bytes - return 0b1111 + return 0b11 << (beat_offset & ~1) + + elif operation in ("sw", "fsw"): + return 0xF0 if beat_offset & 0x4 else 0x0F + + elif operation == "fsd": + return 0xFF + + else: + raise ValueError(f"Unknown store operation: {operation}") + + +def replicate_store_data_for_beat(operation: str, value: int) -> int: + """Position store data on the beat by replication (bus contract). + + The RTL replicates sub-beat store data across all 64 bits and lets the + byte strobes select the addressed lanes ({8{byte}}, {4{half}}, + {2{word}}, dword pass-through — store_queue.gen_write_data). The + expected-write monitor compares the full beat, so the model mirrors the + replication exactly. + + Args: + operation: Store operation ("sb", "sh", "sw", "fsw", or "fsd") + value: Store data (low bits used per the operation's size) + + Returns: + 64-bit beat image with the data replicated across the beat + + Examples: + >>> hex(replicate_store_data_for_beat("sb", 0xAB)) + '0xabababababababab' + >>> hex(replicate_store_data_for_beat("sh", 0x1234)) + '0x1234123412341234' + >>> hex(replicate_store_data_for_beat("sw", 0xDEADBEEF)) + '0xdeadbeefdeadbeef' + >>> hex(replicate_store_data_for_beat("fsd", 0x0123456789ABCDEF)) + '0x123456789abcdef' + """ + if operation == "sb": + byte = value & 0xFF + return int.from_bytes(bytes([byte]) * 8, "little") + + elif operation == "sh": + half = value & 0xFFFF + return half | half << 16 | half << 32 | half << 48 + + elif operation in ("sw", "fsw"): + word = value & MASK32 + return word | word << 32 + + elif operation == "fsd": + return value & MASK64 else: raise ValueError(f"Unknown store operation: {operation}")