Integration & End-to-End Measurement

Rodolfo Azevedo

Institute of Computing, University of Campinas (UNICAMP), Brazil

rodolfo.azevedo@unicamp.br

http://www.ic.unicamp.br/~rodolfo/mo801

Goal of this class

Module 5, Class 3: connecting the accelerator to the bus, the C driver, and measuring speedup.

Simulation proved the accelerator is correct in isolation. Now we put it on the bus, write the driver, replace the software kernel call, and measure. The number you get here is the headline result of the entire course.

At the end of this class, you should be able to:

• Connect the KWS accelerator to the SoC bus using the address decoder from Project 2.
• Write a C driver that configures, triggers, and polls the accelerator via memory-mapped registers.
• Replace the software KWS kernel call with the hardware accelerator and verify bit-identical output.
• Measure end-to-end inference speedup and explain any gap between predicted and measured performance.

Adding the accelerator to the bus

Extend bus_top.sv with one new entry — no other changes needed:

// In bus_decoder:
24'h000203: sel_accel = 1;   // 0x0002_03xx — already reserved in Project 2

// In bus_top:
kws_accel accel (
    .clk(clk), .rst(rst),
    .req(cpu_req), .en(sel_accel), .resp(accel_resp),
    .dmem_addr(accel_dmem_addr),
    .dmem_re(accel_dmem_re),
    .dmem_rdata(dmem_rdata),        // shared with CPU's DMEM reads
    .dmem_ready(dmem_ready)
);

// Response mux: add accel
if (sel_accel) cpu_resp = accel_resp;

The DMEM arbitration is the only subtle part: when the accelerator is in LOAD state, it drives accel_dmem_addr; the bus must stall the CPU's DMEM access until the accelerator's fetch completes. A simple priority arbiter (CPU wins, accelerator waits) is sufficient.

DMEM arbitration: CPU vs. accelerator

When both the CPU (in MEM_READ/MEM_WRITE) and the accelerator (in LOAD) want DMEM at the same cycle, one must wait.

module dmem_arbiter (
    // CPU port
    input  logic [31:0] cpu_addr,  input  logic cpu_re, cpu_we,
    output logic [31:0] cpu_rdata, output logic cpu_ready,
    // Accelerator port
    input  logic [31:0] accel_addr, input logic accel_re,
    output logic [31:0] accel_rdata, output logic accel_ready,
    // DMEM
    output logic [31:0] dmem_addr,  output logic dmem_re,
    input  logic [31:0] dmem_rdata
);
    // CPU wins; accelerator stalls
    always_comb begin
        if (cpu_re || cpu_we) begin
            dmem_addr   = cpu_addr;
            dmem_re     = cpu_re;
            cpu_rdata   = dmem_rdata;
            cpu_ready   = 1'b1;
            accel_rdata = '0;
            accel_ready = 1'b0;   // accelerator stalls
        end else begin
            dmem_addr   = accel_addr;
            dmem_re     = accel_re;
            accel_rdata = dmem_rdata;
            accel_ready = 1'b1;
            cpu_rdata   = '0;
            cpu_ready   = 1'b1;   // CPU has no pending DMEM request
        end
    end
endmodule

• CPU wins policy: simple, avoids CPU stalls on instruction fetch/load/store.
• Accelerator stalls during CPU DMEM access — extends the LOAD phase by the number of CPU conflicts.
• Conflict rate is low in practice: the CPU's outer loops (oh, ow, co) do little DMEM work between accelerator calls.

The C software driver

// accel_driver.h
#define ACCEL_BASE      0x00020300
#define ACCEL_CONTROL   (*(volatile uint32_t *)(ACCEL_BASE + 0x00))
#define ACCEL_STATUS    (*(volatile uint32_t *)(ACCEL_BASE + 0x04))
#define ACCEL_IN_ADDR   (*(volatile uint32_t *)(ACCEL_BASE + 0x08))
#define ACCEL_WT_ADDR   (*(volatile uint32_t *)(ACCEL_BASE + 0x0C))
#define ACCEL_OUT_ADDR  (*(volatile uint32_t *)(ACCEL_BASE + 0x10))
#define ACCEL_LENGTH    (*(volatile uint32_t *)(ACCEL_BASE + 0x14))

void accel_load_weights(const int8_t *weights, int n) {
    // Write n bytes to the accelerator's weight BRAM via the bus
    for (int i = 0; i < n; i++) {
        ACCEL_WT_ADDR = i;
        ACCEL_CONTROL = (weights[i] << 8) | 0x2;  // write-weight command
    }
}

int8_t accel_run(uint32_t input_addr, uint32_t output_addr, int length) {
    ACCEL_IN_ADDR  = input_addr;
    ACCEL_OUT_ADDR = output_addr;
    ACCEL_LENGTH   = length;
    ACCEL_CONTROL  = 0x1;                    // start
    while (!(ACCEL_STATUS & 0x1));           // poll until done
    return (int8_t)ACCEL_STATUS >> 8;        // result in STATUS[15:8] (fast path)
}

Replacing the software kernel call

In kws_kernel.c, the Conv2D inner loop was:

// BEFORE: software dot product (innermost loops of conv2d_int8)
for (int kh = 0; kh < KH; kh++)
for (int kw = 0; kw < KW; kw++)
for (int ci = 0; ci < C_in; ci++)
    acc += (int32_t)input[...] * (int32_t)weights[...];
output[...] = requantize(acc, ...);

After: replace with one accelerator call per output channel per spatial position:

// AFTER: hardware accelerated
accel_load_weights(&weights_conv[co * KH*KW*C_in], KH*KW*C_in);  // weight row co

uint32_t in_addr  = (uint32_t)&input[(oh*stride)*W*C_in + ow*stride*C_in];
uint32_t out_addr = (uint32_t)&output[oh*W_out*C_out + ow*C_out + co];
accel_run(in_addr, out_addr, KH*KW*C_in);

The outer loops (oh, ow, co) still run on the CPU. Only the inner MAC reduction moves to hardware. This is the classic heterogeneous compute pattern: CPU handles control flow and data movement; accelerator handles the arithmetic kernel.

End-to-end measurement protocol

Use the same profiling instrumentation from M04A03, now comparing two runs:

// Run 1: baseline (software only)
t0 = read_cycle();
conv2d_int8_software(input, weights_conv, act1, ...);
t1 = read_cycle();
uart_printf("Conv2D SW:  %u cycles\r\n", t1 - t0);

// Run 2: accelerated
t0 = read_cycle();
conv2d_int8_accel(input, weights_conv, act1, ...);
t1 = read_cycle();
uart_printf("Conv2D HW:  %u cycles\r\n", t1 - t0);

// Verify correctness
for (int i = 0; i < H_out * W_out * C_out; i++)
    assert(act1_sw[i] == act1_hw[i]);   // must match bit-for-bit
uart_printf("Outputs match: %s\r\n", match ? "YES" : "NO");

Expected results: accelerated vs. baseline

Reference numbers for a 4-wide int8 MAC array at 27 MHz with BRAM weight buffer:

The system speedup (~3.1×) is less than the layer speedup (~4×) because the non-accelerated layers (FC + overhead) remain unchanged — Amdahl's Law. The local BRAM weight buffer is what makes the Conv2D compute-bound; without it, the 32-bit bus would be the bottleneck and the speedup would collapse.

Exact numbers come from your measured baseline (M04A03) and the reference implementation.

Predict before you measure

Before running the accelerated system on the board, predict the cycle count using the timing analysis from M05A02:

Your inputs (from M04A03 baseline and M05A02 timing model):

Fill in these numbers before you run make load. Then measure. Then compare.

Likely sources of gap between predicted and measured:

• Bus overhead: each weight-load call from the CPU adds bus transactions.
• DMEM conflicts: CPU outer-loop loads conflict with accelerator's activation fetches.
• Requantization cycles: the 2-cycle writeback assumption may be optimistic.

"Predict then measure" is the scientific method applied to computer architecture. If your prediction is off by more than 20%, there is something in the hardware you did not model — find it before writing the report.

In-class exercise — gap analysis

A student measured the following after integrating the accelerator:

Conv2D SW:  3,200,000 cycles
Conv2D HW:    960,000 cycles   (predicted: 820,000)
FC SW:         185,000 cycles
FC HW:         185,000 cycles  (not accelerated)
Total SW:   3,501,000 cycles
Total HW:   1,261,000 cycles

Questions (10 min):

1. What is the measured Conv2D layer speedup? Is it close to the 4× theoretical maximum?
2. What is the measured system speedup? How does it compare to Amdahl's prediction?
3. The HW measurement (960K) is 17% higher than predicted (820K). List two specific causes from the hardware design that could explain this gap.
4. If the student ran the FC layer on the accelerator too (at the same efficiency), what would the new total HW cycle count be?

Expected: (1) 3200K/960K ≈ 3.3× (below 4× — due to weight-loading overhead + arbiter conflicts). (2) 3501K/1261K ≈ 2.8× system speedup. (3) Bus transactions for weight loading; DMEM arbiter stalls during CPU outer-loop access. (4) FC: 185K/3.3× ≈ 56K → total ≈ 1,016K + 56K = 1,072K → ~3.3× system speedup.

Debugging mismatches

If act3_sw[i] != act3_hw[i], debug systematically:

1. Single vector, known inputs: run the accelerator on one dot product whose result you computed by hand. If that fails, the MAC unit or accumulator is wrong.
2. Requantization: print the raw int32 accumulator from both SW and HW before requantization. If they match but the int8 output differs, the requantization logic is wrong (check multiplier, shift, saturation).
3. Zero-point: a common off-by-one — check that input zero points are subtracted before the MAC.
4. Waveforms: GTKWave the control FSM — does it enter COMPUTE at the right time? Does done assert?

The bit-exact match requirement has no tolerance. Even one misclassified keyword means the requantization is wrong.

Synthesis report: resources and timing

After integration, run make load on the full system:

Info: Device utilisation:
   SLICE:     1891/4608  (41%)   ← was 27% in Project 2
   LUT4:      3102/9216  (34%)
   DFF:        718/6480  (11%)
   BRAM:         6/26   (23%)   ← +2 for weight buffer + act buffer
   DSP:          5/20   (25%)   ← +4 for the 4-wide MAC array

Info: Max frequency: 31.4 MHz   ← meets 27 MHz with margin

Key checks:

• Timing closure: WNS must be ≥ 0 at 27 MHz. The MAC array's combinational path through the DSP may be the critical path — if timing fails, register the product output (add one pipeline stage).
• BRAM count: 6 total (IMEM + DMEM + weight BRAM + activation buffer) — well within 26 available.

Next class

Roofline, Arithmetic Intensity & Off-the-Shelf Alternatives: situating your measured speedup on the roofline model, understanding why 3.5× and not 7× system-wide, and comparing your design to the Arm Ethos-U55 — what production systems do that this course does not.