Design Space for the Accelerator

Rodolfo Azevedo

Institute of Computing, University of Campinas (UNICAMP), Brazil

rodolfo.azevedo@unicamp.br

http://www.ic.unicamp.br/~rodolfo/mo801

Goal of this class

Module 4, Class 4: interface options, datapath trade-offs, BRAM budgeting, and Project 3 kickoff.

Before writing a line of SystemVerilog, commit to a design point. This class is a structured decision process — interface first, datapath second, memory third — so that implementation in M05A02 is filling in a known spec, not exploring an open space.

At the end of this class, you should be able to:

• Select an accelerator interface style (memory-mapped, custom instruction, DMA) and justify the choice in terms of data volume and software overhead.
• Define the accelerator's register map, including control, status, and data registers.
• Determine the number of MAC units and BRAM allocation that fit within the GW1NR-9C resource budget.
• Produce the Project 3 design specification from the structured decisions made in this class.

Decision 1: interface style

How does the CPU activate the accelerator and retrieve results?

Rationale for memory-mapped: the accelerator needs to consume a 64-element input vector and a 64×64 weight matrix — far more data than two registers can hold. Memory-mapped lets the CPU write data into the accelerator's local BRAM incrementally, then issue a "go" command and poll for "done".

Accelerator register map

Extend the Project 2 memory map at 0x0002_0300:

The CPU writes INPUT_ADDR, WEIGHT_ADDR, LENGTH, asserts CONTROL[0], then polls STATUS[0]. When done=1, results are written to OUTPUT_ADDR in DMEM.

Decision 2: datapath width

How many int8 MACs per cycle?

Tang Nano 9K has 20 DSP blocks total. Project 2 uses 2 (Zmmul + CMAC). Budget for accelerator: up to 16 DSPs — a 16-wide SIMD fits on the device.

Recommended target for Project 3: 4-wide SIMD — manageable complexity, meaningful speedup (~4×), leaves DSPs for expansion.

Decision 3: memory architecture

Where do weights and activations live during computation?

Option A — fetch from DMEM every cycle:

• Simple: the accelerator reads directly from the Project 2 bus.
• Slow: 32-bit bus → 4 bytes/cycle → 4 int8 values/cycle. With 4 MACs/cycle, you need 8 bytes/cycle (4 activations + 4 weights). Bandwidth mismatch: the bus is the bottleneck.

Option B — local BRAM weight buffer (our choice):

• Before computation: CPU DMAs weight row into the accelerator's local BRAM via the bus (N writes of 32 bits).
• During computation: accelerator reads from its own BRAM (wide internal port, e.g. 128-bit) — 16 int8 values per cycle.
• Result: MAC array is compute-bound (not memory-bound) during the kernel.

BRAM cost: one 64×8-bit column of weights = 64 bytes. A 9K BRAM block holds 512 bytes. One BRAM block holds 8 weight rows, which the accelerator reuses across all spatial positions.

BRAM budget

The accelerator uses 2 BRAM blocks: one for weights, one optionally for a double-buffered activation window. Well within budget.

The full system after Project 3

The accelerator is a new peripheral on the existing Project 2 bus. No changes to the CPU, bus, or other peripherals — just a new address-decode entry and a new module.

Project 3 kickoff: deliverables

Goal: implement a 4-wide int8 MAC array accelerator, integrate it via the Project 2 memory-mapped bus, replace the software Conv2D inner loop with the hardware call, and measure the end-to-end speedup.

Grading: correctness (same classification output as baseline) + speedup measurement + roofline analysis. A working 1-wide scalar accelerator with correct results is sufficient for a pass.

• Security: identify one security vulnerability in your design and describe how you would mitigate it (see Security corner below).

The slides that follow cover security, energy, and reproducibility. The two security corners are required — integer overflow is a real implementation risk. The energy and reproducibility corners are optional enrichment.

Security corner: adversarial attacks on ML models

Your KWS classifier takes 1960 int8 values (49×40 MFCC features) and outputs one of 4 classes. An adversarial example is a carefully crafted input that looks (or sounds) normal to a human but causes the model to misclassify.

• In 2018, Carlini & Wagner demonstrated adversarial audio that is indistinguishable from silence to humans but is recognized as arbitrary speech by a speech recognition model.
• For KWS: an attacker could craft background noise that the model classifies as "yes" — triggering a voice-activated system without any human speaking.

Why int8 quantization matters for adversarial robustness

Quantization is not a defense — but it does change the attack surface. Adversarial robustness is an active research area.

Security corner: integer overflow in inference

The int8 MAC accumulates into an int32:

int32_t acc = 0;
for (int i = 0; i < n; i++)
    acc += (int32_t)input[i] * (int32_t)weight[i];  // max per term: 127 × 127 = 16129

• Worst case: all 127 × 127, accumulated over 1024 elements → 16,516,096. Well within int32 range (2³¹ ≈ 2.1 billion).
• But if the kernel is reused for larger layers (e.g., 16K channels), the accumulator could overflow silently in C (undefined behavior for signed int).

Mitigations

• Saturation arithmetic: clamp the accumulator to INT32_MAX / INT32_MIN after each addition.
• Wider accumulator: use int64_t for large channel counts (but costs more on RV32I — needs two registers).
• Static analysis: compute the maximum possible accumulator value at compile time given layer dimensions and quantization range.

For your KWS model, overflow is not a practical risk (max channel count is 64). But in production inference engines, this is a real bug class — TensorFlow Lite has had CVEs related to integer overflow in quantized ops.

Energy corner: cloud vs edge — the energy argument (Optional)

The strongest argument for edge AI is energy:

• The network cost dominates cloud inference energy — even if the server is 100× faster.
• Edge inference at 27 MHz is slow but extraordinarily energy-efficient.
• This is why TinyML exists: not because edge hardware is powerful, but because not sending data is the ultimate optimization.

Energy corner: accelerator energy efficiency

The accelerator changes the energy equation:

• Software (CPU-only): 9.7M cycles × ~50 mW ≈ 18 mJ per inference.
• With accelerator: 2.8M cycles × ~55 mW ≈ 5.7 mJ per inference (accelerator adds ~5 mW but saves 71% of cycles).

The key metric is inferences per joule — not inferences per second:

The accelerator gives 3.1× energy improvement — close to the 3.5× speedup but not identical, because the accelerator adds power even when idle.

Reproducibility note: quantization reproducibility (Optional)

Int8 quantization introduces a subtle reproducibility risk:

• The scale and zero_point values are computed during quantization of the trained model.
• If you re-quantize with a different calibration dataset, you get different scale/zero_point → different int8 weights → different inference results.
• Your kernel hard-codes these values — document which model checkpoint and quantization script produced them.

What to commit in your repository

kws/
├── model/
│   ├── tiny_conv_quantized.tflite  # frozen quantized model
│   ├── quantize.py                  # script that produced it
│   └── README.md                    # model version, dataset, accuracy
├── src/
│   ├── kws_weights.h                # exported int8 weights
│   └── kws_kernel.c                 # inference code
└── test/
    ├── test_vectors/                # known input → expected output
    └── test_inference.c             # end-to-end correctness test

If a classmate cannot reproduce your inference result by cloning your repo and running make test, something is missing.

Next module

Module 5 — AI Accelerators: implementing the 4-wide int8 MAC array in SystemVerilog, verifying it against the C kernel, integrating via the bus, measuring end-to-end speedup, and situating your design on the roofline.