Skip to content

Project 3 - Accelerating TinyML

This is the capstone project. You will run the Keyword Spotting inference kernel on the platform you built in Projects 1 and 2, profile it to find its bottleneck, and design a hardware accelerator for that bottleneck - integrated through the memory-mapped interface from Project 2.

Objectives

  1. Run the KWS int8 inference kernel (provided as C source + exported weights, see Running Example) end-to-end on your platform, using UART to load a precomputed MFCC feature vector and to report the classification result.
  2. Profile the inference using your Project 2 cycle counter and identify the dominant operation (expected: the Conv2D layer of the tiny_conv model, which accounts for the vast majority of MACs).
  3. Design a hardware accelerator for that operation: define its register/memory-mapped interface, datapath, and control.
  4. Implement, verify, and integrate the accelerator into your platform.
  5. Re-run the inference using the accelerator and measure the resulting speedup.

Specification

Baseline run and profiling

Using the cycle counter from Project 2, instrument the inference kernel to report cycles spent in each layer/operation. Identify the single operation that dominates total runtime - this is your acceleration target.

Accelerator design

Your accelerator should:

  • Implement the profiled bottleneck operation (e.g., an int8 MAC array for a convolution/matmul) in hardware.
  • Expose a memory-mapped interface consistent with the bus you designed in Project 2 (e.g., write operands/configuration to registers, trigger computation, poll/interrupt on completion, read results).
  • Be driven from software with minimal changes to the inference kernel (ideally: replace the call to the software implementation of the bottleneck operation with calls to your accelerator's driver).

The complexity of the accelerator is open - it does not need to accelerate the entire operation in one shot (e.g., a small systolic array or a parallel MAC unit processing a fixed-size tile per invocation is a perfectly valid scope), but the interface and the integration must be complete and working.

Optional extensions

If you want to go further:

  • Implement the MFCC feature-extraction front-end on-device (instead of precomputed features via UART).
  • Use the live microphone input on the Tang Nano 9K (if available) for real-time keyword spotting.
  • Accelerate a second operation, or generalize the accelerator to handle multiple layers.

Resource budget

After integrating the accelerator, re-run synthesis and check the Tang Nano 9K's utilization report (LUTs, FFs, BRAM). If your design no longer fits, the accelerator's tile size / buffer sizes are the parameters to scale down - see Design Space & Alternatives, §5.

Deliverables

  • Profiling report for the baseline (cycles per layer/operation).
  • Accelerator SystemVerilog source, with Verilator testbench(es).
  • Updated inference kernel / drivers using the accelerator.
  • Measurement of end-to-end speedup (baseline vs. accelerated), on real hardware.
  • An estimate of bytes moved across the bus per MAC performed by your accelerator, and a short discussion of whether your speedup is compute- or memory-bound - see Design Space & Alternatives, §3.
  • A short (max. 15 pages) report covering: the chosen bottleneck and why, the accelerator's design and interface, the integration, and the results obtained.
  • A short final presentation/demo (see Calendar, Week 15).

Submission

Same pair as Projects 1 and 2 (see Home). Submit via GitHub Classroom, including a link to your repository and any scripts used. Due: see Calendar.

Evaluation criteria

  1. Quality and correctness of the baseline profiling.
  2. Soundness of the accelerator's design and its interface to the rest of the system.
  3. Correct implementation and verification of the accelerator.
  4. Successful end-to-end integration and demonstration on the Tang Nano 9K.
  5. Measured speedup and the clarity of its analysis.
  6. Quality of the final report and presentation.