Skip to content

Running Example: Keyword Spotting on Your Own CPU

Three problems at the edge

In Week 1, we look at three classic TinyML problems to get a feel for the kind of workloads that run on tiny, battery-powered devices:

  1. Keyword Spotting (KWS) - "Is someone saying a wake word ('yes'/'no'/...) right now?" A small neural network classifies short audio clips, typically after converting them to MFCC (Mel-Frequency Cepstral Coefficient) features.
  2. MNIST digit classification - a small CNN classifies 28x28 handwritten digit images. Conceptually simple, no feature-extraction front-end, a classic first example for "AI" in a hardware course.
  3. Anomaly / sensor-data detection - a small autoencoder or MLP flags unusual patterns in accelerometer/IMU data, common in predictive-maintenance applications.

All three share the same shape: a small int8 neural network whose dominant cost is matrix-multiply / convolution (multiply-accumulate, MAC) operations - exactly the kind of computation that benefits from a hardware accelerator.

Why Keyword Spotting

From Week 3 onward, the course converges on Keyword Spotting as its single running example, based on the TFLite Micro micro_speech reference model: a small convolutional network (tiny_conv) over MFCC features, quantized to int8, with a model footprint of ~19 KB.

This gives the course a concrete, end-to-end narrative:

"By the end of the semester, your own RISC-V CPU plus a custom accelerator you designed will classify spoken keywords faster than your CPU alone could."

The model in detail

  • Dataset: Google Speech Commands v2, restricted to a 4-class subset: "yes", "no", _unknown_ (other words), and _silence_.
  • Front-end (computed on the PC, see below): each 1-second audio clip is converted into a 49 x 40 int8 MFCC spectrogram (49 time frames x 40 mel-frequency bins = 1960 int8 values).
  • Model: a small convolutional network (tiny_conv): one Conv2D layer followed by a fully-connected classifier and softmax. Quantized model file: ~19 KB. This is the actual architecture of the TFLite Micro micro_speech example — not a depthwise-separable CNN.
  • Output: a 4-way softmax; on-device, this reduces to an argmax over 4 int8/int32 scores.

This gives every pair a fixed, well-defined contract for the inference kernel: input = 1960 int8 values, output = 1 of 4 labels, with a known, fixed set of weight tensors (one per conv/FC layer) that get exported as C arrays.

Pipeline and scope

flowchart LR
    subgraph PC["On a PC (Week 10)"]
        T["Train KWS model\n(TFLite Micro)"] --> Q["Quantize to int8\n& export weights as C arrays"]
        Q --> F["Pre-compute MFCC features\nfor a handful of test clips"]
    end
    subgraph BOARD["On the Tang Nano 9K (Weeks 7-15)"]
        U["Pre-extracted MFCC\nfeatures via UART"] --> K["Minimal int8 inference kernel\n(conv + fully-connected layers)"]
        K --> R["yes / no / unknown / silence"]
    end
    F -.exported once.-> U

To keep the focus on hardware/software codesign rather than on porting a large ML framework, the course makes three deliberate scoping decisions:

  1. Training and export happen on a PC, using the real TFLite Micro tooling and a real (small) speech-commands dataset. Students see the official, full workflow once.
  2. On the students' own CPU, we do not run the full TFLite Micro C++ runtime. Instead, the exported int8 weights become plain C arrays, paired with a small, dependency-free inference kernel (a handful of loops implementing the Conv2D and fully-connected layers). This avoids needing a C++ toolchain / libc port for a from-scratch RV32I core, while keeping the model, weights, and arithmetic completely real.
  3. MFCC feature extraction is precomputed on the PC and loaded onto the board as input data (via UART, see Project 2). Implementing the MFCC front-end on-device is offered as an optional extension for students who want to go further in Project 3.

How each project uses this example

Project Role of the running example
Project 1 None yet - focus is on getting a correct, working RV32I core on the Tang Nano 9K.
Project 2 The UART peripheral you build is used to load MFCC feature vectors onto the board; the Zmmul and CMAC instructions you add are what the inference kernel's MAC operations will use.
TinyML checkpoint (Week 10-11) You run the inference kernel on your own CPU end-to-end, get a classification result, and profile where the cycles go.
Project 3 You design, implement, and integrate a hardware accelerator for the operation your profile identified as the bottleneck (almost certainly the int8 conv / matmul), and measure the resulting speedup.