Skip to content

Lab 6 — KWS Kernel: From Model to C

Goal

Get familiar with the hand-written int8 inference kernel you will run on your platform and accelerate in Project 3. This lab is a reading and running exercise — no hardware needed yet (that requires the UART and timer from Project 2, which you have, but the kernel can be built and tested on your PC first).

When to do this

This lab is released alongside the M04 classes on TinyML. Complete it before Lab 7 (profiling), which builds directly on the code you explore here. You already estimated the MAC count in Lab 1 — now you will see exactly how those MACs are computed.

The big picture

In Lab 1 you ran TFLite Micro's reference implementation on your PC. That implementation is ~100,000 lines of C++. The course's kernel does the same computation in ~300 lines of C, with no dependencies beyond <stdint.h>. This is what runs on your CPU.

The trade-off: TFLite Micro handles arbitrary models; the course kernel is hand-written for tiny_conv only. But it is small enough to read completely, which is the point.

File layout

1
2
3
4
5
6
7
kws/
  weights.h       — int8 weight/bias arrays + per-channel quantization params
  kws_model.h     — layer shapes as constants (kernel sizes, strides, channels)
  kws_kernel.c    — the inference kernel (conv_int8, fc_int8, kws_infer)
  kws_kernel.h    — public interface
  test_vectors.h  — 4 precomputed 49×40 int8 MFCC inputs + expected labels
  main.c          — glue: load input, run inference, print result

Step 1 — Read the public interface

Activity 1

Open kws_kernel.h. The top-level function is:

1
2
3
// input: KWS_INPUT_FRAMES * KWS_INPUT_BINS int8 MFCC values (row-major)
// returns: predicted class index 0..3 (yes=0, no=1, unknown=2, silence=3)
int kws_infer(const int8_t *input);

And the layer primitive:

1
2
3
4
5
6
7
8
9
void conv_int8(
    const int8_t  *input,   int in_h,  int in_w,  int in_c,
    const int8_t  *weights,
    const int32_t *bias,
    int8_t        *output,  int out_c,
    const int32_t *mult,    // per-output-channel multiplier
    const int     *shift,   // per-output-channel shift
    const int8_t  *zero_point  // per-output-channel output zero point
);

Answer the following (check kws_model.h for the shapes):

  • What are in_h, in_w, in_c for the Conv2D call?
  • What is out_c?
  • Why are mult, shift, and zero_point arrays rather than scalars?
Hint

mult, shift, and zero_point are arrays indexed by output channel because TFLite uses per-channel quantization: each output channel has its own scale factor, chosen to minimise quantization error for that channel's weight distribution. A single global scale (per-tensor quantization) would be less accurate.

Step 2 — Trace a single output element

Activity 2

Open kws_kernel.c and find the innermost loop of conv_int8. The computation for one output element output[oh][ow][oc] is:

1
2
3
4
5
6
7
int32_t acc = bias[oc];
for (int kh = 0; kh < KH; kh++)
  for (int kw = 0; kw < KW; kw++)
    for (int ic = 0; ic < in_c; ic++)
      acc += (int32_t)input[...][kh][kw][ic] * (int32_t)weights[oc][kh][kw][ic];
// Requantize acc → int8
output[oh][ow][oc] = requantize(acc, mult[oc], shift[oc], zero_point[oc]);

Trace this on paper for oc=0, oh=0, ow=0 using the actual values from weights.h (just the first kernel position: kh=0, kw=0, ic=0).

You don't need to compute the full result — just verify you understand the index arithmetic. What is weights[0][0][0][0]?

Step 3 — Count the MACs

Activity 3

You estimated the total MAC count in Lab 1. Now compute it precisely from kws_model.h:

Layer Formula MACs
Conv2D KH × KW × in_c × out_h × out_w × out_c ?
FC in_features × out_classes ?
Total ?

Fill in the table. Does it match your Lab 1 estimate?

Hint

Conv2D: KH=8, KW=20, in_c=1, out_h=42, out_w=21, out_c=8 → 8×20×1×42×21×8 = 451,584
FC: 1280 × 4 = 5,120
Total: 456,704 MACs

Step 4 — Build and run on your PC

Activity 4

The kernel is designed to compile natively (no hardware, no UART):

1
2
gcc -O2 -o kws_test kws_kernel.c main.c
./kws_test

Expected output: all 4 test vectors classified correctly (matching the labels in test_vectors.h). If any label mismatches, there is a bug in the kernel — check the requantize function first.

Use -O2 for the PC build

The native build uses -O2 so the PC reference runs fast. When you later compile for your RISC-V CPU, you will compare -O0 vs -O2 results — both must produce the same classification outputs (optimisation must not change correctness).

Step 5 — The requantize function

Activity 5

Find requantize() in kws_kernel.c. It converts an int32 accumulator back to int8:

1
2
3
4
5
6
int8_t requantize(int32_t acc, int32_t mult, int shift, int8_t zero_point) {
    int64_t scaled = ((int64_t)acc * mult) >> shift;
    int32_t q = (int32_t)scaled + zero_point;
    // Clamp to [0, 127] (fused ReLU) or [-128, 127] (no activation)
    return (int8_t)(q < -128 ? -128 : (q > 127 ? 127 : q));
}

Answer:

  1. Why is scaled a 64-bit value? What would happen if it were 32-bit?
  2. The Conv2D layer uses [0, 127] clamping. Why? (Hint: fused ReLU)
  3. The FC layer uses [-128, 127]. Why the difference?
Hint
  1. acc can be up to 8×20×1×127×127 ≈ 25M, and mult can be up to ~2^31. Their product overflows int32.
  2. ReLU sets all negative values to 0, so the output range is [0, 127] in int8 with the Conv2D's zero point.
  3. The FC produces logits (before softmax), which can be negative — no activation function is fused.

Step 6 — Sketch the accelerator interface

Activity 6

Project 3 asks you to accelerate conv_int8. Before writing any hardware, sketch on paper:

  1. Which loop level would you accelerate? (Output element? Row? Full layer?)
  2. What data needs to travel from CPU to accelerator per invocation?
  3. How would you use the memory-mapped register interface from Project 2?

No exact answer expected — this is your first design sketch. You will revisit it in Project 3 kickoff.

Checklist

  • kws_infer interface understood; layer shapes identified from kws_model.h.
  • MAC count computed precisely and matches Lab 1 estimate.
  • Kernel builds and runs natively; all 4 test vectors pass.
  • requantize function understood (overflow, fused ReLU, clamping range).
  • Accelerator interface sketch done (rough, on paper).

Summary

You read a complete, dependency-free int8 inference kernel and traced the computation from input through quantized output. The conv_int8 function — with its per-channel mult/shift/zero_point arrays — is the inner loop of the whole model and the target of Lab 7 and Project 3. The requantize step is where floating-point scale factors are emulated in integer arithmetic: understanding it is prerequisite for designing a correct accelerator.