Skip to content

Lab 7 — Profiling: Finding the Bottleneck in the KWS Kernel

Goal

Instrument the KWS inference kernel with hardware cycle counters, identify which layer dominates runtime, and produce a profiling report that will motivate and guide the accelerator design in Project 3.

Prerequisites

  • Project 2 working: your CPU runs C code with UART output and the rdcycle CSR.
  • Lab 6 complete: you understand conv_int8 and kws_infer.

Short deadline

This lab is released at Class 21 and due at Class 22 — one class later. It is intentionally short: the instrumentation is 10–20 lines of C, and the output directly feeds into Project 3.

The profiling pattern

Profiling on bare-metal hardware is simple: read the cycle counter before and after each region of interest.

1
2
3
4
uint64_t t0 = read_cycle();
do_something();
uint64_t t1 = read_cycle();
uint32_t elapsed = (uint32_t)(t1 - t0);

You already have read_cycle() from Lab 5. The only new step is deciding where to insert the measurements.

Step 1 — Instrument each layer

Activity 1

Modify kws_kernel.c to record cycle counts for each layer. Add timing variables and read_cycle() calls around each conv_int8 and fc_int8 invocation inside kws_infer:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
uint32_t cycles_conv, cycles_fc;

int kws_infer(const int8_t *input) {
    uint64_t t0, t1;

    t0 = read_cycle();
    conv_int8(input, ...);   // Conv2D layer
    t1 = read_cycle();
    cycles_conv = (uint32_t)(t1 - t0);

    t0 = read_cycle();
    fc_int8(...);            // FC layer
    t1 = read_cycle();
    cycles_fc = (uint32_t)(t1 - t0);

    return argmax(...);
}

Expose cycles_conv and cycles_fc as extern variables so main.c can print them after inference.

Step 2 — Run and record

Activity 2

Compile with -O2 and run on your board. Print the cycle counts for each test vector:

1
2
3
4
Vector 0 (yes):      conv=428314  fc=4891  total=433205
Vector 1 (no):       conv=427998  fc=4887  total=432885
Vector 2 (unknown):  conv=428102  fc=4903  total=433005
Vector 3 (silence):  conv=428201  fc=4896  total=433097

Fill in the actual numbers from your board. Then compute:

Layer Cycles (avg) % of total
Conv2D ? ?
FC ? ?
Other (overhead) ? ?
Hint — expected proportions

Conv2D should dominate: ~451K MACs vs ~5K for FC. Expect Conv2D to take 95–99% of total inference time. The exact cycle count depends on your CPU's multicycle timing (how many cycles per mul, lb, etc.).

Step 3 — Compute cycles per MAC

Activity 3

Divide the Conv2D cycle count by its MAC count (451,584) to get cycles per MAC:

$$\text{cycles/MAC} = \frac{\text{conv_cycles}}{451584}$$

This number tells you how efficiently your CPU executes the inner loop. Compare it to the theoretical minimum:

  • 1 cycle/MAC would require the multiply, accumulate, load, and store all in one cycle — impossible in a multicycle design.
  • What is the realistic minimum for your multicycle CPU? (Count the instructions in the inner loop from Lab 5.)

Step 4 — Project 3 target

Activity 4

Your accelerator in Project 3 should achieve a measured speedup. Compute the targets:

Target Formula Value
Current total inference (cycles) measured ?
Theoretical min with 4-wide MAC total_MACs / 4 ~114K cycles
Theoretical speedup current / (total_MACs/4) ?

Your Project 3 accelerator is a 4-wide int8 MAC array. If it is not memory-bound, it should approach the theoretical speedup. The roofline analysis in Module 5 will tell you where the real ceiling is.

Correctness first

Before measuring speedup, verify that the accelerated kernel still produces the same classifications as the software reference. The cycle count is meaningless if the output is wrong.

Step 5 — Write the profiling report

Activity 5

Write a short report (profiling_report.md, ~half a page) covering:

  1. Cycle counts per layer (table from Activity 2)
  2. Cycles per MAC for the Conv2D layer
  3. Which layer you will target in Project 3, and why
  4. Your predicted speedup (with the 4-wide MAC accelerator)
  5. One potential reason the actual speedup might be lower than predicted

This report is the starting point of the Project 3 design document.

Checklist

  • kws_kernel.c instrumented with rdcycle around each layer.
  • Cycle counts measured for all 4 test vectors; table filled in.
  • Cycles-per-MAC computed for Conv2D.
  • Project 3 speedup target computed.
  • profiling_report.md written.

Summary

You confirmed empirically what the MAC count predicted analytically: Conv2D dominates, and the bottleneck is clear. The cycles-per-MAC number is your baseline — everything Project 3 does is measured against it. When your accelerator runs the same Conv2D in a fraction of the cycles, you will have closed the loop from Lab 1 (running KWS on a PC) to the end of the course (running it faster on hardware you designed).