Alexa Junior: Real-Time Keyword Spotting on FPGA
Year Long Project
Diode
Abstract
This project presents the design and hardware deployment of a real-time, edge-native keyword detection system on the Xilinx Artix-7 FPGA (Nexys 4 DDR). Motivated by the latency and connectivity limitations of cloud-dependent voice assistants, the system implements a Convolutional Neural Network (CNN) capable of classifying 31 keyword classes from the Google Speech Commands dataset — entirely on-chip, with no floating-point unit and no internet connection.
Audio input is pre-processed into Mel-Spectrograms on the host and transmitted to the FPGA via UART at 115,200 baud. CNN weights, originally trained in 32-bit floating point using PyTorch/Keras, are quantized to 8-bit signed fixed-point using a custom fixedpoint8.v module, achieving a 4× memory compression with negligible accuracy loss. The inference pipeline is implemented in Verilog and comprises a MAC convolution engine (convunit.v), sliding window receptive field selector (rfselector.v), ReLU and max-pooling unit (relu_maxpool.v), dual-port BRAM feature buffers (feature_bram.v), and an argmax decoder for one-hot LED output. A four-state FSM (IDLE → LOAD → COMPUTE → DONE) governs end-to-end pipeline control.
The system achieves sub-millisecond inference latency, consuming 65% of available BRAM, 50% of DSP48 slices, and 29% of LUTs — leaving sufficient headroom for vocabulary expansion. Results confirm that 8-bit fixed-point CNNs deliver high-accuracy, real-time keyword classification within severe resource constraints, demonstrating the viability of fully edge-native AI inference on low-cost FPGAs.
Abstract
This project presents the design and hardware deployment of a real-time, edge-native keyword detection system on the Xilinx Artix-7 FPGA (Nexys 4 DDR). Motivated by the latency and connectivity limitations of cloud-dependent voice assistants, the system implements a Convolutional Neural Network (CNN) capable of classifying 31 keyword classes from the Google Speech Commands dataset — entirely on-chip, with no floating-point unit and no internet connection.
Audio input is pre-processed into Mel-Spectrograms on the host and transmitted to the FPGA via UART at 115,200 baud. CNN weights, originally trained in 32-bit floating point using PyTorch/Keras, are quantized to 8-bit signed fixed-point using a custom fixedpoint8.v module, achieving a 4× memory compression with negligible accuracy loss. The inference pipeline is implemented in Verilog and comprises a MAC convolution engine (convunit.v), sliding window receptive field selector (rfselector.v), ReLU and max-pooling unit (relu_maxpool.v), dual-port BRAM feature buffers (feature_bram.v), and an argmax decoder for one-hot LED output. A four-state FSM (IDLE → LOAD → COMPUTE → DONE) governs end-to-end pipeline control.
The system achieves sub-millisecond inference latency, consuming 65% of available BRAM, 50% of DSP48 slices, and 29% of LUTs — leaving sufficient headroom for vocabulary expansion. Results confirm that 8-bit fixed-point CNNs deliver high-accuracy, real-time keyword classification within severe resource constraints, demonstrating the viability of fully edge-native AI inference on low-cost FPGAs.
Report Information
Team Members
Team Members
Report Details
Created: April 8, 2026, 12:38 a.m.
Approved by: None
Approval date: None
Report Details
Created: April 8, 2026, 12:38 a.m.
Approved by: None
Approval date: None