# FPGA Labs

An FPGA is a flexible chip whose internal logic can be configured to realize
the digital circuit we want. Unlike an ASIC, which is manufactured for one
fixed design, an FPGA lets us write a circuit in SystemVerilog, load it onto the
chip, test it, and replace it with another circuit later.

For hands-on experience, we highly recommend buying the inexpensive Tang Nano
20K from either [Amazon](https://www.amazon.com/Tang-Nano-20K-Development-Computer/dp/B0GCVFLFPP/ref=sr_1_1?crid=32SUSOZGEPZC6&dib=eyJ2IjoiMSJ9.Ow-0YEuarWedIbDbBtOwJv4xyhVW5_qqUfOOYW4fjGJ99bRBUHdab_BTUgz_6cyVtW1qZHPo8yTWj7sGpRE0HKkyiMDAd1MSCc4Ea5OlgFsarB_M8y7Nu8sm-REsz0zofY8SMuVfBaJi9QecvRpHNlEv532AEdds7yn9hJ7QXQg.ZnVmNGdJX19GAopy9VviKF0bf9yAC0TmHz8vlWZ4xGQ&dib_tag=se&keywords=speed%2Btang%2Bnano%2B20k&qid=1779923800&sprefix=sipeed%2Btang%2Bnano%2B20k%2B%2Caps%2C286&sr=8-1&th=1),
or [AliExpress](https://www.aliexpress.us/item/3256805394833478.html?spm=a2g0o.productlist.main.1.4b04HoNAHoNAIF&algo_pvid=809e9b1f-24a1-4c4b-b135-129d55ab0ff9&algo_exp_id=809e9b1f-24a1-4c4b-b135-129d55ab0ff9-0&pdp_ext_f=%7B%22order%22%3A%22621%22%2C%22eval%22%3A%221%22%2C%22fromPage%22%3A%22search%22%7D&pdp_npi=6%40dis%21USD%2132.39%2131.89%21%21%2132.39%2131.89%21%402103110517799236779054890ef451%2112000033650315249%21sea%21US%210%21ABX%211%210%21n_tag%3A-29910%3Bd%3A4ca8c57d%3Bm03_new_user%3A-29895%3BpisId%3A5000000204886261&curPageLogUid=zXmtXJ517f75&utparam-url=scene%3Asearch%7Cquery_from%3A%7Cx_object_id%3A1005005581148230%7C_p_origin_prod%3A).
On AliExpress, carefully choose **Bundle: Nano 20K No header**, and
triple-check the delivery address before checkout.

## Overview

We will turn our simulated SystemVerilog circuits into configurations that run
on a real FPGA. The board is described in the official [Tang Nano 20K
datasheet](https://dl.sipeed.com/fileList/TANG/Nano_20K/1_Datasheet/Sipeed%20Tang%20nano%2020K%20Datasheet%20V1.3-en_US.pdf).
It uses a Gowin `GW2AR-18` FPGA; the [board overview](https://wiki.sipeed.com/tangnano20k)
has additional hardware details.
The general FPGA flow is as follows:

1. **Design the circuit in SystemVerilog.** See our [SystemVerilog RTL](https://github.com/abarajithan11/digital-design/tree/main/material/rtl).
2. **Test it in simulation** using a [SystemVerilog testbench](https://github.com/abarajithan11/digital-design/tree/main/material/tb):
   `make sim DESIGN=<design>`.
3. **Translate it into a bitstream** (the configuration loaded onto the FPGA) using
   our [`fpga.mk` build flow](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/common/fpga.mk):
   `make bitstream DESIGN=<design>`.
4. **Program the FPGA** with [openFPGALoader
   Web](https://ofl.trabucayre.com/).
5. **Interact with the circuit on the FPGA:** press its buttons, observe its
   LEDs, or send and receive data from your computer.

### Module Heirarchy

We have built the following module hierarchy to help you quickly implement your own designs in the FPGA.

* **Your SystemVerilog Design**: e.g. [`full_adder`](https://github.com/abarajithan11/digital-design/blob/main/material/rtl/reference/full_adder.sv)
* **An idealized FPGA interface**: [`board_glue`](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/reference/full_adder.sv)
   * Check out the [skeleton of `board_glue` here](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/_skeleton.sv).
   * Each of your designs (e.g. `full_adder`) need to be wrapped in a module named `board_glue` in a file named `full_adder.f` (note that file and module names are different), placed in `material/fpga/tang_nano_20k/top_glue/*/full_adder.f`.
   * The inputs and outputs of `board_glue` represent an ideal FPGA: active high LEDs, clean buttons...etc.
* **Topmost module**: [`board_top.sv`](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/common/board_top.sv)
   * You should not change this file.
   * This module instantiates `board_glue` module. Since the `board_glue`s of all designs have the same interface, this module can accept any of them. When you do `make bitstream DESGIN=full_adder` [`fpga.mk`](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/common/fpga.mk) selects the `board glue` in `top_glue/reference/full_adder.sv`.
   * This module abstracts away the real world behaviors of an FPGA to present an idealized interface to `board_glue`. Some of these abstractions are:
      - Control reset signal when powering on

      ```{raw} html
      <details>
      <summary><strong>Active-high LEDs</strong></summary>
      ```

      The LED lights on the FPGA are electronically wired to be active low. They light up when `LED[i]=0`. This module inverts that into active high, so they light up when you do `LED[i]=1` inside the `board_glue`.

      ```{raw} html
      </details>
      ```

      ```{raw} html
      <details>
      <summary><strong>Button synchronization and debouncing</strong></summary>
      ```

      When you press a button, mechanically the contacts touch and break several times due to vibrations. This causes the `btn[i]` input to oscillate from `0` to `1` and back many times before settling on `btn[i]=1`. Debouncing logic in `board_glue` observes it for a long time, and only gives `btn[i]=1` to the `board_glue` after everything settles.

      ```{raw} html
      </details>
      ```

      ```{raw} html
      <details>
      <summary><strong>UART input synchronization</strong></summary>
      ```

      The `RX` input of UART is not synchronized to any clock (it is in the name). This causes weird behavior if you try to sample it at a clock edge. `board_top` synchronizes it to the clock by registering it twice and gives to the `board_glue`.

      ```{raw} html
      </details>
      ```

      ```{raw} html
      <details>
      <summary><strong>GPIO</strong></summary>
      ```

      Most General Purpose Input Output (GPIO) pins of the FPGA can act as either input or output, and are hence declared as `inout` ports. Dealing with `inout` ports is error-prone, since driving the same inout port from the FPGA (output) and from the outside (input) at the same time can lead to short circuit. `board_top` converts that into `input`, `output` and `output_enable` ports, which helps you avoid human error.

      ```{raw} html
      </details>
      ```

### Environment Setup

The examples use two separate environments:

- Build `.fs` bitstreams and train the neural network inside the course Docker
  container.
- Run the Python UART, audio, and camera tools **outside the container** in a
  Conda environment.

Complete the [Docker setup](https://github.com/abarajithan11/digital-design/)
first. If you have not cloned the repository yet, run:

```bash
git clone https://github.com/abarajithan11/digital-design
cd digital-design
```

All commands below assume that your terminal is in this `digital-design`
directory.

## 1. Full Adder on the FPGA

During discussion, we put the `up_counter` on the FPGA. 
The `full_adder` follows the same flow.
If you have done the `up_counter`, you may skip this.

### 1.1 Design the Circuit

### 1.2 Test in Simulation

Run the
[`tb_full_adder.sv`](https://github.com/abarajithan11/digital-design/blob/main/material/tb/reference/tb_full_adder.sv)
testbench inside the container:

```bash
make enter
make sim DESIGN=full_adder
exit
```

### 1.3 Translate to a Bitstream

Build the bitstream (= FPGA configuration) inside the container:

```bash
make enter
make bitstream DESIGN=full_adder
exit
```

This invokes the commands in `fpga.mk`, which read [`full_adder.f`](https://github.com/abarajithan11/digital-design/blob/main/material/designs/reference/full_adder.f),
removes simulation-only sources, adds [`board_glue`](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/reference/full_adder.sv) and the fixed
[`board_top.sv`](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/common/board_top.sv),
then runs the open-source Apicula flow (`yosys → nextpnr-himbaechel → gowin_pack`) to generate `material/fpga/tang_nano_20k/build/full_adder/full_adder.fs`.

(program-the-fpga)=
### 1.4 Program the FPGA and Interact with your design

```{raw} html
<details>
<summary><strong>Windows 11: one-time WebUSB setup</strong></summary>
```

There are two interfaces on the board: interface 0/A is JTAG programming, and
interface 1/B is UART. Keep them separate during the setup.

1. Download and run [Zadig](https://zadig.akeo.ie/). It is portable and does
   not need to be installed.
2. Connect the Tang Nano 20K to Windows.
3. In Zadig, select **Options → List All Devices**.
4. Select **Dual RS232-HS / USB Debugger (Interface 0)** or **Interface A**.
5. Select **WinUSB**, then click **Replace Driver**.

Do not replace the driver for interface 1/B; that is the UART used by the
Python scripts.

```{raw} html
</details>
```

#### To program a bitstream:

![FPGA Programming](https://media.abapages.com/course-site/fpga_load.png)

1. Open [openFPGALoader Web](https://ofl.trabucayre.com/) in a **Chrome-based browser**. Firefox does not support WebUSB.
2. Go to the section **Automatic Operations**.
3. Leave the cable as default, and select **Tang Nano 20K** as the board.
4. Select **SRAM** for a temporary configuration or **Flash** to keep the configuration after power is removed.
5. Choose the design's `.fs` file. For this example, use `material/fpga/tang_nano_20k/build/full_adder/full_adder.fs`.
6. Click **Program FPGA**. 
7. The first time you do this, it opens this dialog box. Select "USB Debugger" and click "Connect". Note, Firefox does not support WebUSB.
8. A successful run ends with `Execution completed`.

Press **S1** and **S2** to change the full adder's two inputs. LED0 shows the
sum and LED1 shows the carry output.

You can try other simple designs in exactly the same way:

- `and_gate`: [RTL](https://github.com/abarajithan11/digital-design/blob/main/material/rtl/reference/and_gate.sv) · [board_glue](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/reference/and_gate.sv)
- `not_gate`: [RTL](https://github.com/abarajithan11/digital-design/blob/main/material/rtl/reference/not_gate.sv) · [board_glue](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/reference/not_gate.sv)
- `xor_gate`: [RTL](https://github.com/abarajithan11/digital-design/blob/main/material/rtl/reference/xor_gate.sv) · [board_glue](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/reference/xor_gate.sv)
- `mux`: [RTL](https://github.com/abarajithan11/digital-design/blob/main/material/rtl/reference/mux.sv) · [board_glue](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/reference/mux.sv)
- `decoder`: [RTL](https://github.com/abarajithan11/digital-design/blob/main/material/rtl/reference/decoder.sv) · [board_glue](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/reference/decoder.sv)

You can wrap your own module in the same way: add its `.f` file and copy
[`top_glue/_skeleton.sv`](https://github.com/abarajithan11/digital-design/blob/main/material/fpga/tang_nano_20k/top_glue/_skeleton.sv)
to create its matching `board_glue`.

The reusable FPGA files are organized as follows:

```text
material/fpga/tang_nano_20k/
  common/board_top.sv                    fixed board wrapper
  common/board.cst                       fixed pin constraints
  common/fpga.mk                         bitstream build rules
  top_glue/{cpu,reference,system}/       one board_glue per design
  build/<design>/                        generated files; not committed
material/designs/{cpu,reference,systems}/<design>.f
```

`FPGA` defaults to `tang_nano_20k`. A design can be built when it has both a matching `.f` source list and `board_glue`.
The build ignores `tb_*` and `vip_*`
simulation sources. From inside the container, `make bitstream_all` builds every design that has both files.

## 2. UART Serial: Talking to your computer

We will now implement more advanced designs on the FPGA and communicate with them
from the computer. We use UART (**Universal Asynchronous Receiver/Transmitter**),
a simple serial protocol supported by almost every system. UART is easy to use
but relatively slow. We will cover the protocol and its circuits in the
lectures.

```{raw} html
<details>
<summary><strong>Computer and board UART details</strong></summary>
```

On the computer, Python can send and receive UART data through the
[`pyserial` library](https://pyserial.readthedocs.io/en/latest/). Later examples
also need numerical and audio libraries, so we install everything together in
a Conda environment.

The board's USB-to-UART bridge operates at **2 Mbaud**, has a **32-byte buffer**, and has no hardware flow control.
At the board's 54 MHz system clock,
the UART circuits use `CLKS_PER_BIT=27`.
The echo and FIR designs use a two-entry
[`skid_buffer`](https://github.com/abarajithan11/digital-design/blob/main/material/rtl/reference/skid_buffer.sv),
while the host scripts transfer at most 32 bytes at a time and drain replies so
the bridge does not silently drop data.

```{raw} html
</details>
```

### 2.1 Install Miniconda

Open the instructions for your operating system. If Conda is already installed
and `conda --version` works, skip to the next section.

```{raw} html
<details>
<summary><strong>Ubuntu</strong></summary>
```

1. Follow the official [Miniconda Linux installation
   guide](https://www.anaconda.com/docs/getting-started/miniconda/install/linux-install).
   Download the installer that matches your processor.
2. Accept the default install location. When asked whether to initialize
   Conda, answer `yes`.
3. Close and reopen the terminal, then check the installation:

   ```bash
   conda --version
   ```

```{raw} html
</details>
```

```{raw} html
<details>
<summary><strong>macOS</strong></summary>
```

1. Follow the official [Miniconda macOS installation
   guide](https://www.anaconda.com/docs/getting-started/miniconda/install/mac-cli-install).
   Choose the installer that matches your Mac's processor.
2. When asked whether to initialize Conda, answer `yes`.
3. Close and reopen Terminal, then check the installation:

   ```bash
   conda --version
   ```

```{raw} html
</details>
```

```{raw} html
<details>
<summary><strong>Windows 11</strong></summary>
```

Install and run Conda directly on Windows. WSL is still used for the course
Docker container, but it is not used for the Python environment or UART
scripts.

1. Follow the official [Miniconda Windows installation
   guide](https://www.anaconda.com/docs/getting-started/miniconda/install/windows-gui-install).
2. Choose **Just Me**, keep the default install location, and do not select
   **Add Miniconda3 to my PATH environment variable**.
3. Find **Anaconda Prompt (Miniconda3)** in the Start menu, right-click it, and
   select **Run as administrator**. Check the installation:

   ```powershell
   conda --version
   ```

Use the Administrator prompt when creating or updating the environment. After
that, use a regular Anaconda Prompt to activate the environment and run Python
scripts. Change to the Windows location of your `digital-design` checkout
first.

```{raw} html
</details>
```

### 2.2 Create the Environment

On Windows, run the `conda env create` command below from the Administrator
Anaconda Prompt opened in the previous section. Ubuntu and macOS users should
use their regular terminal.

```bash
conda env create -f python-setup/tang-basic.yml
conda activate tang-basic
```

You only need to create an environment once. You do need to activate it in each
new terminal before running a lab script.

Check that the basic packages and the expected Python version are available:

```bash
python --version
python -c "import numpy, scipy, serial; print('FPGA Python environment is ready')"
```

The version should be Python 3.11, and the second command should print the
ready message without an error.

### 2.3 Run the UART Echo Test

This test checks if your computer can talk to your hardware in the FPGA.

1. Simulate `uart_echo`, then build its bitstream inside the Docker container:

   ```bash
   make enter
   make sim DESIGN=uart_echo
   make bitstream DESIGN=uart_echo
   exit
   ```

2. [In Chrome](https://ofl.trabucayre.com/), program
   `material/fpga/tang_nano_20k/build/uart_echo/uart_echo.fs`.
3. Activate the basic environment and run the loopback test:

   ```bash
   conda activate tang-basic
   python material/py/fpga_uart_echo.py
   ```

   If the Python script cannot find or open the board, see [UART access
   troubleshooting](#uart-access-troubleshooting).

The test should finish with:

```text
PASS: echoed 4096 bytes with no loss.
```

## 4. FIR Filter

The FIR filter examples send audio samples to `sys_fir_filter` over UART and
receive the filtered samples back. Start with a saved audio file, then try the
same circuit with live microphone audio.

### 4.1 Offline File Test

This test sends the included audio file through the FPGA and compares every
output sample with the reference file. The script reads (or downloads)
`material/data/chill_sub.wav`, writes `material/data/fpga_out.wav`, and compares
it with `material/data/bass_only_8bit.wav`. These paths are resolved relative
to the script, so it can also be invoked from another directory.

1. Simulate `sys_fir_filter`, then build its bitstream inside the Docker
   container. The simulation generates the files needed by the example:

   ```bash
   make enter
   make sim DESIGN=sys_fir_filter
   make bitstream DESIGN=sys_fir_filter
   exit
   ```

2. [In Chrome](https://ofl.trabucayre.com/), program
   `material/fpga/tang_nano_20k/build/sys_fir_filter/sys_fir_filter.fs`.
3. Activate the basic environment and process the included WAV file:

   ```bash
   conda activate tang-basic
   python material/py/fpga_fir_offline.py
   ```

   If the Python script cannot find or open the board, see [UART access
   troubleshooting](#uart-access-troubleshooting).

The test should finish with a message similar to:

```text
PASS: all 735000 samples match .../material/data/bass_only_8bit.wav.
```

Both scripts detect the Tang Nano UART port automatically. If more than one
serial port is connected, select it explicitly:

```bash
python material/py/fpga_fir_offline.py --port /dev/ttyUSB1
```

Use the path printed by your system; on macOS it will usually begin with
`/dev/cu.usbserial-`, and on Windows it will look like `COM5`.

### 4.2 Live Audio Test

This example records your microphone, sends the audio through the FPGA, and
plays the filtered result through your selected output device. You should hear
mostly bass because the FPGA is running a low-pass filter.

1. Simulate `sys_fir_filter`, then build its bitstream inside the Docker
   container. The simulation generates the files needed by the example:

   ```bash
   make enter
   make sim DESIGN=sys_fir_filter
   make bitstream DESIGN=sys_fir_filter
   exit
   ```

2. [In Chrome](https://ofl.trabucayre.com/), program
   `material/fpga/tang_nano_20k/build/sys_fir_filter/sys_fir_filter.fs`.
3. Connect headphones and start at a low volume. Using speakers near the
   microphone can create loud feedback.
4. Activate the basic environment and list the available audio devices:

   ```bash
   conda activate tang-basic
   python material/py/fpga_fir_live_audio.py --list
   ```

5. If your default microphone and output are correct, start the live filter:

   ```bash
   python material/py/fpga_fir_live_audio.py
   ```

   If the Python script cannot find or open the board, see [UART access
   troubleshooting](#uart-access-troubleshooting).

   Press **Ctrl+C** to stop.

To select different devices, pass the number shown by `--list` or a unique part
of each device name:

```bash
python material/py/fpga_fir_live_audio.py --input 2 --output 5
```

The UART port can be selected in the same command when automatic detection is
not possible. For example, on native Windows:

```powershell
python material/py/fpga_fir_live_audio.py --port COM5 --input 2 --output 5
```

```{raw} html
<details>
<summary><strong>Live-audio troubleshooting by operating system</strong></summary>
```

**Ubuntu:** Select the intended microphone and headphones in [**Settings →
Sound**](https://help.ubuntu.com/stable/ubuntu-help/sound-usemic.html.en). If
`--list` reports that PortAudio sees no devices, install the system audio
support and try again:

```bash
sudo apt update
sudo apt install libportaudio2 libasound2-plugins
```

If the Conda interpreter still cannot see the devices, install the Ubuntu
Python audio packages and run only the live-audio script with the system
interpreter:

```bash
sudo apt install python3-numpy python3-serial python3-sounddevice
/usr/bin/python3 material/py/fpga_fir_live_audio.py --list
/usr/bin/python3 material/py/fpga_fir_live_audio.py
```

**macOS:** The first run may request microphone access. If it was denied, open
[**System Settings → Privacy & Security →
Microphone**](https://support.apple.com/guide/mac-help/control-access-to-the-microphone-on-mac-mchla1b1e1fe/mac)
and allow access for the terminal application you are using. Run the `--list`
command again after changing the permission.

**Windows 11:** Open [**Settings → Privacy & security →
Microphone**](https://support.microsoft.com/en-us/windows/privacy/turn-on-app-permissions-for-your-microphone-in-windows)
and enable **Microphone access** and **Let desktop apps access your
microphone**. Select the intended microphone and headphones in **Settings →
System → Sound**, then run the `--list` command again.

```{raw} html
</details>
```

## 5. CPU

Check out our small CPU designed in 40 lines of SystemVerilog [here](cpu.md).
This example loads a small program into the CPU on the FPGA over UART, runs it,
and sends the data memory back to the computer. 

```py
program = [
   [LOAD,  2, 0x02],       # r2 (counter) = dmem[2] = 10
   [LOAD,  1, 0x01],       # r1 (one)     = dmem[1] = 1
   [LOAD,  0, 0x00],       # r0 (sum)     = dmem[0] = 0
   [ADD,   0, 0, 2],       # r0 += r2
   [SUB,   2, 2, 1],       # r2 -= r1
   [JNZ,   2, 0x03],       # loop to imem[3] while r2 != 0
   [STORE, 0, WATCH_ADDR], # dmem[WATCH_ADDR] = r0  (= 55)
]
```

The program computes `1 + ... + 10` and stores the result in `dmem[4]`. You may write your own program in the assembly specified here.

1. Simulate `cpu_fpga`, then build its bitstream inside the Docker container:

   ```bash
   make enter
   make sim DESIGN=cpu_fpga
   make bitstream DESIGN=cpu_fpga
   exit
   ```

2. [In Chrome](https://ofl.trabucayre.com/), program
   `material/fpga/tang_nano_20k/build/cpu_fpga/cpu_fpga.fs`.
3. Activate the basic environment and load the example program:

   ```bash
   conda activate tang-basic
   python material/py/fpga_program_cpu.py
   ```

   If the Python script cannot find or open the board, see [UART access
   troubleshooting](#uart-access-troubleshooting).

4. Press **S1** when prompted. The CPU runs at about one instruction per
   second, and the LEDs show the opcode. Hold **S2** to show the low six bits of
   `dmem[4]` instead.

When the run finishes, the script prints the returned data memory. The final
line should be:

```text
dmem[4] = 55  (sum(1..10) should be 55)
```

## 6. Neural-Network Accelerator

Assignment 4 combines the UART and neural-network blocks into `sys_nn`. 

```{raw} html
<details>
<summary><strong>The complete path taken by the data</strong></summary>
```

- Handwritten image on PC / from webcam
   - Python (fpga_nn.py) 
   - preprocessing 
   - pyserial
   - USB
- FPGA
   - USB-to-UART bridge 
   - UART RX 
   - dense 1 
   - quant_ReLU 1 
   - dense 2 
   - output qReLU 
   - UART TX 
   - USB-to-UART bridge 
- Your PC
   - USB
   - Python: pyserial → prediction

```{raw} html
</details>
```

The network maps 81 quantized pixels through layers of size `81 → 48 → 10`.
The second qReLU has `RELU=0`, so it requantizes the ten outputs without
discarding negative scores. UART RX receives one 41-byte packed image; UART TX
returns the ten signed 4-bit scores in five bytes. Python unpacks the scores
and predicts the digit with the index of the largest score.

Both host tools: 
[`fpga_nn.py`](https://github.com/abarajithan11/digital-design/blob/main/material/py/fpga_nn.py)
and
[`fpga_nn_camera.py`](https://github.com/abarajithan11/digital-design/blob/main/material/py/fpga_nn_camera.py), crop,
downsample, and quantize the image to the same 9×9 input used to train the
model.

### 6.1 Optional: Train the Neural Network

The trained weights are already provided, so you do not need to train the
network to test the accelerator.
If you want to train it from scratch, run this
from the `digital-design` repository root:

```bash
make enter
python3 py/nn_model.py
exit
```

The course container already includes PyTorch, Torchvision, and Brevitas. The
first run downloads MNIST and trains the model; later runs reuse its checkpoint.

### 6.2 Build and Program `sys_nn`

Complete Assignment 4 through `sys_nn`. From the root of your Assignment 4
checkout, enter the course container:

```bash
make enter
```

Simulate the complete system and build its bitstream:

```bash
make sim DESIGN=sys_nn
make bitstream
exit
```

[In Chrome](https://ofl.trabucayre.com/), program
`fpga/build/sys_nn/sys_nn.fs` using the
[web programmer](#program-the-fpga).

### 6.3 Test an MNIST Image

On the host, open a terminal in the root of your `digital-design` checkout and
activate the basic Conda environment. The script detects the Tang Nano UART
automatically. If detection is ambiguous, use
`python -m serial.tools.list_ports -v` to find it and pass `--port PORT`.

Run a random test image or choose one by its MNIST index:

```bash
conda activate tang-basic
python material/py/fpga_nn.py
python material/py/fpga_nn.py --index 42
```

The script downloads the MNIST test files into `material/data/MNIST/raw` if
they are not already present. It then prints the expected label, the FPGA
prediction, and all ten signed scores. If it cannot find or open the board,
see [UART access troubleshooting](#uart-access-troubleshooting).

### 6.4 Test the Webcam

The script detects the Tang Nano UART automatically. 
The default webcam is device `0`. Pass `--camera NUMBER` if your computer selects a different camera.

```bash
conda activate tang-basic
python material/py/fpga_nn_camera.py
```

In a white paper, write a digit with a dark color and show it to the camera.
The window shows the 9×9 quantized image that the FPGA receives, while the terminal prints the prediction and scores. 
Press **q**, **Esc**, or **Ctrl+C** to stop. 
If the camera cannot be opened, check the camera permission for your terminal or Anaconda Prompt.

The prediction is pretty bad because we can only fit a tiny network into the FPGA in a fully-parallel way.
That is, the whole accelerator accepts a new input and produces a new output every clock cycle.
This has a throughput is 54 million frames per second, which is a massive overkill, but it keeps the SystemVerilog design simple.
Since the input rate is only around 30 frames per second, you can optimize the accelerator such that it takes one input every `INTERVAL` clock cycles, and reuses the multipliers and accumulators when processing it.
This results in much lower area, allowing you to train a bigger model and fill the FPGA.
Engineering is all about trade-offs and constraints.

## Common fixes

- **`conda: command not found`:** close and reopen the terminal. If that does
  not help, return to the Miniconda guide for your operating system and run its
  shell-initialization step.
- **The environment already exists:** activate it instead of creating it again.
  To bring it up to date, run `conda env update -f
  python-setup/tang-basic.yml --prune`.
- **No serial port is found:** see [UART access
  troubleshooting](#uart-access-troubleshooting).
- **The wrong serial port is selected:** rerun the script with `--port PORT`.
- **The live-audio script uses the wrong microphone or output:** run it with
  `--list`, then pass the desired device numbers to `--input` and `--output`.

(uart-access-troubleshooting)=
## UART Access Troubleshooting

macOS and native Windows normally require no UART handoff after programming.
Ubuntu may need its FTDI driver released before Chrome programs the board and
restored afterward.

First, check which serial ports Python can see:

```bash
python -m serial.tools.list_ports -v
```

```{raw} html
<details>
<summary><strong>Ubuntu</strong></summary>
```

If Chrome cannot claim the board, close any programs using `/dev/ttyUSB*` and
release the FTDI serial driver before programming:

```bash
sudo modprobe -r ftdi_sio
```

After programming, restore the driver and check that the ports appear:

```bash
sudo modprobe ftdi_sio
ls /dev/ttyUSB*
```

If Python reports `Permission denied`, add your user to the `dialout` group,
then sign out and back in:

```bash
sudo usermod -aG dialout "$USER"
```

```{raw} html
</details>
```

```{raw} html
<details>
<summary><strong>macOS</strong></summary>
```

No driver restore is normally needed. The board's UART should appear as
`/dev/cu.usbserial-*` or `/dev/tty.usbserial-*`:

```bash
ls /dev/cu.usbserial-* /dev/tty.usbserial-* 2>/dev/null
```

If the script finds more than one possible port, pass the `/dev/cu.*` port
explicitly with `--port`.

```{raw} html
</details>
```

```{raw} html
<details>
<summary><strong>Windows 11</strong></summary>
```

No driver restore is normally needed. Interface 1/B should remain available to
native Windows Python as a `COM` port:

```powershell
python -m serial.tools.list_ports -v
```

If the board is not listed, make sure Zadig replaced the driver for interface
0/A only. Do not replace interface 1/B, and do not attach the board to WSL with
`usbipd`. If automatic detection is ambiguous, pass the port explicitly, for
example `--port COM5`.

```{raw} html
</details>
```