Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

18 episodes of ESP-DL deep learning tutorials for ESP32 - "MCU Embedded AI Development Notes"

Reference document: https://docs.espressif.com/projects/esp-dl/zh_CN/latest/esp32/tutorials/index.html

Use TVM to automatically generate model deployment projects

This case study introduces the complete process of deploying a model using TVM. This project is based on the TVM v0.14.0 branch and is in an experimental state. There is no plan for further iteration and maintenance. Currently, only the conv2d operator of ESP-DL is connected, and other operators may cause exceptions.

Prepare
ESP-DL is a deep learning inference framework adapted to ESP series chips. This library cannot complete the training of the model. Users can use training platforms such as TensorFlow and PyTorch to train the model, and then deploy the model through ESP-DL.
The specific process is

Step 1: Quantification

First, convert the trained model such as tensorflowPyTorch. Taking the TensorFlow platform as an example, you can use tf2onnx in the script to convert the trained TensorFlow model into the ONNX model format.
A series of operations will then be performed on the float32 model to prepare it for quantization.
The quantization tool then accepts the preprocessed float32 model as input and generates an int8 quantized model.

Step 2: Deploy the model

Step 2.1: Prepare for input
Prepare an input image. The size of the input image should be consistent with the obtained ONNX model input size. The model input size can be checked through the Netron tool.

Step 2.2: Deployment Project Build
Use TVM to automatically generate a project for running model inference for a given input.

Step 3: Run the model

Step 3.1: Run Inference
The structure of the project file new_project generated in the previous step is as follows:

├── CMakeLists.txt
├── components
│ ├── esp-dl
│ └── tvm_model
│ ├── CMakeLists.txt
│ ├── crt_config
│ └── model
├── main
│ ├── app_main.c
│ ├── input_data.h
│ ├── output_data.h
│ └── CMakeLists.txt
├── partitions.csv
├── sdkconfig.defaults
├── sdkconfig.defaults.esp32
├── sdkconfig.defaults.esp32s2
├── sdkconfig.defaults.esp32s3
After configuring the terminal ESP-IDF (please note the version of ESP-IDF) environment, you can run the project:

idf.py set-target esp32s3
idf.py flash monitor
Step 3.2: Debug