ST AEKD-AICAR1 User Manual
ST AEKD-AICAR1 User Manual

ST AEKD-AICAR1 User Manual

Evaluation kit for car state classification
Table of Contents

Advertisement

Quick Links

Getting started with the AEKD-AICAR1 evaluation kit for car state classification
Introduction
The
AEKD-AICAR1
is a versatile deep learning system based on a long-short term memory (LSTM) recurrent neural network
(RNN). It is able to classify the car state:
car parked
car driving on a normal condition road
car driving on a bumpy road
car skidding or swerving
The main idea is to define an AI-car sensing node ECU with an embedded artificial intelligence processing.
The system hosts an SPC58EC Chorus 4M microcontroller, which is able to acquire discrete acceleration variations on a
three-axis reference system.
The
AIS2DW12
motion sensor mounted on the
transmitted to the LSTM RNN, which classifies the car state. The classification result is shown on the
touch display.
The LSTM RNN has been implemented and trained using the TensorFlow 2.4.0 framework (Keras) in the Google Colab
environment. The AI-SPC5Studio plug-in has been used to convert the resulting trained neural network into an optimized C
code library, which can run on an MCU with limited power computing resources.
The LSTM RNN training has been performed with several time-series acceleration waveforms recorded on a real vehicle in
motion. The resulting prediction accuracy, calculated by the confusion matrix, is about 93%. Field tests carried-out under all road
conditions with a sedan confirm the adherence of the computed results compared with the real road conditions.
Note:
The
AEKD-AICAR1
UM3053 - Rev 1 - September 2022
For further information contact your local STMicroelectronics sales office.
AEK-CON-SENSOR1
Figure 1.
AEKD-AICAR1 evaluation kit
is an evaluation tool for R&D laboratory use only. It is not destined for use inside a vehicle.
board retrieves inertial data. The acquired data are
UM3053
User manual
AEK-LCD-DT028V1
LCD
www.st.com

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the AEKD-AICAR1 and is the answer not in the manual?

Questions and answers

Summary of Contents for ST AEKD-AICAR1

  • Page 1: Figure 1. Aekd-Aicar1 Evaluation Kit

    UM3053 User manual Getting started with the AEKD-AICAR1 evaluation kit for car state classification Introduction AEKD-AICAR1 is a versatile deep learning system based on a long-short term memory (LSTM) recurrent neural network (RNN). It is able to classify the car state: •...
  • Page 2: Neural Network Basic Principles

    UM3053 Neural network basic principles Neural network basic principles Artificial neural network The artificial neural network works as the human brain neural network. Data are transferred to the neuron through the inputs. Then, they are sent as an output after processing. Figure 2.
  • Page 3: Figure 3. Recurrent Neural Network

    UM3053 Long short-term memory recurrent neural network (LSTM RNN) Figure 3. Recurrent neural network In most cases, the output values of an upper layer are used as the input for a lower level one. This interconnection enables the use of one of the layers as the state memory and allows modeling a dynamic time behavior dependent on the information previously received.
  • Page 4: Designing An Ai-Car Sensing Node

    UM3053 Designing an AI-car sensing node Designing an AI-car sensing node Tool-set introduction An AI-car sensing node is an AI-deep learning system based on an LSTM recurring neural network, which can provide a car state classification: parking, driving on a normal condition road, driving on a bumpy road, and car skidding or swerving.
  • Page 5: Colab Notebook Setup And Package Importing

    UM3053 Colab notebook setup and package importing Figure 5. Project file The document that you are creating is not a static web page. It is an interactive environment that lets you write and execute your Python code to implement an LSTM RNN by using the TensorFlow framework. Colab notebook setup and package importing To implement and train an LSTM RNN, install the Tensorflow 2.4.0 framework on your Google Colab notebook by using the packet index package (PIP) command.
  • Page 6: Ai-Car Sensing Node Life Cycle

    UM3053 AI-car sensing node life cycle • Random This module implements pseudo-random number generators for various distributions. import random • Seaborn Seaborn is a Python data visualization library based on Matplotlib. It provides a high-level interface to draw attractive and informative statistical graphics. import seaborn as sn •...
  • Page 7: Model Training

    UM3053 AI-car sensing node life cycle ## Conv1D based model model = tf.keras.models.Sequential([ tf.keras.layers.Conv1D(filters=16, kernel_size=3, activation='relu', input_shape=(TIMESERIES_LEN, 3)), tf.keras.layers.Conv1D(filters=8, kernel_size=3, activation='relu'), tf.keras.layers.Dropout(0.5), tf.keras.layers.Flatten(), tf.keras.layers.Dense(64, activation='relu'), tf.keras.layers.Dense(4, activation='softmax') _________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv1d (Conv1D) (None, 48, 16) conv1d_1 (Conv1D) (None, 46, 8) dropout (Dropout)
  • Page 8: Figure 6. Acceleration Variations

    UM3053 AI-car sensing node life cycle Figure 6. Acceleration variations Car speed Vx = 0 Km/sec; Car acceleration ax! = 0; Temporal variation of acceleration Δax = 0 2.4.2.1 Training dataset The acquisition of training datasets can be performed using a specific configuration of the AI-car sensing node mock-up (acquisition mode).
  • Page 9: Figure 7. Time-Based Dataset Example

    UM3053 AI-car sensing node life cycle Figure 7. Time-based dataset example The time column contains the recorded acceleration time samples. The other three columns contain the acceleration values measured on the X, Y, and Z axis, respectively (without the gravity offset). To prepare the file for the network training, you have to: •...
  • Page 10: Figure 8. Adding The Status Column

    UM3053 AI-car sensing node life cycle • Add a status column for the car state for each specific acquisition (P = parking; N = normal; B = bumpy; S= skid). Figure 8. Adding the status column • Create a CSV file for each car state through a specific acquisition task. Figure 9.
  • Page 11: Model Fitting And Compilation

    UM3053 AI-car sensing node life cycle states = db['Status'].value_counts() ts_status = db.Status ts_diff_Ax = (db.Acc_x.to_numpy().reshape(-1,1))/9.81 ts_diff_Ay = (db.Acc_y.to_numpy().reshape(-1,1))/9.81 ts_diff_Az = (db.Acc_z.to_numpy().reshape(-1,1))/9.81 ts_time = db['Time'] Use a simple script to reshape the dataset to be compliant with the shape of the LSTM input vector (50 samples of a three-dimensional vector).
  • Page 12: Model Evaluation

    UM3053 AI-car sensing node life cycle model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) model.fit(x_train, y_train, epochs=1000) Fitting and compilation for the RNN generate a complete model. This procedure is quite slow. It can take few seconds up to days, depending on the model complexity and the training dataset size. For the AI-car sensing node, set 1000 epochs and 60% of the available window waveform used for the training.
  • Page 13: Figure 11. Confusion Matrix

    UM3053 AI-car sensing node life cycle • 36 actual normal window waveforms used for the evaluation have been predicted as normal, whereas four have been classified as bumpy. • 18 actual bumpy window waveforms used for the evaluation have been predicted as bumpy (no miss classifications).
  • Page 14: Autodevkit Ecosystem

    UM3053 AutoDevKit ecosystem AutoDevKit ecosystem SPC5-STUDIO-AI plugin The LSTM AI-car sensing node network implemented with the TensorFlow framework inside the Google Colab environment is then converted in a C library compliant with SPC5-STUDIO. On the Google Colab notebook, the LSTM AI-car sensing node network architecture is saved as an *.h5 file. This file is then fed into SPC5-STUDIO-AI to obtain a fast and optimized version of the RNN.
  • Page 15: Figure 13. Install New Software

    UM3053 SPC5-STUDIO-AI plugin Step 2. Insert http://ai.spc5studio.com in the [Work with] field. Figure 13. Install new software Step 3. Click on [Select all] and [Next]. Step 4. Install SPC5-STUDIO-AI. UM3053 - Rev 1 page 15/39...
  • Page 16: How To Import Spc5-Studio-Ai Using The Standard Importing Procedure

    UM3053 SPC5-STUDIO-AI plugin 3.1.1 How to import SPC5-STUDIO-AI using the standard importing procedure Step 1. Create a new SPC5-STUDIO application based on the SPC58ECxx platform. Figure 14. Creating a new SPC5-STUDIO application Step 2. Import the following components: – AutodevKit Init Package –...
  • Page 17: How To Import The Pretrained Lstm Neural Network

    UM3053 SPC5-STUDIO-AI plugin Step 4. Click on [+] to add a new component. Figure 16. Adding a new component Step 5. Select [SPC5 AI Component RLA]. Figure 17. Selecting [SPC5 AI Component RLA] 3.1.2 How to import the pretrained LSTM neural network The following procedure shows how to import the pretrained LSTM neural network for the AI-car sensing node.
  • Page 18: Figure 18. Network List Imported

    UM3053 SPC5-STUDIO-AI plugin Step 1. Select the [SPC5 AI Component RLA] in the project explorer. A new window is opened with the network list imported. Figure 18. Network list imported Step 2. Add a new network by clicking on [+]. Figure 19.
  • Page 19: How To Analyze The Pretrained Lstm Neural Network

    UM3053 SPC5-STUDIO-AI plugin 3.1.3 How to analyze the pretrained LSTM neural network To analyze the pretrained LSTM neural network for the AI-car sensing node, follow the procedure below. Step 1. Select [Analyze] in the [Outline] tab. Figure 21. Selecting [Analyze] UM3053 - Rev 1 page 19/39...
  • Page 20: Figure 22. Clicking On [Analyze]

    UM3053 SPC5-STUDIO-AI plugin Step 2. Click on [Analyze] in the newly opened window. Figure 22. Clicking on [Analyze] If the importing procedure for the LSTM AI-sensing node is correct, a new report is generated. This new report shows the architecture of the neural network and the ROM and RAM memory usage. Neural Network Tools for STM32 v1.4.0 (AI tools v5.2.0) -- Importing model -- Importing model - done (elapsed time 0.574s)
  • Page 21: Spc5-Studio-Ai Api

    UM3053 SPC5-STUDIO-AI API elapsed time (analyze): 0.65s Note: If the report does not appear properly filled, check the generated *.h5 file as it could be corrupted or generated with a Tensorflow framework version different from 2.4.0. In the latter case, uninstall and reinstall Tensorflow 2.4.0 in the Google Colab, as follows: –...
  • Page 22 UM3053 SPC5-STUDIO-AI API • ai_sensing_node_network_create ai_error ai_sensing_node_network_create(ai_handle* network, const ai_buffer* network_config); ai_handle ai_sensing_node_network_destroy(ai_handle network); This is the initial function invoked by the application to create an instance of the neural network. The ai_handle is updated after the network creation with the pointer to the entire structure. From this moment, this handle is passed to all the other neural network related functions.
  • Page 23: Ai-Car Sensing Node

    LSTM neural network computation. The power supply unit. To power the AEKD-AICAR1, use the switch to choose one of the following options: the internal battery pack with six AAA batteries; an external 12 V DC power supply directly connected to the AEK-MCU-C4MLIT1.
  • Page 24: Low-Level Software

    AutoDevKit design flow. The drivers can be easily adapted to any ST MEMS sensor, even if not directly supported by the software library. To configure the AEK-CON-SENSOR1 AutoDevKit...
  • Page 25: High-Level Software

    AutoDevKit design flow. To configure the AEK-LCD-DT028V1 AutoDevKit component for the AEKD-AICAR1 application, follow the procedure below. Step 1. Go to the [SPC58ECxx Platform Component RLA] in the project explorer and add a new [AEK-LCD- DT028V1 Component RLA] from the list.
  • Page 26: Main Application Software

    LCD display: void infotainment_system_init(void); • printing a message (with a specific color) and drawing an icon: void display_print(uint8_t msg, uint16_t color); 4.3.3 Main application software AEKD-AICAR1 application has three operating working modes: • Acquisition • Real-time calculation (default) •...
  • Page 27: Test And Results

    UM3053 Test and results Test and results Environment setup AEKD-AICAR1 sensing node has been installed in a sedan car with a normal shock absorber system. The x-axis of the AIS2DW12 accelerometer must have the same direction of the car motion.
  • Page 28: Schematic Diagrams

    UM3053 Schematic diagrams Schematic diagrams Note: AEKD-AICAR1 kit consists of the following evaluation boards: AEK-MCU-C4MLIT1, AEK-CON-SENSOR1, AEK-LCD-DT028V1, and STEVAL-MKI206V1. You can find their detailed schematic diagrams at the related web pages: • AEK-MCU-C4MLIT1 schematic diagrams • AEK-CON-SENSOR1 schematic diagrams •...
  • Page 29: Bill Of Materials

    UM3053 Bill of materials Bill of materials Note: AEKD-AICAR1 kit consists of the following evaluation boards: AEK-MCU-C4MLIT1, AEK-CON-SENSOR1, AEK-LCD-DT028V1, and STEVAL-MKI206V1. You can find their detailed schematic diagrams at the related web pages: • AEK-MCU-C4MLIT1 bill of materials • AEK-CON-SENSOR1 bill of materials •...
  • Page 30: Appendix A Regulatory Compliance

    UM3053 Regulatory compliance Appendix A Regulatory compliance Notice for US Federal Communication Commission (FCC) For evaluation only; not FCC approved for resale FCC NOTICE FCC NOTICE - This kit is designed to allow: (1) Product developers to evaluate electronic components, circuitry, or software associated with the kit to determine whether to incorporate such items in a finished product and (2) Software developers to write software applications for use with the end product.
  • Page 31: Appendix B Google Colab Code For The Neural Network Training

    UM3053 Google Colab code for the neural network training Appendix B Google Colab code for the neural network training %pip uninstall tensorflow %pip install tensorflow==2.4.0 from google.colab import files uploaded = files.upload() import pandas as pd import numpy as np import tensorflow as tf from sklearn.preprocessing import StandardScaler, MinMaxScaler from sklearn.model_selection import train_test_split , StratifiedShuffleSplit...
  • Page 32 UM3053 Google Colab code for the neural network training ## Conv1D based model model = tf.keras.models.Sequential([ tf.keras.layers.Conv1D(filters=16, kernel_size=3, activation='relu', input_shape=(TIMESERIES_LEN, 3)), tf.keras.layers.Conv1D(filters=8, kernel_size=3, activation='relu'), tf.keras.layers.Dropout(0.5), tf.keras.layers.Flatten(), tf.keras.layers.Dense(64, activation='relu'), tf.keras.layers.Dense(4, activation='softmax') model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) model.fit(x_train, y_train, epochs=1000) model.summary() import matplotlib.pyplot as plt import seaborn as sns %matplotlib inline Y_pred = model.predict(x_test)
  • Page 33: Appendix C Python Script Code For The Data Acquisition And Parsing

    UM3053 Python script code for the data acquisition and parsing Appendix C Python script code for the data acquisition and parsing import serial as sr import matplotlib.pyplot as plt import numpy as np import time from csv import writer def append_list_as_row(list_of_elem): # Open file in append mode with open(file_name, 'a+') as write_obj: # Create a writer object from csv module...
  • Page 34 UM3053 Revision history Table 3. Document revision history Date Revision Changes 20-Sep-2022 Initial release. UM3053 - Rev 1 page 34/39...
  • Page 35: Table Of Contents

    UM3053 Contents Contents Neural network basic principles........... . 2 Artificial neural network .
  • Page 36 UM3053 Contents Revision history ...............34 List of tables .
  • Page 37 UM3053 List of tables List of tables Table 1. List of #defines ..............21 Table 2.
  • Page 38 AEKD-AICAR1 software architecture ........
  • Page 39 ST’s terms and conditions of sale in place at the time of order acknowledgment. Purchasers are solely responsible for the choice, selection, and use of ST products and ST assumes no liability for application assistance or the design of purchasers’...

Table of Contents