Have you ever dreamt of writing code that operates flawlessly across a multitude of devices, irrespective of their underlying hardware? Well, there’s no need to dream any longer! WebAssembly (Wasm) is here to turn that dream into reality, particularly for edge computing applications. This article delves into the realm of Wasm on the edge, demonstrating its versatility through an engaging demo.
A WASM PRIMER
If you’re unfamiliar with Wasm, don’t worry! Picture a versatile tool that converts code into a format that can be understood by a variety of devices. That’s what Wasm is, in essence. Developers write code once, and Wasm ensures it operates on diverse hardware architectures. This results in:
- Reduced Development Time: There’s no need to rewrite code for each device, saving valuable time!
- Simplified Maintenance: Update the code once, and all compatible devices will benefit from the changes.
THE SCALEXTRIC SHOWDOWN: A WASM DEMO IN ACTION
To showcase the flexibility of Wasm at the edge, we’ve constructed a demo using a Scalextric track (imagine miniature car racing!)
Here’s the setup:
- Two Toy Cars
- Two Sets of Cameras:
- A smart Sony sensor with a built-in DSP (Digital Signal Processor) for hardware-accelerated processing.
- A Raspberry Pi with a PiCam.
Both cameras capture car movements simultaneously. The magic happens next:
- Sony Semiconductor Solutions’s IMX500: Leverages its dedicated hardware for image processing and neural network inference (think AI smarts for object detection).
- Raspberry Pi: Uses Wasm and WASI-NN (WebAssembly Standard Neural Network Interface) to run TensorFlow Lite models. WASI-NN essentially allows the Pi to perform machine learning tasks using pre-trained models.
Finally, the results are sent to a dashboard, displaying the car positions and telemetry data, effectively recreating the Scalextric loop.
THE BEAUTY OF WASM: PORTABILITY AND PERFORMANCE
The crucial takeaway here is that both devices, despite their differing architectures (Xtensa for the Sony sensor and ARM for the Raspberry Pi), utilize largely the same Wasm modules. The only distinction is that the Raspberry Pi requires WASI-NN for its machine learning capabilities, due to its less powerful CPU.
This underscores the power of Wasm:
- Portability: Code written for one device can be easily ported to others with minimal adjustments.
- Performance: Wasm offers near-native speeds, making it an ideal choice for edge applications where processing power might be limited.
HOW DO WASM MODULES REALLY WORK?
WebAssembly (Wasm) modules work by compiling high-level source code into a binary format that can be executed at near-native speed by various devices, regardless of their hardware architecture. But, what happens inside the module? Which is the Functional Analysis of this Wasm Module?
SENSOR CONTROL WASM MODULE
This module handles sensor lifecycle management and sensor data acquisition. There are various types of sensor data, including image, depth, and metadata. If the sensor is an IMX500, you can acquire metadata; if it’s a RaspiCam, you can obtain RGB images. This module serves as a data source, passing the sensor data to another module as output. By having an interface that can manage sensor data in a unified manner like this, we can enhance module reusability and achieve sensor fusion with various systems. Therefore, we have created a draft of the wasi-sensor at this repository. If possible, we would like to explore the possibilities of this interface with as many of you as possible.
Visit it by clicking here 👉 https://github.com/midokura/wasi-sensor-discussion
WHICH INPUTS DOES IT RECEIVE? WHAT IS THE OUTCOME?
- Input: None.
- Output: Sensor data.
- IMX500: Inference Metadata.
- RaspiCam: RGB Image
AI POST PROCESSING WASM MODULE
This module analyzes the binary output of AI inference results. It transforms them into a format that is easily processed in subsequent business logic, such as class, score, and bounding box information for object detection. The formats of the binaries output from the IMX500 and those from TensorFlow are compatible, allowing you to use the same module on both systems..
WHICH INPUTS DOES IT RECEIVE? WHAT IS THE OUTCOME?
- Input: Inference Metadata.
- Output: Object Detection Information such as score, bounding box.
FUNCTIONALITY OF THE WASI-NN MODULE
This module carries out preprocessing to execute AI inference and manages the AI inference process itself. In AI inference, we control the backend using the WASI-NN interface. In this demo, we utilized TensorflowLite’s Backend to perform inference on a Tensorflow model. Through WASI-NN, you can load the AI model, set input images, execute inference, and obtain inference results.
Using WASI-NN, you can easily switch this Backend to another one or effortlessly change the Target used for inference from CPU to GPU or TPU.
On the other hand, this module does not exist for IMX500 systems. Everything from loading the model to preprocessing the input image to executing the inference is carried out within the sensor itself.
The WASI-NN in this demo still uses a ‘load’ command to bring the AI model into WASM. However, the newly created ‘load_by_name’ command for Edge AI will make it even more user-friendly for embedded systems. It’s exciting to see progress in the study of WASI, which is well-suited for embedded systems, and we’re eager to contribute to it.
WASM ON THE EDGE: THE FUTURE IS BRIGHT
This demo underscores the immense potential of Wasm on the edge. Developers can craft code for high-performance devices and seamlessly adapt it to function on resource-constrained ones. This leads to accelerated development cycles, simplified maintenance, and ultimately, more efficient edge applications. So, when you’re next building something for the edge, consider harnessing the power of Wasm – it could be the game-changer you’ve been seeking!
