It’s only in recent years that household appliances like the washing machine have been produced with embedded AI technology, but the evolution of deep learning continues to grow, and the MIT press office announced that the MIT researchers may have made big improvements with their newly developed system. Recent tests show that their system has unprecedented speed and accuracy. We could soon see a huge leap forward in technology in the market.
MIT researchers have developed a system called MCUNet with which they hope to bring deep learning neural networks to computer chips in wearable medical devices, home appliances, and the countless other things labeled “Internet of Things” (IoT).
The system designs compact neural networks that deliver unprecedented speed and accuracy for deep learning on IoT devices, despite limited memory and processing power, according to the press release.
The technology could facilitate the expansion of the IoT universe while saving energy and improving data security.
The research will be presented at next month’s conference on Neural Information Processing Systems. The lead author is Ji Lin, who holds a Ph.D. studying in Song Han’s lab at MIT’s Department of Electrical Engineering and Computer Science. Co-authors include Han and Yujun Lin of MIT, Wei-Ming Chen of MIT and National Taiwan University, and John Cohn and Chuang Gan of MIT-IBM Watson AI Lab.
PAPER: “MCUNet: Tiny Deep Learning on IoT Devices”
The following was wwritten by Daniel Ackerman for the MIT news desk who shared the document with various media for immediate release.
Designing a deep network for microcontrollers is not easy. Existing neural architecture search techniques start with a large number of possible network structures based on a predefined model, and then gradually find the one with high accuracy and low cost.
“It can work very well for GPUs or smartphones,” Lin says. “But it’s been difficult to apply these techniques directly to tiny microcontrollers because they’re too small.”
So Lin developed TinyNAS, a neural architecture search method that creates custom-sized networks.
“We have a lot of microcontrollers that come with different power capacities and different memory sizes,” Lin explains. “So we developed the algorithm [TinyNAS] to optimize the search space of the different microcontrollers.
The custom nature of TinyNAS means it can generate compact neural networks with the best possible performance for a given microcontroller, without unnecessary parameters.
“Then we deliver the final, efficient model to the microcontroller,” Lin explains.
To run this tiny neural network, a microcontroller also needs a lightweight inference engine. A typical inference engine carries some dead weight – instructions for tasks it can rarely perform. The extra code is no problem for a laptop or smartphone, but it could easily overwhelm a microcontroller.
“It doesn’t have off-chip memory and it doesn’t have a disk,” says Han. “The whole set is only one megabyte of flash, so we have to manage such a small resource very carefully.” Cue TinyEngine.
The researchers developed their inference engine in collaboration with TinyNAS. TinyEngine generates the essential code needed to run TinyNAS’ custom neural network. Any deadweight code is discarded, reducing compile time.
“We only keep what we need,” says Han. “And since we designed the neural network, we know exactly what we need. This is the advantage of the coded design of system algorithms.
In the group’s tests on TinyEngine, the size of the compiled binary code was between 1.9 and five times smaller than comparable microcontroller inference engines from Google and ARM. TinyEngine also contains innovations that reduce execution time, including in-place deep convolution, which nearly halves peak memory usage. After co-designing TinyNAS and TinyEngine, Han’s team put MCUNet through its paces.
MCUNet’s first challenge was image classification. The researchers used the ImageNet database to train the system with tagged images and then test its ability to classify news. On a commercial microcontroller they tested, MCUNet managed to classify 70.7% of new images – the old state-of-the-art neural network and inference engine combo was only 54% accurate . “Even a 1% improvement is considered significant,” Lin says. “So that’s a giant leap for microcontroller settings.”
The team found similar results in ImageNet tests of three other microcontrollers. And in terms of speed and accuracy, MCUNet beat the competition for audio and visual “wake-word” tasks, where a user initiates an interaction with a computer using voice cues (think: “Hey, Siri”) or simply by entering a room. The experiments highlight the adaptability of MCUNet to many applications.
Promising test results give Han hope that it will become the new industry standard for microcontrollers. “He has enormous potential,” he said.
The breakthrough “extends the frontier of deep neural network design even further into the computing realm of small, power-efficient microcontrollers,” says Kurt Keutzer, a computer scientist at the University of California, Berkeley, who was not involved. to the works. He adds that MCUNet could “bring intelligent computer vision capabilities to even the simplest kitchen appliances or enable smarter motion sensors.”
MCUNet could also make IoT devices more secure.
“A key benefit is maintaining privacy,” says Han. “You don’t need to push the data to the cloud.”
Local data analysis reduces the risk of personal information theft, including personal health data. Han envisions smartwatches with MCUNet that not only detect users’ heart rate, blood pressure and oxygen levels, but also analyze and help them understand this information. MCUNet could also bring deep learning to IoT devices in vehicles and rural areas with limited internet access.
Additionally, MCUNet’s small IT footprint translates to a low carbon footprint. “Our big dream is for green AI” says Han, adding that training a large neural network can burn carbon equivalent to the lifetime emissions of five cars. MCUNet on a microcontroller would require a small fraction of that power.
“Our end goal is to enable tiny, efficient AI with less computational resources, less human resources, and less data,” says Han.