Bringing Big Neural Networks to Self-Driving Cars, Smartphones, and Drones

by Katherine Bourzac,  IEEE Spectrum

Engineers are trying to squeeze outsize AI into mobile systems. 

Artificial intelligence systems based on neural networks have had quite a string of recent successes: One beat human masters at the game of Go, another made up beer reviews, and another made psychedelic art. But taking these supremely complex and power-hungry systems out into the real world and installing them in portable devices is no easy feat. This February, however, at the IEEE International Solid-State Circuits Conference in San Francisco, teams from MIT, Nvidia, and the Korea Advanced Institute of Science and Technology (KAIST) brought that goal closer. They showed off prototypes of low-power chips that are designed to run artificial neural networks that could, among other things, give smartphones a bit of a clue about what they are seeing and allow self-driving cars to predict pedestrians’ movements.

For the human brain, drawing on memories to make associations comes naturally. A 3-year-old child can easily tell you that a photo shows a cat lying on a bed. Convolutional neural networks can also label all the objects in an image. First, a system like the image-recognition champ AlexNet might find the edges of objects in the photo, then begin to recognize those objects one by one—cat, bed, blanket—and finally deduce that the scene is taking place indoors. Yet even doing this kind of simple labeling is very energy intensive.

Until now, neural networks—learning systems that operate analogously to networks of connected brain cells—have been much too energy intensive to run on the mobile devices that would most benefit from artificial intelligence, like smartphones, small robots, and drones. The mobile AI chips could also improve the intelligence of self-driving cars without draining their batteries or compromising their fuel economy.

Smartphone processors are on the verge of running some powerful neural networks as software. Qualcomm is sending its next-generation Snapdragon smartphone processor to handset makers with a software-development kit to implement automatic image labeling using a neural network. This software-focused approach is a landmark, but it has its limitations. For one thing, the phone’s application can’t learn anything new by itself—it can only be trained by much more powerful computers. And neural networks experts think that more sophisticated functions will be possible if they can bake neural-net–friendly features into the circuits themselves.

MIT professor Vivienne Sze says increasing the scale of neural networks also increases their power consumption, with the main power drain being the transfer of data between processor and memory. In collaboration with fellow MIT professor and Nvidia researcher Joel Emer, Sze has developed Eyeriss, a custom chip designed to run a cutting-edge convolutional neural network. The network can run the AlexNet algorithm using 0.3 watts of power instead of the 5 watts to 10 watts consumed by a typical mobile graphics-processing unit. The chip saves energy by placing a dedicated memory bank close to each of its 168 processing engines, and it retrieves data from a larger primary memory bank as infrequently as possible.

Meanwhile, KAIST professor Lee-Sup Kim says circuits designed for neural network–driven image analysis also will help with airport face-recognition systems and robot navigation. His lab demonstrated a chip designed to be a general visual processor for the Internet of Things, which minimizes data movement by bringing memory and processing closer together. It uses only 45 milliwatts of power and lightens the computational load as well.  Read the report.