Next Steps for ActionAI

Some of our earliest work applying ML to video was done in the context of prototyping IoT products like YogAI. A couple years ago, we described a more generalized pipeline called ActionAI. ActionAI was designed to streamline prototyping IoT products using lightweight activity recognition pipelines on devices like NVIDIA’s Jetson Nano or the Coral Dev Board. Since then, NVIDIA has introduced action recognition modules into their Deepstream SDK. They model a classifier using 3D convolutional kernels over the space-time volume of normalized regions of interest, batched over a k-window in time....

 · 4 min · Terry Rodriguez & Salma Mayorquin

Machine Learning on Video

Factors like cheaper bandwidth and storage, expanded remote work, streaming entertainment, social media, robotics and autonomous vehicles, all contribute to the rapidly increasing volume of video data. Nonetheless, performance in benchmark ML video tasks in perception, activity recognition, and video understanding lag behind the image counterpart. In this post, we consider the challenges in applying ML to video while surveying some of the techniques en vogue to address them. The Time Dimension Treating video analytics as a search over space and time, the dimensionality begets additional hurdles to statistical and computational efficiency....

 · 5 min · Terry Rodriguez & Salma Mayorquin