The IDF’s New Artificial Intelligence Can Tell You What’s Happening in a Video

A team of IDF researchers in the C4I Directorate has created an algorithm that can understand videos and describe them on-screen. Artificial intelligence like this is an enormous revolution in how we keep an eye on the battlefield.

Artificial intelligence is all the rage in Silicon Valley these days, with major developers building AI into their products. As tech pioneers, the IDF is no different. In a recent development, a team of IDF C4I Directorate researchers have built an artificial intelligence that will one day be used to keep watch on Israel’s borders. If it detects anything suspicious, it’ll be able to automatically notify the field observers responsible for that area.

The new artificial intelligence works by combining two neural networks. One analyzes each individual frame of a video and determines what is contained in the video, and the second looks for how those elements are moving. The artificial intelligence then takes the data and responds with a caption describing the video. For example: “the brown car is driving on the highway.”

The research team trained the algorithm against 2.4 million images and videos. The neural network took just 14 hours to train using a 2,880 core GPU. This system allowed them to finish training their model in 1/40th of the time it would’ve taken using a traditional CPU.

“For now, the algorithm is just a research project,” explained Major Seffi, the head of the operational data research department in the C4I Directorate. “We’re looking into possible uses in the future. This artificial intelligence may be able to assist IDF field observers by calling attention to things of interest happening on our borders. It’ll enable each soldier to keep watch on a much larger area with relative ease.”