Why you should care

A secretive team of Israeli soldiers is leading the country’s charge in the field of cutting-edge tech.

By Mehul Srivastava

The Daily DoseNOV 27 2018

When Ofir Schlam, co-founder of Taranis, an Israeli agriculture tech startup, was growing up on a farm in Israel, he would regularly wake at 5 a.m. to search through the crop for the tiniest caterpillars, pests and rot. Years later, when he joined the military and was attached to the prime minister’s office, he adapted that skill set to analyze thousands of surveillance images, looking for the smallest anomaly.

One of Schlam’s key senior executives at Taranis, Amihay Gornik, developed his expertise working at large aerospace companies, designing imaging parts for military drones. Gornik figured out a way to make a fast-moving camera think it was standing still by nestling it inside a proprietary pod he had fitted with a gyroscope, which helped cancel out vibrations and resulted in less blur.

At Taranis they insert the pods onto Cessna aircraft, fly them up to about 100 feet and then zoom over tens of thousands of acres of farmland at 120 miles per hour, taking photographs with an off-the-shelf camera. Even at that speed, their software can spot the tiniest pests or signs of disease, and the planes can map thousands of acres in the time it takes for drones to travel just a few. More scale, more caterpillars caught.

The element that makes Taranis possible, says Schlam, is the melding of their uniquely Israeli experiences and the way they have applied them to business. “It’s a small place,” he says. “And yet, it’s not that hard to find someone who’s into farming, into tech and has done this kind of thing before — maybe in the army, maybe at another startup.”

There are good pockets of technology for this stuff in the U.S., Japan, China. But nothing like in Israel.

Jon Medved, CEO, OurCrowd

It is those experiences that have helped such a tiny country become a leader in one of the most promising frontiers in the technology world: computer vision. Despite the unwieldy name, it is an area that has come of age in the past few years, covering applications across dozens of industries that have one thing in common: the need for computers to figure out what their cameras are seeing, and for those computers to tell them what to do next.

The biggest success story is Mobileye, which uses a dozen cheap cameras to see the traffic around prototype autonomous cars and then guides them through traffic. In 2017, Intel paid $15.3 billion to acquire the technology, as carmakers plunge billions of dollars into building self-driving cars.

Computer vision has become the connecting thread between some of Israel’s most valuable and promising tech companies. And unlike Israel’s traditional strengths — cybersecurity and mapping — computer vision slides into a broad range of different civilian industries, spawning companies in agriculture, medicine, sports, self-driving cars, the diamond industry and even shopping.

In Israel, this lucrative field has benefited from a large pool of engineers and entrepreneurs trained for that very task in an elite, little-known group in the military — Unit 9900 — where they fine-tune computer algorithms to digest millions of surveillance photos and sift out actionable intelligence.

“There are a lot of areas we are strong at, but at computer vision, we rock,” says Jon Medved, CEO of OurCrowd, a $750 million crowdfunded investment vehicle-based in Jerusalem, who has put millions of dollars into computer vision startups. “This is the kind of thing that’s really hard to duplicate and Israel is way ahead. There are good pockets of technology for this stuff in the U.S., Japan, China. But nothing like in Israel.”

***

Zebra Medical, backed by $50 million in venture capital funding, uses artificial intelligence to scan millions of MRI and other images from around the world, guiding radiologists to the slightest sign of disease. At medical diagnostics startup FDNA, engineers are figuring out how a picture of someone’s face could reveal rare genetic disorders. Cortica, with about $70 million in funding, is trying to replicate how the human brain sees the world in a quest to teach a computer to understand its environment.

Israeli startups working in computer vision have attracted more than $1 billion in seed and VC funding over the past three years, more than half of it this year alone. That is an increase from $56 million in 2015, according to Start-Up Nation Central, which supports investments in Israeli technology.

This phenomenon is powered by a new generation of sophisticated cameras, especially on smartphones, just as the vast computing power needed to process millions of images has become much cheaper.

The full name for Unit 9900 — the Terrain Analysis, Accurate Mapping, Visual Collection and Interpretation Agency — hints at how it has created a critical mass of engineers indispensable for the future of this industry. The secretive unit has only recently allowed limited discussion of its work. But with an estimated 25,000 graduates, it has created a deep pool of talent that the tech sector has snapped up.

Soldiers in Unit 9900 are assigned to strip out nuggets of intelligence from the images provided by Israel’s drones and satellites — from surveilling the crowded, chaotic streets of the Gaza Strip to the unending swaths of desert in Syria and the Sinai.

With so much data to pour over, Unit 9900 came up with solutions, including recruiting Israelis on the autistic spectrum for their analytical and visual skills. In recent years, says Shir Agassi, who served in Unit 9900 for more than seven years, it learned to automate much of the process, teaching algorithms to spot nuances, slight variations in landscapes and how their targets moved and behaved.

“We had to take all these photos, all this film, all this geospatial evidence and break it down: How do you know what you’re seeing, what’s behind it, how will it impact your intelligence decisions?” explains Agassi, who helps run an association of about 1,000 graduates of 9900. “You’re asking yourself — if you were the enemy, where would you hide? Where are the tall buildings, where’s the element of surprise? Can you drive there, what will be the impact of weather on all this analysis?”

Computer vision was essential to this task, says Agassi. Teaching computers to look for variations allowed the unit to quickly scan thousands of kilometers of background to find actionable intelligence. “You have to find ways not just to make yourself more efficient but also to find things that the regular eye can’t,” she says. “You need computer vision to answer these questions.”

***

When he was in the military, Eran Shir worked in ballistic missiles. To manipulate a missile accurately at several times the speed of sound, Shir and his colleagues used radars, high-definition video and maps to create a precise replica of the missile’s surroundings.

Now, at Nexar, which analyzes traffic and collision data from drivers’ smartphones, he has helped map millions of miles of road around the world. Propped on a dashboard, the camera on the phone tracks the traffic around it and warns the driver of risks while creating a video of the journey for insurance records, as well as alerting cities to potholes or dangerous intersections. With all the phones sharing data, it helps create a more detailed map.

“The Israeli tech ecosystem has been dealing with understanding reality in its various forms for a long time, using vision, lasers, kinematic sensors,” Shir says. “We are [now] dealing with the real problems of the world — health, agriculture, transportation — and our tech system is optimized for that.”

Nexar has made use of other graduates from Israel’s military. It needed physicists to match up the sensors on different phones from around the world and engineers who had worked with missiles and who built helmets for combat pilots. This is “the kind of work where you can’t settle for good enough — you have to get closer,” Shir says.

That sort of sophisticated talent gives Israel a means to compete with the U.S. and China and allows niche companies to thrive, says Aviv Zeevi Balasiano at the Israel Innovation Authority, an arm of the government that vets and invests in startups. The industry received a big injection of knowledge from Russian Jews who migrated en masse to Israel after the collapse of the Soviet Union in the 1990s.

“Some of the knowledge they brought with them was creating computer algorithms and machine engineering, fields which no one in Israel had ever heard about,” Balasiano says. That kicked off the earliest iterations of using computer vision in Israel’s military.

But the field took years to mature. Daniel Gabay, co-founder of Trigo, which wants to automate the checkout in grocery stores, jokes that not so long ago computers could not tell the difference between a cat and a dog. Now computer vision can tell a car to swerve when a child appears in the road or, in Trigo’s case, tell the difference between a bag of Doritos and a bag of regular chips.

Also a graduate of an elite military unit, Gabay is racing against Amazon to perfect the technology for a cashierless store, where sophisticated code helps a small number of cameras figure out what shoppers have in their baskets, and charge them automatically. The system is being tested in several hundred stores in Israel. Gabay is vague about his time in the army, but says he “learned to do the practical things — how can you take vision and create a working system.”

Almost all of the engineers working at Trigo were “cherry-picked from elite military units,” says Gabay.

***

Yet despite their growing reputation, Israeli companies have to remain focused on what they are best at, says Ronny Cohen, who runs Vayavision, rather than spreading themselves too thin. His company is betting that the use of better software will allow self-driving cars to map the environment around them with fewer and cheaper cameras or radars.

“It’s too difficult to do all this alone, so I am not going to build a car, or even tell the car where to go. I just need to build the best environmental model for cars,” Cohen says, describing how he turned down an offer to develop his system for vehicles in Amazon warehouses.

The development of massive databases — from close-ups of farm insects to medical scans to traffic data — has given Israeli companies a valuable head start over rivals. And in an industry where every new image teaches the algorithm something useful, that has made catching up difficult.

It has also created opportunities in unexpected sectors. Physimax, a startup run by Ram Shelev, uses a bank of cameras to analyze the posture of athletes, then suggests changes to their exercise routines and techniques. It is already being used by the U.S. military, the Indiana Pacers and Brazil’s Flamengo soccer team.

“Computer vision is absolutely the thread that ties us to other Israeli companies,” Shelev says. “I need people with the same unique DNA — smart Ph.D.s in mathematics, neural network analysis — to tell a player in the NBA how to improve his jump shot.”

Surveillance complex: Military expertise has also created defense industry

Israel’s developing expertise in computer vision has spawned companies in many of the most dynamic modern industries. But the deep pool of army-trained engineers has also fueled less benign activities: a vast private industry of surveillance and defense sales.

Companies staffed by veterans of intelligence units such as 8200 and 9900 also sell some of the most sophisticated surveillance software to all sorts of countries, some with less than savory human rights records.

Most recently, Herzliya-based NSO Group has faced criticism that it has sold its Pegasus smartphone surveillance software to the United Arab Emirates, Kuwait and Kazakhstan, where it has been used to penetrate the cellphones of political dissidents and human rights activists, according to the University of Toronto’s Citizen Lab. The company says it only sells to governments for the use of preventing crime and terrorism, and that its sales are approved by the Israeli government.

The tiny nation’s defense exports soared to $9 billion in 2017, an increase of 40 percent, on the back of a $2.5 billion deal to sell missiles to India. Prime Minister Benjamin Netanyahu has claimed that demand for Israeli military technology is one of the key reasons for a recent thaw between Israel and its Arab neighbors.

But the fact that these technologies and weapons were often developed for the surveillance of Palestinians, or for fighting Hamas in the Gaza Strip, where thousands of Palestinians have died in four recent wars, has made Israel’s defense exports a target of international criticism.

And the reported use of Pegasus by Saudi Arabia to track the phones of dissidents living abroad, including friends of Jamal Khashoggi, the slain journalist, has pitted one of Israel’s most secretive exports against a global outcry against surveillance.