Deep Learning Goes To The Deep Seas And The Billion-Dollar Tuna Industry

The next frontier for artificial intelligence may involve teaching computers to distinguish albacore tuna from its yellowfin cousin.

The Nature Conservancy, an environmental non-profit, is working with several Pacific Island nations and a big tuna fishing company to more easily count and identify fish caught at sea using cutting edge technology.

The goal is to use trendy artificial intelligence techniques like deep learning to help fishermen reduce the number of protected animals like sharks and turtles that are accidentally caught along with the tuna. The Nature Conservancy hopes that the program could prevent overfishing and help threatened and endangered sea life recover without putting fishermen out of work.

“We have real optimism that data science community can help us differentiate a turtle from a tuna, and flag when a shark comes on board,” said Mark Zimring, a project director for The Nature Conservancy.

Companies like Google(goog) and Facebook (fb) routinely use AI techniques like machine learning and deep learning to identify faces in pictures. The Nature Conservancy believes that the same pattern-recognition technology could be applied to the $6 billion to $7 billion tuna fishing industry, explained Zimring.

Typical commercial fishing near the tropical island nation of Palau, for example, involves what’s known as longline fishing. It involves casting fishing lines—some over 50 miles long—with up to 2,000 hooks.

Hauling in the lines can take 10 hours and is “incredibly hard work,” said Zimring. Afterward, it can take another four to six hours for the individual fish that are caught to be identified, he said.

The problem is that many commercial fishing companies don’t have the money to place someone on board to ensure that all the fish caught are being accounted for, Zimring said. Derrick Wang, the vice president of Luen Thai Fishing, said that fishing operators in the Pacific Island region typically send a physical observer to accompany fishermen about 10 times out of 200 trips in a year. Luen Thai Fishing is the fishing subsidiary of Hong Kong-based Luen Thai International Group with Wang, based in Guam, overseeing Pacific Island operations.

One of the six to eight cameras installed on a fishing boat.

The Nature Conservancy

In 2013, The Marshall Islands fined Luen Thai Fishing $120,000 for violating one of the country’s island laws on illegal shark finning. According to local news media, authorities discovered multiple shark fins and skins on one of the group’s fishing vessels.

Luen Thai Fishing has since adopted a “total ban shark policy” and has “implemented several procedures” to prevent catching sharks, Wang said. He is optimistic that by working with The Nature Conservancy on adopting new technology, the company “will definitely” further prevent any similar fishing problems in the future.

To help give Luen Thai more insight, The Nature Conservancy has installed nearly a dozen boats with electronic monitoring systems that include a set of cameras, sensors, and GPS devices. The system can record most of what takes place on board to prove that the operators did nothing illegal and to back up any compliance data that they must present to officials when they deliver their catch.

“The large fishing companies may not trust their captains to be doing the right thing,” said Matt Merrifield, the chief technology officer of The Nature Conservancy. But with the monitoring system, the companies can show that the fishermen are “operating by the rules,” he said.

While having access to hours footage is useful, it would be a chore for on-shore workers to review it. A 10 hour trip can take six hours to review, The Nature Conservancy said.

To save time, The Nature Conservancy has partnered with the startup Kaggle to help develop an algorithm that can speed up the process. Ultimately, The Nature Conservancy wants the algorithm embedded into a fully automated software tool that workers could use in their fishing operations.

Kaggle is known for hosting data science competitions on behalf of businesses like Walmart(wmt) and General Electric(ge). Companies involved with the competitions typically submit a huge amount of data, like sales data in Walmart’s case, that researchers then use to build algorithms that can predict outcomes like when would a consumer be most likely to buy a computer in a store.

For the tuna fish project, The Nature Conservancy compiled hours of boating footage, and then sliced the video into thousands of images, said Kaggle CEO Anthony Goldbloom. The non-profit added context to the image data by identifying objects in the pictures like sharks, different tuna species, and turtles.

For the contest, Kaggle will give researchers access to the data set, with half of it containing the correctly labeled data and the other half scrubbed of the relevant information, Goldbloom said. It will be up to researchers to build what’s known as convolutional neural networks, essentially software that loosely mimics how the brain learns, that they can then feed the image data into, he said.

Using the correctly labeled data as the source material, researchers can then construct an algorithm that would sift through the unlabeled data and try to identify the fish or other animals in the images. The researcher who builds the most accurate algorithm will win a $150,000 cash prize.

Man reviewing fishing videos. The Nature Conservancy

The large jackpot was intended to offset some of the costs that researchers will incur in developing their algorithms, Goldbloom said. Running these deep-learning experiments will require buying large amounts of cloud-computing resources from companies like Amazon(amzn) and Microsoft(msft) to crunch the data, he said.

Still, this challenge will be difficult for researchers to solve, Goldbloom warned. Fishing images are more difficult to decipher than the images that companies like Google and Facebook used to recognize cats and dogs.

Poor lighting conditions, rain drops hitting the camera lenses, and people obstructing the view of the fish makes the job potentially messy.

“This is a problem to test the limits of machine learning,” Goldbloom said.

But with the rapid advances in artificial intelligence technologies due in part to the rise of computing power and the amount of available data, Goldbloom is confident that researchers can build an algorithm that can at least count the correct number of fish in the pictures. From there, the researchers can fine-tune their algorithms to distinguish turtles and sharks from tuna, and then between the different tuna species.

“My hunch is that this should be possible,” Goldbloom said. He expects that after a month, researchers should be “within striking distance” of a workable algorithm. The contest will conclude by April.

For The Nature Conservancy, having a fish-identifying algorithm is just one part of a years-long process in what it hopes to be a full-fledged monitoring system.

For more about artificial intelligence, watch:

“The algorithm and the machine learning is really the kernel we would build a whole piece of operational software around,” said Merrifield.

It will be up to the artificial intelligence community to prove that machine and deep learning is “really as good as they say it is,” he said.

SPONSORED FINANCIAL CONTENT

You May Like

Read More

Sign Up for Our Newsletters

Sign up now to receive FORTUNE's best content, special offers, and much more.