Pinterest adds visual search for elements in images and through your camera

Pinterest said today it’s launching three new products today that will point out specific elements in pictures — whether viewed live through a camera or through a typical image search — and use them as a jumping point for search.

All of these are designed to keep Pinterest coming back over and over to discover ideas based on images. Pinterest has been increasingly trying to close the gap from a user initially viewing an image to being able to jump to ideas and products with a single step, and adding these new in-image search capabilities is another step toward that.

“Early information technology used words to connect ideas, like hyperlinks,” co-founder and chief product officer Evan Sharp said. “Search engines we built today have drafted on that, they rely on words to get you answers to your questions. But when it comes to searching for ideas, words aren’t the right way. Sometimes you don’t really know what you’re looking for until you see it.”

So let’s break down each product, starting with the most important one, Lens. That gives users a way to open their camera, look at any image and Pinterest’s Lens feature will automatically pick apart the objects in an image. That can drill down into foods, animals, or even patterns like hexagons. That gives users the ability to start searching for related elements through that. Lens is launching in beta today on iOS.

The main reason why this is so critical is that it means Pinterest may be able to capture that brief moment that a customer might have to just make an impulse purchase. That moment can be incredibly fleeting, and lowering the friction toward seeing something in the real world and making that purchase can capture that in a way that other companies may not be able.

Pinterest is also updating its visual search when it comes to finding specific products, isolating each product within an image. So if you’re looking at a pin from a company that may be selling a jacket, it will also pick up the image of the boots and let you jump to them. Users can also jump to additional related content to those products or elements in the photo. With most of Pinterest’s content coming from Pinterest, this gives Pinterest a way to seamlessly jump through products — and offers businesses a way to build awareness for their other products.

Instant Ideas adds a small little circle to the bottom of each pin, allowing them to jump straight into related elements and gather additional ideas related to that topic. This one seems pointed toward getting users to find products and ideas that they’ll save on their Pinboards — like recipes or potential styles.

Pinterest has largely become synonymous with visual search, which has become the company’s specialty and point of differentiation against other networks. With 150 million users, Pinterest is geared toward getting people to come in and start sort of wandering around to discover ideas and products they might not have known they wanted.

However, we’re starting to see some of these tools trickle down into other services, though maybe in a different fashion. Houzz, for example, breaks down specific products in a photo of a room or home that users can purchase. There are startups like Clarifai want to equip small businesses with similar visual search tools, though they take more of a metadata and tagging approach that can train their algorithms. And there’s always Google, which has invested heavily in visual search, but has yet to necessarily weaponize it in the same way Pinterest has for potential advertisers.

Nevertheless, these Pinterest products are a potential gold mine for those marketers. Pinterest is able to potentially engage with users at different points in their purchasing lifetime. Whether that’s in the mode where they are looking to discover ideas — and build brand awareness — to drilling them into finding a specific product and buying it, Pinterest offers a wide range of advertising products to get at each part of the customer’s shopping timeline.

Pinterest is going to have to solidify its pitch that it is one of the best visual search companies in order to continue to woo advertisers, which may still be treating Pinterest as more of a curiosity than a consistent ad buy. Pinterest is going to have to battle Snap, which is expected to go public next year, as a tool for building brand awareness and capture a potential customer’s attention at the beginning of their shopping lifetime. And there’s always Facebook, which has become a mainstay of marketers.

That’s going to come through a combination of new ad products — like its new addition of search ads — and also by improving its suite of products that it can present to advertisers as unique and differentiated from traditional ad buys. Pinterest, while growing quickly, was a bit off targets it initially set in early 2015 and has to figure out how to re-adjust its expectations as to what kind of advertising and consumer products marketers want.

“These three new products make anything in the world an entry point to the 100 billion ideas in Pinterest,” Pinterest CEO and co-foudner Ben Silbermann said. “Together they create a whole new discovery experience that’s unlike anything that’s out there today. You can get ideas whether you’re opening the app or walking through town. The more people the use it, the better the results become, the more we can recommend inspiring ideas.”