Google’s Featured Snippets Algorithm Is Smart Enough To Detect Third Party Image Manipulation and Act Quickly [Case Study]

Ever since featured snippets landed in the SERPs, I’ve been heavily analyzing their impact. They have always fascinated me based on their unique SERP treatment, the amount of traffic they can drive, and how the featured snippets algorithm works. And If you’ve read any of myposts about featured snippets, then you know the algorithm is extremely temperamental. It always seems to be looking for the best answer, and can change on a dime.

In addition, we also know that machine learning is being used by Google to surface the best possible “upshot” for featured snippets. That was explained in an article about Google’s Pygmalion team. So, the algo is continually learning, changing, and looking for the best possible featured snippet for any given query. It’s dynamic, to say the least.
April 2017 – A Big Change To Featured SnippetsIn April, we picked up a pretty big change in how featured snippets were working. There seemed to be a major uptick in the number of featured snippets that were displaying third party images. Typically, a featured snippet that contains an image would display that image from the same page or site being surfaced. For example, like this:

After the update in April, many featured snippets started showing images sourced from third party sites. So, you could technically have a featured snippet from site A, but the image could be sourced from site B. For example, like the following:

Algorithms Are Not PerfectAlthough this seemed to happen in a limited fashion before April of 2017, there was a distinct uptick with the April update. With the change, it seems Google is trying to surface the best image for a featured snippet, no matter where that image comes from. And that can lead to some embarrassing situations. Remember, algorithms aren’t perfect.

For example, check out these images for larger-brand featured snippets, which were surfaced from third party sites:

Big Question: Is Image Manipulation Possible?The good news is that the featured snippets algorithm can move quickly and snippets can change just as fast. The bad news is that many sites suddenly had featured snippets with images sourced from third party websites. And those third parties could even include competitors, low-quality sites, etc.

Based on what I was seeing during my analysis, I started wondering if the images being sourced from third party sites could be manipulated by those third parties. For example, if your image was being sourced for someone else’s featured snippet, could you edit that image to say, and show, whatever you wanted? And then have that displayed in the other site’s featured snippet? That would be crazy, but seemed totally possible to me.

I wanted to test it out, but I needed to be able to control the image being surfaced in a featured snippet for another site. I waited for an opportunity, and then the SEO gods gave me a gift (sort of).

The Situation – How Dare You Google?For a pretty long time I’ve had the featured snippet for “how to demote sitelinks”. I published a tutorial in 2014 that walked site owners through the somewhat confusing process of demoting sitelinks in Google Search Console. The post became pretty popular and ended up with the featured snippet.

Here was my featured snippet:

Then Google had to throw a wrench into the situation and removed the ability to demote sitelinks last year. So, I updated my post with the news and waited to see if the featured snippet would go away. My tutorial didn’t really fit anymore since you cannot walk through that process after Google removed the ability to demote sitelinks.

But I seemed to maintain the featured snippet for a while. That’s until recently when Google swapped out my blog post for another covering a similar process. Yes, my beautiful featured snippet had gone away. So sad. :)

BUT WAIT… was I really gone??

No, I wasn’t completely removed. The image for the featured snippet was still from my website! What?!? Google removed my post as the featured snippet, replaced my post with a tutorial from another site, yet still used the image from my post. I felt violated.

Here was the new featured snippet, but with an image sourced from my blog post!

But remember that test I wanted to run? Not all is lost… I could now run my test since one of my images was being used in a featured snippet for another site. I moved quickly to launch my test.

Setting Up The Test – “Hey, I’m down here!”I fired up Photoshop and opened the image being sourced for the featured snippet from my blog post. It was a good image to test this on, since there was a decent amount of white space. I simply added text notifying users that they could read MY POST which was located down the SERP (and that it was from gsqi.com). Hey, it’s my image and I can do what I want!

Here is what the image looked like after editing. Subtle, right? :)

Then I used fetch as Google to request that my post get indexed quickly. Then I waited. I didn’t have to wait long for Google to index the post and my new image. That happened pretty quickly. It was only about an hour before I saw the new image indexed.

But… the featured snippet image was still the old version (without the text)! The featured snippet algorithm seemed to lag behind what was already indexed. The old image was being used even when the new one had been indexed already. Very interesting.

Here you can see that new image was indexed in Google images, but the featured snippet still used the old image!

Key Takeaway #1:The featured snippets algorithm seems to lag behind. Even though the new image had been edited, indexed, and showing in Google images, the featured snippet still showed the old image.

So now I just needed to wait and test the featured snippet every hour until a change occurred (if a change occurred). The next morning I noticed a change, and it surprised me.

“Clever girl…” The Featured Snippet Changed But…The next morning I checked the snippet to see if the image would reveal my subtle message. Low and behold, the featured snippet image changed… but not exactly the way I thought it would! The featured snippet algorithm seemed to detect manipulation and swapped out that image for another one, BUT STILL FROM MY POST.

Wow, I was extremely impressed with the sophistication of the algo. All of this happened really quickly… And the algo seemed to understand that the image changed and that it was now being manipulated. So it changed the image to another one from my post. Very clever Google. I’m impressed.

After detecting manipulation, Google swapped out the edited image for another from my blog post!

Key Takeaway #2The featured snippet algorithm is smart enough to detect manipulation. And when it does, it can easily swap out the edited image for another (even if that new image is from the same site). The algo is making decisions just like a human would, and in near-real time.

Reversing Course – What will the algorithm do?After seeing the featured snippet algorithm outwit me, I decided to quickly change my image back to the original version without the message. I removed the text explaining to users that they should read my post, re-uploaded that image, and then used fetch as Google again to request indexation. Then I waited.

The next morning I checked the featured snippet, and boom, the original image was back! Holy smokes. The featured snippet algorithm was smart, really smart. The algo detected that the image was not being manipulated and decided to put it back in the featured snippet.

So what are we dealing with here?? It’s human-like.

After detecting the image was not being manipulated anymore, Google added the image back in the featured snippet:

Key Takeaway #3The featured snippets algorithm was smart to enough to understand that the image was not being manipulated anymore, so it returned that image to the featured snippet right after it was indexed again.

Summary – Google’s Algorithms Keep Getting SmarterI was blown away to see how Google’s featured snippet algorithm was able to detect manipulation, act quickly to remove an image, and then reverse course when it detected the manipulation had stopped. Remember, we’re talking about an algorithm here!

This shows us that Google’s algorithms keep getting smarter and more human-like. We know featured snippets are using machine learning to surface the best possible snippet for the SERP, and now we know that process might be used for images as well.

If your images are being used for featured snippets from other sites, understand that you probably can’t trick Google’s algorithms (for long anyway). If you manipulate the image, it can easily be swapped out. The algorithm is like quasi-human watching the SERPs and ready to take action, when needed. That’s ultra-impressive, but scary at the same time.

Hey, thanks. I’ve been dying to test this out and the opportunity presented itself. I was eagerly waiting to see if the revised image would show up in the featured snippet. The algo seemed to be making human-like decisions, and very quickly. Impressive and scary at the same time! :)

Cooper Fellows

Have you considered leaving the manipulated one up? The algo could very easily have created a computation hash of the image, once that changed (due to the new text) it may have fallen back to a less desirable image while it evaluated the first one (either automatically or manually). I also wonder if they are using character recognition on the featured images to avoid this exact thing?

Great points and hard to say if leaving the edited version up would have changed anything. I believe it was flagged algorithmically based the image manipulation (just due to how the situation unfolded).

As soon as I uploaded a clean version, it popped back into the featured snippet. I’m going to look for more examples where I can test this. Stay tuned. :)

I noticed that many featured snippets that didn’t have images suddenly had them after the April update (but they were sourced from third party sites). I do also think that Google wants to provide the best image it can for each featured snippet, even if it’s not from the same article/post. One thing is for sure… there was a big uptick in third party images after the April update. :)

Richard

As an aside, there are some examples of featured snippets extracted from expired domains still appearing weeks or months after the expiry.

A good follow-up to this would be to test what changes trigger this behaviour. If you crop the image differently do you see the same outcome? If you add less obtrusive text what happens?

Yes, great ideas. I’ll be on the lookout for more opportunities to test this. Stay tuned. :) BTW, do you have any examples of featured snippets from expired domains? I’ve seen featured snippets from pages that 404, but not from expired domains. Thanks.