If you have searched on Etsy today, you may have noticed something different about your search results. Today we switched the default sort order to relevancy, while previously we had been displaying the most recent results. As we wrote about in the blog [1], forum [2] and discussed in an Online Lab [3] we’ve been focused on improving search sorted by relevancy for the past several months, and we’ve made some significant improvements [1].

We recognize this is a big change to both buyer and seller site experience, and we did not take it lightly. Defaulting to sort by relevancy is something that you have been asking for in the forums for quite some time, and it has been top of mind for us as well. In this post, we’ll break down our thinking and the process that led us to make this change, both at your suggestion and with your help!

For months now, we’ve been working on the changes to search by relevancy and running experiments to evaluate whether search sorted by relevancy is really the best overall move for the marketplace. We also recognize that the search experience has to be optimized for the best experience for two different types of members: buyers and sellers. Though search is primarily a tool for buyers, with a good search experience the whole marketplace wins: a buyer finds what they were looking for and a seller makes a sale. With that in mind, we have performed tests to look at how many items shoppers are purchasing, and the diversity of shops that are presented to them.

Drilling down into those tests, here’s how we decided to make the change. We’ve run dozens of A/B experiments and side-by-side studies [4] which compare member behaviors when given two different search results: sorted by relevancy and sorted by recency. We evaluated how many items a buyer clicks on from a search and compared the number of items clicked and hearted when the results were sorted by recency (the previous default) versus sorted by most relevant. The results spoke for themselves: when shoppers were given results sorted by relevancy they were 4.4% more likely to click on a listing from a search results page. We also saw shoppers viewed 5.5% more listings per search visit when they were given results sorted by relevancy, versus sorted by recency. We also found a similar increase in the number of items a shopper favorited. The results spoke for themselves: when shoppers were given results sorted by relevancy, they were 4.4% more likely to click on a listing from a search results page. We also saw shoppers viewed 5.5% more listings per search visit when they were given results sorted by relevancy, vs. sorted by recency. We also found a similar increase in the number of items a shopper favorited: listing views and favorites were significantly higher when results were sorted by relevancy. And as you can guess, sales go up with increases in listing views and favorites. In a blind side-by-side comparison of the search results sorted by relevancy and sorted by recency, shoppers preferred the sort by relevancy results 15 to 1.

What’s the meaning behind these results? With search sorted by relevancy, the results of the search are, well...more relevant! See for yourself — try preforming a search for “desk” and check out the results. Then click on the sort by “recency” in the sort by bar at the top of your results — this will show the default results page you saw yesterday. With recency sort, chances are you may see a lot of items that are not in fact desks: desk lamps, desk chairs or other items that go in or around desks. If you’re searching for a desk, you really don't want to weed through results including stationary, pencils or other small desk related items, do you?

One of the other benefits of search by relevancy is that you will see a wider range of shops selling the item you are looking for. For example, in search sorted by relevancy, we optimize for showing as many relevant items from multiple shops so that a single shop doesn't dominate the results. In a search sorted by recency, we cannot control the diversity of the results since we are simply showing you the most recent items to be listed or renewed on Etsy. As a result, your search results page sorted by most recent could be dominated by one shop, even when several other shops were offering equally relevant items, which makes it difficult for smaller shops to get some visibility. It also leads to massive shopper frustration — most shoppers leave when they can’t find what they want, and sellers can’t make any money without shoppers! Now, we break up items from the same shop when there are others items that are equally relevant so that all relevant items get a better chance at making it into the first pages of search results. Of course, if you preform a very specific search, you may still see multiple items from the same shop on your homepage, as we will aim to give you the most relevant results to your search as possible.

For sellers, we recognize that there may be some confusion around how to make your items show up in the search results sorted by relevancy, and that’s why we’ve given you some tips and tricks in this forum post [2]. Previously, you were able to list new items and renew current ones in order to give your items a boost in the search results sorted by recency, and our relevancy algorithm isn't quite as straightforward. However, recently listed items will still receive placement across the site. Namely, we have added a section a the top of search results pages sorted by recency. This section is reserved for the four most recent items, with the ability to see more without leaving the page, by clicking the arrows. Recently listed items will also continue to appear at the bottom of the Etsy homepage, and the category pages will continue to be sorted by most recent. And of course, if you ever want to view your search results with the most recent first (like you viewed them yesterday) you can click on “most recent” in the sort by bar.

We will continue to post articles on search optimization in the coming weeks as we want to be as helpful as possible in helping you get your listings on top. For seller tips on revising your listings check out this forum thread [2] and this blog post [1].

We will also be streaming a Q&A with the Etsy Search Admin in the Online Labs on Thursday, August 11th 3 p.m. ET. [5]

Comments or questions? Please post them in this thread and we’ll answer the most pressing ones during our Q&A.

We appreciate your trust and patience with us in making this change. Together we are building the most diverse and exciting handmade marketplace!

16
Highlighted Responses

ie: all other things being equal, will I show up ahead of an older listing in relevancy if I renew?

---

Great question. Recency does play a part in relevancy. You can read more about this, here:

“Even when viewing search results sorted by relevancy, recency of the item matters, especially when searches are broad and return a lot of results. When a broad search (like “dress”) returns over 100,000 results, we give some priority to the most recently posted items. We also have a “most recent” section at the top of the search results where we will display the top 4 most recent items.”

What I see is "RECENTLY LISTED" as "featured" across the top, and "RELEVANT" down beneath that. (in all of my shops it appears the same when I search, and I have signed in and out, and re-loaded my browser)

_____

In the post above we mentioned this:
" recently listed items will still receive placement across the site. Namely, we have added a section a the top of search results pages sorted by recency. This section is reserved for the four most recent items, with the ability to see more without leaving the page, by clicking the arrows. Recently listed items will also continue to appear at the bottom of the Etsy homepage, and the category pages will continue to be sorted by most recent."

pinemeadowcrafts says
The sense of community is not as strong as it was in the past and it feels like the big time sellers that bring Etsy the most money are the ones that matter. What about us little guys that struggle to make a profit but are here because we love what we do? Etsy, we are not bringing you the big dollars like the resellers and the sellers of the licensed characters but don't we still matter? We may not be bringing you a lot of money individually but I think us "little people" are what make Etsy so special. Don't you want to continue to be the selling forum for us too? Don't you want to be like the shoe company that people are proud to support?
___

Makes me sad to hear this.
I wanted to clarify my post. Today we changed to relevancy to establish fairness between sellers big and small in the marketplace, so that sellers who paid more money to renew their items wouldn't displace relevant content in the marketplace and so that all items that are properly titled and tagged have a fair chance to rank at the top of search results. We do not in fact prefer big or small sellers, that has nothing to do with our algorithm, the point is that sort by relevancy levels the playing field. We feel that showing more variety gives buyers more chances to find shops that interest them.

Jalycme says
I think "Reserved listings" should have less relevancy in the title. I did the girl green dress search too and 3 of the items on the first page were all from the same seller and they were reserved listings (those are the first couple of words in the listing too)... I don't understand that.

i did read the OP and thread, but am a bit fuzzy today, so forgive me if my question has been asked/answered...

do the new attributes on the listing page contribute to an item's relevancy?

---

Great question, modflo! At the moment we're collecting that info, but it's not yet being used in search (so nope, it doesn't affect an item's relevancy). When and if we do decide to start using this info we'll start with small experiments to determine the best ways to incorporate the new attributes to improve search. As we get closer to releasing search improvements that leverage this new information to everyone, we will post an announcement in the forums. You can read more about attributes and see some Q&A's, here:

Right now, we do not plan to change search results back to default to sort by recency because we feel relevancy is the strongest search experience. As always our goal is to provide the best experience for the marketplace, and we're doing this behind the scenes by reading your feedback (especially here in the Forums), observing buyer and seller behavior on the site (see the OP for more info on relevancy stats and we'll continue to evaluate this closely) and doing experiments to determine what features and settings will connect buyers and sellers as quickly/accurately/happily as possible.

Admin--is it possible to answer if tags with one word or more than one word have more weight? Or does it just go by each word whether or not they are in the same tag box?

Thank you!

---

Hey peaseblossomstudio,

Let me pull this text from Natalie's relevancy blog article as I think it gets right at what you're asking:

"Tags: Tags (or “keywords”) have always been an important factor in having your items found in search results, as they are your way of providing a short and explicit description of your item. We have also added an additional search feature that gives exact word pairs more weight. When your search includes two or more words, we prioritize results that have those exact words next to each other. For example, if I search for “pink unicorn,” results that contain “pink unicorn” in the title or tags will appear higher than those who title an item “Pink Flowers and Unicorn.”

In regards to the many questions I have seen around tag stuffing policy I want to let you know that the Search Team and policy teams have been looking at this carefully, comparing what the policy says to what works best in relevancy-based search. Ideally what we want is both a product (search) and a policy that best serve the marketplace. This may mean changing the rules for what kinds of multi-word tags are allowed. We want to be sure we get it right, in the best interest for both shoppers and sellers.

I do wish we'd worked things out a bit sooner in regard to clarifying this matter, to have avoided this confusing situation. I appreciate the keen attention to detail and respect for the rules that you and other sellers have demonstrated in the Forums on this topic.

We are working as quickly and carefully as we can to ensure we do the right thing. I appreciate your patience.

"Natalie, as a larger seller I don't think that what you just said is quite fair. If I am selling well on your site and am able to renew my sold items and come up well in the search results, why should that be a problem? If you are going to be manipulating the search results so that smaller sellers are coming up better than the larger ones, this is totally unfair.

And right now, the search results that I am looking at is totally favoring one shop (23 listings on the first page of search results). Will this be fixed?"

Agree with above. For many larger sellers Etsy is their sole means of support and they put in the time. Lots and lots and lots of time working at their job. Most of the larger sellers I know work 16 plus hours a day. Myself included.

Relevancy is not working as intended. Results are heavily diluted with non-relevant content. The idea is good but at this time it is punishing sellers who do relist a lot of sold items. It is the first day but so far, the same issues are still a major problem.

--------------

Hi MeandMatilda - thank you for sharing your feedback. We will continue to improve relevancy as we move forward. You shouldn't be seeing the same seller with a bunch of listings on the first page of a search- we have implemented an improvement to relevancy where the search results are diversified so that buyers won't see results from the same seller bunched together. This might be happening if there are only a few sellers in that search- is that possible here? Would you mind convoing me a link to the search that you did? We would like to look into this, as that shouldn't be happening.

Great question! This is an appropriate result because it is a listing for a portrait of a child, pet, etc. Having the exact phrase "childrens portrait" is helping it to be more relevant for the search term.

Okay, I've been researching this since a few hours ago, and here's what i think I've figured out:

1. Look at your Shop Stats. Pick the top 3 phrases that people use to find your shop.

2. Split your listings up 3 ways so that each listing has one of those phrases as the first words of your title. Example: my top 3 search phrases are "tissue pom pom", "paper pom pom", and "tissue pom." Each one of my listings now has 1 of those phrases as the first words in the title.

3. Include ALL 3 of those phrases in EACH listing as a tag. I know this isn't exactly following the rules right now, but mine are at least phrases that make sense instead of a tag something like "pink coral dress coat".

Making those changes has taken me from page 5 to page 1 when I search each phrase. I know my buyers will find me because that's what THEY are using to search for me.

Hope this helps some people - I was freaking out, but I'm glad I figured out how to get myself back on top.

***Edited to add: those phrases (for me) are pretty specific, so that might be why I'm coming out on page 1. For someone who's top search is "sterling silver bracelet", you might have more competition to get on the first page. I also renew about 12 items a day right now, and the items I renewed today are definitely the closest to the top. So, renewing DOES have some effect.

Admin, I know you are working to make a better search experience, but this change does raise some concerns. There are always unforeseen consequences with even the best-intentioned changes.

My two biggest concerns are these:

1.) Items with unique titles (which make sense for artwork), rather than descriptive titles (like "Pink Unicorn Tshirt") are never going to be found if titles are given weight over tags/descriptions.

This puts artwork at a particular disadvantage, while it would hopefully work well for functional objects. To succeed in the new search, I fear my titles might need to change from "Kitten Bandit Print" to "Print Cat Girl Cute Superhero Halloween" or something equally awful.

2.) The results for search terms will remain much more static than in the past, when listings where rolling by chronologically.

Ie: if one searches for "Alice in Wonderland" under the current system, there are 16,000+ resulting listings, but the searcher is likely to always see the same search returns on the first 5 pages (out of thousands of possibilities). Perhaps this is the Etsy algorithm of relevance at work, but it seems it will just permanently bury those other 16,000 listings.

Thoughts?

-------

Hi!

Thanks so much for sharing your thoughts - you make some excellent points.

1. If search traffic is a major source of traffic in your shop, you might consider putting a few relevant keywords first in your title, then follow this with the title of the piece. That said, this is a good point about Art, and we are looking into ways to improve this.

2. We definitely agree that search results should change and stay fresh rather than being static. We have kept recency as a factor in relevancy to help keep the results changing. It's important to us to try to find the "best fit" results while keeping the results fresh.

Thank you for sharing all of your feedback about the default switch to sorting by relevancy. We appreciate you taking the time to learn about it, and we’ve gotten a lot of good questions and suggestions. As search is certainly never a finished project, your feedback is needed in order to continue to improve. We’ve been watching this forum for the past day and collecting the most frequently asked questions, and their answers below.

1) I’m seeing items when I perform a search in one marketplace (vintage, handmade or supplies) that I don’t think belong there, why?

We determine which marketplace an item belongs in based on the top 3 questions asked in the listing process in the about this item section. In this section the seller must answer 3 questions:
1. Who made it?
2. What is it?
3. When was it made?

The answers to these questions determine whether they appear in vintage, supplies or handmade searches, and depending on the answers, an item can appear in more than one search marketplace. For example, if you answered “who made it” with “I did,” and you answered “when was it made?” with a date over 20 years ago, your item may appear in both the vintage and handmade searches. Another example I recently found was handspun yarn-- which would appear in both supplies and handmade marketplace.

If you are curious as to whether an item is handmade, vintage and/or a supply, you can scroll to the bottom of the item’s live listing and it will say “supply, handmade or vintage” (or a combination of the two like “handmade supply”) in the “About this item” section, below the payment methods on the item page.

2) I’m not seeing the diversity of shops that you mentioned in your post: “in search sorted by relevancy, we optimize to show many relevant items from multiple shops, so that a single shop doesn’t dominate the results.” What’s going on?

Our primary goal is to show shoppers the most relevant search results first. After that, we aim to show as many diverse shops as possible, provided all other factors are equally relevant. This is best seen when there are more than 100,000 results for a search. We are constantly exploring ways to improve search results and making sure we are displaying as diverse relevant content as possible at the top of our list.

3) Some of you have mentioned that the results to relevant searches are very stagnant...

Again, our primary goal is to show shoppers the most relevant search results first. As we mentioned in our blog post last week, when a broad search (like “dress”) returns over 100,000 results, we give some priority to the most recently post items. Broad searches return lots of results, and we view recency as a tie-breaker when there are lots of equally relevant results. Stagnant results are not ideal for shop owners and we are currently evaluating how best to proceed, especially when there are less than 100,000 results for a search.

4) Does the order of tags matter? Do I need to put words in my tags that are also in my title?

Unlike the order of words in a listing title, the order of tags does not matter. Lifted from our blog post last week [1]:
….as a first order of business when listing an item, you should make sure you have accurate and descriptive titles for all of the items you list. Additionally, we look at words at the beginning of your title first (just like your buyers do!) and pay less attention to words toward the end of the title. In practice, you should make sure the first words of your title explain what your item is.

Tags should accurately describe your item, think of them as common words that buyers will search for in looking for your item. You may consider putting the most important keywords in both your title and tags. Most importantly, make sure your item titles are easily readable for potential buyers to understand, click, and buy.

5) There have been a lot of questions about what our policy is on tagging and tag stuffing.

The search and policy teams have been looking at this very carefully, comparing what the policy says to what works best in relevancy-based search. Our top priority is a search experience and a policy that best serve the marketplace. This may mean changing the rules for what kinds of multi-word tags are allowed. We want to be sure we get it right, in the best interest for both shoppers and sellers. Looking back, we wish we would have worked things out a bit sooner in regard to clarifying this matter, to have avoided this confusing situation.

We appreciate the keen attention to detail and respect for the rules the community has demonstrated in the Forums on this topic. We are working as quickly and carefully as we can to ensure we do the right thing and hope to have clarification very soon. I appreciate your patience.

Thanks again for all of your questions, comments and feedback regarding the relevancy launch. We could appreciate it all very much, and are working hard to create the best possible search experience for buyers and sellers in the marketplace.

For sellers who are wondering how search sorted by relevancy has affected their shops, we recommend checking out shop stats to learn about how shoppers are finding your items.

As always, we want to remind you that search on Etsy is by no means a solved problem, and the launch of search sorted by relevancy is not the end of our search story. We will continue to improve search, and we will continue to do testing and experiments, as well as taking your feedback and suggestions into account -- so keep it coming!

Also, we will be a Q&A in the Online Labs later today: Thursday August 11th at 3pm EST where we’ll be answering your most pressing questions about search sorted by relevancy. Join us here: www.livestream.com/etsy?rsvptoeventid=410816.

So today in the live web broadcast they admitted the a relevancy search does not make sense in the jewelry catagory sowhy don't they change it back?????
______________________________

OMG. I'm still catching up... Did they REALLY say that?! *head/desk*

----

Just to clarify: we did not say that relevancy search does not make sense for 'jewelry" searches. We did explain that recency is a factor in the relevancy ranking of broad searches with more than 100,000 items. A search for the keyword 'jewelry' is an example of a broad search.

-Frank

2582
Responses

for instance: if you do a search, and there is one item that is the first, "most relevant" item on the page...and it gets clicked the most (because it is the first item on the page), does it become even MORE relevant?
Thanks
___________________________________________

Ahhh, so Etsy *likes* the idea of being able to use multi-word tags, but jumped the gun on calculating that into relevancy.

I'm going to guess that the tagging policy gets changed. It really does make sense that if you're selling a sterling silver bracelet, you should be able to use one tag that says "sterling silver bracelet". That way if someone searches for that exact term, you're going to come up. Too bad that it's against the rules for the time being.

I do wish we'd worked things out a bit sooner in regard to clarifying this matter, to have avoided this confusing situation. I appreciate the keen attention to detail and respect for the rules that you and other sellers have demonstrated in the Forums on this topic.

We are working as quickly and carefully as we can to ensure we do the right thing. I appreciate your patience.

----------

We want you to be careful and get it right too, and appreciate your answer. But this begs the question: why is it being rolled out as the default when it's still got lots of problems?

Though I agree with someone's statement earlier that the category bleed issue is much more distressing to me than the tag stuffing, and I haven't yet seen that issue addressed (or did I miss it?). Is someone working to correct this search-wide issue?

"Natalie, as a larger seller I don't think that what you just said is quite fair. If I am selling well on your site and am able to renew my sold items and come up well in the search results, why should that be a problem? If you are going to be manipulating the search results so that smaller sellers are coming up better than the larger ones, this is totally unfair.

And right now, the search results that I am looking at is totally favoring one shop (23 listings on the first page of search results). Will this be fixed?"

Agree with above. For many larger sellers Etsy is their sole means of support and they put in the time. Lots and lots and lots of time working at their job. Most of the larger sellers I know work 16 plus hours a day. Myself included.

Relevancy is not working as intended. Results are heavily diluted with non-relevant content. The idea is good but at this time it is punishing sellers who do relist a lot of sold items. It is the first day but so far, the same issues are still a major problem.

--------------

Hi MeandMatilda - thank you for sharing your feedback. We will continue to improve relevancy as we move forward. You shouldn't be seeing the same seller with a bunch of listings on the first page of a search- we have implemented an improvement to relevancy where the search results are diversified so that buyers won't see results from the same seller bunched together. This might be happening if there are only a few sellers in that search- is that possible here? Would you mind convoing me a link to the search that you did? We would like to look into this, as that shouldn't be happening.