Cross-wiki Search Result Improvements/Testing

The Discovery department is developing the ability to provide search results from other Wikimedia projects within the same language (commonly called cross-wiki or inter-wiki searches).

For example, if you were on the French Wikivoyage and searched for milk you would be presented with the expected matching articles from the French Wikipedia and results from other French Wikimedia projects (i.e.: fr.wiktionary and fr.wikiquote).

This page is an attempt to outline upcoming A/B tests that the search team would like to run and, as always, we invite all contributors to openly discuss these tests on the talk page.

After a conversation with the community, we decided to run tests on Persian, Catalan, Italian and Polish Wikipedias: testing began in mid February 2017.

Update (9Mar2017): due to a higher zero results rate than expected on the originally selected wikis, we'll expand our testing to include French, German, Russian, and Arabic wikipedias for testing.

"Why this test?" We want to find out if adding relevant search results from other wiki's (or projects) would help users discovery additional relevant information and data for the query that they typed in.

"What is this test?" This A/B test will have a test group that will be shown a new right sidebar that will contain relevant search results that were found from cross-wiki searching. A control group will see the currently existing search results page.

Note: not all search queries will return snippet results from the sister projects; some of the projects might display or all of them might display, depending on the search query that was entered.

Test group users will see:

additional search results from sister wikis in a right sidebar

each result for the sister wiki(s) will display:

the top ranked result from any wiki that contain relative search results

an icon that denotes which wiki the result is from

article name of the search result

description of the search result

typical bolding of the search result term(s)

link below the search result that is labeled 'more results'

this link will open a new browser tab and display a search results page for the original search term on that sister project

up to 3 images will be displayed that are relevant to the original search term

display a link that will open a new browser tab and display a search results page of multimedia for the original search term from the native wikipedia that the user is on

for example, if a user searched for 'rainbow' on English Wikipedia, and clicked on the more multimedia link, the user will be displayed search results for multimedia for 'rainbow' on English Wikipedia in a new tab.

The test ran from February 9 - 22, 2017 and the data is being analyzed in T156300. The first draft report show that due to a couple UI bugs and a high zero results returned on the original 4 wikipedias, we had a lack of conclusive results. Therefore, we'll re-run this test; follow ticket T160004 to view the new test criteria. Namely, we'll add in four more wikis to test on and fix the UI bugs we had earlier.

"What is this test?" This A/B test will have a test group that will be shown additional information that will be relevant to individual results on the existing search results page. This additional information will be links and metadata of related pages and links to related categories that are similar to the individual search results. A control group will see the currently existing search results page.

The A/B test was started June 29, 2017 but due to a minor bug that was fixed on July 12, 2017; the official testing period was from July 12 - July 20, 2017.

A second A/B test to display only the language links wrapped up on Sep 21, 2017, after running for a week. Unfortunately, there was only one clickthrough to an article that was written in another language, as the report documented.

"What is this test?" This A/B test will have a test group that will be shown thumbnails for the search query results and updated metadata for each search result. A control group will see the currently existing search results page. The images used in this test will be displayed using the PageImages API extension, similar to what is being used on the Wikipedia portal page in the search box type ahead feature.

"Why this test?" : The Wiktionary widget is a cool little experimental API that pulls information from English Wiktionary and will display a variety of things: pronunciation, a short definition, what part of a sentence it is (noun, adjective, verb) and samples of usage. Depending on the word, you might receive all of that information, some of it, or nothing at all - it all depends on what is in Wiktionary for that word.