Monday, Gizmodo published an article about the curation practices behind Facebook’s Trending module, and to what extent a curator’s personal biases affect what’s shown to Facebook’s billion-plus active users. (As you can imagine, this has caused some controversy.) Although Gizmodo focused on the role of human curators, those of us who work with algorithms and machine learning have had to confront the fact that biases can end up deeply encoded into supposedly objective systems, including Klout’s own topic classifier -- the system that identifies your Expert topics, as well as the relevant articles for the Explore tab. Here’s a quick roundup of how we in Klout’s Data Science team think about keeping the topic system as unbiased as possible.

Klout’s topic classifier in 30 seconds or less

The major inputs to our topic system are:

The classifications we’re applying:

An ontology of nearly 10,000 human-curated topics

An underlying dictionary of over a million named entities and concepts

The data being classified, which includes:

Social media profiles (to be analyzed for topic expertise and interest)

URLs published on social media and elsewhere (to be analyzed for topical content and served in the Explore tab)

Even without getting into the weeds of the pipeline that brings those two types of input together -- more on that below -- critical readers will already be able to spot a few areas where we’re vulnerable to bias. Let’s walk through them.

Any classification system contains value choices

Should “Autism” be placed under “Diseases” or “Neurology”? Is “the Tea Party movement” distinct from “conservative politics”? Is “Wizard Rock” really a thing? How we answer these questions shapes the experience of our users, and inevitably gives our ontology a point of view.

Longtime users may remember the early days of Klout, when topics were a, shall we say, messy combination of user-submitted tags and data-mined concepts. As the Data Science team worked on regularizing and improving the ontology, we’ve relied on the following principles:

The ontology is a living thing; we should always have tools in place for updates

The ontology should always have up-to-date guidelines defining:

Scope -- what portion of the world we’re describing. (In our case, as much of it as possible.)

Granularity -- at what level of detail we’re describing it. (Do we need to include every actor? Television show? Heavy metal subgenre?)

Voice -- the tone we use in describing it. (Do we use scientific names? Full legal names for persons? Slang?)

A little redundancy won’t hurt -- our system can support topics with some conceptual overlap, so err on the side of inclusion. (For example, both “Gun Rights” and “Gun Control” are topics in the Klout ontology.)

Users are the best source of feedback -- our users have a broader range of perspectives than we do; make it easy for them to alert us to problems

Even so, any time your application or audience changes, it’s important to reassess your classification scheme. One major flaw in Klout’s topic ontology is that it was developed for a U.S. audience, and still needs significant work for other countries and languages.

Staying alert for sins of omission

In addition to the human-curated topics in the ontology, we also use a dictionary of concepts and entities derived from Freebase. Freebase is a widely-used resource in the data science world, but “widely-used” is not the same thing as “perfect”, by any means. The biggest issue with Freebase is what it leaves out; like Wikipedia, it was collectively sourced, and like Wikipedia, it’s biased toward the interests of its editors, and sparse in some areas like cosmetic products and fashion terms, requiring us to develop ways to supplement the dictionary. The moral of the story: it pays to look critically at any pre-packaged data set you plan to use.

Boosting inclusivity

Next, let’s consider the URLs we collect for the Explore tab. The majority are URLs that have been shared on social media, which means they are dominated by the topics most discussed on social media: politics, celebrity news, music, etc. What we sometimes call “niche topics”, like molecular biology, or Wicca, or wheelchairs, naturally are present in fewer URLs. Does that count as a bias? It’s unclear, but it’s not a good end-user experience and risks making some users feel marginalized. As a result, we’ve had to develop backup strategies to increase coverage for less common topics.

The fuzzy line between human bias and business logic

One of the more ironic tidbits in Gizmodo’s article was that Facebook’s curators were told to suppress news about Facebook -- that is, to interfere with the Trending algorithm to avoid the appearance that Facebook was interfering with the Trending algorithm. But that kind of decision is probably familiar to the product managers in the audience, whose goal it is to preserve the user experience. Similarly, a discovery feed like our Explore tab might recommend porn, or spam, or hate speech, and need to be tuned or overridden. To make it even more complicated, the definition of porn, or spam, or hate speech may change from region to region. Keeping those decisions from being made inconsistently or thoughtlessly is really difficult, but our approach has been to define a single owner who both documents the rules and is accessible to discuss individual cases. As others have pointed out, Facebook’s mistake may not have been having curatorial tools, but isolating the employees using them.

Fine, but what about the actual topic algorithm?

Eagle-eyed readers will have noticed that we haven’t touched on the nuts and bolts of how Klout’s topic system actually assigns topics. The challenges of data modeling and debugging machine learning algorithms are pretty well surveyed elsewhere, and how we handle those challenges at Klout would require a dedicated blog post. However, there’s less discussion of how to handle human biases when collecting training or validation data -- how people’s points of view get encoded into the data a given algorithm is trying to approximate. The two approaches often recommended could be described as micromanaging versus crowdsourcing; either a) have an in-house process that includes well-defined guidelines, trained judges, and a reconciliation process for disagreements, or b) have lightweight guidelines but a large number of judges, in the hopes that individual biases will be muted. There are tradeoffs to either approach; our team has recently been relying mostly on in-house validation data, mostly because it’s friendly to our development schedule. But what’s more important, in our experience, is that the potential weaknesses of the training/validation data are known and discussed and documented ahead of time, so that they can be distinguished from problems with the model itself.

No system is perfect, and keeping out bias takes continual work. Although a focus on documentation, consistency, and validation will take you a long way, the very best defense against unintentional bias is a diverse team, who can bring multiple points of view. Want to come work with us?

Sarah Ellinger is the Lead Data Analyst for Klout/Lithium’s Data Science team. She is responsible for overseeing the content of the topic ontology, as well as monitoring the performance of the topic classification system. Sarah attended U.C. Berkeley’s School of Information and has over a decade of experience in taxonomy and web content classification at tech companies large and small. She can be found on Twitter discussing information science and Game of Thrones spoilers @sarahellinger.

You can free pyramid solitaire card game however, many people like play alone as there's no need to search for other players, or to worry about how other players will feel when you beat them click here to play free pyramid solitaire card game http://solitairetimes.com/pyramid

You can free pyramid solitaire card game however, many people like play alone as there's no need to search for other players, or to worry about how other players will feel when you beat them click here to play free pyramid solitaire card best Android split application

Such intelligent work on the subject and ideal way of writing here. I am really impressed! This post is a helpful overview of the particular topic and very actionable. Interesting approach! This is really great, unique and very informative post, I like it. thanks imo for pc, kik for pc.

The point that we as Facebook users ever desired neutrality speaks to a belief in digital democracy. That is the difference that Facebook have themselves fixed up. Paper Help Service - Ordercollegepapers.Com They intentionally positioned themselves as a distribution system, which is explicitly not an editorial object.

All things considered, whenever your application or crowd changes, it's imperative to reassess your characterization conspire. One noteworthy blemish in Klout's subject metaphysics is that it was created for a U.S. group of onlookers, and still needs critical work for different nations and dialects.

Dear customers we will talk with you now on how to work on us to buy and sell شراء اثاث مستعمل جدةused and what are the most important steps Which we adopt We have many and many channels of communication with customers, the most important are the channels of communication through the Internet and through the site of the company buy used furniture and also our pages on social networking sites such as Facebook and Twitter, where the latest news of the company and also We communicate with customers directly and respond to all inquiries …. Second: It is also a channel of communication with customers and the most important but direct شراء الاثاث المستعمل بجدة is through your visit to any of our branches that have already mentioned with the previous address and be dealing directly as you have used شراء اثاث مستعمل بجدةthat you are trying to sell or you have the desire to buy either Our furniture is used in exhibitions of used محلات شراء الاثاث المستعمل بجدة . Third: After contacting us in any of the above mentioned forms and in the case that the client has used furniture that wants to dispense with and sell a representative of our shops to go to the client’s house and preview the purposes that the client wants to sell and also assess them and determine the appropriate price and offer them to the customer and we are our customers We take into account that we give our customers the highest prices in the market at all, but within the appropriate limits of the value of furniture used by them as we do not never achieve the purposes of customers to win their confidence in the sales and purchase and in the case of agreement between our representative and the customer pay We agreed to send a team of workers trained on the operations of jaw and installation for domestic purposes and remove the removal of objects from the house of the client to our workshops … ارقام شراء الاثاث المستعمل بجدة Fourth: We never show the furniture used in our exhibitions before passing on the workshops for maintenance of our ارقام محلات شراء الاثاث المستعمل بجدةwhere we have very professional technicians In the maintenance of home furniture, where if the house furniture have any kind of damage or شركة شراء اثاث مستعمل بجدة we do good repairs to it and re-renewal and tidy and organize and fit in the appropriate shape us and our company until it is presented in the appropriate manner that fits well from the Saudi oil shelf Home furniture

Curation has been everywhere throughout the news for as long as couple of weeks. At that same keynote Apple presented Apple News, and keeping in mind that the introduction made it sound somewhat like those client produced radio stations... Custom Assignment help

Spilled records indicate how Facebook, now the greatest news wholesaler on the planet Assignment Writing Service depends on antiquated news esteems over its calculations to figure out what the most blazing stories will be for the 1 billion individuals who visit the interpersonal organization consistently.

In our enjoy, is that the potential weaknesses of the schooling/validation facts are known and mentioned and documented in advance of time, so that they may be distinguished from issues with the model itself. Pay Someone To Do My Essay

At Assignment, Dubai, the first batch of 6 Journalism students will be graduating this year, I am one of them. The 3 years of undergraduate studies have been like a chase to the finish line. We do not have the specific study materials, but we excel in making use of the most appropriate materials relevant to the topics of the assignment. Life is stressful because a journalist’s routine involves a lot of running behind people, trying to reach the important people, getting different views about a specific assignment topic, or just a general talk. Its like digging deep into the matter. UK Essays

In this article, I have concentrated on websites that attention on giving English substance. This article gathers the rundown of ten most went by means of Custom Essay Writing Service UK. The entire rundown is made of different pastime reports and research.

Hey, you used to write excellent, but the last few posts have been kinda boring… I miss your super writings. Past several posts are just a little out of track! come on! happy anniversary wishes for sister