Posted
by
timothy
on Thursday September 16, 2010 @04:59PM
from the suckling-at-the-boob-tube dept.

An anonymous reader writes "Netflix has released some statistics about its users, showing that more than one percent of its customer base has rated 5,000 shows or more, and a few hundred users have rated over 50,000. A reporter for The Atlantic tracked down a few of those extreme users to find out why they do it. Mike Reilly, a producer, heard about the Netflix prize, and wanted to test the limits of the movie recommendation algorithm. Lorraine Hopping Egan has rated about 6,500 movies, but she still uses word of mouth when trying to decide what to watch."

5k is a lot more plausible for a serious film/TV buff, especially if they went through rating things they'd seen long before Netflix. Probably a mix of bots and real users, at that level. 50k, OTOH, seems pretty much bot-only territory.

I've rated about 3600 titles... but honestly, you don't have to watch an entire movie all the way through to give it a one star "I Hated it" or two star "I didn't like it". Sometimes I'll play a low rated movie for 15-20 minutes or so just to see if it's an under-appreciated gem -- or I'll play a 3 star movie and skip around waiting for it to pick up, then close out and rate it w/ 2 stars because I didn't enjoy it.

5,000 titles isn't that impressive when you consider every TV show, documentary, and mo

I've rated about 3600 titles... but honestly, you don't have to watch an entire movie all the way through to give it a one star "I Hated it" or two star "I didn't like it".

It feels like I have seen hundreds of movies where a brilliant ending changed my impression of the movie from "huge waste of time" to "OMG, that was very clever, I'm going to be thinking about that for a long time!".

True, but I wouldn't do that. If I start to watch a movie, I will finish watching it, no matter how bored I am, because for me the ending and the final impressions left by movie can change it into something that was definitely worth watching, even if it was two hours of boredom at the time.

So what does this mean? It probably just means that in a perfect world, there would be different ratings for "hated it, so I turned it off" and "finished watching it, hated it". Imagine an extreme example where a movie wa

>>>you don't have to watch an entire movie all the way through to give it a one star "I Hated it" or two star "I didn't like it".

Then you shouldn't rate it all. If you've not finished it, don't rate it (same you do not judge a book by its cover). I've discovered that some movies/TV shows are uninteresting but then suddenly turn interesting during the final climax, or sudden plot twist. One of those was Nicole Kidman's "The Others" which is now one of my favorites (seen it 4 or 5 times). Others

If I had rated every children's show (Barney for example) with one star instead of simply clicking "not interested", I would easily have over 4,000 rated by now & it's only my second month of Netflix.

The children's show recommendations are out of hand. It's natural for someone to rate their kids' favorite shows high so that they get recommendations for them, but it's really easy for someone with very similar taste that doesn't have any kids at all to not show even a whisper of the same interest in children's shows. They need to have a checkbox in the account settings that says "have kids" or "don't have kids", and then just not even try to equate ratings between the two groups.

There'd still be people that matched your tastes. The problem is that people rating for themselves and also for their kids are basically merging two viewpoints, and as a result it's applying those recommendations to other people that share one of those viewpoints. I think dividing the two groups is an easy solution that lets everyone get what they want.

If you watch and rate 5 movies/shows per day it only takes 27 years. That's if you don't commit suicide before then.

I once took a weekly history of film class. In one session we watched 3 buster keaton movies in a row, then a Jackie Chan moviethat lifted several scenes right from Keaton classics. By the third Keaton feature in a row, laughs were few and far between because we were literally laughed out after a couple of hours. The JackieChan flick (maybe Police Story?) brought us back around, though, the f

Not the film class I took - we often watched the movies over the span of two class sessions.Okay, we had long films (The Seven Samurai being the longest) and a 2hr session; conversely, I guess you had short films and a 3 or 4 hour session.

It's a simple filter. If someone has a degree then you know at least that they can turn up on time at least a couple of times a year. And probably can read and count to 100.

Yes lots of people without degrees would be better at the job then them. But more of the other people without degrees would be worse. And for a job that really doesn't require a degree you likely don't really care that much if you miss the "best" person because they get caught in the filter, if said filter also removes a large number of

Or watching more than one show per day. Or having watched them in the past, long before Netflix was around, and rating them in their system.

Say you watched five films a day every day(and one and a half hours times five is seven and a half hours a day) it would still take you 28 years to watch 50,000 films, so this figure is only just the right side of impossible, and it's certainly not remotely plausible.

TV shows and movies. Five 30 minute TV shows (sans commercials) is less than two hours. Mix in one movie a day with 4 shows and you have (90+(4*22))=172 minutes, or around 3 hours without commercials (DVD sets or rentals) or 4 hours with commercials. That is possible, although it would be rare. I do know some people who watch more TV than that.

Keep in mind that some of us have been watching TV for several decades now, so I could rate every Brady Bunch, Gilligan's Island, Andy Griffith Show, etc. ever mad

Watching all those movies back to back would take approximately 8.5 years alone. There's no possible way anyone could do that and still have enough time to post to Netflix about it! It has to be either bots randomly rating movies with accounts, or users rating movies based primarily on their title, synopsis, and whether they are likely to want to watch the movie or not.

You're assuming that they're all full-length movies. They can be episodes of TV shows, which can be only 10 minutes, giving 0.9 years. You're also assuming that one netflix user means one person watching the rentals. A family might rent two things for the adults and one for the children and watch them concurrently, but only rate things with one account, which would be 0.3 years back-to-back. If you watch a lot of TV, that's quite feasible spread over 2-3 years.

Sure Slashdot has changed. Back then, people wondered when Slashdot would publish some news for nerds and stuff that matters. Now people wonder when Slashdot stopped publishing news for nerds and stuff that matters. Totally different.

Assuming we rate 50,000 movies at 2 hours a movie, this comes out to approximately 11.4 years of straight time. (i.e. no sleep). This does not include the amount of time to rate these items. I know tv shows are less than two hours, but if these ratings are for a series, as opposed to a simple episode, then even more time will be needed.

I often rate movies I have seen in the past 30 years, including many movies I never finished watching. I also rate series based on if I have watched them, dismissed them, or plan to watch them. I'm not sure how the metric of "how long it would take to watch everything rated" matters anymore than "how long it would take to meta-rank every subdomain of.com"

I've been watching movies and TV for four times that long, so I would only have needed 6 hours a day to reach that benchmark. And, believe it or not, there are actually people in the world who are even older than I am!:)

Of course, you don't necessarily have to watch an entire movie to decide you don't like it--especially with the Netflix rating system, where the primary purpose is to encourage it to suggest other things you might like, so giving low ratings to things you'd never even consider watching isn

I have something like 3000 items rated and my experience is items average a lot less than two hours. I have a lot of 20 minute show episodes rated. Also it's important to note that more than one person use a single account and with steaming you can pump through shows pretty quickly.

My theory is they had to rate that many movies to get the stupid Netflix rating system to make suggestions for them. I was well into 1000+ ratings before it'd even occasionally try. Horrible system.

Several hundred Netflix members have rated more than 50,000 filmed entertainment programs. 50,000! To watch all those at a pace of one movie or TV show per day, it would take 136 years.

But those users are just the extreme end of a broader behavioral pattern. About a tenth of one percent (0.07%) of Netflix users -- more than 10,000 people -- have rated more than 20,000 items. And a full one percent, or nearly 150,000 Netflixers, have rated more than 5,000 movies. By contrast, only 60 percent of Netflix users rate any movies at all, and the typical person only gives out 200 starred grades.

This is typical where you give people the choice of rating something. Same goes for music. People on a music torrent tracker rate every single torrent uploaded, even without ever downloading it - just because they don't like the artist,. and make sure nobody else does either.

Or they do the opposite and rate every single song by his "best" artist a 5/5, even if the song is total shite.

This is more of an internal social conflict rather than some mathematical dillema, it's just people being people (and by people, I mean dicks).

Of course, that's standard anywhere the internet allows you to leave comments on anything. People will post to things they know nothing about just for the sake of seeing their own post counts go up or for the small fame of seeing their screennames out there somewhere, or they'll post on something that even slightly leans to their biases in an attempt to just increase the population of their side represented in the posts underneath.

Hopefully their recommendation system doesn't take into account raters who fall more than 2 standard deviations outside the mean.

Exactly... extreme outliers are the easiest to discard.

It would be more nefarious if there were an organized effort to astroturf for certain films by using a cracked database of valid logins, or by creating a bunch of trial accounts, or something like that.

Hopeably, their recommendation system is based on people whose ratings are vaguely similar to yours, not some absolute measure of popularity. Otherwise, all those, "based on your interest in X and Y, you may like Z" messages are horribly misleading!:)

I have rated 2223 items at Netflix, 99% feature length movies. Since becoming a member I have received and watched 1034 movies. I own a little over 2000 dvds and Blu-Rays. Yes, I watch a lot of movies (and obviously have no life). But, that is my main entertainment; I do not subscribe to cable or satellite tv, I am over 60 years old, and have seen a -lot- of film over the years. And no, I won't tell you where I live or how the alarm system works. So yeah, I suppose I can see how someone could have rated -maybe- 5000 movies/shows. But, 50,000? I can not quite fathom that. Perhaps someone with even less of a life than mine spends all day compulsively rating movies based entirely on their descriptions and cast lists? Robots? The studios and/or MPAA? As a side note, Netflix recommendation engine seems to have no clue whatsoever what I am likely to enjoy, probably due to my wide range of preferences. I am constantly amused at some of the stuff they suggest and have noticed that the old pre-contest engine, for my tastes, was much more likely find stuff I like and potentially rent.

I believe my rate count the last time I checked was around 1800, so I can identify with you. I've had a couple long nights where I'd just rate movies for the fun of it. My queue got quite a bit longer during that process, as I came across movies I'd meant to see but never did, etc.

One thing that caught my attention in the summary and the title was the use of the word "show." If we're including tv shows, and we're including ratings of "not interested", then how much larger does my rate count get if i deci

I suppose I can see how someone could have rated -maybe- 5000 movies/shows. But, 50,000? I can not quite fathom that.

Consider the possibility that you HATE show XYZ. Netflix, however, keeps recommending it to you, no matter how many times you rate a disk in the series poorly (I've seen this myself). It's not at all hard to imagine seeing two or three episodes, and rating 100 disks as 1-star. Of course that's a rather extreme example, but the point is valid, and it's certainly not hard to get from watchin

You don't really rate disks though, you rate entire seasons of shows unless they're somehow not sold as season collections. Most shows are only going to go about 5 seasons or so, so you'd really have to go overboard to get to 50,000 ratings even that way.

You go to http://slashdot.org/firehose [slashdot.org] and look at the articles by clicking on their titles, maybe follow the links they contain to see if the summary is correct and the links work and aren't a trap or anything, then you click the + or the - and pick a category for your reasoning from the inadequate list.

The idea is that when stories like this one come up that are (a) dull, and (b) poorly written, and (c) so is the summary, you can have a voice in saying whether it's forced upon the rest of/. or just scrolls off the bottom of the Firehose, never to be seen again until the inevitable dupe is posted.

But clearly, that ratings system isn't doing a bit of good, because, dayum...

because it's a pain to use after the fact on the computer, when watching on my PS3.

Things netflix needs: To know where credits start in a show, and ignore that you have stopped watching partway though the credits, and not offer you with "resume blah blah" when you have 3 minutes of credits left.

Yes i have seen it show up there, and i know i can rate them there as well. I was thinking more for stuff i have seen outside of netflix. Kinda a pain to rate on the ps3, and i hardly need to login on the computer these days. My dvd cue is ~80 long, and i can search the streaming stuff from the ps3.

It really depends on what they mean by rated - if they include 'not interested' (aka the rating count they show on the "movies you'll love" page), then I have 17344 ratings. However, I estimate I've probably only watched about 2000-3000 movies and TV series (not all through Netflix, of course).

Why do I do it? Because I like to be able to see on my queue when things will be added/removed from watch instantly. Seems like a lot of work, but it really isn't if you sit browsing through descriptions while you

I haven't posted this question to the Netflix folks yet, but I can't believe no one has asked for it. If you are a Netflix person, please take this as a minor request from a customer who just wants to see you do better.

Million dollar prizes for nifty relational search algorithms are neat and all, but how about one simple thing that shouldn't cost more than two weeks developer time and would be a really nice feature: be able to sort a queue, specifically, be able to sort based on date rel

I prefer IMDB for rating stuff. I have a list of about 500 movies I want to see so I don't really need more recommendations right now. I also have been very interested in short subject pieces (short attention span?) lately ranging from animated to pre-1900 Edison etc. stuff. There's a lot of DVDs of these but if I want to keep track of what I've seen and what I thought of individual "movies" I have to go somewhere other than Netflix.

IMDB is very good for this since they're about 98% complete in my current

Watching that many is certainly possible, but recalling them all seems unlikely.

World record holder Gwilym Hughes got it by watching ~14 films a week from 1953 to 2008. He said: "People think that I'm glued to the television set 24 hours a day but I'm not because I'm a member of about 10 organisations. I watch films from about 9pm until about 12. Sometimes I could set up one on the televisions in the study. It works out about 10 to 14 films a week."

His favorite movies is also one of my favorites -- Lawrence of Arabia.