(1) Sun Aug 07 2011 11:57The MST3K-IMDB Effect, Quantified:
Sometimes when I rewatch an MST3K episode I go to the movie's IMDB page to learn more about it. Inevitably I'm annoyed by the comments of people who give these movies one-star reviews solely on the basis of having watched an edited version on MST3K. But even greater than my annoyance is my desire to quantify the phenomenon. Today, I have quantified it.

What does being on MST3K do to a movie's IMDB rating? My best guess is that it knocks 2.9 stars off what what the rating would have been if the movie hadn't been on MST3K. But read on to see how I came up with that number, and why it depends on the director.

Disclaimer

I am not a statistician. I'm not even a data scientist. I know how to get data out of the Internet. I know the difference between mean, median, mode, and standard deviation. And that's about it.

Example

Here's an example in case you're not familiar with the MST3K-IMDB effect, which there's no reason you should since that's a name I just made up for it. Consider "Speech: The Function of Gestures", a short film directed by Arthur H. Wolf. It's got 5 votes and an IMDB rating of 5.2. Now here's another short film in the same series, "Speech: Platform Posture and Appearance". Same director, same writer, same lead actor, but this film had the misfortune to be double-billed with Red Zone Cuba on MST3K. As a result, it's got 98 votes and an IMDB rating of 1.6.

Call me skeptical, but I've watched both films and I'm not convinced there's really a three-and-a-half star difference between them. Another film in the series, "Speech: Using Your Voice", was also featured on MST3K, but in a less memorable episode ("Earth vs. the Spider"), and it struggles along with an IMDB rating of 2.4.)

Methodology

Since the "Speech" films are part of a series, it makes sense to suppose that the difference between them is mostly due to the MST3K-IMDB effect. Of course, most films aren't part of a series. So I went by
director instead. I picked up the filmography of every director who
directed a film that was on MST3K. I split their films into two
lists, "Normal" (not featured on MST3K) and "MST" (featured on
MST3K). The "Normal" set only includes films that had enough IMDB
votes to be given a rating. I included shorts and episodes of TV
shows. This isn't perfect, because IMDB's plain-text data dump
sometimes (but not always) gives a director's credit where their
website gives a writer's credit. But it's close enough.

I took the average rating of the "Normal" list and the "MST"
list. The difference between the two averages is how much it
hurt that director to have one of their films featured on
MST3K. As we'll see see, some directors were hurt a lot, and some of
them shrugged it off, both for interesting reasons.

For the sake of comparison, the mean rating for a movie on IMDB at large is 6.4 stars, the median is 6.6 stars, and the standard deviation is 1.6 stars. However, "one star" is not a consistent unit of measurement. I'm considering redoing this table with normalized percentiles, but I'm not convinced there's a big demand for that, so for now you get stars.

Data

Here's a big table with the data for every director who had at at least five films in the "Normal" set and at least one in the "MST" set. Normalm and Normalstd are the mean and standard deviation for the IMDB ratings of that director's non-MST films, and Normaln is the sample size. MSTm, MSTstd, and MSTn are the same thing for the director's MST film(s).

Effect1 is what we're looking for: for a given director, how many stars does a film lose just from being on MST3K? But wait! What if the director made some good stuff and some bad stuff, and only the bad stuff ended up on MST3K? The MST3K set would have lower ratings, but it wouldn't be because of MST3K. That's where Effect2 comes in, and that's why the table is sorted by Effect2. I'll explain Effect2 after you get a look at the data.

The Incredibly Strange Creatures Who Stopped Living and Became Mixed-Up Zombies!!? (1964)

Rebane, Bill

2.7

0.7

8

2.1

0.6

2

0.6

0.8

Monster a-Go Go (1965), The Giant Spider Invasion (1975)

Burke, Martyn

5.8

1.5

7

4.4

-

1

1.4

0.9

The Last Chase (1981)

Sachs, William

4.3

1.1

9

3.2

-

1

1.1

0.9

The Incredible Melting Man (1977)

Warren, Jerry

2.4

0.6

8

1.8

-

1

0.6

0.9

The Wild World of Batwoman (1966)

Maetzig, Kurt

5.4

1.3

13

4.0

-

1

1.4

1.1

Der schweigende Stern (1960)

Buchanan, Larry

3.3

1.2

25

2.0

-

1

1.3

1.1

The Eye Creatures (1965) (TV)

Yuasa, Noriaki

5.0

1.6

12

3.1

-

1

1.9

1.1

Gamera tai daiakuju Giron (1969)

Gordon, Bert I.

4.2

0.8

13

3.2

0.6

8

1.0

1.2

Beginning of the End (1957), Earth vs. the Spider (1958), King Dinosaur (1955), The Amazing Colossal Man (1957), The Magic Sword (1962), Tormented (1960), Village of the Giants (1965), War of the Colossal Beast (1958)

Brannon, Fred C.

5.8

1.3

43

4.2

-

1

1.6

1.2

Radar Men from the Moon (1952)

Mikels, Ted V.

3.3

1.3

19

1.8

-

1

1.5

1.2

Girl in Gold Boots (1968)

Wood Jr., Edward D.

4.0

1.0

15

2.8

0.8

2

1.2

1.2

Bride of the Monster (1955), The Sinister Urge (1960)

Zarindast, Tony

4.2

2.0

10

1.7

-

1

2.5

1.2

Werewolf (1996) (V)

Bradley, David (I)

4.9

1.9

6

2.4

-

1

2.5

1.3

12 to the Moon (1960)

Ludwig, Edward

6.1

1.0

33

4.7

-

1

1.4

1.3

The Black Scorpion (1957)

Clark, Greydon (I)

3.5

1.2

19

1.9

0.0

2

1.6

1.3

Angels' Brigade (1979), Final Justice (1985)

Franco, Jesus

4.1

1.2

166

2.5

-

1

1.6

1.3

The Castle of Fu Manchu (1969)

Eason, B. Reeves

5.8

0.9

46

4.5

-

1

1.3

1.4

Undersea Kingdom (1936)

Pyun, Albert

4.5

1.5

43

2.5

-

1

2.0

1.4

Alien from L.A. (1988)

Sturges, John

6.5

0.7

42

5.5

-

1

1.0

1.4

Marooned (1969)

Neumann, Kurt (I)

6.2

0.9

51

4.8

-

1

1.4

1.4

Rocketship X-M (1950)

Rou, Aleksandr

6.8

1.5

14

4.5

-

1

2.3

1.5

Morozko (1965)

Zens, Will

4.4

1.8

7

1.6

-

1

2.8

1.6

The Starfighters (1964)

Corman, Roger

5.4

1.4

44

3.2

0.8

6

2.1

1.6

Gunslinger (1956), It Conquered the World (1956), Swamp Women (1956), Teenage Cave Man (1958), The Saga of the Viking Women and Their Voyage to the Waters of the Great Sea Serpent (1957), The Undead (1957)

Now, for the explanation of Effect2. From Normalstd we know how likely this director is to make a film that's substantially better or worse than their average. If they made one bad film that was on MST3K, and there was no MST3K-IMDB effect for that director, the rating for that film would most likely be within two standard deviations of the director's average. But if there were a strong MST3K-IMDB effect for that director, the rating for the MSTed film would be much lower than the director's other bad films. So, Effect2 is: how many standard deviations below Normalm is MSTm?

Let's look at the extremes of the list. First, the directors with very low Effect2:

Mario Bava seems to have actually benefited from having his movie Diabolik appear on MST3K! Looking at the IMDB reviews, it seems that the nostalgic 60s-spy-movie fans have taken the upper hand over the MST3K-IMDB effect and given this film a rating in keeping with the director's other films.

At the top but in non-anomaly territory, we have Pietro Francisci,
who directed a bunch of sword-and-sandal movies
(Normalm=5.0), including two of the Hercules
movies shown on MST3K (MSTm=4.8). These movies are
nothing special but they're pretty fun, such that being on MST3K
barely hurts them at all.

On the other hand, we have Ray Dennis Steckler, who directed
movies so bad (Normalm=3.1) that having one of
them appear on MST3K (MSTm=2.1) barely hurts the rating
at all.

And this is the big thing I learned doing the project: you can
calculate the MST3K-IMDB effect, but you must also look at the
director's average movie rating to see what it means. A low
Effect2 just means that being on MST3K doesn't hurt a
director's ratings very much. It doesn't say anything about the movie's
quality.

OTOH, a director with a high Effect2 is probably worth a
second look in a non-MST3K context.

Look at the director at the
bottom, David Elliot, with Effect2 of
14.1. He worked with Gerry Anderson,
creator of "Thunderbirds", and Invaders From The Deep is made up of recut episodes of "Stingray", a "Thunderbirds"-type show. Cutting episodes of a TV show into a movie is never a good
idea, but "Stingray" has an IMDB rating of 7.9, which is quite a way from the MSTm of 2.1.

I'd couldn't even remember Invaders From The Deep being on
MST3K. Investigation reveals
it was on the very first non-pilot episode of MST3K. This episode is lost and no one's seen it for over
twenty years, so why is Effect2 so large? Could it
be that a bunch of MST3K fans gave this movie a one-star rating
without even seeing it on MST3K?

William
Morgan made the questionable decision to direct the Ed Wood-penned The
Violent Years (MSTm=2.6), but he also directed
six-star movies like Mr. District Attorney and Disney's
cartoon compilation Fun and Fancy-Free
(Normalm=6.2). This is a case where it's hard to
isolate the MST3K-IMDB effect. It's probably there, but we're
looking at a guy who made one really bad film in an otherwise
average career, and that bad film also ended up on MST3K.

Douglas Williams
directed Overdrawn at the Memory Bank, a movie that was
pretty good for made-for-TV but whose rating (MSTm=2.1)
is almost ten standard deviations away from his stellar
Normalm of 8.5, which he got from directing episodes
of Fraggle Rock.

Robert
Carlisle directed Last Clear Chance
(MSTm=2.3), a short film that was a hoot on MST3K, but
he also directed a series of shorts called "Unusual Occupations"
that looks interesting. ("A daisy grower invents tinted daisies; a
woman makes 3-D seaweed art[.]") (Normalm=7.3)

Alan J. Levi
directed TV shows from "Battlestar Galactica" in the 1970s to "Airwolf" in
the 1980s to "Columbo" and "Quantum Leap" in the 1990s to "NCIS"
today. (Normalm=7.2). But when the mediocre "Gemini
Man" (IMDB rating: 6.4) had some of its episodes re-cut into Riding
With Death, suddenly the premise becamse ridiculous and the director
earned an MSTm of 1.7.

I always
enjoyed Aleksandr
Ptushko's lush Soviet epics, and IMDB agrees
(Normalm=7.2). But put one of his movies on MST3K and it
instantly loses three stars. (MSTm=4.2)

And so on. The MST3K-IMDB effect is real--ninety percent of the
directors in this table have an Effect2 of more than one
standard deviation, and for sixty percent of them, it's more than two standard deviations. But it doesn't affect all directors equally.

Let's close out by taking a look at some of MST3K's favorite
directors.

Bert I. Gordon has a Normalm=4.2, which is
pretty bad. But his record 8 MST3K movies have an MSTm of
3.2, which isn't much worse. Put one of his movies on MST3K and it
loses only one star.

Roger Corman, the B-movie king, has a pretty respectable (if you're Roger Corman) Normalm=5.4, but the same
MSTm as Bert I. Gordon: 3.2.

Ed Wood isn't especially well-known for being on MST3K but he does
provide a pretty clear example of the MST3K-IMDB effect. His fifteen
non-MST movies have a Normalm=4.0. His two MST movies
have a MSTm of 2.8. Is The Sinister Urge (IMDB rating: 2.11)
really the worst movie Ed Wood ever directed?

Unfortunately, there is no way to calculate the MST3K-IMDB effect
for the notorious Coleman Francis: since every film he directed
was on MST3K, we can't calculate Normalm.

Conclusion

I'm still annoyed by those one-star reviews, but I understand them a little better now. When you watch, say, "The Function of Gestures", you enjoy it for its camp value, you have fun with it, and you give it a relatively good rating. But when you watch "Platform Posture and Appearance" or "Using Your Voice" on MST3K, you're watching someone else making fun of it, you have fun at its expense, and you give it a bad rating as a sign of solidarity with the MST3K characters.

Finally, I'd like to thank IMDB for, in a relic of its geeky past,
making plain-text dumps
of its data available. It's a strange feeling to have a file open
in an Emacs buffer that lists nearly every movie ever made. (There are
about 2 million, if you're curious.) Now that I have the data and
scripts to process it, I may run other cinematic experiments in the
future. One thing I would like to see added is IMDB links for the people and movies. It's a pain to look all these things up, which is why there aren't as many links in this post as you'd think.