Friday, August 14, 2009

For years now, lots of folks have been saying that hip hop is dead. I’ve always preferred to use the words “declining” or “dying” (notice the difference) or even “has cancer”. It might be a bit grimmer since it’s describing an ongoing, depressingly downward spiral as opposed to an occurrence of the past; but it’s more accurate in my book. And I might just be able to prove its validity!

I was honored to be a part of Jeff Weiss’ ‘Top 50 Rap Albums of the '00s’ project, featuring a roster of well-regarded hip hop experts whom I read, occasionally converse with, but most of all respect. I certainly didn’t agree with everybody’s lists, but I could understand where they were coming from. Soulja Boy, Hurricane Chris, Gorilla Zoe, Huey, Mims, Rich Boy – none of these guys were on anybody’s list. Not even at #50 just for kicks. Hip hop certainly has a wide spectrum, proof positive by the fact that no two lists looked identical (though of course there were many similarities – top ten entries to be specific). In short, not all of the contributors’ lists looked like my ideal list, but they were all reasonable. And that’s what makes them all so important – as the old saying goes, “the more, the merrier” – for a study I decided on toiling with: proving hip hop’s decline over the decade.

My hypothesis, as you could probably gather by now, was to prove that hip hop has declined in quality over the years. I went about trying to prove this by collecting the data from Weiss’ ‘Complete Voters Ballots’ page. First, I copy-and-pasted everybody’s list and spent several minutes removing the numbers in front of the entries (nearly 1,600 total). It was a strenuous task, but I did it while listening to Pete Rock & CL Smooth, so it wasn’t that bad. Next, I plugged them all into a Microsoft Excel spreadsheet and alphabetized them. The next step was to count each album and remove any duplicate rows on the spreadsheet. For instance, 50 Cent’s Get Rich or Die Tryin’ was chosen by twenty-one people, so I deleted 20 of the rows with identical content and put the number “21” to the right of the sole cell which read “50 Cent – Get Rich or Die Tryin’”. Are you following me? I did this for each album (over 450 albums in total)!

So now what I had was an Excel file with two columns: one for the artist name/album title, the other for the amount of times it was featured on somebody’s list (i.e. how many total “votes” it received). Now I had to do the real tough work. I had to Google search each and every album title to verify the year it was released. It was a strenuous task, but I did it while listening to Poor Righteous Teachers, so it wasn’t that bad. (Not so) soon afterwards, I had the two aforementioned columns, as well as a third column for all of the release date years. Next up, I used the Sort feature to arrange the entries by year.

Next, I created yet another column which served the purpose of counting the total number of the second column (the vote total per album) per year. To explain, here is an example: the year “2001” had 35 selected albums. And the total sum for all votes for all albums in the year 2001 was 138 (Jay-Z’s The Blueprint contributed to nearly one quarter of the total vote count for that year). I moved all of this new data (I no longer needed the artist name/album title column) to a newer, simpler Sheet which looked like this:

As you’ll notice, the years 1999 and 2000 are combined. This is because a few entries (a total of five, to be exact) from 1999 were included on some peoples’ lists. Albums like MF DOOM’s Operation Doomsday and Q-Tip’s Amplified, for instance, fit into this small category. I decided to throw them into the 2000 group. My rationale for doing this was that if I left 1999 as its own group, it wouldn’t be represented well as a full year in and of itself, barring other great 1999 releases that would have been in peoples’ consideration had the ballot explicitly accepted 1999 releases. Besides, the numbers are so marginal, it wouldn’t make much of a difference. I know that doesn’t sound very scientific, but for all intents and purposes it’s the most practical choice. I mention this all purely for the sake of clarity and honesty. Still, you may ask: “Doesn’t this tilt the count in your hypothesis’ favor?” No! I’ll explain soon enough!

Next up, I created two graphs to visualize the data tables above. Here is Figure 1A:

Figure 1A illustrates a steady rise in the amount of albums per year which were included in everyone’s shared/combined lists. Keep in mind that A) 1999 and 2000 are merged, and B) 2009 is not yet complete. A rise in the amount of albums seems to suggest a rise in the quantity of albums released. In other words, more albums, more choices/options to be included on the list. Below is a chart which aims to illustrate levels of quality per year (not quantity) of hip hop releases:

You might be asking: how can you measure “quality” with just numbers? Here’s my methodology for measuring quality in this test: if an album gets a lot of votes as opposed to another album with very few votes, the album with more votes earns a higher “quality” score than the other. Therefore albums like The Blueprint, which earned lots of votes (The Blueprint landed on all but one person’s list), are more “quality” then, say, Jay-Z’s The Dynasty: Roc La Familia which only earned six people’s votes. In other words, “quantity” equals the total number of albums per year, whereas “quality” equals the total votes of all albums per year.

These two graphs served a purpose, yes, but they didn’t fully satisfy me. I wanted to prove beyond a reasonable doubt that there was a strong correlation to be found here. I did. By dividing the decade into two segments, I could create an easier way to view the charts. I split the decade as follows: 1) 1999/2000 through 2004, and 2) 2005-2009. Below is a chart which displays the rise in quantity of releases from decade half one to decade half two:

In essence, Figure 2A is a simplified version of Figure 1A. You’ll notice that the number 218 (the total number of all albums selected between the release dates of 1999/2000 to 2004) equals the first half of numbers on Figure 1A and so on for the second half. The jump from 218 and 236 may seem insignificant, but it’s worth noting that that’s an 8.25% rise. Keep in mind that this is including A) the extra albums from 1999, and B) the fact that 2009 is technically represented as only a half of a year. Fixing this disadvantage would tilt the incline even more. This is what I was talking about a few paragraphs back. But this isn’t the graph that’s most telling. This is:

This is the decline I’ve been talking about and searching for (what can I say, I had a hunch). This chart displays the decline in votes for the releases of the second half of the decade. The collective of all votes for all selected albums released from 1999/2000 to 2004 is 881, compared to 2005 – 2009’s 676 vote total. That’s a 23.27% drop. Quite significant if you ask me! Even if you were to A) remove all votes for albums from 1999, and B) double the votes for 2009 (being generous, since 2009 is roughly only halfway through), we’d still be facing an 875 to 698 “win” for the first half of the decade. That’s a still-significant 20.23% slide. Now we might know why Dr. Dre is apprehensive about dropping Detox: “Never let me slip, ‘cause if I slip, then I'm slippin’”.

Again, let me break it down: the sum total of all votes by all 30+ participants in Weiss’ ballot showed a disparate preference for early-to-mid-2000’s hip hop as opposed to mid-to-late-2000’s hip hop. Hip hop has placed quantity over quality, plain and simple. So let’s be clear, once and for all: Hip Hop Is Dead Declining! What are you gonna do about it?

P.S. Even though I am Jewish and I do wear glasses, I don’t consider myself a statistics expert. Therefore, if you have any questions or concerns about my procedures, or just downright found a flaw with the way I looked at this data, please lemme know! Otherwise, Nate Silver, eat your heart out (j/k)!

At last someone's done the geeky stuff so people can visualize things a lot easier. The problem is there's too much major label filler (Shit!). All the labels are worried about is sales. It seems that when a label emerges and makes quality music which they and the artists really seem to care about (In my opinion, one example being Rawkus), they eventually fold or get taken over by a major label and never seem to reach the same quality again.

Interesting piece and I think you make a valid point. To play devil's advocate though (because playing devil's advocate is fun), I think there's more than just a decline in quality at play, more specifically what I like to call "critical creep".

I remember a few years ago when you'd ask Hip-Hop fans for a specific year when rap "fell off" they'd name 97 as the tipping point when Hip-Hop went from a pure unmolested art form to an evil shiny suited blingy commercial monster. And older heads were actually likely to pick an even earlier year. Today, that argument sounds nuts (Camp Lo! CNN! Wu-Tang! Biggie! etc...) but that's what it was. Then people slowly warmed up to 98 (Outkast! Jay! Lauryn!) and so on and so on.

My point is that as time moves on, people get more nostalgic for the music of bygone years and begin to reappraise it in a softer focus. I’m pretty sure if (when?) we try this experiment again in a few years the results won’t tip so strongly towards the beginning of the decade as people begin to reevaluate the records they were bumping a few years later. The same way it’s hard to stay mad at Puffy 12 years after the shiny suit era, our anger for Lil Jon (and subsequently, Souljah Boy) will fade in time.

(Although on the real? There are a couple of rotten years if you aks me)

Man, this is some serious work. Nicely done, Ivan.I wouldn't be able to pinpoint a specific year where shit was bad; hell, 2005 was when I started to listen to hip-hop seriously. I think if we did this list in another two decades that the second half of the 00s would get a little more love.

i'm with sach o...no matter when u do this, people will pick the stuff that came out longer ago, not because it's 'better', but because they know it better. and then, i know i find stuff from 2003 that i didnt hear then that changes my top 10 year lists sometimes (well obviously i do not make up a majority, most people stick to what they know), but that has to count for something.

I think I will always look fondly on the Blueprint-era. Nothing will match that because that was MY time.

Even though quality is undeniably declining, I reckon I still wouldnt love anything else as much.

From Chronic to Blueprint, all these records are now in the Hip-Hop museum safe from criticism, and always in our hearts.

The new shit will never match up because music is ever changing. A lot of younger cats probably love the new shit and look down on 'the classics'.

We had lames like Hammer, Lil Jon, Puffy, Nelly?.....but they were 'our' lames so they're not as wack as Soulja Boy Fuck 'Em and Flo-rida.

That said, even just in terms of flow and lyrics, you can still compare quality from different eras. Every era has its BS but the top rap music from now isnt in the same league as the classics from even 6-7 years ago.

Ivan hits the nail on his head.

The 4-eyed, Jewish Geek! Im playing......cos you're the guy who will end up richer than all of us!

This was interesting, and I appreciate the effort you put in -- you should have hit me up and I would have sent you the spreadsheet I used to calculate the results.

I disagree with your conclusion though. I think it's only natural that consensus solidifies over time. Our voters considered that there were many albums worth voting for from recent years (your "quantity" chart), but they only came to an agreement over which ones with the benefit of time.

Hip-hop is as strong as ever. It ain't going anywhere.

And I will say, even though I didn't vote for Rich Boy, if we had done a songs list, "Let's Get This Paper" would definitely have been in there. ("Crank Dat Soulja Boy" probably would have too.)

To quote Tony Soprano "It's good to be in something from the ground floor. I came too late for that, I know. But lately I'm getting the feeling that I came in at the end. The best is over" That's how I sometimes feel about hip-hop.

Hope came in the form of a black president and hard financial times (in which hopefully peoples materialstic bullshit values changed in common mode values) but as of yet, no renaissance has occured...

But here's the deal, hip-hop is still alive, overall there is still quality music been moved , it just not rife in the mainstream, that's what happens when hip-hop becomes the new rock when record labels push that shit (and pimp your culture) it's like a production line and saturation occurs, {side note, smh at the south (and I'm not talking about Bun B or killa mike) sorry just felt like i had to say something, there is some poor music coming out from that region)

"east gave brith to it, the west made it gangster and the south killed it"

My point is, hip-hop isn't dead, hell it hasn't even declined, it just become main stream.

Whinston Churchill - "The best argument again democracy is a five-minute conversation with the average voter"

In hip-hop that translates to lisiting to 5-mintes of Hot-97/Z100/BET

My advice is, spend the time listening/supporting good quality music (as you ivan clearly did while compiling this post).

At the end of that day, that is all you can do

also Ivan man, ya gotta start running some mad venn diagrams on peepz, hit em with the what's hot and not graphicaly!

Jonathan: I see your point, but it's worth noting that a lot of people (really, a LOT) included obscure or otherwise not-so-well-known mixtapes and other releases on their list, most of which were recent or fell into the decade half two category. Blame it on short-term memory or the simple fact that these newer albums are still on heavy rotation, this MIGHT be enough to refute your counterpoint.

Sach O (and all who responded to his comment): Yep, you're probably right about that. But then again, when we look at the history of film making, there's a "golden era" (and so on...) which is pretty much universally accepted as a specific time span. Not so much in hip hop. Maybe in a few years...?

This stuff is bad research and bad math. 1) The only reason the "quality" decreased was because the guy put 1999 and 2000 together, making its "quality" seem higher than it actually was.2) What hip hop reviewers consider good is not necessarily an indication of hip hop's health.3) 2009 seems to go down only because the year isn't over yet.

"Even if you were to A) remove all votes for albums from 1999, and B) double the votes for 2009 (being generous, since 2009 is roughly only halfway through), we’d still be facing an 875 to 698 “win” for the first half of the decade. That’s a still-significant 20.23% slide."

Rap and hip hop beats are very much in demand by the young people nowadays. The young generation of today's era love to burn the dance floor with such beats. These beats are very much versatile in nature just like the rap beats. The rap and hip hop tracks are very much popular all over the world because these beats have been developed from the hip culture, which has lots of bass and treble