Hot Pod: There’s a new (and problematic) way to measure which podcasts are the most popular

Welcome to Hot Pod, a newsletter about podcasts. This is issue eighty-eight, published September 20, 2016.

Another public-facing podcast ranker. It’s troublesome, though if you’re a podcast publisher you best pay close attention nonetheless. This one’s going to be long, so either skip it or strap in.

Here’s the deal: Podtrac, the decade-old podcast measurement (and until its recent restructure, advertising) company, announced a new podcast ranker yesterday, one that aspires to display the top 20 podcasts in the industry based on monthly downloads. This is the second such public-facing ranking that the company has released in recent months; In May, Podtrac pumped out a chart that ranked podcast publishers against each other based on network-wide monthly downloads.

That initial ranker suffered from two glaring flaws. First, it can’t be considered adequately representative of the podcast industry because of its incomplete sampling. (The original report purports to cover of “90 percent of the top podcasts.”) And second, there’s a general lack of transparency around its sampling methodology. (Said “top podcasts” category isn’t clearly defined, and it isn’t clear who is and isn’t included.) The publisher ranker’s initial May 2016 sample did not include important publishers like Panoply, Gimlet, Earwolf, The Ringer, and Wondery. That’s not to say that they would all show up in the top 10 if they were included, mind you; I’m just making a point about representation, and many of them remain excluded at this writing.

This new show ranker, which was reportedly assembled due to advertiser demand, suffers from those same fundamental issues, plus some new complexities that further interrupt itss capacity to serve as a trustworthy conveyor of value in the podcast industry.

Let’s break this down:

1. Yesterday’s new chart ranks individual podcasts based on “Unique Monthly Audience” (as determined by Podtrac’s internal measurement rules), but the chart itself does not explicitly display actual download numbers. I view this as an incredibly odd — and even counterproductive — choice. The omission strips the chart of important granular analytical value, and patrons of the chart are placed in a position where they wouldn’t even be able to, say, discern the scale of the difference between two consecutively-ranked podcasts, which can go a long way in properly conveying the shape and form of the competitive landscape.

Interestingly enough, Velvet Beard, Podtrac’s VP of podcast analytics, tells me that this omission came out of a compromise with certain publishers who are reticent to disclose their show numbers.

2. That reticence is further reflected in the eighth ranking on the new chart, which awkwardly reads: “Publisher declined to list show.” This state of affairs comes out from a crucial distinction between the two Podtrac rankers: While the original podcast publisher ranker lists publishers that explicitly measure their podcasts with Podtrac (an arrangement understood by Podtrac as permission for inclusion into their ranker), this new show-oriented ranker does not require explicit publisher participation in the company’s measurement services for inclusion.

It was explained to me that part of the ranker’s methodology involves some internal modeling that doesn’t actually require publishers to opt into their measurement system for download size assessment. Which, you know, the more I think about it, is a choice that would creep me out if I was a publisher, because not only are we left with a situation where an external body has taken upon itself to tell the story of my audience for me — without my explicit acknowledgment and consent — it’s also a story based on their terms, the foundations of which may well be different from my own. And that means something in an industry that lacks a universally standardized and enforced measurement paradigm.

That mystery eighth podcast (whose identity was included in the initial press release sent to me, and was scrubbed in a followup version after I attempted to verify) isn’t the only show that was included in the ranker without given permission; the Joe Rogan Experience, which came in on the eleventh slot, appears to be a non-participant as well.

3. Beard tells me that the company has been consistently trying to reach out to publishers to get them involved with the ranker. “We send out emails, but not everybody writes us back,” she said.

I suppose there are strong strategic reasons why some publishers would not want to get involved in Podtrac’s ranking system. To begin with, you have the table stakes concern that a publisher who chooses to be listed would be ceding its monopoly over how it tells the story of its own downloads. Which would be fine for some…and less so for others, particularly those who make it a practice of fluffing their numbers, a very real problem in this industry that lacks mature measurement standards and an independent third-party that can serve as a check against bad practices.

But even for those whose goods are sound, there’s simply too much of a perceived risk to anoint Podtrac as that third-party due to the company’s current relationship with Authentic, its ad sales arm that was spun off as a sister company earlier this summer. The two companies still share leadership and infrastructure, which presents a strong disincentive for some publishers who would be understandably uneasy ceding parts of their narrative to company that’s structurally connected to a potential competitor. The golden rule applies: It’s not the actual conflicts of interest, it’s the perception of potential conflicts of interest that matters.

For what it’s worth, Beard tells me that the company’s long-term hope is to effectively decouple Podtrac from Authentic to mitigate such concerns. However, she also notes that the team has to first figure out how to make its business — which currently doesn’t make any money off these rankings — financially independent before any significant decoupling can happen.

Look, Podtrac’s industry rankers need a lot of work before they can be considered a genuine representation of the emerging podcast industry, and for what it’s worth, I do think the Podtrac team is operating with civil intent. (And to some extent, I really do hope they pull it off.)

But let’s be real here. In a medium whose defining problem is its lack of measurability — which therefore generates an advertising environment starved for every little bit of information — Podtrac’s good-enough rankers are bound to gain some traction among advertisers either way.

And it looks like things may be panning out in that direction: ahead of the IAB Podcast Upfronts a few weeks ago, I was speaking with Jason Hoch, the chief content officer of HowStuffWorks, which uses Podtrac for analytics verification and is listed on the industry ranker, and he noted that the original ranker drew a tremendous amount of new advertising attention to his network. “The in-bounds we got from that were amazing,” he told me.

So I’ll say this: It appears increasingly imperative that podcast publishers start engaging with Podtrac in order to win back their audience narrative (and the narrative of the industry). I’m not the biggest fan of how Podtrac has gone about doing things — their lack of methodological transparency remains troubling, and the whole including-podcasts-without-explicit-permission thing feels kinda dirty — but they are, regardless, materially contributing to the publisher-advertising relationship.

And in case you’s still interested: According to Podtrac, the top three podcasts in August 2016 are, in downward order, This American Life, Radiolab, and Stuff You Should Know.

A leadership change at NPR One. The public radio mothership’s buzzy listening app, NPR One, is losing Sara Sarasohn, its managing editor, who is leaving the organization after 24 years of service.

Tamar Charney will reportedly step in as interim editorial lead. Charney was hired back in January to serve as the app’s “local editorial lead,” a role that involves connecting the app with local public radio stations across the country. While she will take over many of Sarasohn’s duties, she will continue focusing on her original responsibilities as well. The team remains rounded out by content programmer Viet Le, along with an NPR One-specific product team led by product manager Tejas Mistry and content strategist/analytics manager Nick DePrey.

Sarasohn, who has worked multiple positions on All Things Considered and the NPR arts desk throughout her lengthy career, is leaving public media for a position at a Silicon Valley startup, and though she declined to provide specifics, she noted that her new gig isn’t involved in the audio world.

An internal staff note announcing Sarasohn’s departure indicates that she leaves NPR One in a strong position. According to the memo, “NPR One’s audience reaches record highs with each new month, more than 80 stations are contributing content to it, and the typical listener uses the app up to 12 times per month.”

It also noted that the organization will relay more information about the app’s future in the coming days.

The NPR One app — which flirts with aspirations of being “the Netflix for podcasts” and is marketed as “the Pandora of podcasts” — is reportedly considered to be “the most exciting thing to have happened at NPR in years.” I’m broadly a fan of it myself, but I do struggle to view the app itself as somehow central to NPR’s digital future. The ecosystem of content and technology that’s being built beneath the app, however, is another story. (Side note: Between you and me, my main consumption modality with public radio nowadays is my Amazon Echo, which has come dangerously close to being my only source of verbal interaction on most days.)

Sarasohn’s last day is September 25.

Relevant: In what is probably a yuge coincidence, news of Sarasohn’s departure comes about a week after NPR announced that it was picking DC-based WAMU’s The Big Listen, a broadcast about podcasts, for national distribution. Hosted by Lauren Ober, the show is one of several audio programs currently floating about podcastland that seeks to alleviate the medium’s discoverability problem through linear, performative curation. That list includes the CBC’s Podcast Playlist and Gimlet’s Sampler.

Tangentially relevant: WNYC Studios now has a third VP of on-demand content: Tony Phillips, a former BBC veteran of 27 years. (My whole life, basically.) His most recent role at the British radio mothership was “Editor, Commissioner, and Producer.” Phillips will expand the leadership layer, which also includes Paula Szuchman and Emily Botein, into a trifecta.

Mid-October will mark WNYC Studio’s first full year of operation.

Cable podcasts. CNN, the cable news heavyweight and source of all my anxieties, is pushing deeper into podcasts with the announcement of two new podcasts:

1. The Daily DC, a daily morning political news digest show featuring CNN political director David Chalian; and

2. Party People, described as a “look at the 2016 race from a rightward perspective.” The show is hosted by two CNN contributors, Republican communications strategist Kevin Madden and The Federalist editor Mary Katharine Ham.

Curious observers might be interested to know that CNN’s foray into original podcasting is largely orchestrated by one Tyler Moody, a VP at the company, and that the podcasts are being hosted — and represented in the ad sales market — by New York-based podcast CMS company Palegroove.

And speaking of right-leaning politics podcasts: Fox News is making a weekly television show out of I’ll Tell You What, an elections podcast hosted by The Five cohost Dana Perino and Fox News digital politics editor Chris Stirewalt. The show will be limited-run, airing Sunday evenings until the elections in November. According to The Washington Post, the podcast’s conversion into the television format “represents Fox News’s first new programming initiative since longtime network chairman Roger Ailes resigned in July.”

Side note. A few weeks ago, I wrote a column observing what appears to be a dearth of explicitly conservative political and election-related podcasts, which briefly led me to consider this state of affairs being a function of early adopter demographics. Since then, I’ve regularly received recommendations from readers of more conservative-oriented shows, and while I believe my original observation still holds, I will say that one podcast in particular has found its way into my primary rotation: Radio Free GOP with Mike Murphy. It’s really polished, and really, really engaging, and it’s worth a try regardless of where you are on the spectrum.

Did you read Ken Doctor’s columns? You really should. The five-part series published on Nieman Lab all throughout last week did an amazing job laying out the current state and potential future(s) of the professionalizing layer of podcasts from the 30,000-feet view more than I ever could.

The series contained a bunch of novel findings that are incredibly useful for incremental observers like myself — for example, the fact that digital native Gimlet currently scores 5 million downloads monthly across its 6 shows (many of which are off-season at the moment) and now has a 55-person headcount (damn!); that NPR, Midroll, and PodcastOne each account for $10 million in sales; that 50 percent of WNYC’s sponsorship revenue now comes from digital as opposed to terrestrial sources, a good chunk of which is driven by podcasts.

But Doctor’s columns also laid out an analogy that connects what many podcast publishers/networks are doing these days to the long-established digital media strategy of aggregation. It’s a connection that hasn’t previously occurred to me, but it has become to me an essential framework in gaming out the probable trajectory — and potential pitfalls — of many of these emerging podcast companies.

Anyway, hit ’em up.

“You can’t compare it to anything else that exists in the industry right now,” said Rena Unger, the IAB’s director of industry initiatives, in the latest episode of The Wolf Den. “Podcasts took one of the boldest moves. You’re not doing your own individual upfronts, you were sharing one stage. You have 12 competitive companies that take off their competitive hats and say, ‘Let’s work together to elevate the space and increase our pie together.'”

Unger was responding to my, uh, critique of the recent Podcast Upfronts, which largely comes out of anxieties that were pinpointed almost perfectly by Chris Bannon, Midroll’s chief creative officer and co-host of The Wolf Den, who replied to Unger: “Yeah, it’s funny. There’s a great spirit of collaboration, but I think what Nick fears in his writing is that will disappear. That it will become some sort of commercial shark-tank in which where we race to the bottom in some way together.”

Or borrow a passage from Doctor’s final entry in his podcast series: “The phrase of the moment, both from some in the trade and from many of the millions of listeners who’ve become podcast addicts, seems to be: Don’t screw it up.“

I’m getting that tattooed.

Bites:

There’s a budding audio/podcast platform company floating about the Bay Area called Tiny Garage Labs that’s founded by former Netflix operatives and a former Planet Money correspondent. I’d keep an eye out on their blog — word on the street something’s coming real soon. (Tiny Garage Labs)

Made a quick mention of this two Hot Pods ago, but it’s more or less confirmed now: Panoply now reps MTV Podcasts, which joins the network with two new shows — Lady Problems and Videohead.

“A lot of sports podcasting simply reuses talk-radio formats. From the way you sound, this is clearly going to be something different.” “Yeah, I am hoping we can pull it off.” ESPN Films and FiveThirtyEight senior producer Jody Avirgan talks to Adweek about the upcoming 30 for 30 podcast documentaries. Lots of interesting nuggets in there. (Adweek)

“The web is built on hyperlinks, with each link a pathway to discovery, an endorsement, a reference. Podcasts could be like that too.” Jake Shapiro, CEO of RadioPublic, published what appears to be a manifesto for the upcoming listening app that will make up his team’s first independent foray into the podcasting marketplace. (RadioPublic blog)

If you haven’t been keeping up with APM’s new investigative podcast, In The Dark — along with everything that’s been happening with the actual case it’s examining — you really should. Vulture has a great interview here with Madeleine Baran, the investigative journalist who drives the show. (Vulture)

At the Online News Association 2016 conference in Denver last week, WNYC’s Delaney Simmons and NPR’s Mathilde Piard gave a presentation on their respective organizations’ attempts to wield social media tools as a points of audio distribution. (Journalism.co.uk)

This version of Hot Pod has been adapted for Nieman Lab, where it appears each Tuesday. You can subscribe to the full newsletter here. You can also support Hot Pod by becoming a member, which gets you more news, deeper analysis, and exclusive interviews; more information on the website.

Quah, N. (2016, Sep. 20). Hot Pod: There’s a new (and problematic) way to measure which podcasts are the most popular. Nieman Journalism Lab. Retrieved February 18, 2018, from http://www.niemanlab.org/2016/09/hot-pod-theres-a-new-and-problematic-way-to-measure-which-podcasts-are-the-most-popular/

Chicago

Quah, Nicholas. "Hot Pod: There’s a new (and problematic) way to measure which podcasts are the most popular." Nieman Journalism Lab. Last modified September 20, 2016. Accessed February 18, 2018. http://www.niemanlab.org/2016/09/hot-pod-theres-a-new-and-problematic-way-to-measure-which-podcasts-are-the-most-popular/.