FiveThirtyEight: An Assessment
FiveThirtyEight, led by founder and editor-in-chief Nate Silver, is not afraid to challenge the conventional wisdom. That, after all, is how Silver made his bones when political analysis was just a hobby back in 2008. He continues in that vein today, of course. Consider, for example, this recent and very provocative piece entitled "As The Race Tightens, Don’t Assume The Electoral College Will Save Clinton." Within the body of the item, Silver also revealed that his two models have grown a bit more bearish on the Democratic nominee as August turns to September, giving her a 70% chance to win, and a 74% chance to win the White House. Both of these assertions—the poll numbers, and the Electoral College revelation—certainly run contrary to what just about everyone else (including us) is saying.
Now before proceeding, let us note that while Silver and his team work for ESPN, they aspire to be much more scholarly than other media outlets. And in the world of the academy (of which both V & Z are a part), it is both apropos and expected for scholars to offer critical assessments of one another's work. Check any journal, for example, and the back pages will be full of reviews of the latest publications and/or research. And so, with that said, we start with the judgment that this particular article is of poor quality. The headline strongly implies that if Hillary Clinton drops to a two or three or four point lead in the polls, that she might very well lose the Electoral College. This is misleading, at best, and disingenuous at worst, since that's not what the text actually argues. The actual argument is that Hillary Clinton's numbers in swing states track fairly well with her national polling numbers. The implication is that if the national polls turn into a tossup, the swing states will turn into a tossup too, and that Clinton might be in trouble.
There are a number of criticisms that might be made of this argument. To start with, it's not exactly profound to observe that swing states track fairly well with national preference polls. Of course they do; that's why they are swing states. Beyond that, the whole analysis is not telling us what is happening, merely what could happen if Trump pulls even in the polls. But he's rarely done so this year, has never done so since the Democratic convention, and there's currently no particular reason to believe that he will do so in the future. And finally, even if the swing state polls do end up a dead heat across the whole country, we must recall that Clinton needs considerably fewer of those swing states than Trump does. If we hypothetically imagine each of the 12 or 13 swingiest states as a pure coin flip, Clinton may only need two or three coin flips to go her way, depending on which states we're talking about. In fact, about 86% of the various coin flip combinations end with a Clinton win, about 12% end with a Trump win, and about 2% end in a tie. So, in fact, if the polls do tighten, it is exceedingly likely that the Electoral College will save Clinton.
Further—and this may seem ungenerous, but it bears noting—the article is simply not very well written. In Silver's early days as a psephologist, and certainly through his three years with the New York Times, his writing was elegant, sometimes even breathtaking. His book had its moments (quite a few of them, in fact), and these days he and his staff still occasionally hit a home run. But in general, they have dispensed with what their third grade math teacher told them: "Show your work." Explanations of data are often hurried, unclear, or non-existent. Quite regularly, tables or graphs are dumped into the middle of a piece with minimal discussion. A site can publish brief and simple pieces (like 95% of the items on this site) or it can produce lengthy and complex pieces (like, say, Larry Sabato's Crystal Ball), but it can't do both within the same item. FiveThirtyEight tries to occupy a middle ground between the two, and most of the time, it just doesn't work. Tyler Cowen, at Marginal Revolution, put it succinctly: "[T]hese are 'tweener' pieces, too superficial for smart and informed readers, yet on topics which are too abstruse for the more casual readers." Others have also weighed in with critical reviews, including The New York Times' Paul Krugman, Salon's Elon Green, The Week's Ryan Cooper, and The New Republic's Leon Wieseltier. Most of these commentaries were produced shortly after the site's ESPN launch in 2014, but the site's content and style have not substantively changed in the two years since, so the critiques still ring true. A large percentage of the content produced for the site these days has the same basic problems as the piece discussed above.
So, what has happened at FiveThirtyEight? Here we must move into the realm of supposition, though supposition that (we hope) is well-founded. We would suggest that the first major problem is, for lack of a better term, one of expectations. By 2012, Silver was—at age 34—on top of the world. He was arguably the most respected political analyst in the country (among liberals, at least), and was ensconced as the #1 political pundit at the newspaper of record. Most aspiring journalists work their whole lives without ascending to such heights, and he did it in just five years. By that time, he had won countless awards, had been named to Time's list of the 100 most influential people in the world, and was even—thanks to a six-figure book advance and the sale of his baseball-analysis business—a millionaire.
Then, ESPN came knocking, with the offer of an even bigger platform than the Times, and a vastly larger budget. Silver was not merely content with moving his political blog, however; he (and likely ESPN) wanted something bigger and splashier. In fact, the plan was to produce something rather revolutionary. But the problem with trying to top yourself after you've already been on top of the world is that it's extremely hard to do. We think of Lorne Michaels, who created one of the most influential shows in the history of television in Saturday Night Live, and then followed up with The New Show, which lasted five episodes. Or Orson Welles, whose first feature film—Citizen Kane—tops most critics polls, but also set a standard that Welles never quite managed to equal. Or Ralph Ellison, who was so intimidated by trying to follow up on Invisible Man that he never published again during his lifetime (even though he produced more than 3,000 pages of manuscripts).
And we need not limit ourselves to other media to help understand the challenge that Silver faced when he signed on the dotted line with ESPN. Not long after poaching FiveThirtyEight from the New York Times, ESPN hired Jason Whitlock to create a site called The Undefeated, which would blend sports, politics, and culture, and would be targeted at black readers. Whitlock's efforts were nothing short of a train wreck, however, resulting in several contributor defections, blowback from several of the staffers who stayed, and a scathing 10,000-word takedown on the sports media site Deadspin. By the time the site finally launched in 2016, Whitlock was long gone from ESPN, cashiered because of his failure to bring the ambitious project to fruition.
Of course, not everyone fails to rise to such big challenges. Scorsese followed Taxi Driver with Raging Bull, the Beatles followed Revolver with Sgt. Pepper's Lonely Hearts Club Band, and Dickens followed A Tale of Two Cities with Great Expectations. And thus Silver, hoping to be in this group, moved forward with his groundbreaking new approach. In part, that meant asking new and provocative and even funky kinds of questions. Not just about politics, but also "Can You Bake The Optimal Cake?" and "What's Changed in Harper Lee’s Hometown Since 'Mockingbird,'" and "What If An Evil Commissioner Turned Hank Aaron's Homers Into Singles?" and "Searching for the Best Burrito in America." These are all real headlines from the site. And they're all certainly interesting subjects for discussion, though not that different from what WhatIf? or Freakonomics (among others) have been doing for years.
The other thing Silver determined to change when he moved to ESPN was his approach. He published a manifesto outlining his thinking; he also gave an interview to the New Yorker. In both places, he was incredibly disdainful of both the media's political coverage (particularly that of Politico and his former employers at the Times) and also of academic political analysts. As Krugman puts it in the piece linked above:
Silver seems to have taken the wrong lesson from his election-forecasting success. In that case, he pitted his statistical approach against campaign-narrative pundits, who turned out to know approximately nothing. What he seems to have concluded is that there are no experts anywhere, that a smart data analyst can and should ignore all that.
It is not surprising that Silver took such an iconoclastic approach—again, that's how he made his reputation. But the experts aren't always wrong, and sometimes there are good reasons that things are done the way they are. To employ a crude parallel, there are a great many short films that are 5- to 15-minutes in length, and a great many features that are 90-180 minutes in length, but 55-minute movies are rare. There are 2- to 7-minute songs, and 30- to 60-minute symphonies, but fairly few 14-minute compositions. And there are 5- to 10-page short stories, and 200- to 1,000-page novels, but not too many 125-page books. It seems fair to suggest that journalism in general, and political analysis in particular, have settled at the levels that they have (short and sweet; long and complex) for a reason: because those styles work. Point is: Perhaps Silver should go back and think critically about his manifesto, and should consider retooling FiveThirtyEight's approach.
There may also be a second major issue with FiveThirtyEight, which we will describe as one of economy. We start this part of the discussion by noting that every one of us who is writing about politics this year benefits from a horse race. "Things are the same as they were yesterday" is not a story. "Clinton extends her lead" and "Trump makes up ground on Clinton" are. Similarly, we also benefit from finding things that are new and different to talk about. Sen. Bernie Sanders (I-VT) and his rallies tended to get relatively little media coverage; not because of any particular bias against him, but because they were all the same. You can only write, "10,000 young, mostly white people show up to cheer Sanders" so many times. Hillary Clinton, evenhanded and cautious as she is, also tends to give us relatively little to talk about much of the time. With Donald Trump, on the other hand, it's several new and outrageous and previously unheard of things almost every day. Hence his dominance of the headlines.
Point is, all the political sites have a certain bias towards "dog bites man." However, there is reason to believe the bias is unusually strong for Silver and his crew. Many political sites and prognosticators—NBC News, the Wall Street Journal, Fox News, Bloomberg Politics—are part of organizations for whom political coverage is part of their core mission. Others—Sabato's Crystal Ball, the Harvard Political Review—are part of (and are supported by) universities. Still others—HuffPo, Breitbart, Politico, The Hill—are already stable, self-sustaining businesses. And a few—this site, Sam Wang's Princeton Election Consortium—are side projects of academics who already have day jobs. The point is that while we all like page views and clicks, none of these sites is—as far as we know—facing an immediate existential crisis. Page views could go up or down by 50%, and most or all of the above would keep on trucking.
For FiveThirtyEight, by contrast, this may not be true. Only ESPN president John Skipper knows for sure, but there are three things that should have Silver and his staff at least somewhat nervous. The first is that for ESPN, politics (and burritos and Harper Lee and the optimal cake) are not part of their core business. If ABC or Fox or the Washington Post ceased all political coverage tomorrow, that would be earth-shattering. If ESPN does so, it's a sidebar.
The second concern is that ESPN is in dire financial straits. They've lost 14 million subscribers in the last five years, with 4 million of that coming in just the last year. They're already far and away the most expensive cable channel in the country, so there's no more blood to be squeezed from that stone, and rights fees for live sports just keep going up and up. The picture is bleak enough that it's dragging down the stock of parent company Disney, who has responded by demanding hundreds of millions of dollars be cut from the budget.
The third problem is that Bill Simmons' Grantland is no more. This long-form sports and culture site was the biggest of ESPN's so-called "verticals," and it paved the way for the acquisition of FiveThirtyEight and the creation of The Undefeated. But in an era of cost-cutting, Simmons' $5 million salary was too much for ESPN to bear (especially in view of his large ego and big mouth) and so he left for the greener pastures of HBO. Grantland, which was wildly popular, and drove traffic to the other verticals, was shuttered just days later.
The point is that it would be very easy for ESPN to decide, sometime after November 8, that FiveThirtyEight is a luxury that can no longer be afforded. Of course, that calculation depends on how much money the site is losing, or whether or not it's losing money at all. The more clicks they get, and the more publicity they get, the further they move from the chopping block. It's understandable that under these circumstances, Silver & Co. would—either consciously or subconsciously—be working particularly hard to be provocative and to stand out from the crowd. Indeed, the motivation is so strong that their analysis really should be taken with a grain of salt until we see that ESPN is truly committed to them, and that they're truly not going anywhere (in other words, if they survive to 2020). Alternatively, Silver might consider leaving ESPN, and finding a partnership where he's not a tiny data-analysis fish in a big sports-broadcasting pond.
This assessment may seem like an attack on a competitor, but really it is not. Surely, there is room for both this site and FiveThirtyEight in the blogosphere. If anything, it's a form of tough love, in hopes that a site that was once a pleasure to read can return to being a pleasure once again.
--Zenger