Incensed by an article from HuffPost’s Washington Bureau Chief Ryan Grim criticizing 538.com and its prediction model, 538’s election prognosticator Nate Silver unleashed a furious tweetstorm on Saturday.
Grim accused Silver of “just guessing” by “changing the results of polls to fit where he thinks the polls truly are, rather than simply entering the poll numbers into his model and crunching them” as opposed to other models that simply look at the hard numbers.
RELATED: Massive Final Weekend Election 2016 Update: Lots of Links
Silver calls this unskewing a “trend line adjustment.” He compares a poll to previous polls conducted by the same polling firm, makes a series of assumptions, runs a regression analysis, and gets a new poll number. That’s the number he sticks in his model ― not the original number.
He may end up being right, but he’s just guessing. A “trend line adjustment” is merely political punditry dressed up as sophisticated mathematical modeling.
Guess who benefits from the unskewing?
By the time he’s done adjusting the “trend line,” Clinton has lost 0.2 points andTrump has gained 1.7 points. An adjustment of below 2 points may not seem like much, but it’s enough to throw off his entire forecast, taking a comfortable 4.6 point Clinton lead and making it look like a nail-biter.
Grim notes that the HuffPost pollster gives Clinton 98 percent chance of winning while the NY Times has Clinton’s chances at 85 percent.
RELATED: Colbert Asks Nate Silver to Predict What Will Happen if Trump Won’t Concede: WATCH
Silver’s model has it as a much tighter race with Clinton currently at 65.5%.
Grim accused Silver of “ratcheting up the panic” unnecessarily, and in favor of Trump:
Silver’s guess that the race is up for grabs might be a completely reasonable assertion ― but it’s the stuff of punditry, not mathematical forecasting.
Punditry has been Silver’s go-to move this election cycle, and it hasn’t served him well. He repeatedly pronounced that Trump had a close to 0 percent chance of winning the Republican primary, even as he led in the polls.
Silver ripped into Grim on Saturday afternoon:
This article is so fucking idiotic and irresponsible. https://t.co/VNp02CvxlI
— Nate Silver (@NateSilver538) November 5, 2016
The reason we adjust polls for the national trend is because **that's what works best emperically**. It's not a subjective assumption.
— Nate Silver (@NateSilver538) November 5, 2016
It's wrong to show Clinton with a 6-point lead (as per HuffPo) when **almost no national poll shows that**. Doesn't reflect the data.
— Nate Silver (@NateSilver538) November 5, 2016
Every model makes assumptions but we actually test ours based on the evidence. Some of the other models are barley even empirical.
— Nate Silver (@NateSilver538) November 5, 2016
There are also a gajillion ways to make a model overconfident, whereas it's pretty hard to make one overconfident.
— Nate Silver (@NateSilver538) November 5, 2016
If you haven't carefully tested how errors are correlated between states, for example, your model will be way overconfident.
— Nate Silver (@NateSilver538) November 5, 2016
Not just an issue in elections models. Failure to understand how risks are correlated is part of what led to the 2007/8 financial crisis.
— Nate Silver (@NateSilver538) November 5, 2016
There's a reasonable range of disagreement. But a model showing Clinton at 98% or 99% is not defensible based on the empirical evidence.
— Nate Silver (@NateSilver538) November 5, 2016
We constantly write about our assumptions and **provide evidence** for why we think they're the right ones. https://t.co/IhLKXdxGGK
— Nate Silver (@NateSilver538) November 5, 2016
That's what makes a model a useful scientific & journalistic tool. It's a way to understand how elections work. Not just about the results.
— Nate Silver (@NateSilver538) November 5, 2016
The problem is that we're doing this in a world where people—like @ryangrim—don't actually give a shit about evidence and proof.
— Nate Silver (@NateSilver538) November 5, 2016
The philosophy behind 538 is: Prove it. Doesn't mean we can't be wrong (we're wrong all the time). But prove it. Don't be lazy.
— Nate Silver (@NateSilver538) November 5, 2016
And especially don't be lazy when your untested assumptions happen to validate your partisan beliefs.
— Nate Silver (@NateSilver538) November 5, 2016
Grim concludes:
We’ll have to wait and see what happens. Maybe Silver will be right come Election Day ― Trump will win Florida, and we’ll all be in for a very long night. Or our forecast will be right, she’ll win nationally by 5 or 6, and we can all turn in early.
If he’s right, though, it was just a good guess ― a fortunate “trend line adjustment” ― not a mathematical forecast. If you want to put your faith in the numbers, you can relax. She’s got this.
(h/t/ politico)
UPDATE: Silver was on ABC News this morning talking about how he sees things shaping up:
FiveThirtyEight's Nate Silver on what he's watching for in final days of the 2016 election https://t.co/4kY3nSyXfd https://t.co/sgK2IVMFfc
— ABC News (@ABC) November 6, 2016
