Dynasty, in Theory: Revisiting Preseason Expectations

Comparing the predictive power of preseason ADP to early-season performance.

Let’s talk about science.

You may not have heard, but science today is grappling with something of a reproducibility crisis. Over the last several years, it has become more and more apparent that a shockingly large number of its findings are not consistent when experiments are run a second time.

The reasons for this are easy to point out. Because of how tests for statistical significance typically work, we would expect scientists to find a “statistically significant relationship” where there really was none entirely by chance in around one out of every 20 experiments.

The problem is that more than 1/20 of the juiciest findings we read about are going to suffer from this flaw, though. Why? Simply put, science has a publication bias. Prestigious journals crave to publish surprising, new, and unintuitive findings. Nobody wants to run a story that, yes, people really do get mad when you poke them with a stick, (p = 0.001).

Studies in the 1/20 “false positives” bucket are dramatically more likely to get published than the 19/20 “true negatives”, so they’re disproportionately represented in the literature. Additionally, scientists rarely gain prestige by double-checking someone else’s work, (and journals are rarely interested in publishing negative results), so some of these errors aren’t found for years— if ever.

Here at Dynasty, in Theory, we’re committed to doing the hard, unglamorous work of Real Science(TM). Two years ago, I wrote about how preseason ADP predicted late-season performance better than performance in the first four weeks. Last year, I re-ran the test and found that, for 2013, both preseason expectations and early-season performance carried roughly equal predictive power.

As you may have guessed, it’s “Revisiting Preseason Expectations” week. Let’s run the results for 2014!

Methodology:

Once again, I have compiled a list of the top 24 quarterbacks, 36 running backs, 48 wide receivers, and 24 tight ends according to MFL’s 2014 preseason ADP. From that list, I have removed any player who missed more than one of his team’s first four games or more than two of his team’s last twelve games. As always, we’re looking by team games, rather than by week, to account for players with a week 4 bye.

I have used standard scoring, because that was the easiest for me to look up with the databases I had. For the remaining players, I tracked where they ranked at their position over the first four games and over the final twelve games. Finally, I’ve calculated the correlation between preseason ADP and stretch performance, as well as the correlation between early performance and stretch performance.

Here’s the data:

Quarterback

ADPPlayerFirst 4 GamesLast 12 Games
1 Peyton Manning 2 8
2 Aaron Rodgers 6 1
3 Drew Brees 10 6
4 Andrew Luck 1 2
5 Matthew Stafford 8 17
6 Tom Brady 28 4
8 Matt Ryan 4 11
9 Cam Newton 25 10
10 Colin Kaepernick 11 15
11 Russell Wilson 3 5
12 Jay Cutler 7 16
13 Tony Romo 19 9
15 Philip Rivers 5 14
16 Andy Dalton 14 19
17 Ben Roethlisberger 16 3
19 Alex Smith 13 20
20 Ryan Tannehill 20 7
21 Joe Flacco 12 13
22 Eli Manning 9 12

The average delta between preseason ADP and stretch performance was 5.26 spots. The correlation was 0.422.

The average delta between early performance and stretch performance was 7.74 spots. The correlation was -0.019.

Running Back

ADPPlayerFirst 4 GamesLast 12 Games
1 LeSean McCoy 31 11
2 Jamaal Charles 27 7
4 Matt Forte 14 4
5 Eddie Lacy 43 3
6 Marshawn Lynch 2 6
8 DeMarco Murray 1 2
10 Arian Foster 19 5
12 Le\'Veon Bell 3 1
13 Alfred Morris 7 16
19 Frank Gore 15 17
20 Toby Gerhart 46 65
21 Shane Vereen 37 25
25 Chris Johnson 28 38
26 Bishop Sankey 47 40
27 Trent Richardson 20 46
28 Joique Bell 41 12
32 Steven Jackson 34 22
34 Lamar Miller 11 13
35 Darren Sproles 16 34
36 Fred Jackson 17 26

The average delta between preseason ADP and stretch performance was 9.85 spots. The correlation was 0.568.

The average delta between early performance and stretch performance was 13.30 spots. The correlation was 0.472.

Wide Receiver

ADPPlayerFirst 4 GamesLast 12 Games
2 Demaryius Thomas 5 4
3 Dez Bryant 11 2
5 Julio Jones 4 12
7 Jordy Nelson 2 5
8 Alshon Jeffery 26 10
9 Antonio Brown 1 3
10 Randall Cobb 6 9
11 Keenan Allen 48 44
12 Vincent Jackson 47 38
13 Larry Fitzgerald 72 43
15 Andre Johnson 49 45
16 Pierre Garcon 32 56
17 Cordarrelle Patterson 28 100
18 Roddy White 35 24
19 Michael Crabtree 25 59
21 DeSean Jackson 46 17
22 T.Y. Hilton 37 8
23 Michael Floyd 44 31
25 Julian Edelman 21 28
26 Emmanuel Sanders 14 6
27 Torrey Smith 65 14
28 Jeremy Maclin 7 11
29 Marques Colston 55 27
30 Mike Wallace 20 22
31 Sammy Watkins 31 25
32 Eric Decker 29 30
34 Reggie Wayne 23 69
35 Golden Tate 30 13
36 Kendall Wright 61 35
37 Terrance Williams 19 52
38 Kelvin Benjamin 8 23
39 Mike Evans 43 7
40 Riley Cooper 86 54
41 Dwayne Bowe 70 61
42 Anquan Boldin 53 16
43 DeAndre Hopkins 12 20
44 Reuben Randle 59 34
45 Tavon Austin 89 73
47 Hakeem Nicks 58 79

The average delta between preseason ADP and stretch performance was 17.41 spots. The correlation was 0.333.

The average delta between early performance and stretch performance was 18.67 spots. The correlation was 0.477.

Tight End

ADPPlayerFirst 4 GamesLast 12 Games
1 Jimmy Graham 3 5
3 Rob Gronkowski 10 1
4 Vernon Davis 17 54
6 Jason Witten 24 6
9 Greg Olsen 8 4
11 Zach Ertz 11 21
12 Martellus Bennett 2 7
13 Ladarius Green 39 58
14 Antonio Gates 6 2
16 Charles Clay 33 12
17 Heath Miller 12 9
20 Travis Kelce 7 8
21 Delanie Walker 4 13
22 Coby Fleener 14 3
23 Tim Wright 56 15
24 Andrew Quarless 22 25

The average delta between preseason ADP and stretch performance was 12.06 spots. The correlation was -0.051.

The average delta between early performance and stretch performance was 12.31 spots. The correlation was 0.416.

Conclusions

In 2014, preseason ADP was much more predictive at quarterback, slightly more predictive at running back, slightly less predictive at wide receiver, and much less predictive at tight end. Over all four positions, the correlation between preseason ADP and stretch performance was 0.466. The correlation between early performance and stretch performance was 0.560.

All correlations are down dramatically from the aberrationally high ones of 2013 and more in line with the ones I found from 2012 and before. 2014 also marked the first time where, over all positions, early season performance was significantly more predictive than preseason ADP, (though it still varied wildly on a position-by-position basis).

I'll be interested to see next year whether 2014's performance was a trend or whether, like 2013s unnaturally high correlations, it was merely a 1-year aberration. Either way, I believe the data still supports the conclusion that four games is typically the tipping point around which early performance and preseason ADP tend to carry similar predictive weight.

One last note must be made; because I am excluding players who missed substantial time, these data are subject to selection bias. Players who lose their job due to underperformance rather than injury will not be counted, and they would surely drive the correlations for early-season performance up. As an example, consider Johnny Manziel, who was drafted on average as one of the top 24 quarterbacks, but who was not counted in these results because he couldn't win the starting job.  

I've included all of the data used, so please feel free to peruse at your leisure and draw your own conclusions.


More articles from Adam Harstad

See all

More articles on: Dynasty

See all

More articles on: Strategy

See all