Welcome to the fifth installment of the Quarterback Streaming Challenge. As a quick recap: previous research of mine has found that quarterbacks average fewer points per game in weeks they were started than they do in weeks they were left on the bench. This finding challenges one of the underlying principles of quarterback streaming, which states that by assiduously selecting favorable matchups, one could get production from a player that exceeds his usual total.
Faced with this disconnect, I've designed a controlled experiment to test the hypothesis. I have invited 21 participants to try their hand at streaming quarterbacks based on a real fantasy football industry experts league. These participants are largely writers and contributors at various fantasy football sites— seven here at Footballguys, nine from elsewhere around the web— with five additional at-large spots awarded to fans.
This is primarily an attempt to measure the performance of the Late Round Quarterback strategy, as well as our ability to predict matchups without the benefit of hindsight. With that said, I'll be publishing a recap of how things stand every week so that it can serve as a roadmap for those of you who find yourself streaming in your own fantasy leagues.
As I mention in the introduction, data suggests that playing matchups is hard. Anecdotally, I've mentioned results in the past weeks that confirmed this. (For example, four out of five Matt Ryan owners benched him before his 500-yard game against Carolina.) This past week gave another example, as of the five teams that rostered Marcus Mariota, two cut him and two more benched him in advance of his biggest game of the season.
Anecdote is no substitute for data, though, which is why we're running this project; to gather enough data to reach empirically-supported conclusions. And while the season is far too young to make any sweeping analysis, there is one question I want to get an early peak at. Each team rosters two quarterbacks at a time. Ignoring all of the other moves they could have made, I'm curious what percentage of the time the participants have made the correct call just between the two options already on their roster.
If quarterback production was perfectly unpredictable, we should expect to see participants starting the correct quarterback 50% of the time. If quarterback production was perfectly predictable, we should expect to see participants starting the correct quarterback 100% of the time. So we can get a very rough feel for predictability by seeing how close the observed value is to either 50% or 100%.
With 21 participants over five weeks, there should be 110 pairs to compare; however, several of those pairs feature situations where one of the two quarterbacks did not play, (either due to injury or to bye). Tossing out those pairs, we are left with 103 player pairs. Among those 103 pairs, the participants selected the correct quarterback 64 times, a 62% success rate.
That's certainly above the 50% threshold, which suggests there is some predictability to quarterback performance. But it's low enough to hint at a great degree of unpredictability, too.
Simply comparing hit-rate, however, misses out on the importance of upside. If you start quarterback A every week, and quarterback B outscores quarterback A by 0.1 four times, while quarterback A outscores quarterback B by 20 in the other game, your "hit rate" will only be 20%. At the same time, the value of that huge game more than justified the tiny losses in the other four weeks.
We can look at production in a macro sense to account for this by counting the number of participants whose starting quarterback averages more points per game than their bench quarterback. Again, 50% implies random chance, while 100% implies perfect predictability. The number of participants who over the course of the five weeks started the correct quarterback in aggregate is 13, which gives us... an identical 62% success rate.
Over the entire sample, the participants have averaged 18.40 points per game by quarterbacks they have started and 16.79 points per game by quarterbacks they have benched. This is a significant and meaningful difference, and I don't mean to suggest otherwise. We are demonstrably better at playing matchups than random chance alone would suggest.
As I said, my purpose here— as my purpose in this project as a whole— is simply to put hard data to these intuitions, to measure things like "how much better than chance alone". And the data so far quantifies what we already knew; playing matchups is hard.
This week has been much busier for waiver claims. Here are your hot pickups of the week:
- Colin Kaepernick ($37)
- Colin Kaepernick ($26)
- Marcus Mariota ($16)
- Brian Hoyer ($15)
- Marcus Mariota ($15)
- Colin Kaepernick ($13)
- Marcus Mariota ($6)
- Brian Hoyer ($2)
- Marcus Mariota
- Marcus Mariota
- Marcus Mariota
- Brian Hoyer
And the declared starters:
- Marcus Mariota (x6)
- Brian Hoyer (x5)
- Matt Ryan (x2)
- Tyrod Taylor (x2)
- Brock Osweiler
- Colin Kaepernick
- Alex Smith
- Joe Flacco
More articles from Adam HarstadSee all
More articles on: AnalysisSee all
Fantasy Overview - Week 7 - Haseley
What You Need to Know From Week 6 - Bloom
Normalized Strength of Schedule: Week 7 - Lee