Dynasty, in Practice: Coaches are Smart

A defense of a profession that is frequently under fire.

Some weeks I like to write articles with coy titles that dance around the topic for paragraphs before finally getting to the point. Some weeks I like changing up the format just to keep everyone on their toes. This is one of the latter weeks.

There’s a trendy strain of thought right now that the people who best know how to coach in the NFL are somehow not the people who coach in the NFL. Whether it’s snarky twitter bots advising coaches on in-game decision-making or dedicated fans making playing time suggestions, the expertise of coaches is frequently called into question.

I want to call that tendency into question, instead. Coaches are not perfect; they get things wrong all the time. They are not infallible or above being questioned. But NFL coaches are still the biggest experts on NFL football. They are smart, experienced, and capable. And we should sometimes give that more weight.

Why Should Coaches Be Good At Their Job?

We have a lot of reasons to believe that coaches *SHOULD* know what they are doing. Let’s start with the fact that only four head coaches today have served at least ten seasons with their current team. Bill Belichick is in his 16th season with the Patriots, Marvin Lewis is in his 13th year with the Bengals, Tom Coughlin is in his 12th year with the Giants, and Mike McCarthy is in his 10th season with the Packers.

Collectively, those four coaches have won 62.7% of their games with their current team. They have combined for seven championships, with all but Lewis owning at least one ring. And while Lewis might seem an outlier, it’s important to remember that he’s the only Cincinnati coach with a winning record since Forrest Greg, who coached 57 games from 1980 to 1983. Measured against those who preceded him, he has been a resounding success.

What does this mean? It is exceedingly difficult to remain employed as a head coach in the NFL. Bad coaches are found out and replaced at an alarming rate. And even here, “bad” is relative; there are 32 head coaching positions at any given time, but hundreds of assistant coaches and position coaches in the NFL, and thousands of coaches in the college ranks, and tens of thousands of coaches in high school.

The head coaches who make it to the NFL represent a fraction of a fraction of a percent of the overall coaching pool, and only a fraction of that group will survive at their position for any appreciable length of time.

Yes, NFL coaching searches tend to be cautious, often recycling former head coaches who were halfway decent in their former job rather than taking a chance on an unproven new hire. There is still a great degree of nepotism within the ranks. While these factors make coaching in the league less of a meritocracy, they only matter at the margins; by and large, the NFL is still a tremendous meritocracy.

Indeed, given the ruthless, results-oriented nature of coaching in the NFL and the massive pool of potential applicants waiting for a chance to fill any vacancy, we should expect NFL coaches to be exceedingly good at their jobs.

This is impressive in and of itself, given the vast range of skills required to succeed as an NFL head coach. Coaches must hire staffs and delegate just the right amount of authority, putting each individual member in a position to succeed. They must motivate 50 grown men at various points in their life through a brutal 6-month physical, mental, and emotional grind.

Coaches must oversee the training and development of an entire roster playing a variety of different positions. They are responsible for the development and implementation of an overall offensive and defensive vision, as well as individual week-to-week gameplans. And then, on game day, they must manage all of these various elements in real-time while adapting to ever-changing game situations.

Further, as the above example demonstrates, even after bucking the odds to get a job in the first place, head coaches are then faced with severe selective pressures. Coaches who do not perform are weeded out. Any behaviors that contribute to underperformance would therefore be selected against, while behaviors that contribute to overperformance would be selected for.

You have to be really good to get a head coaching job. You have to be even better to keep it.

The Importance of Proper 4th Down Strategy

There is a large body of research suggesting that current coaches are too unwilling to keep their offense on the field on 4th down. Indeed, if there is any evidence that coaches are perhaps not great at their jobs, this is the most damning. If you polled a hundred fans, I’d bet this would be the most commonly-cited weakness among current head coaches.

Most coaches have habits that reflect the conventional wisdom of the time when they were learning their craft, and those habits will necessarily lag modern understanding by years or even decades. And many coaches know this; Gary Kubiak, for instance, hired an analytics consultant to speak to him in his headset during games. If that’s not a tacit admission that he could be better at in-game strategy, I don’t know what is.

But let’s take a moment to consider what this undisputed assertion that coaches are bad at 4th-down strategy means. Remember what I said earlier about coaches being subjected to severe selective pressures? If 4th-down decision making were a key contributor to final results, we should expect those selective pressures to apply heavily to it.

If being bad at 4th-down decision-making was really bad for a team’s bottom line, we’d expect coaches who were bad at it to underperform and get fired quickly, while coaches who were good at it overperformed and remained employed. So the fact that virtually the entire profession is bad at it tells me that, in the end, it’s simply not that important to how a team performs for the season.

This isn’t to say that it doesn’t matter. It’s to say that the amount that it matters is absolutely trivial compared to how much every other aspect of a coach’s job matters. Which makes perfect, intuitive sense if we really think it through. Let’s go back to the list of responsibilities I mentioned earlier.

Would you rather have a coach who was really, really bad at recognizing and hiring talented assistants, or one who was really, really bad at 4th-down decision-making? Would you rather have a coach who couldn’t delegate, or who couldn’t figure out when to punt? Would you prefer a coach who couldn’t motivate his players, or one kicked field goals on 4th-and-goal from the 1?

It’s obviously better for a coach to be good at everything, but being good at everything often isn’t feasible, and in terms of tradeoffs that can be made, it seems “poor 4th-down decision-making” is the least damaging. Which is why so many people in the league right now are so bad at it. It’s never been selected very strongly against.

The biggest problem for coaches is that it’s also probably the single most visible aspect of their job. Fans don’t see John Fox working with Adam Gase to develop and install an offense. (Gase stands to potentially be a hot head coaching candidate this offseason; if he's hired, he'll be the fourth Fox assistant to be named head coach since 2012.)

They don’t see Fox working with Jay Cutler to develop as a player and put himself in situations to succeed. They don’t see him managing playing time and practice reps so that when Matt Forte is injured, Jeremy Langford is ready to step him. But they sure see him punting on every drive against the Seattle Seahawks.

And because it’s the most visible aspect of his job, fans often attach too much importance to it. They assume that what they see is all there is. They don’t see a dramatically improved team that’s 5-6 against the toughest schedule in the NFL, (according to Pro-Football-Reference’s SoS values). They don’t see a team with a transformed defense and a very consistent offense.

They see a team with a coach whose game-day skills are the butt of national jokes. And they assume that the league has passed that coach by, despite three straight 12-win seasons and a Super Bowl berth with two franchises. It’s easy to say that Peyton Manning deserves a lot of the credit for that, but the last time Fox had a quarterback other than Manning under center, he won the AFC West, (and a playoff game!), with Tim Tebow under center.

This is John Fox’s third straight dramatic turnaround of a franchise. He took over a 1-15 Carolina squad, took them to 7-9 in his first year, and made the Super Bowl in his second. He took over a 4-12 Denver squad, improved them to 8-8 in his first year, and went 13-3 in his second. And now he’s taken over a 5-11 Chicago squad and already matched last year’s win total despite a brutal schedule.

And he’s widely considered the worst coach in the NFL on 4th-down strategy. So again I ask, just how important is 4th-down strategy in the grand scheme of things if a coach can be that bad at it and still be that good of a coach?

What Is Proper 4th Down Strategy, Anyway?

We’ve spent long enough questioning coaches’ 4th-down decision-making. Let’s inspect for a moment the modern understanding of “proper” decision-making. Several studies have suggested that teams dramatically underrate the cost of a lost possession and should almost always keep their offense on the field. A high school coach who never punts has become something of a cult hero.

This analytic-based understanding has become so ubiquitous that there’s now a Twitter bot whose raison d'être is chiding coaches for their conservative decision-making.

What I don’t see very often is people questioning— seriously questioning— whether this understanding is actually correct. So I’d like to do that for a second.

I understand the numbers and the data being used to reach these results. But as we all know, analytics aren’t perfect, and the numbers you choose and the analysis you use will have a dramatic impact on the final conclusions. And failing to account for key variables could completely throw off your results.

Let’s consider for a second a coach who I believe I can uncontroversially call “smart”. Bill Belichick is famously a student of the game; his library of football books is widely considered the largest and most comprehensive in the world. Belichick is known for the relentlessness with which he searches out any tiny scrap of thought or knowledge that could give him the tiniest edge on a football field.

Belichick is certainly no stranger to analytics, either. Belichick’s closest confidante is a mysterious man named Ernie Adams. Belichick has been working with Adams since 1991; former Browns GM Ernie Accorsi once said of Adams “I was never really sure what Ernie did, but you could tell he was important to Bill. He was upstairs during the games and he had some say in things. He was an analytics guy before there were analytics.”

When questioned about whether he’s read some obscure study on football strategy, (including the ubiquitous 4th-down studies), Belichick always responds that he has. His brilliance is unquestioned, as is the forward-thinking nature of his approach and his comfort with analytics. Indeed, in many ways, Belichick serves as a vindication of the current analytic consensus that coaches should go for it on 4th down more often, since Belichick goes for it more often than nearly any of his peers.

At the same time, Belichick also serves as a stunning rebuke to the current analytic consensus, because he does not go for it nearly as often as the consensus suggests he should. Why not? We can’t say he’s not bright enough to understand. We can’t say he has disdain for numbers. We can’t say he’s worried about job security, or he’s afraid to take risks, (remember, this is the man who famously called for his team to take an intentional safety against the Denver Broncos).

Instead, we must conclude that Belichick has seen the numbers and he simply disagrees with them. And then we must ask who we think is more likely to be right about proper 4th-down strategy, the people producing the numbers, or Bill Belichick?

Before we ask ourselves that question, let’s consider one more anecdote. Chip Kelly came into the NFL with much fanfare, vowing to run his team scientifically to a degree never before seen. He brought in a host of new analytics, including player GPS tracking to monitor for injuries before the first symptoms even showed.

Kelly was also famous for his 4th-down aggressiveness in college, and his success was championed as a triumph of reason and analysis. Many expected him to come into the NFL and revolutionize decision-making processes that had grown stale over decades. (Though Kelly himself was quick to disagree.)

Instead of changing the NFL, the NFL changed Chip Kelly. His 4th-down decision-making in the pros has been much more conservative than it was in college. I would suggest that this wasn’t out of any fear for his job security; Kelly had no problem enacting a whole host of other, more controversial policies. And it’s certainly not because he doesn’t know or understand the numbers.

Instead, I think the examples of Bill Belichick and Chip Kelly suggest that there’s far more to proper strategy than is currently being considered. That the current analytical consensus, which has for too long gone unchallenged, is missing something rather crucial.

I would suggest that perhaps the right thing to do on 4th down is closer to what Bill Belichick is actually doing than it is to what an algorithm suggests over Twitter every Sunday.

Braess’ Paradox and Complex Optimizations

Enough about what coaches obviously get wrong, let’s look for a minute at something that coaches actually get right.

With the Philadelphia Eagles, Andy Reid was often the butt of national jokes, criticized for seeming to forget for long stretches of time that he even has a running back on the field. His in-game decision-making, like John Fox’s, was often called into question.

2004-2006 was the most pass-heavy stretch of Reid’s career in Philadelphia. In those three seasons, his Eagles ranked 9th, 3rd, and 8th in passing attempts and 31st, 30th, and 27th in rushing attempts. Over the three-year strech, Reid’s Eagles threw on 59.8% of their offensive plays. The league as a whole threw just 53.4% of the time.

So far in 2015, though, the league is passing on 58.0% of its offensive plays. Reid, it turns out, was not absent-minded; he was a decade ahead of his time. Indeed, Reid’s Chiefs find themselves passing on 55.8% of their offensive plays, a value that looks almost like a throwback in today’s NFL.

With the passing game in ascendence, the pendulum has now swung, and some have begun to question whether perhaps coaches don’t throw often enough. After all, the pass is much more efficient than the run in terms of moving the football down the field.

The answer to that question must first deal with something called Braess’ Paradox. To explain Braess’ Paradox, we must take a quick detour and discuss traffic.

Imagine that you and nine of your neighbors all work at a nearby power plant, and there are two routes that you can take to get there. The first road is wide and clear, but a bit out of the way; this route will take you ten minutes regardless of the number of cars on the road. The second route is a narrow but direct alleyway. Because it is so small, it is prone to traffic jams, and the amount of time the route takes is equal to the number of cars that take the route.

If you’re heading out to work in the morning and your only goal is getting there as quickly as possible, which route will you take? Obviously the narrow, direct road; worst case scenario, all ten drivers take it, and even then it’s not any longer than the broad, winding route.

Assuming all of your neighbors feel the same, they’ll all take the narrow winding route, and the average commute time will be ten minutes. This is what’s known as the Nash Equilibrium— the point where everyone is acting in their best interest, and no one is individually better off by changing their decision.

But let’s say that the government rules that five of your neighbors are forced to take the long way. Those neighbors still take ten minutes for their morning commute. The remaining five neighbors, though, find their drive cut in half; they reach work in just five minutes. Across all neighbors, the average commute time has dropped to 7.5 minutes. And if the government rotated which five neighbors were ordered to take the long way, then the end result is that by denying drivers the chance to act in their own best interests they wind up better off.

That’s Braess’ Paradox in a nutshell. Individuals do what is in their own best interest, but sometimes what’s best for the population as a whole is to force some individuals to behave slightly less efficiently.

What does this have to do with football? Imagine that passing is like the narrow, direct road while running is like the broad, winding road. Every time a team passes the ball, it’s like adding another car, and every individual pass attempt becomes less efficient, (because defenses now know more of the offense’s plays and are primed to expect the pass more often).

If coaches called their most efficient play on every down, they would quickly reach the Nash Equilibrium where passing and rushing were both equally efficient. This would maximize the efficiency of each individual play, but it would not maximize the efficiency of the offense as a whole.

Instead, coaches must sometimes call less effective plays so that their most effective plays don’t lose all of their potency. This high-wire act involves balancing the needs of any individual down against the needs of the offense as a whole, and it is a really, really difficult optimization process.

Speaking of really difficult optimization problems, let’s take another quick detour. At the University of Iowa, scientists have developed something that has come to be known as the Iowa Gambling Task, or IGT. In it, participants are given four decks of cards to choose from. Some cards in these decks represent winnings, while others represent losses.

The four decks are all different in terms of how the wins and losses are distributed. The subject’s task is to draw cards in a manner that maximizes total winnings. And the scientists find that after 40 or 50 card draws, most participants are able to figure out which decks are the “good” decks and limit themselves to those piles.

The interesting thing, though, is that when scientists measure stress, they detect indicators when the participant is about to select a “bad” deck after as few as 10 trials. The subconscious mind, it seems, has managed to figure out the truth four times faster than the conscious mind.

Let’s go back to our pass:run optimization problem. How good are coaches at it? Well, consider this post by Chase Stuart at his website, Football Perspective. In it, Chase questioned why teams ran the ball, given that the pass was so much better. And at the end, he produced a chart that featured the league-wide pass:run ratio in black, and the net difference between per-pass efficiency and per-run efficiency in red.

Pass:Run Ratio

Take a minute to consider that graph. The two lines move in near-perfect lockstep. NFL coaches, for nearly 65 years, have been reacting to increases in relative passing efficiency with perfectly commensurate increases in relative passing frequency. As I said, that’s a tremendously difficult optimization problem, and coaches as a whole have been performing it entirely unconsciously— much like the participants in the IGT.

While individual coaches will frequently see their run:pass balance questioned, it’s truly striking how well the profession as a whole has managed to intuitively find the perfect balance across the league, season after season after season.

Indeed, as with the IGT, we overrate the importance of conscious knowledge. I would wager that, if asked, coaches could not explain how they settle on the precise play ratio they use. And yet their subconscious still manages to aggregate decades of experience to reach the correct answer.

Apportioning Playing Time

There’s one last criticism that coaches frequently face that I want to discuss today: the criticism of how they dole out playing time.

Sometimes we see a talent who is quite obviously transcendent, and yet the coaching staff refuses to use it. I think most prominently of Jamaal Charles, who in 2010 was a first-team AP All Pro at running back… despite not even leading his team in carries. Kansas City opted to give Charles 230 carries, while giving 245 to Thomas Jones.

Jamaal Charles averaged 6.4 yards per carry. Thomas Jones averaged 3.7. I’m not a huge fan of yards per carry as a measure of player quality, but a disparity that large is far too striking to ignore. I would hold this up as a stunning display of coaching incompetence.

At the same time, I remember the plight of one of my favorite players. From 2004 to 2007, Mewelde Moore served as the backup running back for the Minnesota Vikings. He was dynamic as a rusher and a receiver, averaging nearly five yards per carry and nearly ten yards per reception when called upon. But he was rarely called upon unless there was an injury ahead of him on the depth chart.

At first, I thought this was a result of coaching incompetence. Mike Tice, Minnesota’s then-coach, was never considered great. In Moore’s third year, Minnesota fired Tice and hired Brad Childress, a sharp offensive mind coming from Andy Reid’s staff in Philadelphia.

When Childress ignored Moore, I held out hope that he’d get a chance in free agency. In 2008, Moore was signed by the Pittsburgh Steelers to back up Willie Parker. And when Willie Parker struggled and missed time, Moore was once again a revelation, outperforming those ahead of him on the depth chart and giving the entire offense a spark. And for his efforts, Mewelde was rewarded with… a spot back on the bench.

At the time, I thought that this was another case like Jamaal Charles where the coaches simply didn’t know what they had. But that belief became less and less tenable as more and more coaches viewed Moore as a change-of-pace back. In his first six years, Moore played for three head coaches and four offensive coordinators, and none opted to feature him despite his obvious brilliance when he played.

This, in my mind, is a perfect example of “What You See Is All There Is” thinking on my part. Moore’s coaches were smart. They saw orders of magnitude more from Moore than I did; seeing him every day in practice, watching every touch several times in film review. They surely had a much better feel for his strengths and weaknesses than I did.

If they were refusing to feature him they surely had a reason. Perhaps they felt Moore could not handle the wear and tear of a larger workload. Perhaps this was their way of optimizing for Braess’ Paradox; by using Moore more, they would have worn away what made him so special.

One other thing to consider is that all players are not built equally. While I am immediately leery of any claims that a player is “injury prone”, the natural variation in human genetics should dictate that some players are simply better able to hold up to a heavy workload than others. And if head coaches are smart, we’d expect they should be able to identify which players those are.

This idea that coaches are able to optimize playing time in this way seems again to be a testable hypothesis. So I did just that. I pulled the list of every running back from 2005 to 2014 who appeared in at least 6 games and averaged at least 14 touches per game. This left me with a sample of 304 player-seasons.

In theory, every time a player touches the ball he is at heightened risk for injury, as 11 large and athletic men focus on bringing him to the turf. We would expect that running backs who received more touches therefore missed more time to injury.

One possible confound is the idea that an injury itself could make running backs look like they received a lighter workload. Imagine a running back who averaged 20 touches per game for nine weeks, then was injured on the first touch of the game in week 10. Because of the injury, his per-game average would fall from 20 to 18.1.

To control for that, for players who missed games I also calculated a “hypothetical touches per game”. Hypothetical touches per game operated under the assumption that a running back who missed time got injured on the very first carry of their final game, and then calculates touches per game in the remaining contests. This is obviously a very conservative hypothetical; few backs truly get injured on their very first carry.

I have then divided my 304-season sample into 50-season groups, (slightly more in the case of ties). For each group, I have recorded minimum touches per game, maximum touches per game, average touches per game, average hypothetical touches per game, average games played, percent of the sample who played 16 games, percent of the sample who played 14 or more games, and average touches per missed game.

 LowHighT/GHyp T/GAvg G%16%14+T/MG
Group A 22.75 28.56 24.14 24.87 14.84 0.66 0.80 308.9
Group B 20.38 22.69 21.51 22.44 14.62 0.46 0.78 227.8
Group C 18.92 20.36 19.69 20.87 13.66 0.30 0.64 115.0
Group D 17.31 18.88 18.15 19.19 13.65 0.39 0.71 105.2
Group E 15.81 17.27 16.53 17.58 13.58 0.30 0.60 92.8
Group F 14.00 15.80 14.87 15.67 13.93 0.37 0.67 99.8

In both touches per game and hypothetical touches per game, the average difference between groups is approximately 1.85, suggesting missed time isn’t skewing the numbers very much. But the real takeaway comes from the final four columns.

In all three measures of missed time, (games played, %16, %14+), the top group finishes first and the second group finishes second. Despite receiving the heaviest workload, no group of running backs was as likely to remain healthy as the workhorses.

Looking at touches per missed game, the results are especially dramatic. The highest-workload running backs averaged nearly three times as many touches per missed game as the bottom 66%. The second-highest workload group averaged more than double the touches per missed game of the bottom 66%.

If there was any question whether coaches were performing some sort of behind-the-scenes optimization— distributing workload not just based on ability, but on a player’s ability to handle it— this should put that immediately to rest.

No coach has ever made this optimization as explicit as Bruce Arians did about Andre Ellington his rookie year. Ellington was a shooting star, making big play after big play. The calls to give him more touches were rising by the week. And Arians finally admitted that he was concerned about overworking Ellington and shortening his career as he had with Willie Parker’s in Pittsburgh.

That comment is illustrative. As is the fact that Ellington saw a dramatically increased workload last year and that was accompanied by a commensurate decline in his effectiveness. As is the fact that Ellington is back in a limited role this year, routinely making big plays, and starting to build a consensus that he should be getting more touches.

So again, what does this mean? Let’s consider Lamar Miller, the current title-holder of “player most fans think should be getting the ball more often”. Over the first four weeks of the season, head coach Joe Philbin reduced Miller’s workload week by week, with his carries falling from 13 to 10 to 7 to 7. When Philbin was fired over Miami’s bye week, many assumed that unleashing Lamar Miller would be the top priority of interim head coach Dan Campbell. And the first two weeks after the bye seemed to bear that out with Miller getting 19 and 14 carries.

But the two weeks following the bye were also massive blowouts for the Dolphins, who called more running plays overall as a result. Teams with bigger leads call more runs, a concept that has come to be known as “game scripts”. And after those huge two weeks, Miami reverted to more typical game scripts over their last four. The result for Miller has been 9, 12, 16, and 7 carries, (11 per game).

Indeed, it seems Campbell couldn’t wait to begin talking up backup running back Jay Ajayi as soon as he returned from IR, suggesting he needed more carries going forward. The new coach seems almost as leery of giving Lamar Miller carries as the old coach was. (In fairness, Campbell has substantially increased Lamar Miller’s role in the passing game.)

This raises an important question. Is Lamar Miller the next Jamaal Charles— a player whose value was overlooked and who was criminally underutilized early in his career? Or is he perhaps the next Mewelde Moore, or Felix Jones, or Tatum Bell, or Jerious Norwood, or Leon Washington, or any of a number of other players who have looked like sensational talents on reduced workloads but never took the next step to a higher volume.

After all, of all of a player’s stats, none is more stable and consistent from sample to sample than his usage. Which again suggests that coaches are seeing something inherent in the player and tailoring a specific usage plan to best fit him.

Who Cares if Coaches Are Smart?

The point of Dynasty, in Practice is to provide practical advice for dynasty owners. The idea that coaches are smart hardly seems to fall under that umbrella. But, in fact, assuming that coaches know more than we do, (even if they are sometimes wrong), carries sweeping practical ramifications.

If we assume that coaches don’t know what they’re doing, it’s easy to tell a compelling story for why Lamar Miller is currently undervalued. He’s obviously a star to everyone who watches him play. If we assume that coaches do know what they’re doing, though, it’s just as easy to tell a compelling story for why Miller is currently overrated. Because the current consensus discounts the fact that perhaps there’s a reason Lamar Miller finds carries so hard to come by.

If we find ourselves advancing a theory for player value that relies on the idea that we know his strengths better than his own coaches do, I'd suggest that perhaps it's time to think long and hard about our position. Coaches, after all, are pretty smart.


More articles from Adam Harstad

See all

More articles on: Dynasty

See all

More articles on: Strategy

See all