Photo: Grace Hollars, USA Today Sports

Jeff Saturday was one of the best centers of his generation. An undrafted free agent, Saturday started a pair of games as a guard in his second year before earning the starting center job in year three.

Saturday would go on to start 186 out of a possible 192 games for Indianapolis over the next twelve years, serving as an anchor over one of the most consistently great teams in history, a team that won 12 or more gains an NFL-record seven straight years. Saturday snapped the ball to the league MVP in four different seasons, making five pro bowls along the way and twice being named a first-team All-Pro.

After 2011, Saturday's contract with the Colts was up and he signed with the Green Bay Packers in free agency. And in 2012, voters once again honored him as one of the best centers in the NFL, sending him to his sixth pro bowl. The biggest problem, of course, is that Saturday wasn't even the best center in Green Bay; the Packers benched him for the last two games of the season because of his poor play.

Saturday is probably the best example of the biggest flaw with the pro bowl— the tendency for players past their prime to coast on reputation, especially at positions with few statistics. Coach John Fox once said that it took a year to get into the pro bowl and another year to get out. Young players have to play for a while at a pro bowl level before earning the name recognition that led to pro bowl berths. Old players have to play for a while below that level to lose the name recognition again.

While this is the best-known flaw, it's hardly the only one. For instance, pro bowl voting closes before the season is even finished, sometimes leading to situations like Richard Sherman's in 2012; the league announced it was suspending Sherman for failing a test for performance-enhancing drugs during the peak of pro bowl voting. Sherman contested his suspension and had it overturned... but not until voting was closed.

In the end, the Pro Bowl did not honor Sherman as one of the six best cornerbacks in the NFL that year. But the Associated Press's All-Pro team did honor him as one of the two best.

And beyond problems with positional designations, there's the simple fact that a fixed number of pro bowl berths can't reflect the individual contours of each season. Most commonly, there are more pro bowl berths at quarterback than there are quarterbacks worthy of a pro bowl berth, leading to situations like 2009 where Vince Young made the pro bowl with 1800 passing yards, 10 passing touchdowns, and 7 interceptions.

Less commonly, there are more spectacular performances than there are playoff berths; in 1995, Isaac Bruce had the second-most receiving yards of any player in history, yet he didn't make the pro bowl and he arguably wasn't even a snub.

The four pro-bowl receivers from the NFC that year had 122/1848/15, 123/1686/14, 122/1371/17, and 111/1603/10 receiving. In addition to Bruce, Robert Brooks failed to make the pro bowl despite 100 receptions, 1500 yards, and 13 touchdowns; there simply weren't enough spots to go around.

In 2003, LaDainian Tomlinson had 2370 yards from scrimmage, (which led the NFL), scored 17 touchdowns, and brought in 100 receptions. He didn't make the pro bowl that year, either. He likewise wasn't a “snub” in the general sense of the word.

Indeed, Jamaal Lewis rushed for 2,000 yards, Priest Holmes set the single-season touchdown record, Clinton Portis had 1900 yards in just 13 games for a playoff team, (Tomlinson's Chargers were 4-12), and Tomlinson had to watch the Pro Bowl from the contiguous United States like most of the rest of us. There just weren't enough berths even to ensure that every historically great season got one that year.

And even looking beyond a season-by-season basis, certain players for some reason simply get no respect from the pro bowl. Fred Taylor topped 1500 yards five times and didn't make the pro bowl in any of those years; it wasn't until he had 12,978 career yards from scrimmage that he made his first, (and only), pro bowl.

And Taylor is lucky compared to some. There have been 45 players to top 10,000 receiving yards in their career. Those 45 players have made an average of 5.5 pro bowls. 40 out of 45 made three or more. 44 out of 45 made at least one. And poor Joey Galloway stands alone as the best receiver to never earn an invite.

Hopefully, all these stories illustrate a very important point: as a measure of player quality, “number of pro bowl berths” is extremely flawed in a number of different ways.

And yet, as a student of history, I keep using it to compare players across eras, (despite the inevitable “LOL pro bowls” that I know will follow). Why? Because every measure of player quality is flawed. Just because it's flawed doesn't mean it can't be useful.

Another Terrible Metric

Consider the league MVP award. In truth, there are a handful of different bodies that hand out MVP awards, all of which are equally qualified, but for some reason, the Associated Press's MVP award has earned a place in popular consciousness as “the” MVP award.

Can Antonio Brown win league MVP? He would be first WR to do so https://t.co/BmzPO1JTZ7

— Ed Bouchette (@EdBouchette) December 14, 2017

 

Antonio Brown has a chance to be the first wide receiver to ever win NFL MVP, @greggrosenthal writeshttps://t.co/WRn0n4bOal pic.twitter.com/omATDexyvr

— Around The NFL (@AroundTheNFL) December 12, 2017

 

Antonio Brown has a 0% chance of becoming the first receiver ever named MVP.

He does have an outside shot at becoming the fifth receiver named MVP, though, after Don Hutson (‘41 and ‘42), Harlon Hill (‘55), Lance Alworth (‘63), and Jerry Rice (‘87).

— Adam Harstad (@AdamHarstad) December 13, 2017

Now, “MVP” is an insanely high standard, especially if we're limiting ourselves strictly to the AP MVP. Obviously not winning a single Associated Press MVP doesn't mean a player isn't great, (see: Drew Brees, Jerry Rice, Dan Fouts). But winning an MVP award certainly means a player was tremendous.

Except we're talking about flawed measures, so you know that's not going to be the case. I mentioned Dan Fouts never won an Associated Press MVP award. He did win one from the Pro Football Writers of America and the Newspaper Enterprise Association in 1982, though. That was a strike-shortened year, but Fouts was on pace for 5125 passing yards, which would have broken the single-season NFL record.

(The single-season record, for what it's worth, had been set by Fouts himself in 1981. Before that, it was set by Fouts in 1980. That record broke the one previously set by... Dan Fouts in 1979.)

So if Fouts was so marvelous, what paragon of football excellence did the Associated Press name its most valuable player? That would be... erm... a kicker. Mark Moseley had a great season, even a historically great season, one of the best kicker seasons the league has ever seen. But to be named MVP over what was essentially a 5,000-yard passer?

And remember, Fouts was essentially on a record-setting pace for a fourth consecutive season. Not to suggest Moseley's year was a fluke, but he only made the pro bowl one other time in his career, (whoops, there I go using pro bowl berths as a measure of player quality). Clearly, "AP MVPs" is a pretty flawed metric.

(Other players with more AP MVP awards than Dan Fouts include Brian Sipe and Bert Jones, who combined for two career pro bowls. There I go again.)

Everything Is Awful!

You could apply this level of scrutiny to anything and find major flaws in it. Football is a chaotic sport with small sample sizes and lots of entanglement; weird outcomes are basically de rigueur. That's a large part of what makes it so great, in my opinion.

The thing about pro bowl berths, (and MVPs, and fantasy points, and yards from scrimmage, and touchdowns, and quarterback wins, and everything else that some people use to support a point and other people laughingly tear down), is that while they don't perfectly correlate with player quality, they still correlate with player quality.

Imagine a hypothetical league with ten players and three pro bowl berths. The best player makes it 95% of the time, the second-best makes it 75%, the third-best makes it 55%, the fourth-best 40%, the fifth-best 15%, the sixth-best 8%, the seventh-best 6%, the eight-best 4%, the ninth-best 2%, and the worst player makes it 0% of the time.

This seems like an extremely flawed stat. The second-best player in the league misses the pro bowl a quarter of the time! One time out of fifty, the second-worst player is named to the pro bowl! How awful!

But pro bowls still tell us useful information. 88% of pro bowlers are top-4 players and 93% are top-5. No pro bowler was the worst player in the league. The “average pro bowler” ranks 2.6th in the league, (a perfect process would lead to an average rank of 2nd).

In this hypothetical league, can you say a player with one pro bowl is better than a player with zero? Absolutely not, at least not with any degree of confidence. But is a player with ten pro bowls better than a player with two? Yes, that's a virtual certainty.

If you demand perfection from a metric before you'll accept it you'll find yourself waiting a long time in the dark. The question isn't whether something is unassailable, it's whether that thing provides value or context that can help us gain a greater understanding. This need for context is why I'll pair “career pro bowl berths” with other metrics when making comparisons, like top-5 finishes and total yards.

The picture painted by several data points combined is more complete and more robust than that painted by any of those data points in isolation.

In our hobby, it's really quite important to know which players are actually good and which players are actually not. It's also quite difficult. After Todd Gurley's rookie year, many were convinced he was really good. After his sophomore year, many were convinced he was really not. The owners who clung to the belief that he was good have been richly rewarded in year three.

There's no magic bullet that will ever let us perfectly sort the good players from the bad ones. But if we combine enough flawed metrics, we can usually at least make a dent in the problem, we can get closer to where we need to be.

After all, the goal of fantasy football has never been to be perfect. It's simply been to be better than the other guy.