Director of Football Data and Analytics National Football League
New FeatureThe Extra Point
Welcome to the Extra Point, where members of the NFL's football data and analytics team will share updates on league-wide trends in football data, interesting visualizations that showcase innovative ways to use the league's data, and provide an inside look at how the NFL uses data-driven insight to improve and monitor player and team performance.
Down one point with 10 seconds remaining in an NFC Wild Card game against the Philadelphia Eagles, the Chicago Bears’ Cody Parkey attempted a 43-yard field goal. Parkey’s kick, eventually ruled to be partially blocked, deflected off the left upright then the crossbar, before falling into the end zone for a miss.
A deeper look shows that the so-called “Double Doink” was more difficult than a typical 43 yarder. Parkey not only was kicking in a game-winning scenario in the playoffs in January in Chicago, but he also was kicking north in Chicago — one of the most difficult kicking directions in the NFL.
Since 2000, kickers in Soldier Field have made 63.2% of 40-50-yard field goal attempts when kicking north, the lowest of any stadium/direction combination in the league. But don’t blame poor field goal kicking — when kicking south, kickers in Soldier Field have connected on 77.6% of attempts from that same distance, which is roughly the league average.
Turns out, there are six NFL stadiums where it’s notably more difficult to kick towards one end zone than the other.
In addition to Soldier Field, kickers have an easier time facing south at Arrowhead Stadium in Kansas City and Gillette Stadium in New England. Alternatively, kickers at the former home of the Raiders (Oakland Coliseum), Heinz Field in Pittsburgh and Raymond James Stadium in Tampa Bay kick better when facing north.
The following chart shows estimated success rates for field goal attempts between 40-60 yards at each of those six stadiums, separated based on kicker direction. Lines for kickers facing north are shown in red, while kickers facing south are shown in blue.
For Parkey, his 43-yard field goal attempt kicking towards the north end zone of Soldier Field yields roughly the same chance of success as a 49-yard field goal towards the south end zone.
Digging deeper within each stadium provides evidence for why some of these differences exist.
Soldier Field lies near Lake Michigan, which is known for unpredictable wind. The direction of the Soldier Field wind tends to point south about twice as often (67%) as it points north.
Arrowhead Stadium is symmetrical, but both ends of the stadium are relatively open. The stadium sees south-facing winds about 62% of the time, making it more difficult to make field goals when kicking to the north. This potentially affects extra points, too — kickers facing south at Arrowhead have hit 98% of extra point attempts since 2015, compared to 92% of north-facing attempts.
Gillette Stadium features a distinctive lighthouse in the open, north end zone, making conditions more difficult when compared to kicking into the closed end zone to the south. After a December 2018 game against the Buffalo Bills, Patriots coach Bill Belichick cited the open, lighthouse end of the stadium as a reason that he kept his offense on the field for a fourth-down attempt. Kickoffs are also impacted. Since 2011, when the NFL moved the kickoff to the 35-yard line, 48% of kickoffs to the south end zone at Gillette Stadium have gone for a touchback, compared to 40% to the north.
Heinz Fieldin Pittsburgh, like New England, has an open end zone to the south. The open end faces the Allegheny River, and on field goals of 40 yards or longer in this direction, Heinz Field kickers have hit just 57%, compared to 75% when facing north.
Raymond James Stadium in Tampa features winds that more often than not blow north, leading to more successful field goal attempts in that direction. South-facing field goals of 50 yards or fewer have been successful 80% of the time, compared to 88% when kickers faced north. Gusts out of the south were so strong before a 2018 game that the netting behind the end zone got tangled up with the uprights.
Oakland Coliseum, the former home of the Raiders, is more of an enigma. Since 2003, field goal attempts facing south have been 6.7% less likely to go in when compared to the north — the largest difference in the league. The Coliseum is symmetrical, and the winds disproportionately point west (83% of games), so there is no clear advantage from the wind or a closed end of the stadium. The Coliseum was also home to baseball’s Oakland Athletics — throw in the A’s dirt infield, and kickers faced a variety of conditions that could have affected kicking.
Are coaches aware of these differences?
Assessing if team behavior changes based on kick direction is not straightforward, but the average field goal distance in each direction may be telling. For example, If worries about kicking north in Chicago impacted coaching decisions, perhaps the average field goal distance for north-facing attempts would be lower.
We checked the average field goal distance when kicking north or south in all current and former stadiums, and none of the six examples above ranked in the top-15 in terms of differences. In fact, New England and Pittsburgh both had the two smallest differences in the league — less than 0.05 yards for an average distance between north and south end zones.
If coaches have picked up on the different trends, it does not appear to be showing in terms of average distance.
Going for it on Fourth Down Becoming the New Norm for NFL Coaches
In 2013, a New York Times project compared the fourth-down decisions made by NFL head coaches to a statistically-driven “4th-Down Bot.” At the time, there was a wide disparity between what the bot suggested and what the coaches did in live game situations.
Seven years later, the increased use of football analytics has also led to it becoming more normal for teams to go for it on fourth down with five or fewer yards to gain. In 2019, teams went for a first down 26.2% of the time on non-high leverage fourth downs (fourth-and-five yards or fewer to gain), the highest rate in at least two decades.
The charts below mimic the ideas of the bot’s originators. The charts use data from 2000 to present when games are close, and show current head coaches with two or more years of head coaching experience. The colors of the rectangles are determined by using a weighted sum favoring recent years. The most aggressive coaches are in the top left of the chart and the least aggressive are in the bottom right.
Philadelphia Eagles head coach Doug Pederson was the coach who was most likely to go for it in these situations, especially in fourth-and-one situations.
Interestingly, Tampa Bay Buccaneers head coach Bruce Arians, known for his downfield passing offense, is among the least aggressive head coaches on fourth down and less than five yards, while Baltimore Ravens head coach John Harbaugh, known for his focus on the run, is among the most willing to go for it.
When looking at the 11 most aggressive coaches, seven of them made the playoffs last year. Additionally, three out of the remaining four — Mike Tomlin, Dan Quinn and Ron Rivera — led their teams to a Super Bowl in previous seasons.
Probability models for the new 14-team playoff system
Last week, the NFL owners voted to expand the postseason from six to seven teams per conference. In the new format, the No. 1 seed is now the only team with a first-round bye. The No. 2 seed will face the No. 7 seed on Wild Card weekend. The third through sixth seed opening matchups remain the same as they were under the 12-team format that went into effect in 1990.
The graph below shows how each seed’s estimated probability of winning the Super Bowl changes under the new playoff format. The dots represent the probability of winning the Super Bowl under the old format, the arrow extends to the new probability under this format. The text on the right shows the change in probability.
Statistical models inspired by this paper were used to determine a distribution of each team’s strength, the value of home-field advantage and the value of having a bye. For each game, these factors were used to determine each team’s probability of winning any given matchup. Under this model, we simulated the playoffs 100,000 times for each format with a variety of possible team strengths for each seed to account for how each postseason can differ. The number of Super Bowl wins by seed is divided by the number of simulations to calculate the probabilities.
Only the No. 2 seed sees a decrease in the probability of winning the Super Bowl — primarily because they lose a first-round bye and now play an additional game, introducing another chance to be eliminated. The probability for the No. 1 seed increases because it now has a lower chance of playing the No. 2 seed in the Conference Championship, and a greater chance of playing a lower-seeded team. The probabilities of the remaining seeds all increase because they are now more likely to avoid higher-seeded teams and to host a playoff game in the Divisional or Conference Rounds. Since the No. 7 seed didn’t exist before, its probability increases from 0% to 2.9%.
Interestingly, the No. 5 seed has a slightly higher probability than the No. 4 seed in both formats. This is likely because the fifth seed often has a better record (and underlying team strength) as the best of the 12 non-division winners, while the No. 4 seed has the worst record of the four division winners.
This new format makes earning the top seed more important and gives two additional teams a chance to compete for the Lombardi Trophy, which should lead to even more exciting regular season and playoff football.
Big Data Bowl Winning Paper Leads to New Drill at the NFL Combine
This past weekend at the NFL’s Scouting Combine in Indianapolis, defensive linemen were tested in a timed “Figure 8” drill (also called the “Hoop” drill). In a Figure 8 drill, linemen start next to one of two adjacent circles. They sprint halfway around the first circle, before making a complete loop around the second circle, then around the remaining section of the first circle ending back at the starting line.
Check out LSU defensive end Rashard Lawrence in the clip below.
Lawrence also picks up and discards two towels on the ground. The towels mimic where the defenders’ hands often end up on a pass rush.
Defenders are likely used to this drill from their high school or college practices, and with good reason — the path closely mimics player movement on the field.
Here’s how the Football Operations data and analytics team helped propose the Figure 8, with a hat tip to the College Division winners of the 2019 Big Data Bowl.
The 2019 winning paper, “Routes to Success,” was written by four students from Simon Fraser University — Dani Chu, Lucas Wu, Matthew Reyers and James Thomson. Though the original paper tracked the most common wide receiver patterns, the algorithm easily translates to other positions. Last summer, the students worked with the NFL Football Operations team to modify it for defensive linemen.
Using Next Gen Stats player tracking data, Chu, Wu, Reyers and Thomson generated the most common movements on run and pass plays for all positions on the field. Here’s an animation showing how all defensive end movement on pass plays can be grouped, initially from one for each team, to one for each of nine common movements, shown at the end of the animation.
The nine most common movements for defensive ends are shown below. For example, the most common defensive end movement (top left) is an outside pass rush, lasting an average of 4.38 seconds and ending around five yards upfield. Alternative paths are shown in the remaining clusters. For example, clusters seven and nine show the ends rushing upfield before sprinting sideways.
Look familiar? Most of the common patterns feature players moving in curved patterns — curved patterns that are also represented in the Figure 8 drill. Armed with this data, the NFL implemented a version of the Figure 8 drill at the International Combine in Germany before bringing it to Indianapolis this past weekend.
The Figure 8 was one of several new drills at the 2020 Combine. It was a concept that started with Next Gen Stats and came to fruition thanks to the hard work of students at last year’s Big Data Bowl.
2020 Big Data Bowl Recap
The second annual Big Data Bowl, powered by Amazon Web Services (AWS), focused on predicting the outcomes of rushing plays during the 2019 season. Participants were provided with the NFL's Next Gen Stats, including speed, direction, and location information for all 22 players on the field at the moment a ball carrier receives the ball, and were tasked with predicting where the ball carrier would end up.
This year, six collegiate finalists presented their work to NFL club analytics staff at the NFL Combine in Indianapolis. Three honorable mention papers were also named. Below is a summary of each presentation and a link to the complete entry.
Matt Ploenzke (Harvard)
Ploenzke used Next Gen Stats data to build interpretable model inputs based upon football-specific domain knowledge, ultimately highlighting the importance of ball carrier downfield acceleration and unblocked tackler distance and spacing.
Key stat: Among roughly 40 input variables, a ball carrier’s “effective acceleration” was the most important for estimating yards gained on a handoff play.
Kellin Rumsey, Brandon DeFlon (University of New Mexico)
The battle between blocker and defender is often decided by leverage. In this paper, Rumsey and DeFlon define offensive and defensive leverage, and study the statistical properties of these metrics.
Key stat: In the first six weeks of the 2017 season, Blake Martinez (Green Bay Packers) was among the league’s best at generating defensive leverage. Martinez finished the season with the third-most solo tackles (96).
Pash and Powell used kinematic data such as player positions and velocity to determine zones of control for both the offensive and defensive teams at the time of the handoff. These zones of control predict the probabilities of yards lost or gained and quantifies the risk involved with plays.
Key stat: Robert Woods (Los Angeles Rams) and Raheem Mostert (San Francisco 49ers) outperformed the model predictions the most, averaging nearly three more yards than predicted over the 2017 and 2018 seasons.
Namrata Ray, Jugal Marfatia (Washington State University)
Ray and Marfatia measured the open space of the rusher at three time intervals — handoff, after a half-second, and after one second — to understand the association between open space and yards gained. Results indicated that the difference in the open space between the time of handoff and after a half-second or full second was a strong predictor of the number of yards gained.
Key Stat: Yards gained by the rusher increases by four yards on average for every one percent increase in the additional open area created within a half-second of the handoff.
Using an advanced machine learning algorithm, Stern assessed the value of initial space created for the ball carrier by the offensive line. That space was then linked to linemen grades, and standardized by accounting for the number of defensive backs, linebackers and defensive linemen on the play, the defensive strength of the opposing team, and the running direction of the running back.
Key Stat: In 2018, New Orleans Saints center Max Unger received a top five grade according to Stern’s space grade rank for centers, despite being the 31st-graded run blocker by Pro Football Focus.
Brighenti computed each team's control of the field at the moment of the handoff to predict the outcome of rushes. Brighenti found that offensive control at the running back's expected point of intersection with the line of scrimmage was the most important predictor of run yardage.
Key Stat: The critical factor separating successful and unsuccessful plays is ownership of the run gap at the line of scrimmage — even on plays gaining more than 10 yards, the difference in field control past the line of scrimmage was almost negligible.
What the Top Predictions Looked Like in the NFL's Big Data Bowl
As part of the NFL’s second Big Data Bowl, more than 2,000 data scientists from all over the world competed on Kaggle to predict rushing play outcomes over the last five weeks of the 2019 regular season.
Data scientists Phillip Singer and Dmitry Gordeev from Austria captured the top prize of $50,000 with their highly technical approach. Nicknamed The Zoo on Kaggle, Singer and Gordeev’s team led throughout the contest, besting second place by the same scoring margin that separated second place and 24th place.
The Zoo posted their predictions on Kaggle, and today’s Extra Point assesses how closely their predictions matched what happened on the field. Note: each prediction was made only using run plays from the past. In statistical terms, these are considered out-of-sample predictions.
What was being predicted?
The goal of the Kaggle contest was to predict where every ball carrier would end up on a handoff play. Participants were given several characteristics of the play, including the NFL’s Next Gen Stats, and game and play traits such as the down, distance, starting yard line and which players are on the field.
For example, on this Cordarrelle Patterson carry for the Chicago Bears in Week 1 against the Green Bay Packers, Patterson barely had a chance of reaching the line of scrimmage. The Zoo assigned the following probabilities:
loses 3 yards or more: 9.4%
loses 2 yards: 19.4%
loses 1 yard: 19.9%
gains 0 yards: 21.4%
gains 1 yard: 11.5%
gains 2 yards: 6.9%
gains 3 yards or more: 7.2%
Patterson lost 2 yards on the play — one of the three most likely outcomes predicted by The Zoo’s model.
While predictions on Patterson’s carry were grouped near zero yards, on this Raheem Mostert 41-yard touchdown run against the Carolina Panthers in Week 8, The Zoo gave the San Francisco 49ers back a 10.5% chance of reaching the end zone — making it more likely that he would score than Patterson was to pick up three yards. According to The Zoo, Mostert had a 61% chance of picking up 10 yards or more on his carry.
How accurate was The Zoo across all handoff plays?
Looking at all Big Data Bowl plays from Weeks 1–12 provides a sense of how precise these predictions were. The following chart groups each carry into a predicted number of yards, and averages the observed yardage gained within each group of predictions. The black dotted line corresponds to settings where predictions perfectly matched reality.
By and large, The Zoo’s predictions reflected what eventually happened on 2019 carries. For example, there were precisely five carries where The Zoo’s model predicted that the most likely yardage gained would be 20 yards — on those carries, the ball carriers averaged 19.2 yards. The Zoo’s most common yardage prediction was four yards, and of the 1,543 carries where four yards was the prediction, ball carriers averaged 3.98 yards.
Predicting football outcomes is challenging, and the predictions made as part of the Big Data Bowl were no different. The Zoo’s most likely rushing distance on each play was an exact match on only 17.6% of plays, which means that more than four out of every five times, the offenses over or underperformed expectations. But being able to better understand what goes into successful plays — separating Patterson’s carry from Mostert’s, for example — gives us deeper insight into what drives ball carrier success.
On February 26, The Zoo and other Kaggle top finishers will join finalists from the Big Data Bowl collegiate contest and present their results to NFL team analytics staff at the Scouting Combine in Indianapolis.
Why Fourth-and-15 from the 25? Insight into the NFL’s experiment with an onside-kick alternative
In the 2020 Pro Bowl, a team scoring a touchdown or field goal will be able to attempt a fourth-and-15 offensive play from its own 25-yard line to try to keep possession of the football. If the offense converts, it keeps the ball; if it falls short, the defending team takes over at the dead ball spot.
Why this experiment? In today’s post, we’ll explain how the league used analytics to inform this potential onside kick alternative.
The Football Operations data and analytics team’s first research into an onside kick alternative began before the NFL’s offseason meetings in March of 2019, when the Denver Broncos proposed a rule change that would give scoring teams the option of using one fourth-and-15 play each game. Under Denver’s proposal, the scrimmage play would occur from a team’s own 35-yard line. Denver was motivated by the decline in onside kick recovery rates. Kicking teams historically recovered onside kicks between 15% and 20% of the time in a given season. In 2018, in part to changes on the kickoff play, that number dropped below 10%.
The league’s challenge: How to give teams an opportunity to maintain possession by using game play instead of the onside kick.
The first aspect of game play we looked at was how often teams convert on third and fourth down given various yards to gain. But plays aren’t simply “convert” or “not convert” — on several plays, penalties either give the offense the first down, or require the offense to start over from a different position.
Here’s a chart that shows the complete set of scrimmage play outcomes, using plays from 2002 through 2018. Only plays run with a score differential of eight points or fewer are included, and those run within the last two minutes of each half are dropped.
The light red band reflects the percentage of plays where the offense picks up a first down on the play, while the dark red corresponds to first downs picked up by penalties (defensive holding and defensive pass interference, for example, lead to automatic first downs). The dark blue areas correspond to what we termed as “do-overs,” in which the offense keeps possession but needs to attempt a play from a new line of scrimmage. Finally, the areas in light blue are defensive stops.
In reality, the closest match for an onside kick is to follow a team attempting a fourth-down play until either it has picked up a first down or a defensive stop is made, while also accounting for the do-overs. In other words, if 100 teams attempt a fourth-and-15 but pick up an offensive holding penalty, at least one of those teams is going to convert the ensuing fourth-and-25.
Accounting for do-overs, we settled on the following chart to compare Denver’s proposal of a fourth-and-15 with the historical onside kick recovery rate of 13.2%.
Denver’s proposal is slightly more forgiving than the onside kick (note: we used only expected onside kicks in this calculation), although the differences are within a few percentage points. A perfect comparison for the historical onside play would be a fourth-and-17 scrimmage play.
Lastly, we searched for the best yard line to use for the scrimmage play. Although Denver suggested the 35-yard line — the same as where teams kick off — teams that pick up first downs on scrimmage plays typically gain more yards than just the line to gain. Teams converting on fourth-and-14 to fourth-and-16 typically end up eight yards past the line to gain when they convert. Our suggestion then — and the yard line that we’re testing at Pro Bowl — will push offensive teams back 10 yards to their own 25-yard line.
Punters in 2019 Were Performing Better Than Ever
While the improvement of NFL placekickers over the last few decades has beencelebrated repeatedly, another group of specialists is performing better than they ever have: punters.
Check out the following stats comparing regular season play from 2000 to 2019:
Nearly one-quarter of punts (23.3%) from near midfield — between the 48-yard lines — resulted in a touchback in the 2000 season. In 2019, that rate dropped to just under 10% (9.8%).
There were 110 total regular season touchbacks in 2019, 55% fewer than in 2000 when there were 247.
The net punting average was 5.8 yards higher in 2019 than it was in 2000 — even after accounting for differences in field position between the two seasons of play.
The punter with the lowest per-punt average in 2019 — Dallas’ Chris Jones, at 41.6 yards — would have been ranked No. 17 in 2000 — or league average.
The combination of punt distances and net punting averages going up, along with touchback rates going down, means that offensive teams were more likely to be pinned back deep in their own territory in 2019.
The following chart shows the likelihood of an offensive team starting inside its own 20-yard line after receiving a punt. The lines show different territories of the field, each corresponding to the area from where the opponent punted.
For example, when a team punted from between its own 31 - and 40-yard lines, only about one in four punts in the early 2000s would pin the receiving team inside its own 20-yard line. In 2019, it was closer to one in two (45.1%). Similarly, when a team punted from between its own 41-yard-line and midfield in 2019, 70.1% of punts pinned the receiving team inside its own 20 — about 1.4 times the rate from 2000.
It’s not just the 20-yard line that teams are finding themselves behind. On kicks from near midfield, the likelihood of the receiving team starting from inside its own 10-yard line went up by a factor of 1.7, from 17.5% in 2000 to 31.0% in 2019.
More Girls are Playing Tackle Football
Tackle football has traditionally been a boys sport, but that’s starting to change. Spurred in part by girls-only tackle football leagues in Utah and Texas, 47 of 50 states saw an increase in the percentage of girls who play high school tackle football in 2018 compared to a decade ago.
The following heatmaps show the percentage change of high school 11-person tackle football players that were girls using participation data from NFHS.
In California, the percentage of girls playing was 4.7 times higher in 2018, jumping from 0.14% of players to 0.65% of players. California’s 593 girls playing tackle football was the highest total among all states. Nearby New Mexico had the highest percentage, with girls making up more than 1 percent of all players in 2018.
On the other side of the country, 204 girls played tackle football in 2018 in New Jersey, a rate that was 6.3 times higher than in 2008. Throughout the Northeast region, the rate of girls on teams grew by a factor of 5.9, with large jumps in participation in Rhode Island, New Hampshire and Vermont.
Across the country, 2,404 girls played tackle football in 2018.
Tackle football isn’t the only way girls participation continues to grow. Georgia in 2019 became the fourth state to sanction girls flag football as an official high school sport — an effort that was led by the Atlanta Falcons and the Arthur M. Blank Family Foundation — and the Tampa Bay Buccaneers just allocated $250,000 to create the first ever girls flag scholarship program.
The Biggest NFL Comebacks of the Last Decade
With only 11 postseason games left in the last season of the 2010s, let’s use our win probability model to look at which teams overcame the longest odds to win games over the last decade.
While Super Bowl LI provided the most memorable comeback of the last decade, with the New England Patriots erasing a 28-3 halftime deficit to upend the Atlanta Falcons, our model identified four other games where teams overcame longer odds to win.
Here are the top 10 least likely comebacks of the decade. Details of each comeback are provided below.
1. Philadelphia Eagles vs. New York Giants, Week 15 2010. Comeback odds: 1 in 252.
With 8:17 remaining in the game, the Eagles trailed 31-10. A 65-yard touchdown pass, successful onside kick, and a Michael Vick touchdown run cut the deficit to seven with just over four minutes remaining. After a Giants drive stalled, the Eagles drove the length of the field to tie the game with 1:16 remaining. After another New York drive stalled, DeSean Jackson returned the ensuing punt 65 yards for the game-winning touchdown as time expired. The game became known as the “Miracle at the New Meadowlands,” and it’s our most surprising comeback of the 2010s.
Here’s the win probability plot showing each team’s chances throughout the game.
2. Tampa Bay Buccaneers vs. Carolina Panthers, Week 11 2012. Comeback odds: 1 in 165
With 1:02 left in the fourth quarter, the Buccaneers trailed 21-13 and had the ball at their own 20-yard line with no timeouts. Josh Freeman led an 80-yard touchdown drive — including a two-point conversion — to force overtime. Tampa won the game 27-21 with a touchdown on the opening drive of overtime.
3. New Orleans Saints vs. Washington Redskins, Week 11 2017. Comeback odds: 1 in 159
With 4:14 in the fourth quarter, Washington led New Orleans, 31-16. The Saints converted a big third down and eventually scored a touchdown to cut the lead to 31-23. A Redskins three-and-out was followed by another New Orleans touchdown and two-point conversion that tied the game. The Saints won in overtime, 34-31.
4. Detroit Lions vs. Dallas Cowboys, Week 4 2011. Comeback odds: 1 in 153
With 10:30 left in the third quarter, Dallas had the ball and a 27-3 lead. On the next play, Tony Romo threw a pick-six to make it 27-10. Another Romo pick, followed by a rejuvenated Lions offense scoring a touchdown, touchdown, field goal, and a touchdown on its second-half drives, led to a 34-30 Detroit win. This game is the only one on our list where the lowest odds of winning came in the third quarter.
5. New England Patriots vs. Atlanta Falcons, Super Bowl LI 2016. Comeback odds: 1 in 142
This game is known as the 28-3 comeback, but New England’s lowest odds of winning actually came with 8:31 left in the game. The Falcons had the ball and a 28-12 lead when Dont’a Hightower forced a fumble and a sack, which led to a New England touchdown and two-point conversion. Another sack and holding penalty ended another Falcons drive. The Patriots tied the game with another touchdown and two-point conversion and scored a touchdown on the first overtime possession to win 34-28.
6. Oakland Raiders vs. Cleveland Browns, Week 4 2018. Comeback odds: 1 in 141
With 1:41 remaining, the Raiders trailed the Browns by eight when they turned the ball over on downs. Oakland forced a three-and-out and got the ball back with 1:28 left. The Raiders drove downfield for a touchdown and two-point conversion to force overtime, where they won on a 29-yard field goal.
7. Seattle Seahawks vs. Green Bay Packers, NFC Championship 2014. Comeback odds: 1 in 136
At the 4:57 mark of the fourth quarter, the Packers had the ball and a 19-7 lead. After Seattle forced a three-and-out, a Russell Wilson touchdown cut the lead to 19-14 with 2:09 left. The Seahawks recovered an onside kick and drove down the field for a touchdown and a 22-19 lead with 1:19 remaining. The Packers kicked a field goal to force overtime, but Wilson connected with Jermaine Kearse on a 35-yard game-winning touchdown pass, which sent Seattle to Super Bowl XLIX.
8. Indianapolis Colts vs. Detroit Lions, Week 13 2012. Comeback odds: 1 in 99
With 4:24 remaining in the fourth quarter, the Colts trailed the Lions by 12 points. A Detroit drive stalled at midfield and Indianapolis forced a punt. An Andrew Luck touchdown pass made it 33-28 Lions with 2:39 left. Indy’s defense held and they got the ball back at their own 25 with 1:07 left and no timeouts. Luck led a game-winning drive that culminated with a touchdown pass to Donnie Avery as time expired.
9. Kansas City Chiefs vs. San Diego Chargers, Week 1 2016. Comeback odds: 1 in 89
With 12:53 remaining in the fourth quarter and the Chiefs trailing 27-10, Kansas City quarterback Alex Smith threw an interception. A missed Chargers field goal, and three straight scoring drives from Kansas City mixed with two failed San Diego drives led to the Chiefs tying the game with a minute left. Kansas City won the game in overtime.
10. Denver Broncos vs. Miami Dolphins, Week 7 2011. Comeback odds: 1 in 80
With 5:50 left in the fourth quarter and the Broncos trailing the Dolphins 15-0, Denver punted the ball to Miami. After forcing a three-and-out, Tim Tebow led the Broncos on an eight-play touchdown drive to cut the deficit to 15-7. Denver recovered the onside kick and Tebow orchestrated a game-winning drive, culminating with a Broncos touchdown with 17 seconds remaining. Denver won in overtime.
When are teams being more aggressive on fourth down?
More than ever, NFL offenses are staying on the field on fourth down. In the 2019 season, the rates with which teams have gone for it on both fourth-and-1 and fourth-and-2 plays are higher than in any season during the last two decades.
To answer the question of when teams are going for it — even after knowing the distance needed for a first down — we need additional information. Where on the field is the team? What is the score? How much time is left?
Using statistical modeling, we estimated the likelihood that a team would go for it on fourth down — given distance, score, yard line, and time remaining — for each of the last 15 regular seasons. Here’s a plot that compares the likelihood that a team will go for it in 2019, relative to the average from the 2005—2018 seasons, shown across the length of the field (left to right). The plot assumes that the game is tied at the start of the second quarter, when the increase in fourth-down aggressiveness has picked up the most. Each line corresponds to a different distance category.
The uptick in fourth-and-1 aggressiveness has primarily happened in two scenarios — when an offense is around midfield and once an offense has fewer than 10 yards to go for a touchdown. Increases in all other go-for-it rates are primarily centered around the opponent’s 40-yard line. As an example, teams at the opposing 40-yard line when facing a fourth-and-2 are going for it 25.1% more often in 2019 relative to past seasons.
As additional anecdotes, in 2019 here are the seven fourth-down attempts where a team would’ve gone for it less than five percent of the time.
Baltimore successfully went for it on fourth-and-3 at the Kansas City 9-yard line during Week 3. The game was tied with five minutes left in the first quarter. Go for it probability: 2.9%
Minnesota successfully went for it on fourth-and-4 at its own 31-yard line during Week 4 against Chicago. The Vikings were trailing by 16 with two minutes left in the third quarter. Go for it probability: 3.1%
New England unsuccessfully went for it on fourth-and-7 at the Kansas City 27-yard line during Week 14. The Patriots were trailing by 10 with seven minutes left in the second quarter. Go for it probability: 3.1%
Los Angeles Chargers unsuccessfully went for it on fourth-and-goal from the Minnesota 15-yard line during Week 15. The Chargers were down 15 points at the start of the fourth quarter. Go for it probability: 3.3%
Chicago unsuccessfully went for it on fourth-and-9 at the Los Angeles Rams' 31-yard line during Week 11. The game was tied with nine minutes left in the first quarter. Go for it probability: 3.8%
Baltimore successfully went for it on fourth-and-1 at its own 29-yard line during Week 15 against the New York Jets. The Ravens were up 21 points with two minutes left in the third quarter. Go for it probability: 4.1%
Buffalo unsuccessfully went for it on fourth-and-10 at the Philadelphia 29-yard line during Week 8. The Bills were down 11 points with one minute left in the third quarter. Go for it probability: 4.9%
The most conservative decision according to our model? During Week 1, the Pittsburgh Steelers chose to kick a field goal on a fourth-and-goal from the New England one-yard line, when trailing by 20 points. Our model estimates that a typical team would have gone for it 95.2% of the time.
Teams Taking More Time Off Play Clock in 10-minute OTs
After receiving the ball to start overtime Week 14 against the New York Giants, the Philadelphia Eagles drove down the field and scored a touchdown. The drive was quick — Philadelphia took only eight plays to go 75 yards — however, the Eagles used almost as much of the 10-minute overtime as they could.
Philadelphia’s eight plays took nearly five minutes off the game clock, with the Eagles snapping the ball with seven, nine, five, two, three, one, and five seconds left on the play clock on their last seven plays, all of which took place with a running clock. Over the past decade, fewer than one in 35 touchdown drives of similar length and number of plays have taken so much time off the clock.
Although just one example, Philadelphia’s lack of urgency to start overtime reflects an interesting league-wide trend that seems to have appeared over the last three seasons — since the NFL switched from a 15-minute to a 10-minute overtime period, when teams get the ball to start overtime, they take more time off the play clock.
Here’s a beeswarm plot that shows the amount of time left on the play clock for every regular-season, first-possession overtime play, comparing the 15-minute overtime (2010 – 2016) to the 10-minute session (2017 – Week 14 of 2019). Only plays with a running clock are shown.
With the shorter OT, teams are snapping the ball an average of 2.2 seconds later in the play clock. In the plot, the red dots are slightly shifted to the left relative to the blue dots. For example, 32.5% of plays started with a running clock are started with less than five seconds on the play clock, compared to 17.5% of plays played with a 15-minute overtime.
There are multiple reasons for snapping the ball later in the play clock, but offenses seem to be recognizing that if they take more time off the clock and don’t score a touchdown, they leave their opponent less time to tie or win the game. Interestingly, a related trend has appeared on the very first play of overtime — teams are passing less often. In games played with 15-minute overtimes, the first play of the first possession was a run 38.2% of the time (34 of 89). In games with 10-minute OTs, the first play has been a run 56.8% of the time (21 of 37). This uptick in run plays comes despite the league as whole passing more often on first down in other game situations.
Win Probability Models for Every NFL Team in 2019
In Football Operations, one of our primary roles as a data and analytics team is to supply the league with metrics that help us better understand the game. One framework that we often use is win probability.
Models to estimate win probability have been around football for more than a decade, with several researchers (including Brian Burke, Trey Causey, and a trio of statisticians from Carnegie Mellon) having developed versions of their own. These models help assess each team’s chance of winning at any given point in a game.
Models use familiar inputs including score, down, distance, and field position, and also more subtle variables, including which team kicked off to start the game and the number of timeouts each team has left. During the 2019 offseason, win probability models were used by the National Football League’s Competition Committee to assess which penalty types have the biggest impact on game outcomes. In addition to penalties, we can use win probability formulas to derive metrics of game competitiveness and excitement.
The following chart highlight’s each team’s win probability curve through every game of the 2019 season (shown in gray), as well as an average win probability curve (shown for each team in its primary team color).
To find the teams that have played in the most exciting games, look for the grey curves with the biggest swings. Detroit, Indianapolis, Seattle and Tampa Bay have each averaged more than seven exciting plays per game — defined as those with at least a 10% swing in win probability — and their curves reflect that many games aren’t decided until the fourth quarter, when big swings in win probability tend to happen. Alternatively, most New England, Washington and Baltimore contests have been decided by the start of the fourth quarter and each of those teams averages fewer than four exciting plays per game.
We can also split into different time segments. On a per-quarter basis, no one has been better than Baltimore in the first quarter. The Ravens have typically started the second quarter with a 68% win probability, which implies that they earned an average of +18% in win probability per-game in the first quarter alone. Kansas City has dominated the second quarter (+17% per-game), Chicago has controlled the third quarter (+14%), and Green Bay has closed out games in the fourth quarter (+15%).
Which were the most exciting games of the 2019 season?
The Week 10 (link) Arizona and Tampa Bay game and the Week 5 (link) Rams at Seahawks game are tied as the most back and forth games, with the Week 14 thriller between San Francisco and New Orleans (link) right behind. And although it wasn’t as high scoring as those previous contests across the entirety of the game, Chicago’s win in Week 2 over Denver featured a flurry of changes near the end. In the final two minutes, there were five plays — a Denver fourth down conversion, Denver touchdown, Denver two-point conversion, Chicago completion into field goal territory, and a Chicago game-winning field goal — that swung each team’s chance of winning the game by at least 25%; no game to date has had more than three such plays.
Where’s the best spot on the field to complete passes… and not get picked off?
In analytics circles, the middle of the field is generally considered to be the best place to throw if you want your receiver to catch the football. But a quarterback’s job goes way beyond completing passes. A QB also has to read the defense, avoid pressure and sacks, prevent interceptions and pick up first downs.
Turns out, not only is the middle of the field the best place for a receiver to catch a pass, it’s also the best place for an opponent to intercept a ball, too.
The following plots look at the likelihood of a completion (left plot) and an interception (right plot) based on the pass distance (in yards) and direction (left, middle, right). Regular season games from 2010 through Week 13 of 2019 are included (bin sizes with smaller samples are dropped).
In both plots, lighter colored areas correspond to more completions or more interceptions. Both middle columns feature lighter colors when compared to the columns on the left and right. That implies that throwing down the middle yields higher rates of completions, but it also yields higher rates of interceptions.
More specifically, a pass thrown 20 yards down the middle of the field has about the same completion percentage as one thrown 10 yards towards either sideline. But those same passes thrown down the middle of the field are also about four times as likely to be picked off. Alternatively, nearly one in five passes are picked off when the ball is thrown 30 yards down the middle of the field, which is about three times the rate of interceptions on the sideline.
As more years of tracking data are observed, we’ll be able to more precisely identify the best parts of the field to complete passes (Ex: Where your favorite team likes to throw the ball), as opposed to relying on play-by-play categorizations. For now, know that when a pass is thrown to the middle of the field, it’s less likely to be incomplete.
Frank Gore Keeps on Running
Over the last two decades, the median age of running backs has dropped two full years, from 26 years, 7 months (in 2000) to 24 years, 7 months (2019).
Meanwhile, Frank Gore, at the age of 36, just keeps on running.
Check out the following chart, which highlights the age distribution of ball carriers in each season with at least 150 carries (or 100, in 2019). Player ages are shown as of September 1 in each season. While the curves have moved slowly left over time, the trend of getting younger holds for all but one player — Gore.
Initially drafted in 2005 by the San Francisco 49ers, Gore was the third youngest running back in his rookie season, when he logged 127 carries for 608 yards. He’s topped 125 carries in each season since. In contrast, of the 20 running backs that were likewise drafted in 2005 with Gore, only one — Darren Sproles — recorded a single rush after the 2014 season.
Over the last four seasons, Gore has finished as the oldest back with at least 125 carries. It’s no surprise that with Gore being so productive for so long, his place on the NFL’s all-time rushing list is secure; this past weekend, Gore passed Barry Sanders for No. 3 on the NFL’s career rushing leaderboard.
Aaron Rodgers Takes Advantage of Free Plays Better Than Anyone
Aaron Rodgers is the NFL’s career leader in passer rating, a two-time MVP and seven-time Pro Bowler. But there’s one thing he’s even better at than passing the ball — taking advantage of the “free play.”
As the quarterback is calling signals, an early jump by a defensive player is a five-yard defensive offside penalty. If the center snaps the ball before that defensive player is a threat, the offense can still run a play. This instance is called a "free play" because even under the worst-case scenario — say, an interception — an offense can still go back and take the initial five-yard penalty.
Since 2006, Rodgers leads all quarterbacks with 84 passes thrown on free plays. And no one has attempted more deep free shots than Rodgers, either. Check out the plot below, which highlights the air yards each quarterback has thrown for after drawing an offside penalty from the defense.
Each of the 170 quarterbacks with at least one free play in his career is shown in the plot with the grey lines, with six standout quarterbacks highlighted.
Rodgers stands alone. Beginning in 2010, he has thrown for approximately 225 air yards a season on offside calls alone. More precisely, roughly once every other game Rodgers earned a free shot downfield, averaging 25 air yards on these throws. No quarterback is within 900 yards of Rodgers in total on this stat since 2006.
Rodgers owns the top three individual seasons in air yards on free plays, peaking in 2015, when he attempted 16 passes for a total of 407 yards downfield (25.4 air yards per free play). Of those 16, seven were complete, including two touchdown passes to James Jones.
“Free play — Rodgers has made a living on this,” said ESPN’s then-Monday Night Football announcer Mike Tirico when calling one of Jones’ touchdown grabs in 2015 (video below). “Draw the offside, know it’s a free play, take the shot downfield.”
While Rodgers has owned air yards on free plays over the last decade, one young quarterback has started on a blazing path — Kansas City Chiefs signal caller Patrick Mahomes. In 2018, Mahomes tossed 15 passes after getting a free play, completing nine, and throwing the ball a total of 234 yards downfield.
Which quarterbacks have been the least aggressive off of an offside call?
Current Minnesota Vikings quarterback Kirk Cousins has thrown just 13 career passes off a free play, which traveled a total distance of 87 yards downfield (6.7 air yards per free play). Deshaun Watson (six passes, 56 air yards, or 9.3 air yards per free play), Kurt Warner (12 passes, 48 air yards, 4.0 air yards per free play) and even Peyton Manning (17 passes, 121 air yards, 7.1 air yards per free play) stand out as the opposites of Rodgers.
How Do NFL Coaches Use Their Challenges?
After an unsuccessful challenge of an incomplete pass on Thursday Night Football against the Los Angeles Chargers, Oakland Raiders head coach Jon Gruden fell to 0-for-7 on challenges in the 2019 regular season. But Gruden is not the only one with limited success — through Week 10, 13 of the league’s 32 head coaches have yet to win a coaches’ challenge this season.
With that in mind, here’s a chart showing how frequently coaches throw a challenge flag (x-axis) versus how successful they are on those challenges (y-axis). The data shows all games for each coach dating to 2004, the first year in which the league allowed up to three successful challenges per team.
In the top right of the graph, Green Bay Packers head coach Matt LaFleur has been the most frequent/successful challenger (winning four of seven challenges), though that has come with only ten games worth of data. More robustly, John Harbaugh, Sean Payton, Bruce Arians, Pete Carroll and Doug Marrone stand out as coaches with multiple seasons under their belts who frequently, but relatively accurately, use coaches’ challenges.
Alternatively, among coaches with at least half a challenge per game, only Freddie Kitchens of the Cleveland Browns has yielded a lower success rate than Gruden. And in the top left, Jason Garrett of the Dallas Cowboys boasts a low challenge frequency (0.29 per game), but when he has challenged, he has successfully overturned the decision on the field more than 50 percent of the time (23-for-45).
Among non-active coaches, and using games after 2004, count former Seattle coach Mike Holmgren as one of the most passive challengers (0.28 per game), with Mike Shanahan of the Denver Broncos (0.52 per game) on the more aggressive side. Meanwhile, Jim Caldwell (Indianapolis Colts) won 63 percent of his challenges, compared to 27 percent for Norv Turner.
Visualizing the Special Teams Gunner
NFL special teams players may not be household names, but they play a valuable role in their team’s success. One key special teams player is the punt-team “gunner” – a player who lines up near the sideline, with the goal of moving downfield as fast as possible to eventually tackle the punt returner. The main challenge of being a gunner – there’s usually at least one defender in your way.
"You're trying to beat two guys in what amounts to a running street fight at a sprint,” said Steve Tasker, to Esquire Magazine. Tasker would know – as a seven-time Pro Bowler and five-time first-team All-Pro, he’s considered one of the best gunners in NFL history.
Though it’s difficult to highlight Tasker’s effectiveness today without the help of video, we can use Next Gen Stats to compare and contrast how today’s gunners operate.
For example, we took the player from each team who has played the gunner role most often for his team in 2019 (Weeks 1 to 7). Below is each player's path, standardized so that they are starting at the same line of scrimmage (moving from the bottom of the image to the top), and using only punts in which each gunner was defended by one player only. Players are arranged in order by who covered the most vertical distance on their coverages in the first 4.5 seconds after the punt, with Justin Bethel (formerly with the Baltimore Ravens, now with the New England Patriots) traveling the furthest in the top left, and George Odom (Indianapolis Colts) traveling the shortest in the bottom right.
One aspect that stands out is how while some gunners only line up on certain sides (Ameer Abdullah on the Minnesota Vikings favors the offense’s left side, while the Cleveland Browns’ KhaDarel Hodge lines up on the right, for example), others like New England’s Matthew Slater are equally likely to line up on either side. In the chart, the typical gunner appears on about 25 punt plays.
Although tough to determine in the chart, the effectiveness of certain gunners in moving downfield is interesting. Players in the top row, for example, are about 28 yards downfield, roughly six yards further ahead than players in the bottom row.
Finally, with Bethel recently signing with the Patriots, New England now boasts two of the league’s fastest gunners (Slater is a seven-time Pro Bowler).
Other interesting gunner tidbits?
Matthew Slater, Josh Bellamy (New York Jets) and Brandon Wilson (Cincinnati Bengals) each have been double teamed by defenders 15 times, leading the league.
Against single defenders, no one travels quicker in the play's first two seconds than Johnny Holton Jr. of the Pittsburgh Steelers. At that point, Holton is an average of 7.7 yards downfield, typically about 0.4 yards in front of the next fastest gunner at that point (Slater).
When facing two defenders, Dee Virgin (Detroit Lions) and Trenton Cannon (New York Jets) are the only two gunners to average being at least 20 yards downfield at the 4.5-second mark after a snap.
The Miami Dolphins lead the league in the rate with which they double team at least one gunner, doing so on 62 percent of punts. Alternatively, the New Orleans Saints put two defenders on a gunner on only 20 percent of opponent punts.
What Numbers Tell Us About Preseason Play Time
One aspect of football that has recently been a point of discussion is the role of preseason games. As of the 1978 season, teams have almost always played four preseason games in each season.
To assess how preseason games may have changed over the past decade or so, our Football Operations data and analytics group analyzed team behavior dealing with the playtime of players who started for their teams during Week 1 of the regular season. When looking at this group of eventual starters, how often did they play in each of the four preseason weeks?
The following chart shows the percent of possible snaps for Week 1 starters in each week of the preseason, as well as Week 1 of the regular season. Averaged across all positions, starters typically play about 80-85 percent of the available snaps in a regular season game, a rate which has mostly stayed consistent over time. This is important, as it suggests that any drop in playtime is not due to coaches using player rotations more often.
Alternatively, playtime in preseason games for Week 1 starters has dropped substantially over the last decade. In the figure, the lines in blue have been trending downwards for several years, reaching all-time lows for playtime in 2019.
Some facts that stood out:
Among preseason Week 3 games (abbreviated as Pre W3 in the chart), Week 1 starters played nearly 50 percent of available snaps in 2010. In 2019, Week 1 starters played 23.6 percent of available snaps.
Week 1 starters only played 1.7 percent of available snaps in preseason Week 4 games in 2019.
The year-over-year drop in preseason participation of Week 1 starters was larger this past year (2019 vs. 2018) than in any previous period over the last decade.
In 2019, there were 85 regular season Week 1 starters that did not play a single snap during the preseason. From 2008 to 2018, the league averaged 24 such players per season.
Altogether, snap time percentages among Week 1 starters dropped to new lows in 2019, information that is valuable as the league assesses the role of preseason games in the context of the larger framework of NFL scheduling.
Where Do Ball Carriers Tend to Move?
Where do ball carriers tend to move?
The primary theme of the NFL’s second Big Data Bowl, a competition that crowdsources analytical insight into football data, is to predict performance on rush plays. From the moment a ball carrier takes a handoff, where will he end up?
As part of the contest, participants receive snapshots of Next Gen Stats player tracking data from every 2017 and 2018 handoff play. This data includes the speed, angle and direction in which each of the 22 players on the field were moving. One potential input for competitors to consider using in their predictions is the angle in which the ball carrier was moving when he got the ball. As an example, players moving towards the sideline may simultaneously be more likely to be stuffed for a loss and to pick up big yardage (if, for example, he were to outrun defenders).
One cool plot that can be made from the data (see the statistical code for yourself) is a sonar chart. Initially popularized in soccer by Eliot McKinley, sonars allow us to identify the angles that players tend to move. With ball carriers, the size of each sonar band corresponds to the frequency with which ball carriers are moving in each binned direction group, and the color reflects how successful they are with the ball. Success rate is determined by down and distance (e.g., a three-yard run play on first-and-10 is not a success, while it is a success on third-and-two).
Here’s a sonar for each player who led his team in carries during the 2017 and 2018 regular seasons. Bands are pointing up, corresponding to players moving from the bottom of the screen to the top, and consist of data from each the past two seasons.
Compare the sonars of Gus Edwards (Baltimore Ravens) and Alvin Kamara (New Orleans Saints). Both boast success rates around 60%, but Edwards did most of his damage up the middle, while Kamara tended to get the ball off-tackle. Alternatively, as you’ll see near the bottom right of the chart, then-Atlanta Falcons running back Tevin Coleman (outside runs) and New England Patriots back Sony Michel (up the middle) also offer a nice contrast.
Information provided in sonars is just one of the many features available for Big Data Bowl contestants to use as they develop their algorithms. Stay tuned for the end of the 2019 regular season, where a live leaderboard will help determine the most accurate predictor of ball carrier success — and the Big Data Bowl title that comes with it.
A Conversation with Two Big Data Bowl Finalists
The 2020 Big Data Bowl is here! And to help mark the occasion, we’re flashing back to the inaugural event to chat with two finalists from the 2019 contest – Nathan Sterken and Adam Vonder Haar. Both contestants earned a trip to the NFL Scouting Combine to present their reports, with Nathan taking home the open entry grand prize and Adam being a finalist.
In addition to their success at the Big Data Bowl, they both also earned new jobs in football analytics following the competition – Adam with the Dallas Cowboys, and Nathan with Telemetry. We spoke with them about their reports, memories from presenting at the Combine and their new roles.
Michael Lopez (Director of Football Data and Analytics, NFL): Tell us a bit about your background in analytics and football.
Nathan: I've always loved football. My playing career peaked in fourth grade so my football background is as a fan. I grew up near Ann Arbor so the University of Michigan is my favorite team. I've been doing analytics professionally for over 10 years in a few industries — from analyzing employment data at the Labor Department to polling and TV data with political campaigns and online behavior with technology companies.
Adam: My interest in analytics and football began about five years ago, around the same time DraftKings and FanDuel were becoming more popular as an alternative to traditional redraft fantasy leagues. I had always enjoyed playing fantasy football with my friends growing up, but it wasn't until I got interested in doing daily fantasy more seriously that I forced myself to learn to code because the copying and pasting into Excel wasn't cutting it anymore. At some point, what was initially a means to an end in coding became what I enjoyed the most, and the analysis became more interesting to me than my results in fantasy football.
Michael: Why did you enter last year’s Big Data Bowl?
Nathan: Lots of reasons. I'd wanted to get my hands on the NFL player tracking data ever since reading about it; I was interested by the research prompt I answered (identify the best receiver routes), since I’d never seen any large-scale analysis of that topic; and since the finals were at the Combine, I figured this was my best shot at getting to run the 40 (and to meet people working in this space).
Adam: The biggest reason I entered last year’s Big Data Bowl was pure curiosity about the data and excitement about the process of working with this new data. Working with tracking data is difficult, and if you don’t enjoy the process at least a little bit, it’s easy to give up on it. I also saw it somewhat as a challenge to myself and as a learning opportunity to improve my skill set. Lastly, there was definitely some motivation around potentially going to the Combine and getting my work in front of teams, but both of those felt like such a long shot to me that I couldn’t rely on them for my main motivation.
Michael: Give us your one paragraph elevator pitch — what was your entry about last year?
Nathan: I demonstrated that you can automate some of the classification work that is typically done by hand and then analyze plays at scale — specifically, I trained a computer to recognize receiver routes using the player tracking data and then computed efficiency metrics for route combinations. The finding: throw it deep (or at least threaten to).
Adam: My entry last year was centered around exploring several potential techniques to discover how an offense can get receivers more open. I used several different methods to classify receiver routes, looked at how different combinations of routes result in greater separation for receivers, and used convex hulls to characterize defender spacing.
Michael: What is one memory of the event that stands out as you look back?
Nathan: Seeing head coaches in the hallways was fun, but the stand-out moment was discussing my research with the analytics staff from several clubs and the other folks who attended the event. I made connections with several clubs and got to meet a couple of the authors I cited in my paper.
Adam: One of my favorite memories of last year’s event was meeting the many brilliant, like-minded individuals who also submitted entries. Being able to talk football and football analytics specifically in-depth, listening to novel ideas everyone had, and, in general, just hanging out with each other at the Combine made the experience especially memorable.
Michael: Next Gen Stats can be tricky. What would your recommendation be for a first-timer dealing with NFL player tracking data?
Nathan: Football coaches have the best advice here: take it one play at a time. If you start by focusing on one play (preferably one where you can find video for it so you know exactly what happened) and doing some simple plots and statistics, you can get your arms around the format of the data without worrying about the size of the overall data set. That said, you’ll want to do that exploration in a programming language like Python or R because you’ll need to be able to quickly scale up your analysis to crunch through each play.
Adam: As tempting as it can be to dive into analysis immediately, I think it’s important to take your time cleaning and understanding the data as much as possible first. The tracking data is not without anomalies and errors occasionally, and even the good data often must be modified so that everything is oriented and normalized consistently. Spending the effort to prepare the data thoroughly makes the analysis much easier when you get to that point.
Michael: Tell us a bit about your new roles with Telemetry and the Dallas Cowboys, respectively. How have you been able to help?
Nathan: Telemetry helps clubs with game preparation and player analysis by making it quick and easy to find video of any type of play by any player. Specifically, they use the player tracking data to create a rich set of annotations on each play and then pair that with a searchable video database. I’ve helped them by building modeling pipelines that use the player tracking data to identify additional aspects of the play, like the type of coverage that was played by the defense.
Adam: I am now a Football Research Analyst with the Dallas Cowboys. Our team’s responsibilities range from scouting and personnel, to assisting coaches with game planning and opponent scouting, to sports science and player health. I spend a lot of time working with the tracking data specifically, trying to discover ways we can use it to make our processes more efficient and improve our ability to make evidence-based decisions.
What Can Player Tracking Data Tell Us About the Onside Kick?
Throughout NFL history, it wasn’t uncommon for kickoff teams to line up like they were kicking deep, only to shock both fans and opponents with an onside attempt. Famously, Thomas Morstead of the New Orleans Saints surprised the Indianapolis Colts at the start of the second half of Super Bowl XLIV with an onside kick (one that the Saints recovered). In total, from the start of the 2010 season through the 2017 season, the league averaged 9.4 surprise onside attempts per season.
In 2018, however, there were only five surprise attempts. Through the first four weeks of 2019, we’ve only seen one.
Prior to the 2018 season, NFL Player Health and Safety and special teams coaches worked together to reimagine the kickoff play — one designed to increase player safety — resulting in a multitude of changes regarding player alignment and allowable blocking types. Concussions on kickoff plays dropped by 35 percent in 2018, but the formation redesign potentially made it more difficult to recover onside attempts. In addition to the drop in surprise onside kicks, the kickoff team has only recovered 6.1 percent of non-surprise attempts since the start of 2018, a rate lower than the league’s historical average of 12 percent. In 2019, teams are 0-for-9 at onside kick attempts through Week 4 (not including recoveries that were nullified by penalties).
During last year’s offseason meetings, the NFL’s Competition Committee debated the merits of a follow-up rule change, proposed by the Denver Broncos, that would have given teams the opportunity to use one scrimmage play per game in place of an onside kick. Although the proposal was not adopted, our football data and analytics crew was able to provide the Committee with insight into how and why onside kicks have changed.
Check out the below animation, which overlays Next Gen Stats tracking data on onside attempts for the kickoff teams and the football, averaged over the course of each of the 2017 (pre-kickoff changes) and 2018 seasons (post-changes). Dots in red correspond to players in 2017, while the blue dots represent players in 2018. Only onside attempts to the kickoff team’s right side are included.
Three differences stand out.
First, there are six red dots on the right side, compared to five blue dots. As part of the kickoff rule changes, five players must now line up on each side of the kicker.
Second, players on the kickoff team may no longer get a running start, which is why the blue dots (2018 season) begin the play standing still.
Finally, and perhaps most importantly, without the running start, players from the 2018 season are no longer able to get down the field as quickly, giving them less of an advantage over the receiving team.
Altogether, each of the above findings provide an explanation for why we are seeing fewer surprise onside kicks and lower recovery rates. Although Denver’s proposal for an alternative did not pass last year, the NFL will continue to use analytical insight to drive potential rules changes.
Q&A with 2019 Big Data Bowl College Winners
The first-ever Big Data Bowl brought together talented members of the analytics community – from college students to professionals – to crowdsource innovative and data-driven football ideas. For today’s Extra Point, we interviewed the college entry winners from Simon Fraser University – Matthew Reyers, Dani Chu, Lucas Wu and James Thomson – to hear more about their experiences and findings. The group’s first-place report, Routes to Success, modeled play success rate and expected points under various passing combinations.
Michael Lopez (Director of Football Data and Analytics, NFL): Tell us a bit about your background in analytics and football.
James: I’ve played football since high school, first as a defensive tackle and back, later in rec league flag football in college. It’s been my favorite game to play since I was 14. I got interested in sports statistics during my undergraduate degree in statistics, when I started following baseball more closely. I quickly turned to football, and since the beginning of my master’s in statistics, I have been active in football statistical research.
Dani: I’ve played sports my whole life, but my introduction to football was mainly through fantasy football. While I have played football in a casual setting, I have not played in an organized environment. I started statistical research in sports in 2016. I have a background in math, computer science and statistics.
Matt: In elementary school, I thought I was going to be the next NFL star. It took all of five games to realize that wasn’t the case. I played for a few more games after that but stuck with other sports. When I eventually got to University, I stumbled across statistics and a group of people that liked sports as much as I did. Since that class four years ago, I have been applying statistics to as many sports problems as I can get my hands on, regardless of the sport.
Michael: Why did you enter last year’s Big Data Bowl?
Dani: I was excited to work with the NFL’s tracking data. I found the question that was asked technically challenging and important for football teams to understand. I thought it would be a great way to challenge us and demonstrate our skills.
James: The opportunity to work with data more detailed than typical play-by-play data I was used to was too good to pass up.
Matt: Tracking data offered a new and exciting field of study for me. I had the chance to view football in a way not many others had been able to and to possibly unearth insights for the first time. I felt myself to be on the cutting edge of analytics and found that to be one of the big drivers of joining this competition.
Lucas: Player tracking data provides us a great opportunity to analyze sports in a way more granular fashion. We’re able to answer a lot of interesting questions and to extract meaningful insights from the tracking data. This is a great venue for us to bring our analytics/data-science skills into sports and showcase the potential of sports analytics.
Michael: Give us your one paragraph elevator pitch – what was your entry about?
Dani:Our project could be broken down into three sections: Route Identification, Route Combination Evaluation and Quantification of Openness/Field Control. Route Identification is useful for a variety of reasons, including tagging game film, building novel receiver metrics and for evaluating route combinations. Route Combination Evaluation could be used to help inform new plays or determine the efficiency of current plays. Finally, the quantification of field control can be used to help evaluate both the design and execution of both running and passing plays.
Michael: What is one memory of the event that stands out as you look back?
Dani: It’s hard to choose one. The dinner the night before with the other finalists was a great way to get to know everyone. The walkthrough of the presentation rooms beforehand was great. To see our names on the banner was surreal. Seeing the crowd and knowing most of the NFL teams were represented was awesome. Finally, walking by [Dallas Cowboys head coach] Jason Garrett in the hallway was very cool.
James: Standing in the presentation room looking around at all of these amazing people who we were competing with was surreal. Everyone's work was so inspired. There are so many good ideas and avenues to explore this data that I had never thought of until I got there.
Matt: If I had to focus on one standout feature, it would have to be the meet-and-greet poster conference after the presentations. Being from a smaller school and Canadian, I rarely have the chance to talk with NFL representatives and affiliates about the inner workings of the game. What I learned from my conversations in that brief hour-long session has fueled many new research questions going forward.
Lucas: It was awesome to be selected as one of the finalists and fly to Indianapolis to present our ideas to NFL personnel. The conversation and the feedback we got from them were valuable for us to develop our ideas further down the road.
Michael: NextGenStats can be tricky. What would your recommendation be for a first-timer dealing with NFL player tracking data?
Dani: My recommendations would be to:
Start small. Build out your work for one play. Write a function for that one thing and then generalize it to all plays.
Get rid of unnecessary data as soon as possible. Drop columns you aren’t using and rows you don’t need.
Build small functions to do useful things and give yourself a toolbox/package of functions to work with.
James: Make a function for every different data transformation you do. Keep them very general so that no matter where you are with your code it will work. It’s a complicated data set, but it’s simple to manage with smaller steps.
Matt: Visualize frequently. We had sample code for creating GIFs of plays that proved invaluable. When we were working, we chose a few plays that we could also find on YouTube and built our intuition about the data and its real-life counterpart. Tracking data is nearly uninterpretable in table formats — I feel it needs to be seen to be understood.
Lucas: Start with the most intuitive things. For instance, visualize the movement of players and ball, then move on to distance travelled, speed and acceleration, etc. It’s also a good idea to do pair-programming, where you always have someone looking after you.
Michael: Tell us a bit about your summer job working with the NFL.
Dani: Working for the NFL this summer was unreal. We got to work on technically challenging material and are helping inform change in the NFL. It is great to see an investment from the league in data-driven changes.
James: I never thought that I would find myself working for the NFL, let alone while I was still a student. It was great to talk about how our work could be used to help make decisions based on player movement in the NFL.
Matt: The NFL gave us a wonderful opportunity to expand our ideas even further after the Big Data Bowl this summer through a project with them. We had the opportunity to explore more positions than just eligible receivers, describing movement patterns across the game. Our work felt important to the game and the NFL, truly putting us at the cutting edge of analytics. I have nothing but positives to say of this wonderful experience.
Lucas: It’s incredible to see our Big Data Bowl submission leading to our opportunity working with the NFL. We are able to explore the movement patterns of different positions. It’s truly exciting to further develop our statistical methods, learn how to leverage cloud computing and deliver our analytics results to the NFL.
Where Your Favorite Team Likes to Pass the Ball
When an NFL quarterback drops back to pass, where is he likely to throw the football?
For decades, scouts would scour film looking for insight into answering this question. Left-handed quarterbacks would presumably favor the left side of the field, whereas other passers were seen as preferring the deep ball. Most recently, Carolina Panthers quarterback Cam Newton came under questioning for not throwing downfield at all.
With the advancement of NFL Next Gen Stats, which tracks every player – on every play, anywhere on the field – at a rate of roughly ten frames per second, scouts are no longer the only football personnel who can measure pass locations.
The following two charts use Next Gen Stats data to compare each team’s passing locations from the 2018 season to the overall league average. For technical followers, two-dimensional kernal density estimation was used to smooth the data.
First, here is each AFC team. In the top left, we see that the Buffalo Bills’ chart features several deep throws, perhaps not surprising given that Buffalo quarterback Josh Allen can chuck it 75 yards. Alternatively, the Oakland Raiders and their quarterback Derek Carr tended to focus on shorter throws near the sideline (see bottom right).
WHERE AFC TEAMS THROW
Next, here is each NFC team.
In the bottom right of the below graphic, the Seattle Seahawks, led by Russell Wilson, stand out as a team that, instead of throwing deep or short, tended to mostly throw to the left side of the field. Alternatively, in the top-right, the Tampa Bay Buccaneers, with both Jameis Winston and Ryan Fitzpatrick behind center, featured a greater rate of intermediate throws (5-20 yards) than any other team in the league.
WHERE NFC TEAMS THROW
While pass location for each team is one interesting way of using player tracking data, the league also uses it to inform other aspects of the game, including rule changes and the position of officials. For now, as your favorite team steps back to pass, know that wherever the ball lands, the NFL is tracking it.
Using Data and Analytics to Identify High-Impact Fouls
As the NFL began its offseason, several club rules proposals sought to expand the use of replay to include penalties. In response, the NFL Competition Committee used data and analytics to identify fouls that have the biggest impact on game outcomes.
To assess impact, the committee used win probability, also known as game winning chance. Below is one example. The black line (New Orleans Saints) and gold line (Pittsburgh Steelers) in the animation show the game winning chance for both teams at each point in the game during their 2018 Week 16 matchup.
The game was back-and-forth. New Orleans led at halftime, only to see Pittsburgh regain the lead late in the third quarter. With 1:25 remaining in the game, a Michael Thomas touchdown reception led New Orleans to a 31-28 win. Amidst all this action, two of the seven most impactful plays — as judged by the change in game winning chance from one play to the next — were defensive pass interference calls.
First, at the end of the first quarter, this call on Pittsburgh’s Joe Haden increased New Orleans’ game winning chance by 14 percent, from 43 to 57 percent.
Then, with two minutes left, this defensive pass interference foul by Haden on Thomas increased New Orleans’ win chances by 21 percent.
In reviewing game winning chance data across several seasons, the committee found that both defensive and offensive pass interference tended to have disproportionately large impacts on changes to each team’s game winning chance.
Armed with this and other information, the Competition Committee prioritized these two fouls as it looked to expand replay to include penalties. Under the new rules, both calls on Haden would be reviewable, as would potential offensive and defensive pass interference calls that were not made on the field.
NFL Football Operations’ mission is to establish a culture of clarity, consistency and credibility in all aspects of the greatest game. Football is an essential piece of America’s fabric, uniting fans, players and communities with a simple yet powerful bond. We honor the game’s history while continuing to innovate and preserve the sport for future generations of fans, players, coaches, teams and officials.