Football Analytics with Python R

1. 1Name the outcome you are evaluating (completion, passing yards, rushing yards, EPA) before touching data. Tukey's point applies: the right, vague question beats a precise answer to the wrong one.
2. 2List the situational variables the passages identify as constraining that outcome: air yards (distance from the line of scrimmage to the intended receiver), pass length (short vs. long), down, and distance (yards needed for a first down).
3. 3Split the sample by the biggest constraint and compare distributions, not just means. In the book's play-by-play data, filter pass_length_air_yards to 'short' versus 'long' and run describe() in Python or summary() in R on passing_yards and epa.
4. 4Read spread as well as center. Check the quartiles, the interquartile range, the standard deviation, and the floor. Long passes have a much wider interquartile range, and their minimum passing yards sit higher because a pass that travels 20+ yards in the air almost never loses yards.
5. 5Read the situational definition of success where it applies. Success is relative to the yards needed: a 7-yard catch on 8 needed is not a first down, and a 5-yard gain on 5 needed is. Use EPA, which uses situational factors to assign a point value to each play, when yards mislead.
6. 6Build the baseline expectation with a model. Use linear regression for rushing yards (adding down and distance in multiple regression) and logistic regression for completion probability, given multiple situational factors.
7. 7Compute the residual: actual minus expected (RYOE, CPOE). This isolates what the player did beyond the situation.
8. 8Test whether the residual is more stable year to year than the raw statistic. Only keep the adjustment if it adds stability.

Compare short and long passes in the book's play-by-play data. Short passes: mean EPA 0.1196, standard deviation 1.43, 25th percentile -0.61, 75th percentile 0.96. Long passes: mean EPA 0.3826, standard deviation 2.19, 25th percentile -0.83, 75th percentile 2.14. Long passes carry a higher mean EPA but a much wider spread. The long-pass median EPA is -0.465, so the typical long attempt loses expected points while a few big hits lift the mean. Passing yards show the same pattern: the long-pass mean is 12.17, the median is 0, the 75th percentile is 26, and the maximum is 98. A quarterback who throws deep therefore faces a different baseline and more variance than one who throws short. Judging both on raw yards per attempt or completion percentage mixes skill with the situation. Modeling the situation first (as CPOE does) and then examining the residual separates the two.

• If two players' opportunities differ on a measurable variable (air yards, down, distance), then compare them against a modeled expectation, not raw totals.
• If the outcome's spread differs across situations (long vs. short passes), then compare distributions and don't rely on the mean alone.
• If a yardage total ignores the yards needed, then switch to a context-aware measure like EPA or a first-down criterion.
• If the situation-adjusted residual (CPOE, RYOE) is not more stable than the raw metric, then don't treat it as a better skill measure.
• If a contextual variable can't be measured with current data, then discard it rather than pretend to control for it.

• Crediting a player for yards the situation supplied, as when the second runner gets all the credit for a 5-yard score after a 65-yard first play. The tell: credit follows the final yards, not the expected-points change.
• Comparing raw completion percentage across quarterbacks with different air-yard profiles. The tell: percentages have risen even among non-elite passers, so the number no longer separates skill.
• Reading a higher mean as better without checking variance. The tell: long-pass EPA mean is higher but the median is negative.
• Controlling for too few variables, such as distance without down. The tell: residuals still correlate with the situation variables you left out.
• Assuming the adjusted metric is better without testing stability. The tell: the residual fluctuates year to year as much as the raw number.

• ▸Outcome metric: name the stat being judged (completion, yards, EPA).
• ▸Player/sample: who and which plays, with the filter used.
• ▸Situation variables: list air yards, pass length, down, distance, and others you can measure.
• ▸Excluded variables: list context you can't measure and have discarded.
• ▸Split results: mean, median, IQR, and standard deviation for each situation bucket.
• ▸Model type: linear (rushing yards) or logistic (completion), plus the variables included.
• ▸Expected value: the model's predicted outcome for the player's plays.
• ▸Residual: actual minus expected (CPOE or RYOE).
• ▸Stability check: compare year-to-year correlation of the residual with that of the raw metric.

1. 1Fix the unit of observation first. Actual performance is one row per play from play-by-play data (the book uses nflfastR in R or nfl_data_py in Python). Name the outcome you are recording: yards gained, pass completed, touchdown scored. Do not mix outcomes in one column.
2. 2Record the raw output untouched. Keep yards gained, the completion flag, and the touchdown flag exactly as logged. Do not adjust, cap, or smooth them. This column is the 'actual' side of every later comparison.
3. 3Attach the situation to every play without altering the output. At minimum keep down and ydstogo (yards left to a first down or score) for rushes, and air yards (distance from the line of scrimmage to the intended receiver) for passes. Raw output is uninterpretable without the conditions it was produced under.
4. 4Aggregate by player and report the sample alongside the total. Sum or average the raw output (for example, yards per carry or yards per attempt) and always print the play count next to it. A rate with no denominator hides how little evidence sits behind it.
5. 5Split passing output by type before you read it. Chapter 2 separates deep passes from short passes to partition a quarterback's production into stable and unstable parts. A single pooled passing number mixes the two.
6. 6Treat the raw number as the numerator of a residual, not as a verdict. RYOE is actual rushing yards minus the model's expected yards. CPOE is actual completion outcome minus modeled completion probability. Compute the actual number cleanly, then compare it with what the situation predicted.
7. 7Test whether the raw figure persists. Compare a player's actual output in one year with the next. The book's regression-to-the-mean example is a back with above-average yards per carry one year, who is statistically expected to drift back toward the league average. If the number does not persist, do not forecast with it.
8. 8Report actual and residual side by side. State what happened (actual), what the situation predicted (expected), and the gap. The actual number alone cannot separate player skill from play context.

Take Adrian Peterson's 2012 season, which the source describes as over 2,000 rushing yards while carrying an otherwise-bad Minnesota Vikings team to the playoffs. That is a clean actual-performance fact: a large season total of rushing yards, and it earned him the last nonquarterback MVP. Stopping there invites the conclusion that the back made the offense. The book's approach asks for more. Rushing is less efficient than passing on yards per play and EPA, and swapping one running back for another does not change rushing production the way swapping quarterbacks, receivers, or linemen does. So you keep the 2,000-plus yards as the actual number, then ask what a model using situation (such as ydstogo in the simple version, tracking data in the Big Data Bowl version) expected on those same carries. The gap, RYOE, is the part attributable to the rusher. You then check whether that gap holds from one season to the next, since an above-average yards-per-carry year is expected to regress. The same logic sits behind Saquon Barkley going second overall in 2018 and drawing criticism from analytics people. The raw production was visible; whether it was player-driven was the open question the book sets out to test in Chapters 3 and 4. The source does not give Peterson's expected-yards figure, so the comparison step stays a procedure, not a number.

• If you are about to rank players on a raw total (yards, completions, touchdowns), then first record the play count and the situation fields, and only then rank.
• If the output is a passing rate, then split it into deep and short passes before reading it. Treat the two parts as separate evidence.
• If a player's raw rate is well above or below league average in a single year, then expect regression toward the average and do not project the full gap forward without a year-over-year check.
• If you want to credit or blame the player rather than the situation, then subtract a modeled expectation (RYOE for rushing, CPOE for passing) and judge the residual, keeping the raw number as context.
• If a metric does not help predict what you care about year after year, then do not build decisions on it, even if it looks decisive in high-leverage moments like third-down conversions.

• Treating a big raw total as proof of player impact. The tell: you cite a season yardage figure (like 2,000-plus rushing yards) with no expected value, no opponent or situation context, and no position-level efficiency comparison.
• Comparing raw outputs across different situations. The tell: a runner with mostly short-yardage carries is set against one with mostly long-yardage carries, and ydstogo never appears in the analysis.
• Pooling deep and short passes into one rate. The tell: a quarterback's yards per attempt moves a lot between seasons and you cannot say which part of the throw mix is moving.
• Projecting a one-year rate forward as if it were a fixed trait. The tell: your forecast keeps the full above-average yards per carry and never shows a regression-to-the-league-average adjustment.
• Over-weighting high-leverage results. The tell: a player's third-down conversions drive the story, but the figure does not repeat from year to year.

• ▸Player and season: name the player and the season or play set being measured.
• ▸Outcome type: state whether the raw output is yards gained, a completion, or a touchdown. Pick one.
• ▸Raw total and rate: enter the untouched actual figure, such as yards per carry or yards per attempt.
• ▸Play count: enter the number of plays behind the figure so the sample size is visible.
• ▸Situation fields captured: list the context kept for each play (down, ydstogo, air yards).
• ▸Pass type split: for passers, record deep versus short separately. Mark N/A for rushers.
• ▸Expected value source: name the model or method that produced the expectation (for example, an RYOE or CPOE model) and its output.
• ▸Residual: actual minus expected (RYOE or CPOE). Note whether it is above or below zero.
• ▸Prior/next-year check: record the same figure for an adjacent season and note whether it regressed toward the league average.

1. 1Define the outcome and the unit of play. Pick one measurable result per play (rushing yards, pass completion, points, draft-pick outcome) and filter to the plays that count. The source's passing example keeps play_type == 'pass' and drops plays where air_yards is NA, so every row has a recorded depth.
2. 2List the situational features that change what an average player produces. The source names down, distance (yards needed for a first down), and air yards for passes. Choose these before fitting, and tie each one to a reason it changes the baseline.
3. 3Match the model to the outcome type. A continuous outcome like rushing yards uses linear regression, simple first and then multiple. A binary outcome like completed or not uses logistic regression (a generalized linear model), which returns a completion probability per pass.
4. 4Fit on the full play-by-play sample, not on one player. The baseline must reflect what the average player does in that situation, so the target player's own results shouldn't define it.
5. 5Compute the residual for every play: observed minus the model's expected value. For completions that is the actual result minus the predicted probability; for rushes it is actual yards minus predicted yards.
6. 6Aggregate residuals by player (mean residual per play, or per pass). The aggregate is the metric: RYOE for rushing, CPOE for passing. Report the sample size beside it.
7. 7Test whether the residual metric is stable. Compare each player's value in one season to the next, and compare that to the correlation for the raw stat (completion percentage, yards per carry). The source runs this test explicitly: 'Is CPOE More Stable Than Completion Percentage?' and 'Is RYOE a Better Metric?'.
8. 8Act on the result. If the residual metric carries over from year to year better than the raw stat, use it to evaluate and predict. If it doesn't, treat it as description rather than prediction, and consider adding or revising features.

The source's motivating problem is that raw yards ignore context. A receiver gains 7 yards when 8 are needed for a first down or touchdown, and no first down results. Another player gains 5 yards when 5 are needed, and the drive continues. Raw yardage ranks the 7-yard gain higher, but the 5-yard gain did more for the offense. A model that conditions on distance to go captures this. The book builds that logic in three steps. Chapter 3 fits a simple linear regression for rushing yards. Chapter 4 adds down and distance to make RYOE. Chapter 5 fits a logistic regression for completion probability from situational factors such as air yards (the distance from the line of scrimmage to the intended receiver). For a pass, suppose the model gives a 0.65 completion probability (an illustrative number, not from the book). A completion yields a residual of +0.35 and an incompletion yields -0.65. Averaged over a quarterback's passes, this is CPOE. The final step is the stability test: check whether a quarterback's CPOE in one season predicts the next better than his plain completion percentage does. The authors call CPOE one of the most predictive quarterback metrics available.

• If the outcome is binary (complete/incomplete), use logistic regression and read the residual against a probability. If it is continuous (yards), use linear regression and read the residual in yards.
• If a context variable plainly changes the average outcome (down, distance, air yards), put it in the baseline. Otherwise the player gets credit or blame for the situation.
• If the residual metric is not more stable year to year than the raw stat, don't use it to project future performance. Use it only to describe what happened.
• If a player has few plays, don't rank him. The source filters aDOT to quarterbacks with 100 or more passes, so apply a minimum-volume cutoff of the same kind to any residual leaderboard.
• If the result comes from high-leverage situations such as third-down conversions, treat it as probably unstable unless the stability test shows it repeats.

• Ranking on raw yards or completion percentage and calling it skill. The tell is that the leader simply had easier situations, such as short distances or shallow targets.
• Using a baseline that omits key context. The tell is that residuals correlate with down, distance, or air yards, meaning the model hasn't absorbed the situation.
• Assuming a residual metric is automatically better. The source treats this as a question to test (Is RYOE a Better Metric? Is CPOE more stable?). The tell is that no year-to-year comparison was run.
• Ranking on thin samples. The tell is a leaderboard dominated by players with very few plays, such as those under the source's 100-pass cutoff.
• Treating outsized results in high-leverage spots as repeatable skill. The tell is a big win-loss effect that vanishes the next season.

• ▸Outcome measured: name the single per-play result (e.g., rushing yards, completion).
• ▸Outcome type: continuous or binary; this decides linear vs. logistic regression.
• ▸Play filter: state the rows kept (e.g., play_type == 'pass', air_yards not NA).
• ▸Context features: list each situational variable (down, distance, air yards) and why it shifts the baseline.
• ▸Model used: simple linear, multiple linear, or logistic.
• ▸Minimum plays per player: set the volume cutoff for reporting.
• ▸Player residual (mean observed minus expected): record the aggregated value for each player.
• ▸Year-to-year stability of the residual metric: record the correlation between consecutive seasons.
• ▸Year-to-year stability of the raw stat: record the same correlation for the unadjusted stat for comparison.

1. 1Write the decision the metric serves in one line, for example 'which quarterbacks to buy next season'. Stability only matters relative to a forward-looking question, so name the outcome you want to predict before you touch the data.
2. 2Pull play-by-play data (nflfastR in R or nfl_data_py in Python) and filter to the plays the metric actually describes. Examples are rushing plays for rushing yards or pass attempts for yards per attempt. Record the filter so the season-to-season comparison is like for like.
3. 3Aggregate the metric to the player-season level. Then line up each player's season N value next to his season N+1 value. Keep only players who appear in both seasons, and note the sample size.
4. 4Measure how well season N predicts season N+1 across all players. Use a scatterplot and a correlation or regression slope. A tight relationship means stable, repeatable production. A cloud means noise.
5. 5Partition the metric into subsets and run the same test on each. The book's example splits passing into deep versus short passes and asks which subset is more stable year to year. Treat the stable subset as the signal and the unstable one as the noisier source of year-to-year swings.
6. 6If a model-adjusted version exists, run the identical stability test on it and on the raw version. The book does this for rushing yards over expected (RYOE) against raw rushing yards, and for CPOE against completion percentage. Compute the adjusted value as observed minus expected (the residual), and compare the two stability results head to head.
7. 7Check outliers from the plot. The book frames EDA as a way to see whether any players break the mold, and then asks what to do with them. Decide whether each outlier is a regression candidate or a genuine exception.
8. 8Write the verdict. A stable metric is one to buy into year after year. An unstable metric, even one that moves win-loss results, should be ignored or sold if it does not help predict what you want.

The source's anchor case is baseball. Walks, strikeouts and home runs, the 'three true outcomes', were the easiest to predict from one season to the next. What batted balls did once in play was noisier and drove most of the perceived year-to-year variance. The lesson is to buy the repeatable outcomes and discount the noisy ones. For football, the RYOE model works the same way. It fits rushing_yards ~ 1 + ydstogo, with an intercept of 3.2188 and a ydstogo slope of 0.1329, on about 92,425 rushing plays. A rush with 2 yards to go is expected to gain about 3.2188 + 0.1329×2 ≈ 3.5 yards (the book rounds the coefficients and writes 3.4). A back who gains 6 on that play has a residual of about +2.5. Summing or averaging those residuals by player gives RYOE. The model's R² is only 0.007, so this expectation is weak and the residual is mostly the raw yards minus a near-constant. That is why the test matters. You then compare each back's RYOE in season N with season N+1, and do the same with raw yards per carry. Whichever correlates better across seasons is the more stable metric. The passages pose this comparison but do not report the resulting stability numbers, so you must run it.

• If a metric's season-N value tracks its season-N+1 value closely, then treat it as true-talent evidence and price it into forward-looking decisions.
• If a result strongly affects win-loss or playoffs (high-leverage outcomes like third-down conversions) but does not repeat year to year, then ignore it or sell it, because it does not help predict the future.
• If an adjusted metric (RYOE, CPOE) and its raw counterpart both exist, then pick the one with the better year-to-year repeatability, and do not assume the adjusted one wins.
• If one subset of a stat (deep passes versus short passes) is far less stable than the other, then separate them and lean on the stable subset to judge talent.
• If a player's current number sits far from his stable baseline, then flag him as a regression candidate rather than extrapolating.

• Confusing impact with predictiveness. You defend a metric because it drove wins, yet it flips from year to year. The tell is that the justification is about past results and never mentions repeatability.
• Assuming residual or 'over expected' metrics are inherently more stable. The tell is no side-by-side stability test against the raw stat. The book treats this as a question to answer ('Is RYOE a Better Metric?', 'Is CPOE More Stable...?').
• Pooling heterogeneous plays into one stat. The tell is a stability result that is mediocre overall but strong in a subset, as with deep versus short passes.
• Treating a weak expectation model as informative. The tell is a low R² (0.007 in the simple ydstogo model), which means the residual is mostly raw yards and adds little context.
• Judging stability from one season pair or a thin sample. The tell is a conclusion resting on a handful of players or a single year-over-year comparison.

• ▸Metric and decision served: name the metric and the forward-looking decision it informs.
• ▸Play filter and data source: state which plays are included and the package used (nflfastR or nfl_data_py).
• ▸Seasons compared (N → N+1): list the season pairs tested.
• ▸Player sample size: count the players present in both seasons.
• ▸Year-over-year relationship: record the correlation or slope between season N and N+1.
• ▸Subset split result: give stability for each subset (e.g., deep vs. short) and name the more stable one.
• ▸Raw vs. adjusted comparison: record the stability of the raw metric and the model-adjusted version (observed minus expected).
• ▸Outliers / regression candidates: list the players who break the pattern and the action for each.
• ▸Verdict (buy / ignore / sell): state the one-line conclusion and its reason.

1. 1Start with the decision, not the data. Write the question the evaluation must answer (for example, 'Is this player's production repeatable in another scheme?'). Tukey's rule applies: an approximate answer to the right question beats an exact answer to the wrong one.
2. 2Split the player's production into stable and unstable components. Run stability analysis: compare each metric in the current season to the same metric in the prior season for the same players. Metrics that repeat are signal; metrics that don't are noise.
3. 3Test any new or adjusted metric against its raw predecessor before you trust it. Compare RYOE per carry to yards per carry, or CPOE to completion percentage, using the same year-over-year stability check. If the adjusted metric is not more stable, the context carries the signal and you are crediting the individual too much.
4. 4Strip out volume when you test efficiency. Use per-play rates with a minimum sample (the passage uses RYOE per carry for players with 50 or more carries) rather than totals, because totals embed both volume and efficiency and muddy the result.
5. 5Adjust for context before you judge the player. Control for situational factors (air yards for passers, expected yards for rushers) and evaluate the residual: actual performance minus the model's expectation for that situation.
6. 6Read volume as a coaching signal and expect efficiency to fall as volume rises. A player who plays a lot is probably good, but heavy usage lets defenses key on him. Check usage, draft status and contract before you read a backup's high efficiency as a talent gap.
7. 7Label each player buy-low or sell-high. Buy-low: strong on stable components, weak on unstable ones. Sell-high: the reverse. Discard high-leverage results (such as third-down conversion rate) that do not predict next year.
8. 8Judge front-office drafting against draft capital, not raw hits. Compare production to what the pick slot predicts, build a 95% confidence interval around the team's average difference, and call a team better or worse only if the interval excludes 0.

The drafting-skill test shows both the method and the humility it demands. Instead of asking whether a team's picks were good, the analysis measures DrAV_OE, a team's draft value over what its pick slots would predict, so a team that always picks late is not penalized. It builds a 95% confidence interval around each team's DrAV_OE. If the interval contains 0, the team is not distinguishable from average. If DrAV_OE sits above the interval, the team did statistically better than average. If it sits below, worse. Only four teams cleared the bar: the Steelers and Packers were significantly better, and the Raiders and Browns significantly worse. Most front offices cannot be separated from average. That matches the research that over a typical GM or coach tenure it is very hard to discern drafting talent. The Raiders' 2018 Khalil Mack trade shows the cost of a poor selection process: two first-round picks came back, but the best pick of the bunch was Josh Jacobs. More picks give more chances, but the hit rate still depends on evaluation accuracy.

• If a metric's year-over-year correlation is no better than the raw stat it replaces, then treat the adjustment as unproven and do not pay for it.
• If a player is strong on stable metrics but weak on unstable ones, then treat him as a buy-low. If the reverse, then treat him as a sell-high.
• If an outcome is high-leverage but does not predict next season (such as third-down conversion), then discount it in acquisition decisions and consider selling it to other decision-makers.
• If a team's 95% interval for DrAV_OE contains 0, then do not credit or blame its front office. Act only when the interval excludes 0.
• If you cannot separate drafting skill over a GM-length window, then favor acquiring more picks (more bites at the apple) over betting on your read of any single pick.

• Judging a front office on raw draft results without adjusting for draft capital. The tell: 'they always pick late' is offered as an excuse, or early picks are praised without comparing to slot expectations.
• Reading small samples as skill or its absence. The tell: a verdict (good drafter, bad drafter) rests on a few seasons and no interval, even though most teams cannot be separated from average.
• Evaluating on volume-contaminated totals. The tell: total rushing yards or total RYOE is used to rank players, so playing time drives the ranking.
• Taking a backup's high yards per carry as proof he is better than the starter. The tell: the efficiency gap is explained by usage; defenses key on the starter, and draft status and contract also matter.
• Trusting a new metric because the reasoning behind it is sound. The tell: no stability test was run, and the argument is that the adjustments are well-founded.
• Rewarding or penalizing a decision by its outcome. The tell: an award or a jeer arrives before the picks play out (the Raiders won a conference award for the Mack trade, yet drafted poorly with the picks).

• ▸Decision question: state the exact acquisition or roster decision this evaluation serves.
• ▸Player and role: name, position, and the situation his production came from (scheme, usage, supporting cast).
• ▸Metric tested: the metric and its raw predecessor (for example, RYOE per carry vs. yards per carry).
• ▸Sample filter: minimum plays required (such as 50+ carries) and confirmation that volume totals are excluded.
• ▸Year-over-year stability result: current-year vs. prior-year relationship for each metric; mark which is more stable.
• ▸Context adjustments applied: situational factors controlled for (such as air yards) and the residual used.
• ▸Usage and efficiency check: playing time, draft status, contract, and whether defenses could key on him.
• ▸Stable vs. unstable profile: strong or weak on stable components, strong or weak on unstable ones.
• ▸Verdict: buy-low, sell-high, or hold, with the one reason that would change it.