Financial Data Analytics with Machine Learning, Optimization and Statistics with Machine Learning, Optimization and Statistics

1. 1Define the risk quantity before choosing a method. Write down whether you need a quantile (V@R calibration), a tail measure (ES), a dependence structure between instruments (credit or joint risk), or an anomaly screen on reported numbers. The passages treat these as separate tasks with separate tools.
2. 2Build returns from prices the way the source does: take the logarithm of the ratio of consecutive prices, tag each row with its year, and keep periods separate. The source analyses 2005–2007 and 2018–2020 separately rather than pooled.
3. 3Test whether the periods you plan to pool share a distribution. Run one-way ANOVA across the series or periods, then Tukey HSD on the pairs, and treat the Bonferroni bound as the upper bound on the p-value for simultaneous pairwise testing. If the means differ, do not pool the data into one risk estimate.
4. 4Fit the dynamics model, then test its residuals. Standardize the returns by the fitted running mean (instantaneous drift) and standard deviation (instantaneous volatility), and run the Ljung–Box test at lag 21 for each index and each year. Also check for autocorrelation up to lag 10. A p-value above 0.05 means you do not reject independence of the residuals.
5. 5Estimate the quantile numerically from the validated model. The source states that estimating quantiles with numerical methods is essential for risk assessment and V@R calibration. Where the likelihood is intractable, use the MLE-based algorithms it introduces, such as EM and MCMC sampling from complex distributions.
6. 6For multi-instrument or credit risk, model the dependence explicitly with a copula, which describes the joint behaviour of several risk factors. Record that dependence models can fail under changing market conditions and unforeseen events, as in the LTCM blow-up (1998) and the 2008 credit crunch.
7. 7Run anomaly screens on reported financial data with Benford's Law and Zipf's Law. Treat a deviation as a flag for auditors and investigators to follow up, not as proof of fraud.
8. 8Compare candidate models on the same footing. For GLMs under the same distributional family, compare deviances, which reduces to comparing likelihoods. Log which models passed and failed, and re-run the diagnostics whenever the data window changes.

The source fits a stochastic diffusion model to the S&P500, FTSE 100, and Hang Seng Index, with an estimated drift and volatility for each year. To check reliability, it standardizes the returns and applies the Ljung–Box test at lag 21, year by year. The p-values all exceed 0.05 except for two S&P500 years (2003, 2007), three FTSE 100 years (2001, 2003, 2006), and one HSI year (2008). In those six index-years the independence of residuals is rejected, so a risk figure for those years carries a documented caveat. In all the other years the standardized returns show no autocorrelation up to lag 10, so the drift and volatility estimates are proper running estimates. A separate check shows why pooling is risky. For 2005–2007 the ANOVA p-value is large and Tukey HSD shows no difference in mean log returns across the three indices. For 2018–2020 ANOVA rejects equal means, HSI vs SPX has a small Tukey p-value, FTSE vs HSI does not reject equality, and FTSE vs SPX has p = 0.1226088, which leans toward unequal. A V@R built on one window therefore does not transfer automatically to another.

• If the Ljung–Box p-value is 0.05 or below for a given index-year, then do not treat that year's volatility estimate as validated. Refit or flag the estimate before computing a quantile from it.
• If ANOVA and Tukey HSD show unequal means between periods or markets, then estimate risk per period or market rather than from pooled data.
• If you rely on a copula or other dependence structure, then state the market conditions it was fitted under, because LTCM (1998) and 2008 show these strategies can fail under changed conditions or unforeseen events.
• If a Benford's Law or Zipf's Law screen flags a deviation, then escalate it for audit follow-up and do not conclude fraud from the statistic alone.
• If you compare candidate GLMs from the same distributional family, then pick on deviance, since the saturated model is the same and the comparison reduces to likelihoods.

• Pooling regimes: one V@R is fitted across periods whose means differ, as 2005–2007 and 2018–2020 do. The tell is a significant ANOVA or Tukey result between the periods that nobody checked.
• Skipping residual diagnostics: a model is used for quantiles without testing the standardized returns. The tell is that no Ljung–Box output exists, or the failed years (such as 2003 and 2007 for the S&P500) go unreported.
• Treating a dependence model as permanent: a copula-based result is trusted outside the conditions it was fitted under. The tell is that no stress-period check, such as LTCM or 2008, is documented.
• Treating an anomaly flag as a verdict: a Benford or Zipf deviation is reported as fraud. The tell is that no investigator or auditor follow-up is recorded.
• Using ordinary regression where the response is binary or otherwise non-normal: the source's example is the blue-chip HSI indicator, which calls for a GLM with a suitable link. The tell is fitted values outside the valid range of the response.

• ▸Risk quantity: V@R, ES, credit dependence, or anomaly screen; state which one.
• ▸Data window and source: the dates and series used, e.g. the index and years.
• ▸Regime check result: the ANOVA and Tukey HSD p-values across the periods or markets, and the pool/don't-pool decision.
• ▸Model and estimation method: the fitted model and how it was estimated (MLE, EM, MCMC, numerical quantile).
• ▸Residual diagnostic: the Ljung–Box lag used, the p-values by year, and the years at or below 0.05.
• ▸Flagged periods: the index-years or segments where the diagnostics failed and what you did about them.
• ▸Dependence assumptions: the copula or joint model used and the market conditions it was fitted under.
• ▸Anomaly screen: Benford or Zipf result, the deviations flagged, and the person assigned to follow up.
• ▸Model comparison: the candidate models, their deviances or likelihoods, and the one chosen.

1. 1Write the decision rule as a single sentence before you look at any returns. Example: 'Buy when the model's fitted probability is ≥ 0.5; otherwise hold.' The source's logistic regression turns fitted probabilities into labels with exactly this 0.5 cutoff (pr=True if fitted value ≥ 0.5). Fix the cutoff in advance. Do not tune it against the outcome you will later report.
2. 2Separate model quality from strategy quality, and score them in two columns. Model quality is accuracy, recall, F-score, or a confusion matrix. Strategy quality is portfolio return, risk-adjusted performance such as the Sharpe Ratio, or the profitability of the trading rule. A model can classify well and still fail as a strategy, so never report the first column as evidence for the second.
3. 3Build the confusion matrix and read the cells separately. In the source's HSI example (680 stocks, 2002), 2 constituent stocks were misclassified as pr=False and 14 non-constituents as pr=True. Ask what each error type costs you in the strategy. A missed opportunity and a bad position are different losses, and one headline accuracy figure hides that difference.
4. 4Check the inputs for outliers before trusting the fit. The source notes the logistic MLE coefficients were extremely large, so fitted values sat near 0 or 1, and it attributes this possibly to outliers. It then suggests building a more robust model with smaller coefficients. Near-0/1 probabilities give you no gradation for sizing positions, so treat them as a warning sign.
5. 5Estimate risk with time-varying volatility, not a constant. The source says volatilities and correlations change rapidly with time. It presents EWMA, ARCH and GARCH as closer to reality than the constant-volatility Black–Scholes–Merton assumption. Use the time-varying estimate in the denominator of any risk-adjusted measure and in any position-sizing step.
6. 6Compare candidate models across repeated resamples, not one split. The source's classifier comparisons (bank churners, default premium data) use boxplots of accuracies, recall rates and F-scores. Take the same approach to the strategy metric: show its spread across resamples, and prefer the rule whose worst cases you can tolerate.
7. 7Stress the strategy against regime change. The source's statistical-arbitrage warning is that dependence-based strategies can be profitable but are subject to market conditions and unforeseen events, as in the LTCM blow-up (1998) and the 2008 credit crunch. Run the rule over a period that contains a stress event and record the worst outcome, not only the average.
8. 8Decide, then record the verdict in one line: strategy outcome metric, benchmark, risk metric, and a keep / resize / retire call. Re-evaluate on new data only. Never judge the rule on the data used to fit it.

Take the source's HSI logistic regression, which predicts whether a stock is an HSI constituent from 2002 data on 680 stocks. The fitted probabilities are converted to labels at 0.5. The confusion table shows 2 stocks misclassified as pr=False and 14 as pr=True, so 16 errors in 680, or about 97.6% correct (derived arithmetic). That is a model-quality result, and it is in-sample, since the labels come from fitted values on the same data. The coefficients are also extremely large, so the fitted values sit near 0 or 1, which the source links to outliers. Suppose you turned the labels into a trading rule. You would still have to measure its return and Sharpe Ratio on data the model did not see, and size positions with a time-varying volatility estimate such as GARCH. The 14 false positives are the cost to watch, because each would be a position taken without cause. The 97.6% figure alone does not establish strategic decision efficacy. Only the out-of-sample, risk-adjusted outcome of acting on the labels does. LTCM (1998) is the cautionary case. A model-driven strategy that worked under normal conditions failed under an unforeseen event.

• If you only have in-sample accuracy or fitted-value results, then label the strategy 'unvalidated' and do not size it, because model fit is not strategy performance.
• If fitted probabilities cluster near 0 or 1 and coefficients are very large, then check for outliers and refit a more robust, smaller-coefficient model before using the probabilities for sizing.
• If two rules have similar average outcomes, then choose the one with the tighter resampled distribution and the better risk-adjusted measure (e.g., Sharpe Ratio), not the higher mean.
• If the risk estimate assumes constant volatility, then replace it with EWMA/ARCH/GARCH estimates before computing any risk-adjusted metric.
• If the strategy depends on dependence structures (copulas, statistical arbitrage), then require a stress test that includes a crisis-like period (LTCM 1998, 2008) before approving it.

• Reporting classification accuracy as strategy success. The tell: the report cites a correct-classification rate and has no return or Sharpe figure.
• Judging the rule on the data it was fit to. The tell: the confusion matrix is built from fitted.values on the training sample, as in the 680-stock example.
• Treating all errors as equal. The tell: a single accuracy number is quoted, with no split between the 2 and the 14 misclassifications and no cost attached to each.
• Assuming volatility is constant. The tell: risk is computed from one sample variance over the whole history, while the source stresses that volatilities and correlations change rapidly.
• Trusting a model through a regime change. The tell: the backtest has no stress period, and the author calls the strategy 'profitable' without noting its dependence on market conditions, as with LTCM and the 2008 credit crunch.

• ▸Decision rule (one sentence): state the exact trigger and cutoff, fixed before testing.
• ▸Model-quality metrics: record accuracy, recall, F-score or the confusion matrix, and say whether they are in-sample or out-of-sample.
• ▸Error-cost map: list each misclassification type (e.g., false positive, false negative) with its count and its cost to the strategy.
• ▸Volatility/risk estimator: name the method used (EWMA, ARCH, GARCH, or constant) and why.
• ▸Strategy outcome metrics: record return, Sharpe Ratio or other risk-adjusted measure, and rule profitability, on data not used for fitting.
• ▸Resample spread: note the range of the outcome metric across resamples (boxplot min / median / max).
• ▸Stress-period result: name the crisis window tested and the worst outcome observed.
• ▸Outlier / robustness check: record whether coefficients or fitted values look extreme and what was done about it.
• ▸Verdict: keep / resize / retire, with the one reason that decided it.

1. 1Fix the observation grid first. For stock data, use closing prices observed daily and treat a year as about 252 trading days. Compute the daily arithmetic return from consecutive closes. Also compute the logarithm of the price ratio, because the fatter-tailed alternatives in Chapter 3 are fitted to that series.
2. 2Arrange and cleanse the dataset before any test. Align dates across series, and keep one return series per instrument (the source's HSBC and Bitcoin examples are each handled this way). Run diagnostics only on the cleaned return series, never on raw price levels.
3. 3Test the Black–Scholes–Merton premise directly. The model assumes returns on different days are iid normal. Start with graphical checks (histogram, Q-Q plot), then apply a formal normality test such as Shapiro–Wilk. The source states the usual outcome with real securities data: normality is invalid because the distribution is fatter-tailed.
4. 4If normality fails, replace it rather than patch it. Fit a fatter-tailed distribution, Student's t, to the log price ratio. Then test whether the new model is valid, using the same Q-Q and goodness-of-fit logic you used on the normal.
5. 5For several assets, test multivariate normality separately. Passing univariate checks does not establish it. Then inspect the sample correlation matrix as a distinct diagnostic.
6. 6Check for volatility clustering. Plot simple moving standard deviations at two window lengths (the source uses 90-day and 180-day on HSBC returns) and look for periods where dispersion stays high or low. Examine sample ACF and PACF to see what dependence remains.
7. 7If volatility moves, model it and diagnose the residuals. Fit ARCH/GARCH (or EWMA) and run the model diagnostics (Figures 7.13–7.17 use HSBC returns). A Shapiro–Wilk test on fitted residuals is acceptable. Footnote 10 argues the dominant terms in numerator and denominator are the true iid noises, so plugging in residuals is justified.
8. 8When a diagnostic statistic is built from fitted residuals, adjust the reference distribution. Footnote 9 says the asymptotic distribution is chi-square with degrees of freedom reduced by the number of estimated parameters. Using the unadjusted count overstates how well the model fits.

The source's own sequence on HSBC returns runs as follows. Start from daily closing prices and form daily arithmetic returns. Test them against the Black–Scholes–Merton claim that returns are iid normal. Chapter 3 reports that real securities data typically reject this because the tails are fatter, so the normal is replaced by a Student's t on the log price ratio. Next, Chapter 7 computes 90-day and 180-day simple moving standard deviations of the same returns. Time-varying levels in those series are the empirical sign of volatility clustering, and they motivate a GARCH fit. The fitted GARCH is then diagnosed (Figures 7.13–7.14), and the same fitted volatilities are plotted against the 90-day and 180-day moving standard deviations (Figure 7.15). A Shapiro–Wilk test on the standardized residuals checks whether the model has absorbed the structure. Footnote 10 supports using fitted residuals in place of true noises. The passages give no numeric results for this example, so the value of the procedure lies in the order: grid, normality, fat tails, clustering, residual diagnostics.

• If a Q-Q plot or Shapiro–Wilk test rejects normality for returns, then do not use a normal-return model such as Black–Scholes–Merton. Move to a fatter-tailed distribution (Student's t) for the log price ratio and re-test.
• If moving standard deviations at 90 and 180 days differ visibly over time, then treat volatility as time-varying. Use EWMA or ARCH/GARCH rather than one constant sigma.
• If you apply a normality test to GARCH residuals, then use the standardized fitted residuals and use the plug-in form. The source justifies this because the true-noise term dominates.
• If a goodness-of-fit statistic is computed from a model with estimated parameters, then reduce the chi-square degrees of freedom by the number of estimated parameters.
• If you are working with several assets, then test multivariate normality and examine the sample correlation matrix. Do not infer joint behavior from per-asset results.

• Assuming iid normal returns because the model textbook does. The tell: Q-Q plot tails bend away from the line and the normality test rejects.
• Reading one fixed-window volatility as the truth. The tell: 90-day and 180-day moving standard deviations disagree and drift, so a single sigma misstates risk in some periods.
• Declaring a volatility model adequate without residual diagnostics. The tell: no residual plots or tests (Shapiro–Wilk, ACF/PACF) are reported after the fit.
• Using the wrong degrees of freedom in a diagnostic test after estimating parameters. The tell: p-values look comfortably high on a heavily parameterized model because the chi-square reference was not reduced.
• Testing normality on price levels or on unaligned, uncleaned data. The tell: nonsensical results driven by trends, date mismatches, or gaps rather than by return behavior.

• ▸Instrument and frequency: name the series and the observation interval (e.g., HSBC daily close; Bitcoin minute price).
• ▸Cleaning notes: record date alignment, missing values, and anything removed before computing returns.
• ▸Return definition: state whether you use arithmetic return, log price ratio, or both.
• ▸Normality result: record the graphical check (histogram/Q-Q) and the formal test with its p-value, and write pass or fail.
• ▸Tail model chosen: write normal or Student's t, and the post-fit check you ran.
• ▸Moving-std windows: list the windows used (e.g., 90 and 180 days) and note whether volatility visibly clusters.
• ▸Dependence check: summarize the sample ACF/PACF reading and the model implied (ARIMA, EWMA, GARCH).
• ▸Residual diagnostics: record the test on fitted residuals, the degrees of freedom used (minus estimated parameters), and the verdict.
• ▸Multi-asset check: record the multivariate normality result and any notes on the sample correlation matrix.

1. 1Write the problem as one sentence naming the output type. The book's own map sorts problems this way: volatility and risk (Ch. 7–8), dimension reduction (Ch. 9), numeric response (Ch. 10 regression), labelled classes (Ch. 10–13), unlabelled grouping (Ch. 14), and flexible function fitting (Ch. 15). Pick the family from this sentence before looking at any algorithm.
2. 2Run the diagnostics first (Ch. 3). Test normality of relative price changes, compare against Student's t, test multivariate normality, and inspect the sample correlation matrix. Record which empirical properties of the stock prices hold. These results decide whether a Gaussian-based model is acceptable or whether you need a heavy-tailed or extreme-value approach (Ch. 8.5).
3. 3Match the model to the diagnosed feature. Time-varying volatility goes to EWMA, ARIMA, then ARCH/GARCH (Ch. 7). Many correlated rates or factors go to PCA (Ch. 9). Tail risk goes to VaR/ES with extreme value theory or historical simulation, plus copulae for dependence (Ch. 8). Labelled outcomes go to GLMs, logistic or Poisson regression, SVM, Bayes classifiers, trees or ANN. No labels means k-means. Start with the simplest member of the matched family.
4. 4Escalate in order: linear (Part II), then nonlinear (Part III). Move from a linear model, perceptron or SVM to Bayesian classifiers, trees and forests, and finally ANN/LSTM only when the simpler model's diagnostics show it is inadequate. Write down which diagnostic triggered each escalation.
5. 5Specify before fitting. Fix the structure: ARIMA/GARCH orders, polynomial degree, number of PCs, tree depth or terminal nodes, k in k-means or k-NN, network layers and learning rate. Fix the loss or impurity measure (entropy, Gini index, misclassification error). Fix the fitting method too. The notes choose Nelder–Mead in R's constrOptim() because it is the only option that needs no gradient function, so specification includes what your optimiser requires.
6. 6Match the evaluation metric to the problem. For classification, read the confusion matrix, not accuracy alone. The book's rare-disease table exists because a rare class makes accuracy misleading. For time-series models, check the residual ACF. For trees, control overfitting through the number of terminal nodes or pruning.
7. 7Adjust the test to the model. For GARCH residuals, the chi-square test on residual autocorrelations uses degrees of freedom reduced by the number of estimated parameters (4 in the notes' example), as with ARIMA diagnostics. Using the unadjusted test is a misspecification of the diagnostic itself.
8. 8Compare against a baseline on the same data and split, and keep the model whose added complexity is justified by a measurable gain. Record the choice, the rejected alternatives and the reason.

Take a credit-default problem like the book's Taiwan credit default dataset and the 15-sample default table in Ch. 13 (Table 13.1), where each sample carries a Y/N label. The output is a class, so the family is classification. First run the Ch. 3 diagnostics on the attributes and check class balance. The book's bank-churner and default-premium class distributions (Fig. 12.6) show why this matters, since recall and F-score can diverge from accuracy. Start with the linear options (logistic regression, then SVM) and score them with a confusion matrix. Then try a classification tree. Its split rule is information gain, and for a continuous attribute such as Taxable Income you choose a threshold from candidate cut-points (Table 13.3). The tree can overfit, so limit terminal nodes or prune, and consider a random forest. Compare the tree and forest with a Bayes-type classifier such as CIBer on accuracy, recall and F-score. The passages do not report the winning model, so the example shows the order of decisions, not a result: diagnose, match the family, escalate only on evidence, and judge by metrics suited to the class balance.

• If the diagnostics reject normality and show fat tails, then use Student's t, an extreme-value approach or historical simulation rather than a Gaussian assumption.
• If volatility clusters or varies over time, then move from a constant-variance model to EWMA, then ARIMA, then ARCH/GARCH, and test GARCH residuals with the parameter-adjusted chi-square.
• If you have many highly correlated inputs such as a yield curve, then reduce dimension with PCA (or use principal component regression) before fitting the downstream model.
• If labels exist and the classes are rare or imbalanced, then select by the confusion matrix, recall and F-score, not accuracy.
• If labels do not exist, then use k-means for grouping. If you have identified classes and need to label a new observation, then use k-NN. Reserve deep networks for when simpler models' diagnostics fail.

• Choosing the model before diagnosing the data. The tell is a Gaussian-based model fitted to returns whose normality test or tail behaviour was never checked.
• Judging a classifier by accuracy on a rare-event problem. The tell is high accuracy with a confusion matrix that misses most of the rare class.
• Letting a tree grow without control. The tell is a deep tree that fits the training data very well but has many terminal nodes and poor out-of-sample results. The book flags overfitting in classification trees (13.6) for this reason.
• Using the wrong reference distribution for residual tests. The tell is applying a chi-square test to GARCH residuals without reducing degrees of freedom by the number of estimated parameters, which gives misleading p-values.
• Jumping to complex models by default. The tell is a neural network or forest with no linear or simpler baseline and no diagnostic that justified the escalation.

• ▸Problem statement and output type: one sentence stating what is predicted or described (volatility, risk, class, number, grouping).
• ▸Data diagnostics: normality and t-fit results, correlation structure, class balance, and which empirical properties hold.
• ▸Candidate family and chapter basis: the matched family (e.g. GARCH, PCA, SVM, tree, k-means, ANN) and why it fits the diagnosis.
• ▸Baseline model: the simplest alternative you will compare against.
• ▸Escalation trigger: the specific diagnostic failure that justifies moving to a more complex or nonlinear model.
• ▸Specification: orders, degrees, number of PCs or terminal nodes, k, network layers, loss or impurity measure, and optimiser (with its gradient requirement).
• ▸Evaluation metrics: confusion matrix, recall, F-score, residual ACF test (with adjusted degrees of freedom), or backtest.
• ▸Overfitting control: pruning, terminal-node limit, or held-out data plan.
• ▸Decision and rationale: the model kept, the alternatives rejected, and the measured reason.

1. 1Inventory the columns by type before transforming anything. Split them into continuous and categorical, as Chapter 12 does when it reports proportions for the first five continuous features and frequencies for the sixth categorical variable of the training data. Every later choice (bin, encode, or compress) depends on this split.
2. 2Cleanse before you engineer. The book builds its cleansed dataset (d3 in Programme 10.11) using Mahalanobis distances (Figure 10.5) to flag outliers. Its classification trees for the 2002 data (Figure 13.8) are shown without outlier removal, which gives you a with/without comparison. Outliers distort PCA directions, bin edges, and thresholds, so remove or flag them first.
3. 3Check redundancy among continuous features with the sample correlation matrix (Section 3.4). Highly correlated groups are the candidates for PCA. Features that are roughly independent gain little from compression.
4. 4Apply PCA to the continuous block and read the scree plot, as the book does for the Boston housing training dataset in both Python (Figure 10.12) and R (Figure 10.13). Keep the components before the elbow. Fit the PCA on training data only, then project the test data with the same loadings.
5. 5Bin a continuous variable by searching candidate cut points against a purity criterion. Chapter 13 does this with the 'Taxable Income' threshold in the credit-default example (Table 13.3) and compares entropy, Gini index and misclassification error (Figure 13.4). Pick the cut that gives the best split under your chosen criterion, and record the group counts on each side (Table 13.2 style).
6. 6Encode each categorical variable from its training-set frequencies (Table 12.4) and cross-tabulate pairs with a contingency table (Table 12.1). Use these to spot rare levels and to see which categorical pairs carry joint information.
7. 7Transform features when the classes are not linearly separable. Figures 11.8 and 11.9 show a quadratic kernel lifting 2D data into 3D so a linear boundary works. Add the transformed feature only if the separation improves on held-out data.
8. 8Validate the engineered set against the raw set. Use the same model, confusion matrix (Tables 10.1–10.5), ROC curve (Figures 10.9–10.10) and lift chart (Figure 10.7). Keep the feature set only if out-of-sample performance holds or complexity falls at equal performance.

Take the Boston housing training dataset the book uses for its scree plots (Figures 10.12 and 10.13). First split the columns into continuous and categorical and remove outliers flagged by Mahalanobis distance (Figure 10.5 shows this diagnostic). Next compute the sample correlation matrix to see which continuous features move together. Run PCA on the continuous block and read the scree plot: the variance explained drops steeply and then flattens, and you keep the components up to the elbow. This is the step that guards against the singularity problem the book illustrates for logistic regression (Figure 10.14), which arises when predictors are nearly collinear. Then bin or threshold a variable where a split is natural, as with 'Taxable Income' in the 15-sample credit-default table (Tables 13.1 and 13.3), choosing the cut that best separates Class N from Class Y. Finally, fit the same model on raw and engineered features and compare confusion matrices and ROC curves. The excerpts name these tables and figures but do not reproduce their numeric values, so the numbers must come from running the book's programmes.

• If several continuous features are highly correlated in the sample correlation matrix, or a regression such as logistic regression shows singularity symptoms, then apply PCA and keep components up to the scree-plot elbow.
• If a continuous variable relates to the label through a threshold (as 'Taxable Income' does in the credit-default tree), then bin it at the cut that optimises entropy, Gini or misclassification error rather than at arbitrary round numbers.
• If a categorical level appears rarely in the training frequency table, then merge it or treat it separately before encoding, because its estimates will be unstable.
• If classes are not linearly separable in the raw space, then try a kernel-style transformation (quadratic or RBF) and keep it only if test accuracy beats the linear version.
• If outliers are flagged by Mahalanobis distance, then cleanse first and refit the PCA and the bins, since both depend on the sample's spread.

• Fitting PCA or choosing bin edges on the full dataset, including test data. The tell is a test score that looks better than the training-only pipeline can reproduce.
• Running PCA on data with uncleansed outliers. The tell is leading components that point toward a few extreme rows, and loadings that change sharply once those rows are removed.
• Keeping too many or too few components because there is no scree plot or other stated rule. The tell is a model that either still shows collinearity and singularity problems or has lost accuracy compared with the raw features.
• Binning at arbitrary cut points or too finely. The tell is leaves or groups with very few samples, as in a tiny 15-sample table, and unstable thresholds between runs.
• Compressing features and then being unable to explain the model. The tell is that you cannot map components back to financial variables, which conflicts with the book's emphasis on interpretability.

• ▸Dataset and target: name the data and the label (for example, credit default Class N/Y).
• ▸Column inventory: list each feature as continuous or categorical, with its training-set proportion or frequency.
• ▸Outlier handling: state the method (such as Mahalanobis distance), the number of rows flagged, and whether they were removed.
• ▸Redundancy check: list the feature groups with high sample correlation and any singularity symptoms.
• ▸PCA decision: record the number of components kept, the scree-plot elbow, and confirm it was fitted on training data only.
• ▸Binning decision: for each binned variable, record the threshold, the criterion (entropy, Gini or misclassification) and the group counts on each side.
• ▸Categorical encoding: record the method used per variable and how rare levels were treated.
• ▸Raw versus engineered comparison: record the confusion matrix, ROC or lift results for both feature sets on held-out data.
• ▸Keep or revert: state the decision and the one-line reason.

1. 1Name the insight you want before fitting anything, and match it to a model family the book treats: a volatility path (EWMA, ARIMA, ARCH/GARCH, Chapter 7), a dependence structure (Gaussian or t-copula, Chapter 8), latent factors (PCA, Chapter 9), a class probability (logistic regression, Bayesian classifiers, trees, Chapters 10, 12, 13), or an anomaly screen (Benford's/Zipf's law, Chapter 4). The target parameter (volatility, correlation, beta, loading, probability) decides the model.
2. 2Diagnose the data before estimating. Chapter 3 tests normality of relative price changes, tests multivariate normality, and inspects the sample correlation matrix. Run these first. A parameter estimated under a normality assumption the data reject is precise but misleading.
3. 3Build a naive benchmark for the same quantity. For volatility, use the 90-day and 180-day simple moving standard deviations, which the book plots against the fitted GARCH volatilities for HSBC returns. A fitted model earns trust only where it tells you something the benchmark does not.
4. 4Fit the model and read the diagnostics, not just the estimates. For time series, check sample ACF/PACF to choose orders and then check the residual plot of the fitted ARIMA. For GARCH, use the diagnostic plots. For stochastic-volatility fits, use normal Q-Q plots of standardized residuals. Residual structure means the parameters are not yet trustworthy.
5. 5For factor models, read the scree plot to decide how many components to keep. Then read the loadings of each retained component and give it a financial meaning (the book does this for the first three principal components of US zero-coupon rates). A component you cannot name is a computed result, not an insight.
6. 6For classifiers, judge probabilities and labels with the confusion matrix, not accuracy alone. The book includes a confusion matrix for a rare-disease test and one from a random classification for contrast. Compare your matrix to the random-classifier baseline. Use lift charts where ranking matters.
7. 7Check dependence and tail claims with Q-Q plots. Compare the Gaussian copula fit and the t-copula fit on the same pseudo-observations, and use the Mahalanobis-distance Q-Q plot to test elliptical assumptions. Keep the copula whose plots show no systematic departure.
8. 8Stress-test for overfitting and stability before reporting. For trees, compare trees of different sizes and prune (Chapter 13.6). Re-fit on a different window or subset and confirm that the key parameter, loading, or split survives. Report only what is stable.

Take the book's HSBC volatility case (Chapter 7). Step one is the benchmark: compute the 90-day and 180-day simple moving standard deviations of HSBC returns. These are the series plotted in Figures 7.2 and 7.3. Step two is to fit a GARCH model to the same returns and plot its fitted volatilities on the same axes as the two moving standard deviations (Figures 7.15 and 7.16). Step three is the diagnostic check (Figures 7.13 and 7.14): if the standardized residuals still show structure, the fitted volatility is not yet a trustworthy estimate. The insight is the comparison. The GARCH path should reveal how quickly volatility responds relative to the lagged windows. The 90-day window reacts faster than the 180-day one, and the model should be judged on whether it adds information beyond both. If the model reproduces only a smoothed version of the moving window, the added complexity has bought no insight. If it passes the residual checks and tracks regime shifts more sensibly, report the fitted volatility with the diagnostics attached.

• If normality or multivariate-normality tests reject, then do not rely on Gaussian-based parameter estimates. Move to a heavier-tailed alternative (the book contrasts Gaussian and t-copulae and uses Student's t for price changes) and compare the Q-Q plots.
• If residual ACF/PACF or residual plots still show structure after fitting, then respecify the model order or family before interpreting any coefficient.
• If a fitted volatility series looks no different from the 90- or 180-day moving standard deviation, then prefer the simpler benchmark.
• If a PCA component's loadings have no clear financial reading and the scree plot shows no elbow, then reduce the number of retained components or stop the factor interpretation.
• If the target class is rare, then judge the classifier by its confusion matrix against the random-classifier baseline rather than by overall accuracy.

• Reading parameters off the output without diagnostics. The tell: no residual plot, ACF/PACF, or Q-Q plot appears anywhere in the analysis.
• Assuming normality by default. The tell: tail events appear far more often than the fitted model implies, and Q-Q plots bend away from the line at the ends.
• Keeping components because the software produced them. The tell: you retain three or more principal components but cannot say what each represents, and the scree plot shows no clear cut-off.
• Trusting accuracy on a rare-event classifier. The tell: high accuracy but a confusion matrix that barely differs from the random-classification matrix on positives.
• Overfitting a tree or flexible model. The tell: a deep tree with many leaves fits the training data perfectly, and its splits change when you refit on a different sample.

• ▸Target quantity: state the parameter, factor, probability, or anomaly you want to understand (e.g., volatility, correlation, class probability).
• ▸Model family and justification: name the model and why it fits that target.
• ▸Distribution diagnostics: record the normality or multivariate-normality test outcome and what the Q-Q plot shows.
• ▸Benchmark: name the naive comparison (e.g., 90-day and 180-day moving standard deviation, random classifier) and its result.
• ▸Fitted result: record the key estimates or loadings, with the model's own diagnostics.
• ▸Residual / fit checks: note ACF/PACF, residual plot, or Q-Q result and whether structure remains.
• ▸Interpretation: write one sentence naming what each factor, loading, or flagged pattern means in financial terms.
• ▸Stability test: record the result of refitting on another window, sample, or pruned model.
• ▸Verdict: state whether the insight beats the benchmark, and list the conditions under which you would not rely on it.

1. 1Name the target and the metric before you fit anything. For a continuous forecast (price, return, volatility), use an error metric such as MSE. For a class label (for example the Class N or Y default label in the credit samples), use classification accuracy plus the TP/FP breakdown.
2. 2Write the base rate of each outcome next to the metric. The source's heart-disease example shows that a '95% accurate' test can still give a low probability of disease after a positive result when only 1% of the population has the disease.
3. 3For classifiers, build the confusion counts (TP, FP, and the negatives) and compute precision = TP / (TP + FP) in addition to overall accuracy. Precision is the quantity that answers 'given a positive call, how often is it right?'
4. 4For parameter-driven forecasts, forecast with the full predictive distribution, not a plug-in point estimate. In the normal-mean case the predictive variance is the original conditional variance plus an extra term equal to the posterior variance of the mean. That extra term is the parameter uncertainty a plug-in forecast ignores.
5. 5Check convergence by refitting on growing sample sizes n. The posterior variance, and with it the extra predictive variance, shrinks toward zero as n grows, so the predictive distribution should approach the true distribution (the book's Figure 1.5 compares the two). If your forecast does not tighten with more data, suspect misspecification.
6. 6When the accuracy figure itself comes from Monte Carlo simulation, judge the estimate by its variance. Reduce that variance by raising the number of simulations or by using a variance-reduction technique such as antithetic variables, which needs a negatively correlated partner variable.
7. 7Repeat the whole estimation procedure many times, as the book's Example 5.1 does with 1,000 repetitions, and record the mean and sample variance of the estimates. A stable mean with low variance means your accuracy number is reliable, not just a single lucky run.
8. 8Report the metric, base rates and uncertainty together. Do not report a single headline accuracy.

Take the book's diagnostic-test example, which is a classification forecast. Prevalence is 1%, the test flags 95% of diseased people as positive, and it correctly clears 95% of healthy people. The test is therefore '95% accurate'. Now ask what a positive result predicts. Per 10,000 people, 100 are diseased, so 95 are true positives. Of the 9,900 healthy, 5% (495) are false positives. Precision = 95 / (95 + 495) ≈ 16%, which is the Bayes posterior the book calls 'fairly low'. The headline accuracy hides this because the rare class is swamped by false positives. The lesson for financial classifiers, such as credit-default labels, is to report precision alongside accuracy and always state the base rate.

• If the outcome class is rare (low base rate), then do not accept overall accuracy as evidence; require precision (TP/(TP+FP)) and the confusion counts.
• If the forecast is a point estimate and parameters were estimated from limited data, then switch to the predictive distribution, whose variance includes the posterior variance of the parameter.
• If the predictive variance does not shrink as the sample size increases, then suspect model misspecification rather than noise.
• If the accuracy number comes from simulation and its variance across repeated runs is large, then increase the number of simulations or add a variance-reduction method such as antithetic variables before trusting it.
• If a test or model looks highly accurate but the positive-call precision is low, then treat positive signals as needing confirmation, not as decisions.

• Trusting headline accuracy on an imbalanced outcome. The tell: accuracy is high, but TP/(TP+FP) is low once you count the confusion table.
• Ignoring the base rate. The tell: you cannot state the prior prevalence of the outcome you are predicting.
• Using a plug-in point forecast and reporting predictive intervals that are too narrow. The tell: the interval omits the extra variance from parameter uncertainty, and realized values fall outside it too often.
• Reading one simulation run as the accuracy of the model. The tell: repeating the run gives noticeably different means, so the estimator's variance is large.
• Expecting more data to fix a bad model. The tell: the predictive distribution does not move toward the observed distribution as n grows.

• ▸Target and type: state what is forecast (price, return, volatility, or class label) and whether it is continuous or categorical.
• ▸Metric chosen: MSE for continuous targets, or accuracy plus precision for classes.
• ▸Base rate: write the prior frequency of each outcome class.
• ▸Confusion counts: record TP, FP, and the negative-class counts (classification only).
• ▸Precision = TP/(TP+FP): compute it and compare it with the headline accuracy.
• ▸Forecast form: mark point estimate or predictive distribution, and note whether parameter uncertainty is included.
• ▸Sample sizes tested: list the n values and whether predictive variance shrank as n grew.
• ▸Simulation repeats and spread: record the number of repeats, and the mean and variance of the estimates.
• ▸Verdict and next action: state accept, revise or collect more data, with one-line reasoning.