Artificial Intelligence in Finance A Python-Based Guide

1. 1Name the task before the algorithm. The passages split it three ways: unsupervised (KMeans clustering, no labels), supervised estimation or classification (labeled features), and reinforcement learning (an agent learns a policy, often represented by a DNN). Write which one you are solving.
2. 2Choose the algorithm by what you know about the functional form. If the form is known and simple, use OLS: it computes well-specified parameters directly. If the relationship is unknown, use a neural network, which approximates a wide range of relationships without knowing the form. Use DNNs (Ch. 7) for feature-to-label mappings, RNNs (Ch. 8) for sequence data, and RL (Ch. 9) for sequential decisions.
3. 3Fix the architecture explicitly and record it. Scikit-learn example: MLPRegressor(hidden_layer_sizes=3*[256], learning_rate_init=0.03, max_iter=5000). Keras example: Sequential with Dense(256, activation='relu', input_dim=1) and Dense(1, activation='linear'), compiled with loss='mse' and optimizer='rmsprop'. Each layer count, unit count, activation, loss, and optimizer is a specification choice.
4. 4Define the data before the model: which raw data, and which preprocessed features and labels. Per the ML blueprint, the features and labels implicitly define the problem (estimation, classification, etc.), so decide features such as lagged returns or technical indicators here and fix them before training.
5. 5Pick the measure of success that fits the problem, such as MSE for estimation. Compute it on training, validation, and test sets, and keep the test set untouched until the end.
6. 6Train incrementally and watch both curves. Neural network weights start random and adjust gradually, so you can train in rounds (Keras example: 100 epochs per round, 5 rounds, MSE printed after each). Log loss and val_loss per epoch; they fall on average, not monotonically.
7. 7Set capacity deliberately and test for overfitting. A model overfits when validation/test performance is worse than training performance. For the OLS benchmark, use a high-but-not-too-high polynomial degree (e.g., deg=9). Compare the DNN against that benchmark on the same test data.
8. 8Treat randomness as part of the spec. Fix seeds (tf.random.set_seed(100), random_state=0) so a configuration can be reproduced and compared.

The passages fit a nonlinear function two ways. First, OLS on polynomial terms recovers the exact quadratic and cubic factors of the original function, which works because the form is known. For relationships not of that kind, MLPRegressor(hidden_layer_sizes=3*[256], learning_rate_init=0.03, max_iter=5000) is fitted with x.reshape(-1,1), and the in-sample MSE is 0.0217: not perfect, but good for a simple configuration. Then a Keras Sequential model (one Dense(256, relu) layer, one linear output, rmsprop, MSE loss) is trained in five rounds of 100 epochs. Round 1 gives MSE 3.09714, and the MSE falls with each round as the approximation improves. In the later financial-style test (Ch. 5), a DNN trained for roughly 1,380 epochs is compared with a deg=9 OLS regression on held-out data: OLS test MSE is 0.00390 and DNN test MSE is 0.000706, about 5.5 times lower. The training-versus-validation loss at the last epochs (0.000500 vs 0.000710 at epoch 1379) is the check that this gain is not simply overfitting.

• If the functional form is known or simple, then use OLS (or a polynomial regression) as the baseline, since it computes its parameters directly; if the form is unknown, then use a DNN.
• If the data is a sequence, then use an RNN; if the problem is sequential decision-making with a policy, then use RL with a DNN to approximate the policy.
• If validation or test error is worse than training error, then the model is overfitting; reduce capacity (fewer layers or units, lower polynomial degree) before adding features.
• If MSE keeps falling across training rounds, then continue training; if loss and val_loss diverge, then stop. Do not expect monotone decline: epoch-to-epoch noise is normal.
• If a neural network does not clearly beat the OLS benchmark on test data, then keep the simpler model.

• Judging a model by training MSE alone. The tell: a strong in-sample fit, such as MSE 0.0217, with no validation or test comparison reported.
• Using a neural network where the form is known. The tell: a 3x256 DNN is built for a relationship an OLS polynomial would recover exactly.
• Setting polynomial degree or capacity too high. The tell: validation error exceeds training error as the degree rises; the passage's advice is 'high—but not too high'.
• Stopping a training run after one noisy epoch, or reading a monotonic trend into the loss curves. The tell: val_loss jumps (0.000685 to 0.001326) and the run is judged on that single point.
• Leaving the configuration undocumented or unseeded. The tell: results change between runs because random_state or the TensorFlow seed was not fixed, and nobody can say which layers, units, or optimizer produced them.

• ▸Task type: unsupervised, supervised estimation/classification, or RL; one line on why.
• ▸Raw data and features/labels: list each feature (e.g., lagged returns, technical indicators) and the label.
• ▸Baseline model: the OLS or polynomial benchmark and its degree.
• ▸Algorithm: DNN, RNN, or RL, and the reason the baseline is insufficient.
• ▸Architecture: layers, units per layer, activations, output layer.
• ▸Training setup: loss, optimizer, learning rate, epochs/rounds, seeds.
• ▸Success measure: the metric (e.g., MSE) and the target value.
• ▸Results: training, validation, and test metric for the baseline and for the AI model.
• ▸Overfitting check: the gap between training and validation/test error, and the action taken.

1. 1Define the features and labels first. Features are the inputs (for example, returns, rolling statistics, volume); the label is the output to learn (for example, next-bar direction). The source says required volume and variety are driven by the structure of these two, so you cannot size the data before you fix them.
2. 2Count the patterns your features can express. In the book's test, 10 binary features give 2^10 = 1,024 distinct patterns, and the model learns whether a 0 or a 1 is more likely for each one. Divide your sample count by that pattern count to get average observations per pattern.
3. 3Set a baseline before training. With balanced binary labels, a naive predictor is right about 50% of the time. The source defines 'big enough' as enough volume, variety and maybe velocity for the algorithm to beat a baseline on prediction, not just fit in-sample.
4. 4Pick the granularity deliberately. Pre-2000s research used mostly end-of-day (EOD) data. Programmatic APIs now serve daily bars (granularity D), minute bars (M1) and sub-minute bars (S30). Finer granularity multiplies rows without lengthening the calendar window. Choose it to match the horizon you want to predict.
5. 5Pull the data through code, not a terminal. Terminal volume cannot be consumed systematically by people, so use an API (for example, Oanda's v20 via tpqoa, with get_history specifying instrument, start, end, granularity and price type). Check the row count with .info() and look for non-null gaps.
6. 6Account for rows lost to feature construction. Rolling windows and lags remove observations. Oanda's 5-hour window at S30 gives 600 bars but only 580 usable rows with window=20. Count the usable rows, not the downloaded ones.
7. 7Add variety by category, not just by length. Table 4-1 lists structured (prices, fundamentals, volumes), unstructured (news, filings, texts) and alternative (web, social media, satellites, Internet of Things) data, each historical or streaming. Add a source only if it changes the information in the features, not merely the row count.
8. 8Compare a small and a large version of the same model on out-of-sample data. Keep the network and the task fixed, change only the data, and read the accuracy gap against the baseline. Treat in-sample accuracy as non-evidence, since a one-hidden-layer network already scores high in-sample on the small set.

The book builds a random binary classification set with f = 10 features and n = 250 samples. Ten binary features yield 1,024 possible patterns, so 250 samples cannot even visit each pattern once. All labels are random, so the only correct prediction is roughly 50% for any pattern. The larger set is grouped by pattern into the same 1,024 combinations, and the per-pattern observation count has mean 12, std 3, min 2 and max 26. Every pattern now has enough observations for the network to learn that 0 and 1 are about equally likely. The source reports that the same network trained on the small, less varied set underperforms the one trained on the large, varied set by more than 10 percentage points. It notes this is huge, since practitioners fight for tenths of a percentage point. The same count-your-rows discipline applies to real data. EUR/USD daily bars from 2018-01-01 to 2020-07-31 give 671 rows. One month of M1 bars (July 2020) gives 30,728 rows. The OandaEnv run at S30 over 08:00–13:00 on 2020-08-11 leaves 580 usable rows after the 20-bar window, with direction labels split 299/281. That is a near-balanced label, but a very thin sample for a network.

• If samples divided by possible feature patterns is below about 1 (as with 250 samples against 1,024 patterns), then do not trust any predictive result. Cut features or add data first.
• If the model's out-of-sample accuracy does not beat the baseline (about 50% for balanced binary labels), then the data is not 'big' for this task, whatever its raw size.
• If you need more observations but not a longer calendar span, then raise granularity (D to M1 to S30) rather than adding another asset. Check that the finer bars still match your prediction horizon.
• If more rows of the same series stop improving out-of-sample accuracy, then add a different data category (unstructured or alternative) rather than more of the same.
• If the gain from more or varied data is only a tenth of a percentage point, then treat it as noise until it repeats out of sample. Gains of 10+ points, as in the book's test, are what actually move the result.

• Judging sufficiency by in-sample accuracy. The tell: high training accuracy and near-baseline performance on held-out data.
• Equating many rows with a big data set. The tell: 30,000 minute bars from a single month cover one regime, so the model is large in volume but thin in variety.
• Ignoring the feature-space size. The tell: adding binary or lagged features without raising the sample count, so many patterns appear zero or one times (the source's small set has more patterns than samples).
• Forgetting rows lost to windows and lags. The tell: you planned for 600 bars and train on 580, or you compute statistics before the window warms up.
• Assuming patterns persist because the examples looked good. The book itself warns its examples suffer from data snooping and overfitting and assume time-series patterns persist. The tell: results that came from repeatedly re-tuning on the same sample.

• ▸Prediction target (label): state the output to learn and its horizon, for example next-bar direction.
• ▸Feature list and type: list each input and whether it is binary, continuous or text-derived.
• ▸Possible patterns / feature-space size: for binary features, 2^f; otherwise note how you discretize or describe the space.
• ▸Raw rows downloaded: row count from the API (.info()), with instrument, start, end and granularity.
• ▸Usable rows after window/lags: raw rows minus those lost to rolling statistics and lags.
• ▸Observations per pattern (usable rows ÷ patterns): flag if below 1 or if min per pattern is under about 2.
• ▸Data categories used: tick the structured, unstructured and alternative types, and historical vs. streaming, in Table 4-1 terms.
• ▸Baseline accuracy: for example 50% on balanced binary labels, plus the label counts such as 299/281.
• ▸Out-of-sample accuracy vs. baseline: small-data run vs. large-data run, with the percentage-point gap.

1. 1Define the baseline before training anything. For random binary data with equally likely labels, an uninformed predictor scores about 50% accuracy no matter which pattern it sees. Write down the baseline number for your task first, because every later score is judged against it, not against 100%.
2. 2Split the data before the first fit. In the source example, the first 70% (`split = int(len(data) * 0.7)`) becomes train and the rest becomes test. Fit on train only, and never let test rows touch training or tuning.
3. 3Report two numbers side by side: in-sample accuracy (`accuracy_score(train['l'], model.predict(train[fcols]))`) and out-of-sample accuracy on test. The distance between them is your overfitting reading. Quality is the test number and the gap, not the training number.
4. 4Compare capacity to data volume. Count the free parameters (for example, three hidden layers of 128 units) against the number of samples (250) and the number of possible patterns (10 binary features). When capacity is large relative to data, expect memorization and treat any high in-sample score as suspect.
5. 5Increase data volume and variety, then rerun the identical train/test protocol. The source states that predictive power 'hinges significantly on the volume and variety of the data.' Keep model and seed fixed so the change in the test score is attributable to data.
6. 6Check whether the model beats the baseline on test. If test accuracy sits at or below the uninformed baseline, the policy has learned no generalizable pattern, whatever the training score says.
7. 7For an RL agent, apply the same logic to the policy. Compare average total reward against the uninformed baseline over repeated epochs (the source's example averages about 65 across 15 epochs). A learned policy is credible only if its reward exceeds the baseline's consistently, not in one run.
8. 8Test any 'theory-like' predictions the same way: compare predicted versus realized values and their correlation, as the source does for expected versus realized returns and Sharpe ratios. Low or negative correlation means the model's internal picture of the market does not transfer.

The source builds a deliberately patternless dataset: 250 samples, 10 binary features, and binary labels all drawn at random with equal probability. There is nothing real to learn, so an honest predictor should score about 50%. An `MLPClassifier` with `hidden_layer_sizes=[128,128,128]` is fit on the full data set and reaches 0.952 in-sample accuracy, which looks excellent. The data is then split 70/30. On the training portion the network scores 0.9714, but on the test portion it scores 0.3867, more than 10 percentage points worse than the uninformed 50% baseline. The 58-point gap between train and test, together with a test score below baseline, shows the network memorized noise (high capacity, small data) and learned wrong relationships. Its learned policy quality is therefore poor, even though the in-sample number would have passed a casual check.

• If in-sample accuracy is high but test accuracy is at or below the uninformed baseline, then reject the model as overfit and do not deploy it.
• If model capacity (layers and units) is large relative to the number of samples, then expect memorization and increase data volume and variety before tuning anything else.
• If the labels are plausibly random or the market is efficient, then set the baseline at chance (about 50% for binary) and demand a test score clearly above it.
• If an RL agent's average total reward does not clearly exceed the uninformed baseline's across repeated epochs, then treat the policy as not having learned.
• If predicted and realized values correlate weakly or negatively (as with the Sharpe ratio at about -0.70), then do not trust the model's forecasts for that statistic.

• Reporting in-sample accuracy as model quality. Tell: a score like 0.95 with no held-out figure alongside it.
• Evaluating on data the model has seen, through tuning or re-splitting after peeking. Tell: test accuracy that looks suspiciously close to training accuracy.
• Using an oversized network on a small data set. Tell: near-perfect training fit (0.97) paired with a collapse on test (0.39).
• Comparing against 100% or against nothing instead of an uninformed baseline. Tell: no chance-level number is written down anywhere.
• Judging an RL agent from a single run or single epoch. Tell: rewards vary widely between epochs (54 to 77 in the source) and the claim rests on one of them.

• ▸Task and label type: state whether it is classification or estimation, and what the labels are.
• ▸Uninformed baseline score: write the chance-level metric (for example, 50% accuracy or baseline average reward).
• ▸Data volume and variety: record sample count, number of features, and the number of possible patterns.
• ▸Model capacity: list layers, units, and the capacity-to-samples ratio.
• ▸Train/test split: record the split fraction (for example, 70/30) and confirm test data was never used in fitting.
• ▸In-sample score: enter the score on the training set.
• ▸Out-of-sample score: enter the score on the held-out test set.
• ▸Generalization gap and verdict: compute train minus test, compare test to baseline, and mark accept, retrain with more data, or reject.

1. 1Define the label before you touch a model: for a trading bot, a binary label of 1 (market up) or 0 (market down) for the next period. Accuracy is then the share of correct predictions over all label values, as in the passages' classification examples.
2. 2Set the uninformed baseline first. With balanced binary labels where every pattern is equally likely to be followed by a 0 or a 1, a baseline predictor is right about 50% of the time. Every accuracy figure you report is judged against this number.
3. 3Split the data before training. The passages use split = int(len(data) * 0.7), with the first 70% as train and the remainder as test. Fit the model on the training sub-set only (model.fit(train[fcols], train['l'])).
4. 4Compute accuracy twice with accuracy_score: once on train and once on test (model.predict(test[fcols])). Treat the in-sample figure as a capacity diagnostic. Only the out-of-sample figure counts as predictive accuracy.
5. 5Compare the test score with the 50% baseline, not with the training score. A test score below the baseline means the model learned wrong relationships, whatever the training score says.
6. 6Check the ratio of model capacity to data volume. The failing example uses three hidden layers of 128 units (hidden_layer_sizes=[128, 128, 128]) on only 250 samples with 10 binary features. Predictive power depends heavily on the volume and variety of training data, so add data or shrink the network before tuning anything else.
7. 7Check whether the data can contain signal at all. In the random example, labels are independent of features, so there is nothing to learn beyond equal likelihood of 0 and 1. Any 'accuracy' above baseline there would be luck, so run the same pipeline on random data as a control.
8. 8Only after the test score clears the baseline should you treat the result as a candidate statistical inefficiency. Predicting market direction out-of-sample is what the source defines as the prerequisite for exploiting it through algorithmic trading.

The source builds a random data set with f = 10 binary features and n = 250 samples, with random binary labels, so the correct baseline accuracy is about 50%. An MLPClassifier with hidden_layer_sizes=[128, 128, 128] and max_iter=1000 is trained on the whole data set and scores 0.952 in-sample, which looks impressive. The data is then split 70/30 (split = int(len(data) * 0.7)) and the model is retrained on the training part only. In-sample accuracy on train is 0.9714285714285714. On the test set it is 0.38666666666666666, more than 10 percentage points worse than the 50% baseline. The gap from about 97% to about 39% shows the network memorized patterns in a small data set instead of discovering a relationship. The only number that measures predictive accuracy is the 0.387, and it fails the baseline test. A bot built on the 0.952 figure would have been trading noise.

• If out-of-sample accuracy is at or below the baseline (about 50% for balanced binary up/down labels), then the model has no demonstrated predictive accuracy, regardless of in-sample score.
• If in-sample accuracy is near 100% and test accuracy collapses, then reduce capacity or add data volume and variety before changing anything else.
• If a model scores well on randomly generated labels, then your evaluation is leaking or broken; fix the pipeline before trusting any result.
• If you only have a whole-data-set accuracy figure, then discard it and re-run with a train/test split, training on the training sub-set alone.
• If test accuracy beats the baseline, then treat it as a candidate inefficiency, not as proof of profit: it is a prerequisite for exploitation through trading, not the exploitation itself.

• Reporting in-sample accuracy as predictive accuracy. Tell: a figure like 0.952 from a model fit and scored on the same data.
• Oversized network on a small sample. Tell: three 128-unit layers on 250 rows, with a train score near 97% and a test score near 39%.
• Comparing test accuracy to training accuracy instead of the 50% baseline. Tell: a result called 'only a small drop' when the test score is actually below chance.
• Assuming the market contains learnable patterns without checking. Tell: the model learns 'patterns' in data where labels were generated independently of features, so any apparent signal is spurious.
• Reading a rising accuracy curve over epochs as evidence of skill. Tell: accuracy improves step-wise during training on the training data (as in the Keras example), with no held-out score reported.

• ▸Label definition: state what 1 and 0 mean (e.g., 1 = next-period market up, 0 = down).
• ▸Baseline accuracy: record the uninformed baseline (about 50% for balanced binary labels) and how you verified label balance.
• ▸Sample size and feature count: write n and f; note the ratio of samples to features.
• ▸Model capacity: list the architecture (e.g., hidden_layer_sizes) and compare it with the data volume.
• ▸Split rule: record the train/test split (e.g., first 70% train, last 30% test) and confirm the model never saw the test rows.
• ▸In-sample accuracy: score on the training set only; use as a diagnostic, not a result.
• ▸Out-of-sample accuracy: score on the test set via accuracy_score; this is the reported figure.
• ▸Gap vs. baseline: out-of-sample accuracy minus baseline, in percentage points.
• ▸Verdict: pass only if out-of-sample accuracy clearly exceeds baseline; otherwise note whether to add data, shrink the model, or revisit features.

1. 1Separate the two questions before you test anything. Statistical inefficiency means the model predicts direction with an edge (the passages give 55% or 60% correct as examples). Economic inefficiency exists only if a trading strategy exploits that edge profitably after transaction costs. Record both results in separate lines, because the first does not imply the second.
2. 2Fix the prediction baseline on out-of-sample data only. In the source's DNN runs, in-sample accuracy (0.6982, 0.7944) sits well above out-of-sample accuracy (0.6476, 0.6590). Carry only the out-of-sample figure into the profitability test.
3. 3Turn each prediction into a position rule before backtesting. The source says the signal is only the beginning, so write down what a prediction triggers: entry, exit, position size, stop loss, take profit. Without these rules you have an accuracy score, not a strategy.
4. 4Run a vectorized backtest first as a fast screen, then an event-based backtest with risk management for the survivors. Part IV orders them this way: vectorized (Chapter 10), then event-based with risk management (Chapter 11), then execution and deployment (Chapter 12).
5. 5Charge transaction costs inside the backtest on every position change. The definition of economic inefficiency requires profit net of costs, so report only net returns, never gross.
6. 6Report absolute return and a risk-adjusted measure such as the Sharpe ratio side by side, both net of costs. The definition requires both, since one can look fine while the other is poor.
7. 7Benchmark against the market over the same period. The source's cautionary case: The Voleon Group, a machine-learning hedge fund with more than $6 billion under management, returned 7% in 2019 while the S&P 500 rose almost 30%. Compute the buy-and-hold return for your instrument and window, and compare your net result to it.
8. 8Treat the result as a hypothesis for live testing, not proof. The source notes that its own examples lack comprehensive empirical evidence and are sometimes tweaked to make a point. Re-run on untouched later data before committing capital.

The source's DNN chapter produces out-of-sample accuracies around 0.65 (adamax 0.6476, nadam 0.6590), and the conclusion calls them promising, with accuracy consistently at 60% or higher. The same conclusion says accuracy is only one side of the coin: an implementable trading strategy must turn these signals into profit. Apply the steps. First, take the out-of-sample figure (about 0.65), not the in-sample 0.7944 for nadam. Second, define the rule: go long on a predicted up day and short on a predicted down day. Third, backtest and subtract transaction costs on each position change. Fourth, report net return and Sharpe ratio next to the buy-and-hold result. The source supplies no net-of-cost result for these models, so the profitability outcome stays unknown until you run it. The Voleon case (7% versus the S&P 500's almost 30% in 2019) shows that a machine-learning approach can lose to the market it tries to beat.

• If out-of-sample accuracy is near 50%, then stop: there is no statistical edge to monetize. If it is 55-60% or higher, proceed to the backtest.
• If in-sample accuracy is far above out-of-sample accuracy (0.7944 versus 0.6590 in the source), then base every profitability estimate on the out-of-sample number only.
• If gross returns are positive but net returns after transaction costs are not, then the inefficiency is statistical, not economic; do not deploy.
• If the net strategy return does not beat a passive benchmark over the same period, then treat the strategy as unproven even if it is profitable in absolute terms (the Voleon 2019 case).
• If a vectorized backtest passes, then confirm with event-based backtesting that includes stop loss, take profit and risk controls before any live deployment.

• Equating prediction accuracy with profit. The tell: the report ends at accuracy figures and has no return, Sharpe ratio or cost line. The source states accuracy is just one side of the coin.
• Reporting gross returns. The tell: the backtest code has no per-trade cost term, so a thin edge looks profitable.
• Using in-sample results as performance evidence. The tell: the quoted accuracy is the higher in-sample number (such as 0.7944) rather than the out-of-sample one.
• Ignoring the benchmark. The tell: a positive return is quoted with no comparison to the market, while the market far outpaced it (7% versus almost 30%).
• Skipping execution and risk rules. The tell: signals map straight to positions with no stop loss, take profit, monitoring or execution assumptions, so the backtest cannot reflect live trading.

• ▸Instrument and test period: name the asset and the exact out-of-sample date range.
• ▸Out-of-sample prediction accuracy: enter the directional hit rate on unseen data only.
• ▸In-sample accuracy: enter it to expose the gap to the out-of-sample figure.
• ▸Signal-to-position rule: state how a prediction becomes entry, exit and size.
• ▸Risk rules: list stop loss, take profit and any other controls applied.
• ▸Transaction cost assumption: state the cost charged per position change.
• ▸Gross return versus net return: enter both, with the net figure after costs.
• ▸Sharpe ratio (net): enter the risk-adjusted result after costs.
• ▸Benchmark return and verdict: enter the buy-and-hold return for the same period, then judge as economic inefficiency, statistical only, or unproven.