Validation Design for Deep Learning Trading Strategies

This notebook examines how validation design affects the apparent performance of a deep-learning trading strategy. Bitcoin daily returns and a deliberately overlapping 10-day forward-return label are used, then three validation settings are compared:

1. a single chronological train/test split;
2. non-anchored walk-forward validation with the overlap left in place;
3. the same walk-forward designs after purging overlapping training labels.

The same three architectures are used throughout: an MLP, an LSTM, and a CNN applied to a Gramian Angular Field (GAF) representation.

Reproducibility

A fixed random seed is used throughout so that the model comparisons are based on one reproducible execution.

# Required packages
!pip -q install yfinance pyts

import os
import warnings

os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
warnings.filterwarnings("ignore")

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import yfinance as yf
import tensorflow as tf

from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_absolute_error, mean_squared_error
from pyts.image import GramianAngularField

from tensorflow.keras import Sequential
from tensorflow.keras.layers import (
    Input, Dense, Dropout, LSTM, Conv2D, MaxPooling2D, Flatten
)
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.optimizers import Adam

# -----------------------------
# Global reproducibility settings
# -----------------------------
SEED = 642
np.random.seed(SEED)
tf.keras.utils.set_random_seed(SEED)
tf.get_logger().setLevel("ERROR")

try:
    tf.config.experimental.enable_op_determinism()
except Exception:
    pass

pd.set_option("display.max_columns", 30)
pd.set_option("display.float_format", lambda x: f"{x:,.6f}")

Step 1a: Data description

We use daily Bitcoin (BTC-USD) prices and returns. The modelling sample is restricted to fewer than 2,000 observations, as required.

Bitcoin is useful for this exercise because the sample contains pronounced changes in both price level and volatility, providing a demanding setting for comparing validation schemes.

TICKER = "BTC-USD"
RAW_OBS_LIMIT = 1900
N_LAGS = 30
N_FWD = 10
ANNUAL_FACTOR = 365.0

raw = yf.download(
    TICKER,
    start="2018-01-01",
    end="2026-01-01",
    interval="1d",
    auto_adjust=True,
    progress=False
)


if isinstance(raw.columns, pd.MultiIndex):
    raw.columns = raw.columns.get_level_values(0)

df = (
    raw[["Close"]]
    .rename(columns={"Close": "close"})
    .reset_index()
    .rename(columns={"Date": "date"})
    .tail(RAW_OBS_LIMIT)
    .reset_index(drop=True)
)

df["date"] = pd.to_datetime(df["date"]).dt.tz_localize(None)
df["Ret"] = df["close"].pct_change()
df = df.dropna().reset_index(drop=True)

print("Raw observations after return construction:", len(df))
print("Sample period:", df["date"].min().date(), "to", df["date"].max().date())
df.head()

Raw observations after return construction: 1899
Sample period: 2020-10-20 to 2025-12-31

Price	date	close	Ret
0	2020-10-20	11,916.334961	0.014844
1	2020-10-21	12,823.689453	0.076144
2	2020-10-22	12,965.891602	0.011089
3	2020-10-23	12,931.539062	-0.002649
4	2020-10-24	13,108.062500	0.013651

Price and return behaviour

The price and daily-return series are plotted below, followed by summary statistics for returns. Kurtosis is reported as excess kurtosis, for which the normal-distribution benchmark is zero.

fig, axes = plt.subplots(2, 1, figsize=(12, 8), sharex=True)

axes[0].plot(df["date"], df["close"])
axes[0].set_title("BTC-USD Daily Adjusted Close")
axes[0].set_ylabel("Price (USD)")
axes[0].grid(alpha=0.25)

axes[1].plot(df["date"], df["Ret"])
axes[1].set_title("BTC-USD Daily Returns")
axes[1].set_xlabel("Date")
axes[1].set_ylabel("Daily Return")
axes[1].grid(alpha=0.25)

plt.tight_layout()
plt.show()

return_stats = pd.Series({
    "Count": df["Ret"].count(),
    "Mean": df["Ret"].mean(),
    "Std. Dev.": df["Ret"].std(),
    "Minimum": df["Ret"].min(),
    "25%": df["Ret"].quantile(0.25),
    "Median": df["Ret"].median(),
    "75%": df["Ret"].quantile(0.75),
    "Maximum": df["Ret"].max(),
    "Skewness": df["Ret"].skew(),
    "Excess Kurtosis": df["Ret"].kurt(),
}, name="BTC daily returns")

display(return_stats.to_frame())

	BTC daily returns
Count	1,899.000000
Mean	0.001531
Std. Dev.	0.030775
Minimum	-0.159747
25%	-0.012980
Median	0.000313
75%	0.015243
Maximum	0.187465
Skewness	0.151224
Excess Kurtosis	3.548208

Step 1b: Deliberately overlapping labels

The target is the cumulative return over the next 10 daily observations:

$$ R_{t,t+10}=\prod_{j=1}^{10}(1+r_{t+j})-1. $$

Adjacent labels overlap heavily. For example, the label at $t$ uses returns from $t+1$ to $t+10$, while the label at $t+1$ uses $t+2$ to $t+11$. They therefore share nine of ten daily returns.

The source of leakage is this overlap across a train/test boundary: a training label near the boundary can contain return information that is also used to construct a test label. The start and end dates of each label window are retained so that this overlap can be measured directly.

# Future returns r_{t+1}, ..., r_{t+10}
future_return_matrix = pd.concat(
    [df["Ret"].shift(-h) for h in range(1, N_FWD + 1)],
    axis=1
)
future_return_matrix.columns = [f"future_ret_{h}" for h in range(1, N_FWD + 1)]

df["Target_10d"] = (1.0 + future_return_matrix).prod(axis=1) - 1.0
df["LabelStart"] = df["date"].shift(-1)
df["LabelEnd"] = df["date"].shift(-N_FWD)

# Next-day return used in the trading backtest.
df["NextRet"] = df["Ret"].shift(-1)

df[["date", "Ret", "Target_10d", "LabelStart", "LabelEnd", "NextRet"]].tail(15)

Price	date	Ret	Target_10d	LabelStart	LabelEnd	NextRet
1884	2025-12-17	-0.019355	0.019252	2025-12-18	2025-12-27	-0.007908
1885	2025-12-18	-0.007908	0.027770	2025-12-19	2025-12-28	0.030901
1886	2025-12-19	0.030901	-0.010956	2025-12-20	2025-12-29	0.002731
1887	2025-12-20	0.002731	0.000975	2025-12-21	2025-12-30	0.003144
1888	2025-12-21	0.003144	-0.012558	2025-12-22	2025-12-31	-0.001486
1889	2025-12-22	-0.001486	-0.011088	2025-12-23	NaT	-0.012160
1890	2025-12-23	-0.012160	0.001085	2025-12-24	NaT	0.002265
1891	2025-12-24	0.002265	-0.001177	2025-12-25	NaT	-0.004306
1892	2025-12-25	-0.004306	0.003142	2025-12-26	NaT	0.000764
1893	2025-12-26	0.000764	0.002376	2025-12-27	NaT	0.005736
1894	2025-12-27	0.005736	-0.003341	2025-12-28	NaT	0.000384
1895	2025-12-28	0.000384	-0.003723	2025-12-29	NaT	-0.007943
1896	2025-12-29	-0.007943	0.004254	2025-12-30	NaT	0.014827
1897	2025-12-30	0.014827	-0.010418	2025-12-31	NaT	-0.010418
1898	2025-12-31	-0.010418	0.000000	NaT	NaT	NaN

Step 1c: Model inputs and trading rule

Each observation uses the most recent 30 daily returns to predict the 10-day forward cumulative return.

The information set is held constant across the three architectures:

• MLP: the 30 returns as a feature vector;
• LSTM: the same returns as an ordered (30\times1) sequence;
• CNN-GAF: the same window transformed into a (30\times30) GAF image.

For the GAF representation, each window is rescaled to ([-1,1]), mapped through

$ \phi_i=\arccos(\tilde{x}_i), $

and represented by

$ G_{ij}=\cos(\phi_i+\phi_j). $

All three models use the same trading rule: take a long position for the next day when the predicted 10-day return is positive; otherwise remain in cash.

# Use the current return plus the previous 29 daily returns.
feature_cols = []
for lag in range(N_LAGS):
    col = f"Ret_lag_{lag}"
    df[col] = df["Ret"].shift(lag)
    feature_cols.append(col)

feat_df = df[
    ["date", "Ret", "NextRet", "Target_10d", "LabelStart", "LabelEnd"] + feature_cols
].dropna().reset_index(drop=True)

assert len(feat_df) <= 2000, "Assignment requires no more than 2,000 observations."

print("Final modelling observations:", len(feat_df))
print("Final modelling period:", feat_df["date"].min().date(), "to", feat_df["date"].max().date())
print("Input dimension:", len(feature_cols))
feat_df.head()

Final modelling observations: 1860
Final modelling period: 2020-11-18 to 2025-12-21
Input dimension: 30

Price	date	Ret	NextRet	Target_10d	LabelStart	LabelEnd	Ret_lag_0	Ret_lag_1	Ret_lag_2	Ret_lag_3	Ret_lag_4	Ret_lag_5	Ret_lag_6	Ret_lag_7	Ret_lag_8	...	Ret_lag_15	Ret_lag_16	Ret_lag_17	Ret_lag_18	Ret_lag_19	Ret_lag_20	Ret_lag_21	Ret_lag_22	Ret_lag_23	Ret_lag_24	Ret_lag_25	Ret_lag_26	Ret_lag_27	Ret_lag_28	Ret_lag_29
0	2020-11-18	0.008988	0.000735	-0.004864	2020-11-19	2020-11-28	0.008988	0.055593	0.047665	-0.007005	-0.015300	0.002548	0.036621	0.026842	-0.002701	...	0.029505	-0.013585	-0.003185	0.017309	0.008085	0.012553	-0.028045	0.044280	0.003382	-0.005866	0.013651	-0.002649	0.011089	0.076144	0.014844
1	2020-11-19	0.000735	0.045138	0.020227	2020-11-20	2020-11-29	0.000735	0.008988	0.055593	0.047665	-0.007005	-0.015300	0.002548	0.036621	0.026842	...	0.013147	0.029505	-0.013585	-0.003185	0.017309	0.008085	0.012553	-0.028045	0.044280	0.003382	-0.005866	0.013651	-0.002649	0.011089	0.076144
2	2020-11-20	0.045138	0.001123	0.053945	2020-11-21	2020-11-30	0.045138	0.000735	0.008988	0.055593	0.047665	-0.007005	-0.015300	0.002548	0.036621	...	0.102319	0.013147	0.029505	-0.013585	-0.003185	0.017309	0.008085	0.012553	-0.028045	0.044280	0.003382	-0.005866	0.013651	-0.002649	0.011089
3	2020-11-21	0.001123	-0.014603	0.008624	2020-11-22	2020-12-01	0.001123	0.045138	0.000735	0.008988	0.055593	0.047665	-0.007005	-0.015300	0.002548	...	-0.000897	0.102319	0.013147	0.029505	-0.013585	-0.003185	0.017309	0.008085	0.012553	-0.028045	0.044280	0.003382	-0.005866	0.013651	-0.002649
4	2020-11-22	-0.014603	-0.000320	0.045242	2020-11-23	2020-12-02	-0.014603	0.001123	0.045138	0.000735	0.008988	0.055593	0.047665	-0.007005	-0.015300	...	-0.047034	-0.000897	0.102319	0.013147	0.029505	-0.013585	-0.003185	0.017309	0.008085	0.012553	-0.028045	0.044280	0.003382	-0.005866	0.013651

5 rows × 36 columns

Common estimation setup

The model architectures, loss function, optimizer, maximum epochs, batch size, trading rule, and evaluation metrics are held fixed across Steps 1-3.

A chronological validation tail is taken from each training sample for early stopping. The fitting portion is purged against that internal validation block so that model selection does not introduce an additional source of label overlap. The outer train/test boundary remains unpurged in Steps 1 and 2 and is treated explicitly in Step 3.

MAX_EPOCHS = 35
PATIENCE = 5
BATCH_SIZE = 32
VAL_FRACTION = 0.15
LEARNING_RATE = 1e-3

gaf_transformer = GramianAngularField(
    image_size=N_LAGS,
    method="summation",
    sample_range=(-1, 1)
)

def build_model(model_name):
    tf.keras.backend.clear_session()

    if model_name == "MLP":
        model = Sequential([
            Input(shape=(N_LAGS,)),
            Dense(64, activation="relu"),
            Dropout(0.20),
            Dense(32, activation="relu"),
            Dense(1)
        ])

    elif model_name == "LSTM":
        model = Sequential([
            Input(shape=(N_LAGS, 1)),
            LSTM(32, return_sequences=True),
            Dropout(0.20),
            LSTM(16),
            Dense(1)
        ])

    elif model_name == "CNN-GAF":
        model = Sequential([
            Input(shape=(N_LAGS, N_LAGS, 1)),
            Conv2D(16, (3, 3), padding="same", activation="relu"),
            MaxPooling2D((2, 2)),
            Conv2D(32, (3, 3), padding="same", activation="relu"),
            MaxPooling2D((2, 2)),
            Flatten(),
            Dense(32, activation="relu"),
            Dropout(0.20),
            Dense(1)
        ])
    else:
        raise ValueError(f"Unknown model: {model_name}")

    model.compile(
        optimizer=Adam(learning_rate=LEARNING_RATE),
        loss="mse"
    )
    return model


def purge_against_test(train_df, test_df):
    """
    Remove training observations whose label interval overlaps the
    earliest label interval in the test set.

    In our non-anchored walk-forward design, all training observations
    precede the test block, so a post-test embargo is not required.
    """
    first_test_label_start = test_df["LabelStart"].iloc[0]
    keep = train_df["LabelEnd"] < first_test_label_start
    return train_df.loc[keep].copy()


def chronological_fit_validation_split(train_df, val_fraction=VAL_FRACTION):
    """
    Reserve the final part of an outer training fold for validation and
    purge fitting observations whose labels overlap that validation block.
    """
    split = int(len(train_df) * (1.0 - val_fraction))
    fit_df = train_df.iloc[:split].copy()
    val_df = train_df.iloc[split:].copy()

    if len(val_df) == 0:
        raise ValueError("Validation set is empty.")

    first_val_label_start = val_df["LabelStart"].iloc[0]
    fit_df = fit_df.loc[fit_df["LabelEnd"] < first_val_label_start].copy()

    return fit_df, val_df


def prepare_X(model_name, X_fit_raw, X_val_raw, X_test_raw):
    """
    Train-only preprocessing.

    MLP/LSTM: one StandardScaler is fit on all observed return values
    in the fitting sample, then reused for validation/test.
    CNN-GAF: each historical return window is transformed into a GAF;
    the transformation uses only values already inside that window.
    """
    if model_name in {"MLP", "LSTM"}:
        scaler = StandardScaler()
        scaler.fit(X_fit_raw.reshape(-1, 1))

        def transform(x):
            z = scaler.transform(x.reshape(-1, 1)).reshape(x.shape)
            if model_name == "LSTM":
                z = z[..., np.newaxis]
            return z

        return transform(X_fit_raw), transform(X_val_raw), transform(X_test_raw)

    if model_name == "CNN-GAF":
        X_fit = gaf_transformer.transform(X_fit_raw)[..., np.newaxis]
        X_val = gaf_transformer.transform(X_val_raw)[..., np.newaxis]
        X_test = gaf_transformer.transform(X_test_raw)[..., np.newaxis]
        return X_fit, X_val, X_test


def fit_predict(model_name, train_df, test_df, seed):
    """
    Fit one fresh model and return predictions in the original target scale.
    The outer test sample is never used for fitting, scaling, or early stopping.
    """
    tf.keras.utils.set_random_seed(seed)

    fit_df, val_df = chronological_fit_validation_split(train_df)

    X_fit_raw = fit_df[feature_cols].to_numpy(dtype=np.float32)
    X_val_raw = val_df[feature_cols].to_numpy(dtype=np.float32)
    X_test_raw = test_df[feature_cols].to_numpy(dtype=np.float32)

    y_fit_raw = fit_df["Target_10d"].to_numpy(dtype=np.float32).reshape(-1, 1)
    y_val_raw = val_df["Target_10d"].to_numpy(dtype=np.float32).reshape(-1, 1)

    X_fit, X_val, X_test = prepare_X(
        model_name, X_fit_raw, X_val_raw, X_test_raw
    )

    # Train-only target scaling improves numerical conditioning while
    # preserving a common target definition across architectures.
    y_scaler = StandardScaler()
    y_fit = y_scaler.fit_transform(y_fit_raw)
    y_val = y_scaler.transform(y_val_raw)

    model = build_model(model_name)

    early_stop = EarlyStopping(
        monitor="val_loss",
        patience=PATIENCE,
        restore_best_weights=True,
        min_delta=1e-5
    )

    history = model.fit(
        X_fit,
        y_fit,
        validation_data=(X_val, y_val),
        epochs=MAX_EPOCHS,
        batch_size=BATCH_SIZE,
        shuffle=False,
        verbose=0,
        callbacks=[early_stop]
    )

    pred_scaled = model.predict(X_test, verbose=0)
    pred = y_scaler.inverse_transform(pred_scaled).ravel()

    return pred, history


def prediction_frame(test_df, pred):
    out = test_df[
        ["date", "NextRet", "Target_10d", "LabelStart", "LabelEnd"]
    ].copy()
    out["prediction"] = np.asarray(pred)
    out["signal"] = (out["prediction"] > 0).astype(int)

    # The prediction at date t uses information available through t.
    # The position is therefore applied to the next daily return.
    out["strategy_ret"] = out["signal"] * out["NextRet"]
    out["buy_hold_ret"] = out["NextRet"]

    return out.reset_index(drop=True)


def predictive_metrics(pred_df):
    y = pred_df["Target_10d"].to_numpy()
    p = pred_df["prediction"].to_numpy()

    return {
        "MAE": mean_absolute_error(y, p),
        "MSE": mean_squared_error(y, p),
        "RMSE": np.sqrt(mean_squared_error(y, p)),
        "Directional Accuracy": np.mean(np.sign(y) == np.sign(p))
    }


def portfolio_metrics(daily_returns):
    r = pd.Series(daily_returns).dropna().astype(float)
    nav = (1.0 + r).cumprod()

    n = len(r)
    total_return = nav.iloc[-1] - 1.0
    annual_return = nav.iloc[-1] ** (ANNUAL_FACTOR / n) - 1.0
    annual_vol = r.std(ddof=1) * np.sqrt(ANNUAL_FACTOR)
    sharpe = (
        r.mean() / r.std(ddof=1) * np.sqrt(ANNUAL_FACTOR)
        if r.std(ddof=1) > 0 else np.nan
    )

    running_peak = nav.cummax().clip(lower=1.0)
    max_drawdown = (nav / running_peak - 1.0).min()

    return {
        "Total Return": total_return,
        "Annualized Return": annual_return,
        "Annualized Volatility": annual_vol,
        "Sharpe Ratio": sharpe,
        "Maximum Drawdown": max_drawdown
    }


def summarize_prediction_frame(pred_df, model_name, experiment):
    row = {
        "Experiment": experiment,
        "Model": model_name,
        "OOS Observations": len(pred_df),
    }
    row.update(predictive_metrics(pred_df))
    row.update(portfolio_metrics(pred_df["strategy_ret"]))

    bh = portfolio_metrics(pred_df["buy_hold_ret"])
    row["Buy-and-Hold Total Return"] = bh["Total Return"]
    row["Buy-and-Hold Sharpe"] = bh["Sharpe Ratio"]

    return row


def plot_strategy_comparison(predictions_by_model, title):
    plt.figure(figsize=(12, 6))

    # All models in one experiment share the same OOS dates.
    any_df = next(iter(predictions_by_model.values()))
    benchmark_nav = (1.0 + any_df["buy_hold_ret"]).cumprod()
    plt.plot(any_df["date"], benchmark_nav, linestyle="--", label="BTC Buy-and-Hold")

    for model_name, pred_df in predictions_by_model.items():
        nav = (1.0 + pred_df["strategy_ret"]).cumprod()
        plt.plot(pred_df["date"], nav, label=model_name)

    plt.title(title)
    plt.xlabel("Date")
    plt.ylabel("Growth of $1")
    plt.legend()
    plt.grid(alpha=0.25)
    plt.tight_layout()
    plt.show()

Step 1c: Single train/test split

The first 80% of observations form the training sample and the final 20% form the test sample. The split is chronological.

No purging is applied at the outer boundary in this step. This preserves the deliberately overlapping labels required for the baseline experiment. Preprocessing parameters are estimated from the training data only, and the test sample is reserved for final out-of-sample evaluation.

The overlap count below reports how many training labels extend into the first test-label interval.

split_idx = int(len(feat_df) * 0.80)

step1_train = feat_df.iloc[:split_idx].copy()
step1_test = feat_df.iloc[split_idx:].copy()

first_test_label_start = step1_test["LabelStart"].iloc[0]
step1_overlap_count = int(
    (step1_train["LabelEnd"] >= first_test_label_start).sum()
)

print("Training observations:", len(step1_train))
print("Test observations:", len(step1_test))
print("Train period:", step1_train["date"].min().date(), "to", step1_train["date"].max().date())
print("Test period:", step1_test["date"].min().date(), "to", step1_test["date"].max().date())
print("Training labels overlapping the test-label period:", step1_overlap_count)

Training observations: 1488
Test observations: 372
Train period: 2020-11-18 to 2024-12-14
Test period: 2024-12-15 to 2025-12-21
Training labels overlapping the test-label period: 9

Step 1c: MLP, LSTM, and CNN-GAF results

The three models are estimated on the same training sample and evaluated on the same test period. Predictive accuracy and trading performance are reported on a common basis.

MODEL_NAMES = ["MLP", "LSTM", "CNN-GAF"]

step1_predictions = {}
step1_histories = {}
step1_rows = []

for model_i, model_name in enumerate(MODEL_NAMES):
    print(f"Training Step 1 {model_name}...")

    pred, history = fit_predict(
        model_name,
        step1_train,
        step1_test,
        seed=SEED + 100 + model_i
    )

    pred_df = prediction_frame(step1_test, pred)
    step1_predictions[model_name] = pred_df
    step1_histories[model_name] = history
    step1_rows.append(
        summarize_prediction_frame(
            pred_df, model_name, "Step 1: Single Split (Leaky)"
        )
    )

step1_summary = pd.DataFrame(step1_rows)
display(step1_summary)

Training Step 1 MLP...
Training Step 1 LSTM...
Training Step 1 CNN-GAF...

	Experiment	Model	OOS Observations	MAE	MSE	RMSE	Directional Accuracy	Total Return	Annualized Return	Annualized Volatility	Sharpe Ratio	Maximum Drawdown	Buy-and-Hold Total Return	Buy-and-Hold Sharpe
0	Step 1: Single Split (Leaky)	MLP	372	0.049896	0.003997	0.063223	0.481183	0.041090	0.040302	0.331878	0.284285	-0.263787	-0.151571	-0.167980
1	Step 1: Single Split (Leaky)	LSTM	372	0.049005	0.003687	0.060718	0.473118	-0.151571	-0.148943	0.424685	-0.167980	-0.321470	-0.151571	-0.167980
2	Step 1: Single Split (Leaky)	CNN-GAF	372	0.049785	0.003804	0.061680	0.473118	-0.151571	-0.148943	0.424685	-0.167980	-0.321470	-0.151571	-0.167980

# Training / validation histories
fig, axes = plt.subplots(1, 3, figsize=(15, 4))

for ax, model_name in zip(axes, MODEL_NAMES):
    hist = step1_histories[model_name].history
    ax.plot(hist["loss"], label="Training")
    ax.plot(hist["val_loss"], label="Validation")
    ax.set_title(model_name)
    ax.set_xlabel("Epoch")
    ax.set_ylabel("MSE loss")
    ax.grid(alpha=0.25)

axes[0].legend()
plt.suptitle("Step 1: Training and Validation Loss")
plt.tight_layout()
plt.show()

plot_strategy_comparison(
    step1_predictions,
    "Step 1: Single-Split Strategy Backtests (Deliberately Leaky Boundary)"
)

Step 1c: Interpretation

Step 1 provides the single-split baseline. Because the label windows overlap at the train/test boundary, these results should be treated as potentially contaminated by leakage.

The three models differ only in how the same 30-day return history is represented. The comparison therefore provides a useful baseline for assessing whether later changes in performance are driven by architecture or by the validation design.

Step 2: Non-anchored walk-forward backtesting

The same three models are re-estimated under two non-anchored walk-forward designs:

• Step 2a: 500 observations for training and 500 for testing;
• Step 2b: 500 observations for training and 100 for testing.

The training window remains fixed at 500 observations and rolls forward by the size of the test window. No outer purging is applied in Step 2, so the deliberately overlapping labels remain present.

def run_walk_forward(model_name, train_size, test_size, purge_outer=False, seed_base=1000):
    fold_outputs = []
    leakage_rows = []

    n = len(feat_df)
    fold_id = 0

    for start in range(0, n - train_size - test_size + 1, test_size):
        train_end = start + train_size
        test_end = train_end + test_size

        train_fold_raw = feat_df.iloc[start:train_end].copy()
        test_fold = feat_df.iloc[train_end:test_end].copy()

        first_test_label_start = test_fold["LabelStart"].iloc[0]
        overlap_before = int(
            (train_fold_raw["LabelEnd"] >= first_test_label_start).sum()
        )

        if purge_outer:
            train_fold = purge_against_test(train_fold_raw, test_fold)
        else:
            train_fold = train_fold_raw

        overlap_after = int(
            (train_fold["LabelEnd"] >= first_test_label_start).sum()
        )

        pred, _ = fit_predict(
            model_name,
            train_fold,
            test_fold,
            seed=seed_base + fold_id
        )

        fold_df = prediction_frame(test_fold, pred)
        fold_df["Fold"] = fold_id + 1
        fold_outputs.append(fold_df)

        leakage_rows.append({
            "Model": model_name,
            "Fold": fold_id + 1,
            "Raw Train Size": len(train_fold_raw),
            "Effective Train Size": len(train_fold),
            "Test Size": len(test_fold),
            "Overlapping Train Labels Before": overlap_before,
            "Overlapping Train Labels After": overlap_after,
        })

        fold_id += 1

    predictions = pd.concat(fold_outputs, ignore_index=True)
    leakage = pd.DataFrame(leakage_rows)

    return predictions, leakage


def run_walk_forward_experiment(
    train_size,
    test_size,
    purge_outer,
    experiment_name,
    seed_offset
):
    preds_by_model = {}
    leakage_frames = []
    summary_rows = []

    for model_i, model_name in enumerate(MODEL_NAMES):
        print(f"{experiment_name}: training {model_name}...")

        preds, leakage = run_walk_forward(
            model_name=model_name,
            train_size=train_size,
            test_size=test_size,
            purge_outer=purge_outer,
            seed_base=SEED + seed_offset + model_i * 100
        )

        preds_by_model[model_name] = preds
        leakage_frames.append(leakage)
        summary_rows.append(
            summarize_prediction_frame(preds, model_name, experiment_name)
        )

    summary = pd.DataFrame(summary_rows)
    leakage = pd.concat(leakage_frames, ignore_index=True)

    return preds_by_model, summary, leakage

Matched random initialization across walk-forward experiments

The same seed sequence is used for corresponding model/fold combinations in Steps 2 and 3. This keeps random initialization and dropout draws aligned across the unpurged and purged experiments, so the before/after comparison is driven by the change in the training sample rather than by a different stochastic fit.

Step 2a: Walk Forward: 500 train / 500 test, unpurged

step2a_predictions, step2a_summary, step2a_leakage = run_walk_forward_experiment(
    train_size=500,
    test_size=500,
    purge_outer=False,
    experiment_name="Step 2a: WF 500/500 (Leaky)",
    seed_offset=0
)

display(step2a_summary)
display(
    step2a_leakage.groupby("Model")[
        ["Overlapping Train Labels Before", "Overlapping Train Labels After"]
    ].mean()
)

plot_strategy_comparison(
    step2a_predictions,
    "Step 2a: Walk-Forward 500/500 — Unpurged"
)

Step 2a: WF 500/500 (Leaky): training MLP...
Step 2a: WF 500/500 (Leaky): training LSTM...
Step 2a: WF 500/500 (Leaky): training CNN-GAF...

	Experiment	Model	OOS Observations	MAE	MSE	RMSE	Directional Accuracy	Total Return	Annualized Return	Annualized Volatility	Sharpe Ratio	Maximum Drawdown	Buy-and-Hold Total Return	Buy-and-Hold Sharpe
0	Step 2a: WF 500/500 (Leaky)	MLP	1000	0.073844	0.009939	0.099697	0.471000	0.098616	0.034925	0.420531	0.293061	-0.646030	1.052911	0.760040
1	Step 2a: WF 500/500 (Leaky)	LSTM	1000	0.070438	0.009324	0.096560	0.479000	-0.022912	-0.008425	0.441511	0.202763	-0.661382	1.052911	0.760040
2	Step 2a: WF 500/500 (Leaky)	CNN-GAF	1000	0.067587	0.008767	0.093634	0.545000	1.052911	0.300215	0.531734	0.760040	-0.661382	1.052911	0.760040

	Overlapping Train Labels Before	Overlapping Train Labels After
Model
CNN-GAF	9.000000	9.000000
LSTM	9.000000	9.000000
MLP	9.000000	9.000000

Step 2b: Walk Forward: 500 train / 100 test, unpurged

step2b_predictions, step2b_summary, step2b_leakage = run_walk_forward_experiment(
    train_size=500,
    test_size=100,
    purge_outer=False,
    experiment_name="Step 2b: WF 500/100 (Leaky)",
    seed_offset=0
)

display(step2b_summary)
display(
    step2b_leakage.groupby("Model")[
        ["Overlapping Train Labels Before", "Overlapping Train Labels After"]
    ].mean()
)

plot_strategy_comparison(
    step2b_predictions,
    "Step 2b: Walk-Forward 500/100 — Unpurged"
)

Step 2b: WF 500/100 (Leaky): training MLP...
Step 2b: WF 500/100 (Leaky): training LSTM...
Step 2b: WF 500/100 (Leaky): training CNN-GAF...

	Experiment	Model	OOS Observations	MAE	MSE	RMSE	Directional Accuracy	Total Return	Annualized Return	Annualized Volatility	Sharpe Ratio	Maximum Drawdown	Buy-and-Hold Total Return	Buy-and-Hold Sharpe
0	Step 2b: WF 500/100 (Leaky)	MLP	1300	0.066808	0.008283	0.091008	0.513846	1.054624	0.224068	0.401884	0.704251	-0.532509	1.399657	0.738030
1	Step 2b: WF 500/100 (Leaky)	LSTM	1300	0.064627	0.008005	0.089470	0.508462	0.852238	0.188943	0.429751	0.617855	-0.592095	1.399657	0.738030
2	Step 2b: WF 500/100 (Leaky)	CNN-GAF	1300	0.063244	0.007701	0.087756	0.541538	0.811174	0.181482	0.446164	0.597092	-0.645881	1.399657	0.738030

	Overlapping Train Labels Before	Overlapping Train Labels After
Model
CNN-GAF	9.000000	9.000000
LSTM	9.000000	9.000000
MLP	9.000000	9.000000

Step 2c: Comparison with Step 1

The table below compares the single-split baseline with both walk-forward designs for each architecture.

The main question is whether the conclusions from Step 1 remain stable when the models are re-estimated and tested across several chronological periods. A material change in predictive error or trading performance would indicate that the single split gives an incomplete picture of out-of-sample behaviour.

step2_vs_step1 = pd.concat(
    [step1_summary, step2a_summary, step2b_summary],
    ignore_index=True
)

display(
    step2_vs_step1[
        [
            "Experiment", "Model", "OOS Observations",
            "MSE", "Directional Accuracy",
            "Annualized Return", "Sharpe Ratio", "Maximum Drawdown"
        ]
    ]
)

for model_name in MODEL_NAMES:
    sub = step2_vs_step1[step2_vs_step1["Model"] == model_name].set_index("Experiment")
    print(f"\n{model_name}")
    for experiment, row in sub.iterrows():
        print(
            f"  {experiment}: "
            f"MSE={row['MSE']:.6f}, "
            f"Sharpe={row['Sharpe Ratio']:.3f}, "
            f"Annualized Return={row['Annualized Return']:.2%}"
        )

	Experiment	Model	OOS Observations	MSE	Directional Accuracy	Annualized Return	Sharpe Ratio	Maximum Drawdown
0	Step 1: Single Split (Leaky)	MLP	372	0.003997	0.481183	0.040302	0.284285	-0.263787
1	Step 1: Single Split (Leaky)	LSTM	372	0.003687	0.473118	-0.148943	-0.167980	-0.321470
2	Step 1: Single Split (Leaky)	CNN-GAF	372	0.003804	0.473118	-0.148943	-0.167980	-0.321470
3	Step 2a: WF 500/500 (Leaky)	MLP	1000	0.009939	0.471000	0.034925	0.293061	-0.646030
4	Step 2a: WF 500/500 (Leaky)	LSTM	1000	0.009324	0.479000	-0.008425	0.202763	-0.661382
5	Step 2a: WF 500/500 (Leaky)	CNN-GAF	1000	0.008767	0.545000	0.300215	0.760040	-0.661382
6	Step 2b: WF 500/100 (Leaky)	MLP	1300	0.008283	0.513846	0.224068	0.704251	-0.532509
7	Step 2b: WF 500/100 (Leaky)	LSTM	1300	0.008005	0.508462	0.188943	0.617855	-0.592095
8	Step 2b: WF 500/100 (Leaky)	CNN-GAF	1300	0.007701	0.541538	0.181482	0.597092	-0.645881

MLP
  Step 1: Single Split (Leaky): MSE=0.003997, Sharpe=0.284, Annualized Return=4.03%
  Step 2a: WF 500/500 (Leaky): MSE=0.009939, Sharpe=0.293, Annualized Return=3.49%
  Step 2b: WF 500/100 (Leaky): MSE=0.008283, Sharpe=0.704, Annualized Return=22.41%

LSTM
  Step 1: Single Split (Leaky): MSE=0.003687, Sharpe=-0.168, Annualized Return=-14.89%
  Step 2a: WF 500/500 (Leaky): MSE=0.009324, Sharpe=0.203, Annualized Return=-0.84%
  Step 2b: WF 500/100 (Leaky): MSE=0.008005, Sharpe=0.618, Annualized Return=18.89%

CNN-GAF
  Step 1: Single Split (Leaky): MSE=0.003804, Sharpe=-0.168, Annualized Return=-14.89%
  Step 2a: WF 500/500 (Leaky): MSE=0.008767, Sharpe=0.760, Annualized Return=30.02%
  Step 2b: WF 500/100 (Leaky): MSE=0.007701, Sharpe=0.597, Annualized Return=18.15%

Step 2d: 500/500 versus 500/100

The 500/100 design retrains the models more frequently and therefore creates more train/test boundaries. With overlapping 10-day labels, this also creates more occasions on which boundary leakage can arise.

The two window choices are compared using predictive error, risk-adjusted performance, and the measured number of overlapping labels. No direction of performance change is assumed in advance.

step2_window_comparison = pd.concat(
    [step2a_summary, step2b_summary],
    ignore_index=True
)

display(
    step2_window_comparison[
        [
            "Experiment", "Model", "OOS Observations",
            "MSE", "RMSE", "Directional Accuracy",
            "Total Return", "Annualized Return",
            "Sharpe Ratio", "Maximum Drawdown"
        ]
    ]
)

step2_leakage_comparison = pd.concat(
    [
        step2a_leakage.assign(Experiment="500/500"),
        step2b_leakage.assign(Experiment="500/100")
    ],
    ignore_index=True
)

display(
    step2_leakage_comparison.groupby(["Experiment", "Model"])[
        ["Overlapping Train Labels Before", "Overlapping Train Labels After"]
    ].agg(["mean", "max"])
)

	Experiment	Model	OOS Observations	MSE	RMSE	Directional Accuracy	Total Return	Annualized Return	Sharpe Ratio	Maximum Drawdown
0	Step 2a: WF 500/500 (Leaky)	MLP	1000	0.009939	0.099697	0.471000	0.098616	0.034925	0.293061	-0.646030
1	Step 2a: WF 500/500 (Leaky)	LSTM	1000	0.009324	0.096560	0.479000	-0.022912	-0.008425	0.202763	-0.661382
2	Step 2a: WF 500/500 (Leaky)	CNN-GAF	1000	0.008767	0.093634	0.545000	1.052911	0.300215	0.760040	-0.661382
3	Step 2b: WF 500/100 (Leaky)	MLP	1300	0.008283	0.091008	0.513846	1.054624	0.224068	0.704251	-0.532509
4	Step 2b: WF 500/100 (Leaky)	LSTM	1300	0.008005	0.089470	0.508462	0.852238	0.188943	0.617855	-0.592095
5	Step 2b: WF 500/100 (Leaky)	CNN-GAF	1300	0.007701	0.087756	0.541538	0.811174	0.181482	0.597092	-0.645881

		Overlapping Train Labels Before		Overlapping Train Labels After
		mean	max	mean	max
Experiment	Model
500/100	CNN-GAF	9.000000	9	9.000000	9
	LSTM	9.000000	9	9.000000	9
	MLP	9.000000	9	9.000000	9
500/500	CNN-GAF	9.000000	9	9.000000	9
	LSTM	9.000000	9	9.000000	9
	MLP	9.000000	9	9.000000	9

Step 3a: Leakage reduction by purging

For each walk-forward fold, training observations are removed when their label interval overlaps the first test-label interval. Formally, a training observation (i) is retained only when

$ t_{i,1}<t_{j,0}, $

where $t_{i,1}$ is the end of its label window and $t_{j,0}$ is the start of the first test label.

This is the purging procedure described in the course material: observations whose labels use information contained in the test period are excluded from the training sample.

An embargo is not required in this specific rolling design because every training block lies entirely before its corresponding test block; there are no post-test training observations in the same fold.

Step 3b: Purged Walk Forward: 500 train / 500 test

step3b_predictions, step3b_summary, step3b_leakage = run_walk_forward_experiment(
    train_size=500,
    test_size=500,
    purge_outer=True,
    experiment_name="Step 3b: WF 500/500 (Purged)",
    seed_offset=0
)

display(step3b_summary)
display(
    step3b_leakage.groupby("Model")[
        [
            "Raw Train Size", "Effective Train Size",
            "Overlapping Train Labels Before",
            "Overlapping Train Labels After"
        ]
    ].mean()
)

plot_strategy_comparison(
    step3b_predictions,
    "Step 3b: Walk-Forward 500/500 — Purged"
)

Step 3b: WF 500/500 (Purged): training MLP...
Step 3b: WF 500/500 (Purged): training LSTM...
Step 3b: WF 500/500 (Purged): training CNN-GAF...

	Experiment	Model	OOS Observations	MAE	MSE	RMSE	Directional Accuracy	Total Return	Annualized Return	Annualized Volatility	Sharpe Ratio	Maximum Drawdown	Buy-and-Hold Total Return	Buy-and-Hold Sharpe
0	Step 3b: WF 500/500 (Purged)	MLP	1000	0.073060	0.009776	0.098874	0.472000	-0.079927	-0.029948	0.421325	0.140091	-0.650671	1.052911	0.760040
1	Step 3b: WF 500/500 (Purged)	LSTM	1000	0.070803	0.009391	0.096908	0.475000	-0.175159	-0.067873	0.436031	0.058218	-0.661382	1.052911	0.760040
2	Step 3b: WF 500/500 (Purged)	CNN-GAF	1000	0.065551	0.008310	0.091161	0.543000	1.140282	0.320146	0.526281	0.791377	-0.689396	1.052911	0.760040

	Raw Train Size	Effective Train Size	Overlapping Train Labels Before	Overlapping Train Labels After
Model
CNN-GAF	500.000000	491.000000	9.000000	0.000000
LSTM	500.000000	491.000000	9.000000	0.000000
MLP	500.000000	491.000000	9.000000	0.000000

Step 3c: Purged Walk Forward: 500 train / 100 test

step3c_predictions, step3c_summary, step3c_leakage = run_walk_forward_experiment(
    train_size=500,
    test_size=100,
    purge_outer=True,
    experiment_name="Step 3c: WF 500/100 (Purged)",
    seed_offset=0
)

display(step3c_summary)
display(
    step3c_leakage.groupby("Model")[
        [
            "Raw Train Size", "Effective Train Size",
            "Overlapping Train Labels Before",
            "Overlapping Train Labels After"
        ]
    ].mean()
)

plot_strategy_comparison(
    step3c_predictions,
    "Step 3c: Walk-Forward 500/100 — Purged"
)

Step 3c: WF 500/100 (Purged): training MLP...
Step 3c: WF 500/100 (Purged): training LSTM...
Step 3c: WF 500/100 (Purged): training CNN-GAF...

	Experiment	Model	OOS Observations	MAE	MSE	RMSE	Directional Accuracy	Total Return	Annualized Return	Annualized Volatility	Sharpe Ratio	Maximum Drawdown	Buy-and-Hold Total Return	Buy-and-Hold Sharpe
0	Step 3c: WF 500/100 (Purged)	MLP	1300	0.066654	0.008266	0.090918	0.520000	1.090251	0.229991	0.401265	0.716696	-0.530241	1.399657	0.738030
1	Step 3c: WF 500/100 (Purged)	LSTM	1300	0.066263	0.008610	0.092793	0.508462	0.825554	0.184109	0.429517	0.608472	-0.592095	1.399657	0.738030
2	Step 3c: WF 500/100 (Purged)	CNN-GAF	1300	0.063279	0.007594	0.087143	0.535385	0.441486	0.108126	0.446693	0.453502	-0.675178	1.399657	0.738030

	Raw Train Size	Effective Train Size	Overlapping Train Labels Before	Overlapping Train Labels After
Model
CNN-GAF	500.000000	491.000000	9.000000	0.000000
LSTM	500.000000	491.000000	9.000000	0.000000
MLP	500.000000	491.000000	9.000000	0.000000

Step 3d: Purged versus unpurged results

The comparison below addresses whether the apparent overfitting associated with overlapping labels is reduced after purging.

Two pieces of evidence are used:

1. the overlap audit confirms whether any training-label interval still enters the test-label period;
2. predictive and trading results are compared with the otherwise identical unpurged walk-forward experiments.

Purging removes a known source of contamination, but it does not imply that performance must fall mechanically. The relevant question is whether the measured results remain stable once the overlap has been removed.

step3_comparison = pd.concat(
    [
        step2a_summary,
        step3b_summary,
        step2b_summary,
        step3c_summary
    ],
    ignore_index=True
)

display(
    step3_comparison[
        [
            "Experiment", "Model", "OOS Observations",
            "MSE", "Directional Accuracy",
            "Total Return", "Annualized Return",
            "Sharpe Ratio", "Maximum Drawdown"
        ]
    ]
)

# Direct before/after changes for each window and model.
pairs = [
    ("500/500", step2a_summary, step3b_summary),
    ("500/100", step2b_summary, step3c_summary),
]

change_rows = []
for window, leaky_summary, purged_summary in pairs:
    left = leaky_summary.set_index("Model")
    right = purged_summary.set_index("Model")

    for model_name in MODEL_NAMES:
        change_rows.append({
            "Window": window,
            "Model": model_name,
            "Leaky MSE": left.loc[model_name, "MSE"],
            "Purged MSE": right.loc[model_name, "MSE"],
            "Change in MSE (%)":
                (right.loc[model_name, "MSE"] / left.loc[model_name, "MSE"] - 1.0) * 100,
            "Leaky Sharpe": left.loc[model_name, "Sharpe Ratio"],
            "Purged Sharpe": right.loc[model_name, "Sharpe Ratio"],
            "Change in Sharpe":
                right.loc[model_name, "Sharpe Ratio"] - left.loc[model_name, "Sharpe Ratio"],
            "Leaky Annual Return": left.loc[model_name, "Annualized Return"],
            "Purged Annual Return": right.loc[model_name, "Annualized Return"],
        })

purging_effect = pd.DataFrame(change_rows)
display(purging_effect)

print("\nLeakage verification after purging:")
for name, leakage_df in {
    "Step 3b 500/500": step3b_leakage,
    "Step 3c 500/100": step3c_leakage,
}.items():
    print(
        f"{name}: maximum overlapping training labels after purge = "
        f"{leakage_df['Overlapping Train Labels After'].max()}"
    )

	Experiment	Model	OOS Observations	MSE	Directional Accuracy	Total Return	Annualized Return	Sharpe Ratio	Maximum Drawdown
0	Step 2a: WF 500/500 (Leaky)	MLP	1000	0.009939	0.471000	0.098616	0.034925	0.293061	-0.646030
1	Step 2a: WF 500/500 (Leaky)	LSTM	1000	0.009324	0.479000	-0.022912	-0.008425	0.202763	-0.661382
2	Step 2a: WF 500/500 (Leaky)	CNN-GAF	1000	0.008767	0.545000	1.052911	0.300215	0.760040	-0.661382
3	Step 3b: WF 500/500 (Purged)	MLP	1000	0.009776	0.472000	-0.079927	-0.029948	0.140091	-0.650671
4	Step 3b: WF 500/500 (Purged)	LSTM	1000	0.009391	0.475000	-0.175159	-0.067873	0.058218	-0.661382
5	Step 3b: WF 500/500 (Purged)	CNN-GAF	1000	0.008310	0.543000	1.140282	0.320146	0.791377	-0.689396
6	Step 2b: WF 500/100 (Leaky)	MLP	1300	0.008283	0.513846	1.054624	0.224068	0.704251	-0.532509
7	Step 2b: WF 500/100 (Leaky)	LSTM	1300	0.008005	0.508462	0.852238	0.188943	0.617855	-0.592095
8	Step 2b: WF 500/100 (Leaky)	CNN-GAF	1300	0.007701	0.541538	0.811174	0.181482	0.597092	-0.645881
9	Step 3c: WF 500/100 (Purged)	MLP	1300	0.008266	0.520000	1.090251	0.229991	0.716696	-0.530241
10	Step 3c: WF 500/100 (Purged)	LSTM	1300	0.008610	0.508462	0.825554	0.184109	0.608472	-0.592095
11	Step 3c: WF 500/100 (Purged)	CNN-GAF	1300	0.007594	0.535385	0.441486	0.108126	0.453502	-0.675178

	Window	Model	Leaky MSE	Purged MSE	Change in MSE (%)	Leaky Sharpe	Purged Sharpe	Change in Sharpe	Leaky Annual Return	Purged Annual Return
0	500/500	MLP	0.009939	0.009776	-1.643265	0.293061	0.140091	-0.152970	0.034925	-0.029948
1	500/500	LSTM	0.009324	0.009391	0.721912	0.202763	0.058218	-0.144544	-0.008425	-0.067873
2	500/500	CNN-GAF	0.008767	0.008310	-5.212582	0.760040	0.791377	0.031337	0.300215	0.320146
3	500/100	MLP	0.008283	0.008266	-0.198468	0.704251	0.716696	0.012445	0.224068	0.229991
4	500/100	LSTM	0.008005	0.008610	7.566399	0.617855	0.608472	-0.009383	0.188943	0.184109
5	500/100	CNN-GAF	0.007701	0.007594	-1.391511	0.597092	0.453502	-0.143590	0.181482	0.108126

Leakage verification after purging:
Step 3b 500/500: maximum overlapping training labels after purge = 0
Step 3c 500/100: maximum overlapping training labels after purge = 0

Consolidated comparison — Steps 1 to 3

Across the full experiment, no architecture dominates under every validation design. CNN-GAF is strongest under the purged 500/500 specification, while the MLP is strongest under purged 500/100. The LSTM generally produces competitive forecast errors but does not convert that advantage into the strongest trading results.

The main finding is therefore about validation rather than model ranking. The same underlying prediction problem produces materially different economic conclusions under a single split, rolling windows, and purged rolling windows. This is precisely why the treatment of overlapping financial labels is part of the model-evaluation problem rather than a separate implementation detail.

all_results = pd.concat(
    [
        step1_summary,
        step2a_summary,
        step2b_summary,
        step3b_summary,
        step3c_summary
    ],
    ignore_index=True
)

display(
    all_results[
        [
            "Experiment", "Model", "OOS Observations",
            "MAE", "MSE", "RMSE", "Directional Accuracy",
            "Total Return", "Annualized Return",
            "Annualized Volatility", "Sharpe Ratio",
            "Maximum Drawdown",
            "Buy-and-Hold Total Return", "Buy-and-Hold Sharpe"
        ]
    ]
)

# Export consolidated tables for the report.
all_results.to_csv("gwp2_all_results.csv", index=False)
purging_effect.to_csv("gwp2_purging_effect.csv", index=False)

print("Saved optional report-support tables:")
print(" - gwp2_all_results.csv")
print(" - gwp2_purging_effect.csv")

	Experiment	Model	OOS Observations	MAE	MSE	RMSE	Directional Accuracy	Total Return	Annualized Return	Annualized Volatility	Sharpe Ratio	Maximum Drawdown	Buy-and-Hold Total Return	Buy-and-Hold Sharpe
0	Step 1: Single Split (Leaky)	MLP	372	0.049896	0.003997	0.063223	0.481183	0.041090	0.040302	0.331878	0.284285	-0.263787	-0.151571	-0.167980
1	Step 1: Single Split (Leaky)	LSTM	372	0.049005	0.003687	0.060718	0.473118	-0.151571	-0.148943	0.424685	-0.167980	-0.321470	-0.151571	-0.167980
2	Step 1: Single Split (Leaky)	CNN-GAF	372	0.049785	0.003804	0.061680	0.473118	-0.151571	-0.148943	0.424685	-0.167980	-0.321470	-0.151571	-0.167980
3	Step 2a: WF 500/500 (Leaky)	MLP	1000	0.073844	0.009939	0.099697	0.471000	0.098616	0.034925	0.420531	0.293061	-0.646030	1.052911	0.760040
4	Step 2a: WF 500/500 (Leaky)	LSTM	1000	0.070438	0.009324	0.096560	0.479000	-0.022912	-0.008425	0.441511	0.202763	-0.661382	1.052911	0.760040
5	Step 2a: WF 500/500 (Leaky)	CNN-GAF	1000	0.067587	0.008767	0.093634	0.545000	1.052911	0.300215	0.531734	0.760040	-0.661382	1.052911	0.760040
6	Step 2b: WF 500/100 (Leaky)	MLP	1300	0.066808	0.008283	0.091008	0.513846	1.054624	0.224068	0.401884	0.704251	-0.532509	1.399657	0.738030
7	Step 2b: WF 500/100 (Leaky)	LSTM	1300	0.064627	0.008005	0.089470	0.508462	0.852238	0.188943	0.429751	0.617855	-0.592095	1.399657	0.738030
8	Step 2b: WF 500/100 (Leaky)	CNN-GAF	1300	0.063244	0.007701	0.087756	0.541538	0.811174	0.181482	0.446164	0.597092	-0.645881	1.399657	0.738030
9	Step 3b: WF 500/500 (Purged)	MLP	1000	0.073060	0.009776	0.098874	0.472000	-0.079927	-0.029948	0.421325	0.140091	-0.650671	1.052911	0.760040
10	Step 3b: WF 500/500 (Purged)	LSTM	1000	0.070803	0.009391	0.096908	0.475000	-0.175159	-0.067873	0.436031	0.058218	-0.661382	1.052911	0.760040
11	Step 3b: WF 500/500 (Purged)	CNN-GAF	1000	0.065551	0.008310	0.091161	0.543000	1.140282	0.320146	0.526281	0.791377	-0.689396	1.052911	0.760040
12	Step 3c: WF 500/100 (Purged)	MLP	1300	0.066654	0.008266	0.090918	0.520000	1.090251	0.229991	0.401265	0.716696	-0.530241	1.399657	0.738030
13	Step 3c: WF 500/100 (Purged)	LSTM	1300	0.066263	0.008610	0.092793	0.508462	0.825554	0.184109	0.429517	0.608472	-0.592095	1.399657	0.738030
14	Step 3c: WF 500/100 (Purged)	CNN-GAF	1300	0.063279	0.007594	0.087143	0.535385	0.441486	0.108126	0.446693	0.453502	-0.675178	1.399657	0.738030

Saved optional report-support tables:
 - gwp2_all_results.csv
 - gwp2_purging_effect.csv

# Compact visual summary of Sharpe ratios across all experiments.
sharpe_table = all_results.pivot(
    index="Experiment",
    columns="Model",
    values="Sharpe Ratio"
)

display(sharpe_table)

ax = sharpe_table.plot(kind="bar", figsize=(12, 6))
ax.set_title("Strategy Sharpe Ratio by Validation Design and Model")
ax.set_xlabel("Validation Design")
ax.set_ylabel("Sharpe Ratio")
ax.axhline(0, linewidth=1)
plt.xticks(rotation=25, ha="right")
plt.tight_layout()
plt.show()

Model	CNN-GAF	LSTM	MLP
Experiment
Step 1: Single Split (Leaky)	-0.167980	-0.167980	0.284285
Step 2a: WF 500/500 (Leaky)	0.760040	0.202763	0.293061
Step 2b: WF 500/100 (Leaky)	0.597092	0.617855	0.704251
Step 3b: WF 500/500 (Purged)	0.791377	0.058218	0.140091
Step 3c: WF 500/100 (Purged)	0.453502	0.608472	0.716696