$npx -y skills add agiprolabs/claude-trading-skills --skill signal-classificationML trading signal classifiers using XGBoost and LightGBM with walk-forward validation, SHAP feature importance, and threshold optimization
| 1 | # Signal Classification |
| 2 | |
| 3 | Predict whether an asset's price will move up or down over a forward horizon using supervised machine learning classifiers. This skill covers the full pipeline: label creation, model training, walk-forward validation, feature importance analysis, and threshold optimization for trading applications. |
| 4 | |
| 5 | ## Why Tree-Based Models Dominate Trading ML |
| 6 | |
| 7 | XGBoost and LightGBM are the workhorses of quantitative trading ML for good reason: |
| 8 | |
| 9 | - **Non-linear relationships**: Financial features interact in complex, non-linear ways that trees capture naturally |
| 10 | - **Robust to feature scale**: No need to normalize or standardize inputs — trees split on rank order |
| 11 | - **Built-in feature importance**: Understand which features drive predictions without separate analysis |
| 12 | - **Fast training and inference**: Train on thousands of samples in seconds, predict in microseconds |
| 13 | - **Handle missing values**: Native support for NaN without imputation hacks |
| 14 | - **Regularization built in**: max_depth, min_child_weight, subsample all prevent overfitting |
| 15 | |
| 16 | Linear models and deep learning have their place, but for tabular trading features with fewer than 100k samples, gradient-boosted trees consistently outperform alternatives. |
| 17 | |
| 18 | ## Classification Types |
| 19 | |
| 20 | ### Binary Classification |
| 21 | |
| 22 | The simplest and most common setup. Predict whether forward returns exceed a threshold: |
| 23 | |
| 24 | - **Up signal**: forward return > +1% |
| 25 | - **Down signal**: forward return < -1% |
| 26 | - **Neutral (excluded)**: -1% to +1% — drop these from training to create cleaner labels |
| 27 | |
| 28 | ```python |
| 29 | import numpy as np |
| 30 | |
| 31 | def create_binary_labels( |
| 32 | prices: np.ndarray, horizon: int = 24, threshold: float = 0.01 |
| 33 | ) -> np.ndarray: |
| 34 | """Create binary labels from forward returns. |
| 35 | |
| 36 | Args: |
| 37 | prices: Array of prices. |
| 38 | horizon: Forward return lookback in bars. |
| 39 | threshold: Minimum return magnitude for a label. |
| 40 | |
| 41 | Returns: |
| 42 | Array of labels: 1 (up), 0 (down), NaN (neutral). |
| 43 | """ |
| 44 | fwd_returns = np.roll(prices, -horizon) / prices - 1 |
| 45 | fwd_returns[-horizon:] = np.nan |
| 46 | labels = np.where(fwd_returns > threshold, 1, |
| 47 | np.where(fwd_returns < -threshold, 0, np.nan)) |
| 48 | return labels |
| 49 | ``` |
| 50 | |
| 51 | ### Multi-Class Classification |
| 52 | |
| 53 | Three classes for finer signal granularity: |
| 54 | |
| 55 | | Class | Condition | Typical threshold | |
| 56 | |-------|-----------|-------------------| |
| 57 | | Strong Up | fwd_return > +2% | High confidence long | |
| 58 | | Mild Up | +0.5% to +2% | Moderate confidence | |
| 59 | | Down | fwd_return < -0.5% | Avoid / short | |
| 60 | |
| 61 | Multi-class reduces per-class sample size. Use only with large datasets (1000+ samples per class). |
| 62 | |
| 63 | ### Probability Calibration |
| 64 | |
| 65 | Raw model probabilities from XGBoost/LightGBM are not well-calibrated. A predicted 0.7 probability does not mean 70% chance of being correct. Use calibration to fix this: |
| 66 | |
| 67 | ```python |
| 68 | from sklearn.calibration import CalibratedClassifierCV |
| 69 | |
| 70 | calibrated = CalibratedClassifierCV(base_model, cv=5, method="isotonic") |
| 71 | calibrated.fit(X_train, y_train) |
| 72 | probs = calibrated.predict_proba(X_test)[:, 1] |
| 73 | ``` |
| 74 | |
| 75 | Isotonic calibration works better than Platt scaling for tree models. |
| 76 | |
| 77 | ## Walk-Forward Validation |
| 78 | |
| 79 | **This is the single most important concept in trading ML.** Standard cross-validation randomly shuffles data, which creates lookahead bias. Walk-forward validation respects time ordering. |
| 80 | |
| 81 | ### How It Works |
| 82 | |
| 83 | ``` |
| 84 | Window 1: [===TRAIN===][GAP][=TEST=] |
| 85 | Window 2: [===TRAIN===][GAP][=TEST=] |
| 86 | Window 3: [===TRAIN===][GAP][=TEST=] |
| 87 | Window 4: [===TRAIN===][GAP][=TEST=] |
| 88 | ``` |
| 89 | |
| 90 | Each window: |
| 91 | 1. Train on past N bars |
| 92 | 2. Skip a gap (embargo) equal to the forward return horizon |
| 93 | 3. Predict on next M bars |
| 94 | 4. Record out-of-sample predictions |
| 95 | 5. Slide forward and repeat |
| 96 | |
| 97 | ### Typical Parameters |
| 98 | |
| 99 | | Parameter | Value | Rationale | |
| 100 | |-----------|-------|-----------| |
| 101 | | Train window | 30 days (720 hourly bars) | Enough data to learn, recent enough to be relevant | |
| 102 | | Test window | 7 days (168 hourly bars) | Enough predictions for statistical significance | |
| 103 | | Step size | 1 day (24 bars) | Overlap test windows for more data points | |
| 104 | | Gap (embargo) | Same as forward horizon | Prevents label leakage | |
| 105 | |
| 106 | ### Walk-Forward Implementation |
| 107 | |
| 108 | ```python |
| 109 | from typing import Iterator |
| 110 | |
| 111 | def walk_forward_splits( |
| 112 | n_samples: int, |
| 113 | train_size: int = 720, |
| 114 | test_size: int = 168, |
| 115 | step_size: int = 24, |
| 116 | gap: int = 24, |
| 117 | ) -> Iterator[tuple[np.ndarray, np.ndarray]]: |
| 118 | """Generate walk-forward train/test index splits. |
| 119 | |
| 120 | Args: |
| 121 | n_samples: Total number of samples. |
| 122 | train_size: Number of training samples per window. |
| 123 | test_size: Number of test samples per window. |
| 124 | step_size: Step between successive windows. |
| 125 | gap: Gap between train end and test start. |
| 126 | |
| 127 | Yields: |
| 128 | Tuples of (train_indices, test_indices). |
| 129 | """ |
| 130 | start = 0 |
| 131 | while start + tr |