LLM-powered quantitative factor mining with evolutionary search
$git clone https://github.com/minihellboy/factorminerInstalls into the current project.
Install factorminer by running `git clone https://github.com/minihellboy/factorminer`, then use it for the current task and follow its documentation at https://github.com/minihellboy/factorminer.
| 1 | # FactorMiner |
| 2 | |
| 3 | **A governed research engine for discovering, evaluating, and documenting |
| 4 | interpretable alpha factors.** |
| 5 | |
| 6 | [](https://www.python.org/downloads/) |
| 7 | [](LICENSE) |
| 8 | [](https://github.com/minihellboy/factorminer/actions/workflows/ci.yml) |
| 9 | |
| 10 | FactorMiner combines a typed formula DSL, LLM-guided search, structured memory, |
| 11 | strict runtime recomputation, and reviewable research artifacts. It follows the |
| 12 | system described in [*FactorMiner: A Self-Evolving Agent with Skills and |
| 13 | Experience Memory for Financial Alpha Discovery*](https://arxiv.org/abs/2602.14670) |
| 14 | and extends it with explicit architecture contracts, stronger validation, and a |
| 15 | model-agnostic agent integration surface. |
| 16 | |
| 17 | FactorMiner is research infrastructure. It proposes and evaluates artifacts; it |
| 18 | does not recommend trades, size positions, bind risk limits, route orders, or |
| 19 | operate an autonomous account. |
| 20 | |
| 21 | ## What is included |
| 22 | |
| 23 | | Surface | Purpose | |
| 24 | | --- | --- | |
| 25 | | Typed DSL and operator registry | Safe, interpretable formulas over OHLCV and registered feature leaves | |
| 26 | | `RalphLoop` | Canonical paper-style generate/evaluate/evolve lane | |
| 27 | | `HelixLoop` | Extended retrieval, debate, canonicalization, and validation lane | |
| 28 | | Policy-based memory | Paper, none, KG, family-, regime-, and edit-aware policies | |
| 29 | | Runtime evaluation | Formula recomputation on the supplied dataset; saved summaries are not trusted as truth | |
| 30 | | Benchmark runtime | Top-K freeze, memory/strategy ablations, CPCV/PBO, cost pressure, and efficiency | |
| 31 | | Research knowledge | Persistent screened sources and hypotheses with bounded retrieval and outcome attribution | |
| 32 | | Evidence packs | Immutable, content-addressed factor evidence with dataset/config/code hashes and integrity verification | |
| 33 | | Research extensions | EDGAR/futures data, crowding, capacity, sensitivity, model-risk, and provenance artifacts | |
| 34 | | Agent gateway | MCP server plus a plugin and managed-agent reference integration | |
| 35 | |
| 36 | The built-in catalog contains 110 normalized paper factors. Named third-party |
| 37 | baselines are not all faithful reproductions; manifests label catalog subsets, |
| 38 | proxies, runtime loops, and saved libraries explicitly. See |
| 39 | [Reproducibility](docs/reproducibility.md) before interpreting benchmark output. |
| 40 | |
| 41 | ## Install |
| 42 | |
| 43 | The recommended contributor setup uses [uv](https://docs.astral.sh/uv/): |
| 44 | |
| 45 | ```bash |
| 46 | git clone https://github.com/minihellboy/factorminer.git |
| 47 | cd factorminer |
| 48 | uv sync --group dev --all-extras |
| 49 | ``` |
| 50 | |
| 51 | For a smaller local environment: |
| 52 | |
| 53 | ```bash |
| 54 | uv sync --group dev |
| 55 | uv sync --group dev --extra llm # add hosted/local LLM providers |
| 56 | uv sync --group dev --extra mcp # add the MCP server |
| 57 | ``` |
| 58 | |
| 59 | The portable default backend is NumPy. The CUDA extra is Linux-oriented; use |
| 60 | `--gpu` only where CUDA is available. A pip editable install also works: |
| 61 | |
| 62 | ```bash |
| 63 | python3 -m pip install -e ".[llm,mcp]" |
| 64 | ``` |
| 65 | |
| 66 | ## Quick start |
| 67 | |
| 68 | No API key is required for the deterministic demo and mock workflow: |
| 69 | |
| 70 | ```bash |
| 71 | uv run python scripts/run_demo.py |
| 72 | uv run factorminer quickstart |
| 73 | uv run factorminer doctor --json |
| 74 | ``` |
| 75 | |
| 76 | `quickstart` writes a small library and static report under |
| 77 | `/tmp/factorminer-quickstart`. To mine directly: |
| 78 | |
| 79 | ```bash |
| 80 | uv run factorminer -o /tmp/factorminer-run mine --mock -n 2 -b 8 -t 10 |
| 81 | uv run factorminer session inspect /tmp/factorminer-run --telemetry |
| 82 | ``` |
| 83 | |
| 84 | For real data, validate the schema first: |
| 85 | |
| 86 | ```bash |
| 87 | uv run factorminer validate-data path/to/market_data.csv |
| 88 | uv run factorminer -c factorminer.local.yaml -o output-real \ |
| 89 | mine --data path/to/market_data.csv |
| 90 | ``` |
| 91 | |
| 92 | The minimum panel fields are: |
| 93 | |
| 94 | ```text |
| 95 | datetime, asset_id, open, high, low, close, volume, amount |
| 96 | ``` |
| 97 | |
| 98 | Identifier aliases such as `symbol`, `ticker`, `code`, and `ts_code` are |
| 99 | accepted. Missing `vwap` and `returns` can be derived by the runtime layer. |
| 100 | |
| 101 | ## Core workflows |
| 102 | |
| 103 | Run the extended research lane: |
| 104 | |
| 105 | ```bash |
| 106 | uv run factorminer --cpu helix --mock --debate --canonicalize -n 2 -b 8 -t 10 |
| 107 | ``` |
| 108 | |
| 109 | Recompute and evaluate a saved library: |
| 110 | |
| 111 | ```bash |
| 112 | uv run factorminer --cpu evaluate output/factor_library.json \ |
| 113 | --mock --period both --top-k 10 |
| 114 | ``` |
| 115 | |
| 116 | Build a composite on explicit fit/evaluation splits: |
| 117 | |
| 118 | ```bash |
| 119 | uv run factorminer --cpu combine output/factor_library.json \ |
| 120 | --mock --fit-period train --eval-period test --method all \ |
| 121 | --selection lasso --top-k 20 |
| 122 | ``` |
| 123 | |
| 124 | Run a benchmark or the standalone Phase 2 report builder: |
| 125 | |
| 126 | ```bash |
| 127 | uv run factorminer --cpu --config factorminer/configs/paper_repro.yaml \ |
| 128 | benchmark table1 --mock --baseline factor_miner |
| 129 | uv run factorminer --cpu benchmark ablation-strategy --mock \ |
| 130 | --baseline factor_miner |
| 131 | uv run python scripts/run_phase2_benchmark.py --mock |
| 132 | ``` |
| 133 | |
| 134 | The CLI also exposes data validation/resampling, visualization, CPCV, portfolio |
| 135 | construction, crowding and sensitivity diagnostics, EDGAR/futures attachment, |
| 136 | research inges |