factor-evaluation

Name: factor-evaluation
Author: minihellboy

95stars28forks

How to install factor-evaluation?

minihellboy/factorminer/factor-evaluation

$npx -y skills add minihellboy/factorminer --skill factor-evaluation

Installs into the current project.

Summary

Evaluate a factor library — recompute Information Coefficient (IC), ICIR, win rate, and turnover on held-out data, and surface train→test decay. Use to judge how good a mined library actually is out of sample. Triggers on "evaluate factors", "compute IC", "how good is this librar

Files · 1

View on GitHub

SKILL.md

1	# Factor Evaluation
2
3	Mining proposes factors; evaluation decides whether to believe them. This skill recomputes a library's metrics on a chosen split and exposes overfitting.
4
5	See `references/metrics.md` for precise metric definitions (IC vs. paper-IC, ICIR, redundancy correlation).
6
7	## Workflow
8
9	### 1. Recompute metrics
10
11	```bash
12	factorminer evaluate output/run1/factor_library.json \
13	--data path/to/market_data.csv \
14	--period test
15	```
16
17	`--period` selects the split: `train`, `test`, or `both`. Always lead with `test` — in-sample IC is not evidence.
18
19	### 2. Read the table
20
21	The output table reports, per factor: `IC Mean`, `Paper IC`, `Abs IC`, `Paper ICIR`, `Win%`, and `Turnover`. The summary block gives library-level means and the IC range.
22
23	### 3. Check decay
24
25	```bash
26	factorminer evaluate output/run1/factor_library.json --data market_data.csv --period both
27	```
28
29	`--period both` adds a decay table (train Paper IC → test Paper IC → delta). A large negative delta is the signature of an overfit factor. Report decay honestly; do not quote the train number as the headline.
30
31	### 4. Rank the survivors
32
33	To shortlist the strongest signals only:
34
35	```bash
36	factorminer evaluate output/run1/factor_library.json --data market_data.csv --period test --top-k 10
37	```
38
39	The top-K-by-IC table is the signal shortlist — the natural handoff to a research-idea workflow that wants to know which quantitative signals are currently working. The MCP `screen_factors` tool returns this same shortlist directly.
40
41	## Interpreting the numbers
42
43	- IC ≈ 0.03–0.05 out of sample is a respectable single factor on liquid universes.
44	- ICIR matters more than IC: a small but stable IC beats a large erratic one.
45	- High turnover quietly erases IC once costs are applied — carry it into `factor-backtest`.
46
47	## Guardrails
48
49	- Never present `train` metrics as the result. The deliverable is the `test` number.
50	- If every factor decays to ~0 on test, the library failed — say so. Do not search for a split that flatters it.

People who liked this also viewed

6 picks

Type

factor-evaluation

95stars28forks

View on GitHub

How to install factor-evaluation?

minihellboy/factorminer/factor-evaluation

$npx -y skills add minihellboy/factorminer --skill factor-evaluation

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Factor Evaluation
2
3	Mining proposes factors; evaluation decides whether to believe them. This skill recomputes a library's metrics on a chosen split and exposes overfitting.
4
5	See `references/metrics.md` for precise metric definitions (IC vs. paper-IC, ICIR, redundancy correlation).
6
7	## Workflow
8
9	### 1. Recompute metrics
10
11	```bash
12	factorminer evaluate output/run1/factor_library.json \
13	--data path/to/market_data.csv \
14	--period test
15	```
16
17	`--period` selects the split: `train`, `test`, or `both`. Always lead with `test` — in-sample IC is not evidence.
18
19	### 2. Read the table
20
21	The output table reports, per factor: `IC Mean`, `Paper IC`, `Abs IC`, `Paper ICIR`, `Win%`, and `Turnover`. The summary block gives library-level means and the IC range.
22
23	### 3. Check decay
24
25	```bash
26	factorminer evaluate output/run1/factor_library.json --data market_data.csv --period both
27	```
28
29	`--period both` adds a decay table (train Paper IC → test Paper IC → delta). A large negative delta is the signature of an overfit factor. Report decay honestly; do not quote the train number as the headline.
30
31	### 4. Rank the survivors
32
33	To shortlist the strongest signals only:
34
35	```bash
36	factorminer evaluate output/run1/factor_library.json --data market_data.csv --period test --top-k 10
37	```
38
39	The top-K-by-IC table is the signal shortlist — the natural handoff to a research-idea workflow that wants to know which quantitative signals are currently working. The MCP `screen_factors` tool returns this same shortlist directly.
40
41	## Interpreting the numbers
42
43	- IC ≈ 0.03–0.05 out of sample is a respectable single factor on liquid universes.
44	- ICIR matters more than IC: a small but stable IC beats a large erratic one.
45	- High turnover quietly erases IC once costs are applied — carry it into `factor-backtest`.
46
47	## Guardrails
48
49	- Never present `train` metrics as the result. The deliverable is the `test` number.
50	- If every factor decays to ~0 on test, the library failed — say so. Do not search for a split that flatters it.

People who liked this also viewed

6 picks

Type