meta-harness-proteus
starsforks
How to install meta-harness-proteus?
001tmf/harness-forge/meta-harness-proteus
$npx -y skills add 001TMF/harness-forge --skill meta-harness-proteusRun one iteration of proteus memory-summary evolution. Called by meta_harness.py.
SKILL.md
| 1 | # Meta-Harness — proteus memory-summary evolution |
| 2 | |
| 3 | Run ONE iteration. Do all work in the main session — do NOT delegate to subagents. |
| 4 | |
| 5 | **You do NOT run benchmarks.** You analyze prior results, prototype a mechanism, |
| 6 | and write new candidate summary compressors. The outer loop (`meta_harness.py`) |
| 7 | scores them on (fidelity, chars) separately, with no model and no network. |
| 8 | |
| 9 | ## What a candidate is |
| 10 | |
| 11 | A summary compressor: it turns one campaign-memory record (a dict — see |
| 12 | `corpus.py`) into the short string injected into the policy's context on |
| 13 | retrieval. The proteus analog of a memory system. The grading is in |
| 14 | `corpus.py::score_fidelity`: the fraction of load-bearing facts (target, |
| 15 | surface, strategy, outcome, quality, difficulty, transfer hint) that survive in |
| 16 | your summary. Context cost = `len(summary)`. |
| 17 | |
| 18 | ## The objective |
| 19 | |
| 20 | Preserve fidelity (>= the floor in `config.yaml`, currently 0.70 worst-record) |
| 21 | while using FEWER characters than `agents/baseline_incumbent.py`. The frontier |
| 22 | is Pareto: fidelity up, chars down. You cannot win by dropping facts — a summary |
| 23 | that loses a required fact loses fidelity and falls off the frontier. |
| 24 | |
| 25 | ## CRITICAL CONSTRAINTS |
| 26 | |
| 27 | - Implement exactly **3** new compressors this iteration. |
| 28 | - Each must change a *mechanism*, not a constant. Bad: "same template, drop the |
| 29 | organism." Good ideas: abbreviation/symbol encoding of fixed vocab |
| 30 | (surface types, outcomes); a key:value micro-syntax instead of prose; |
| 31 | dropping only provably-redundant words; reordering so the highest-value facts |
| 32 | survive truncation; field-name elision where the value is self-identifying. |
| 33 | - **No record-specific hints.** Never hardcode a target name, campaign_id, or |
| 34 | any value from `corpus.py` into a compressor. It must generalize to unseen |
| 35 | records. (This is the anti-leakage rule — load-bearing for proteus.) |
| 36 | - Do not abort early or write "the frontier is optimal". |
| 37 | |
| 38 | ## Workflow |
| 39 | |
| 40 | 1. **Analyze.** Read `logs/evolution_summary.jsonl` (what's been tried), |
| 41 | `logs/frontier.json` (current best), `corpus.py` (records + rubric), |
| 42 | `agents/baseline_incumbent.py` (the system to beat). |
| 43 | 2. **Prototype (mandatory).** Write a throwaway script in `/tmp/` that runs your |
| 44 | compression idea over a couple of `corpus.py` records and checks fidelity by |
| 45 | eye before committing. Delete it after. |
| 46 | 3. **Implement.** For each of 3 candidates: copy `agents/baseline_incumbent.py` |
| 47 | to `agents/<snake_name>.py`, subclass `SummaryCompressor`, implement |
| 48 | `summarize(self, record) -> str`. Import from `candidate_base`. Self-critique: |
| 49 | is this a new mechanism or just a tweaked constant? If the latter, rewrite. |
| 50 | 4. **Validate.** `python -c "import agents.<name>; print('OK')"` from the repo root. |
| 51 | 5. **Write `logs/pending_eval.json`:** |
| 52 | |
| 53 | ```json |
| 54 | { |
| 55 | "iteration": <N>, |
| 56 | "candidates": [ |
| 57 | {"name": "<snake_name>", "hypothesis": "<falsifiable claim about fidelity/chars>"} |
| 58 | ] |
| 59 | } |
| 60 | ``` |
| 61 | |
| 62 | Output: `CANDIDATES: <name1>, <name2>, <name3>` |
| 63 | |
| 64 | ## Interface |
| 65 | |
| 66 | ```python |
| 67 | from candidate_base import Record, SummaryCompressor |
| 68 | |
| 69 | class MyCompressor(SummaryCompressor): |
| 70 | def summarize(self, record: Record) -> str: |
| 71 | ... # pure, deterministic, no I/O, no LLM |
| 72 | ``` |
People who liked this also viewed
6 picksType
- lark-openapi-explorer飞书/Lark 原生 OpenAPI 探索：从官方文档库中挖掘未经 CLI 封装的原生 OpenAPI 接口。当用户的需求无法被现有 lark-* skill 或 lark-cli 已注册命令满足，需要查找并调用原生飞书 OpenAPI 时使用。SkillsJul 2026
- lark-skill-maker创建 lark-cli 的自定义 Skill。当用户需要把飞书 API 操作封装成可复用的 Skill（包装原子 API 或编排多步流程）时使用。SkillsJul 2026
- azure-cost-optimizationOfficial agent plugin providing skills and MCP server configurations for Azure scenarios.SkillsJul 2026
- vercel-react-native-skillsVercel's official collection of agent skillsSkillsJul 2026
- mcp-builderGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate externaSkillsJul 2026
- gws-sharedgws CLI: Shared patterns for authentication, global flags, and output formatting.SkillsJul 2026
