$npx -y skills add K-Dense-AI/scientific-agent-skills --skill hypogenicAutomated LLM-driven hypothesis generation and testing on tabular datasets. Use when you want to systematically explore hypotheses about patterns in empirical data (e.g., deception detection, content analysis). Combines literature insights with data-driven hypothesis testing. For
| 1 | # Hypogenic |
| 2 | |
| 3 | ## Overview |
| 4 | |
| 5 | Hypogenic provides automated hypothesis generation and testing using large language models to accelerate scientific discovery. The framework supports three approaches: HypoGeniC (data-driven hypothesis generation), HypoRefine (synergistic literature and data integration), and Union methods (mechanistic combination of literature and data-driven hypotheses). |
| 6 | |
| 7 | ## Quick Start |
| 8 | |
| 9 | Get started with Hypogenic in minutes: |
| 10 | |
| 11 | ```bash |
| 12 | # Install the package |
| 13 | uv pip install hypogenic |
| 14 | |
| 15 | # Clone example datasets |
| 16 | git clone https://github.com/ChicagoHAI/HypoGeniC-datasets.git ./data |
| 17 | |
| 18 | # Run basic hypothesis generation |
| 19 | hypogenic_generation --config ./data/your_task/config.yaml --method hypogenic --num_hypotheses 20 |
| 20 | |
| 21 | # Run inference on generated hypotheses |
| 22 | hypogenic_inference --config ./data/your_task/config.yaml --hypotheses output/hypotheses.json |
| 23 | ``` |
| 24 | |
| 25 | **Or use Python API:** |
| 26 | |
| 27 | ```python |
| 28 | from hypogenic import BaseTask |
| 29 | |
| 30 | # Create task with your configuration |
| 31 | task = BaseTask(config_path="./data/your_task/config.yaml") |
| 32 | |
| 33 | # Generate hypotheses |
| 34 | task.generate_hypotheses(method="hypogenic", num_hypotheses=20) |
| 35 | |
| 36 | # Run inference |
| 37 | results = task.inference(hypothesis_bank="./output/hypotheses.json") |
| 38 | ``` |
| 39 | |
| 40 | ## When to Use This Skill |
| 41 | |
| 42 | Use this skill when working on: |
| 43 | - Generating scientific hypotheses from observational datasets |
| 44 | - Testing multiple competing hypotheses systematically |
| 45 | - Combining literature insights with empirical patterns |
| 46 | - Accelerating research discovery through automated hypothesis ideation |
| 47 | - Domains requiring hypothesis-driven analysis: deception detection, AI-generated content identification, mental health indicators, predictive modeling, or other empirical research |
| 48 | |
| 49 | ## Key Features |
| 50 | |
| 51 | **Automated Hypothesis Generation** |
| 52 | - Generate 10-20+ testable hypotheses from data in minutes |
| 53 | - Iterative refinement based on validation performance |
| 54 | - Support for both API-based (OpenAI, Anthropic) and local LLMs |
| 55 | |
| 56 | **Literature Integration** |
| 57 | - Extract insights from research papers via PDF processing |
| 58 | - Combine theoretical foundations with empirical patterns |
| 59 | - Systematic literature-to-hypothesis pipeline with GROBID |
| 60 | |
| 61 | **Performance Optimization** |
| 62 | - Redis caching reduces API costs for repeated experiments |
| 63 | - Parallel processing for large-scale hypothesis testing |
| 64 | - Adaptive refinement focuses on challenging examples |
| 65 | |
| 66 | **Flexible Configuration** |
| 67 | - Template-based prompt engineering with variable injection |
| 68 | - Custom label extraction for domain-specific tasks |
| 69 | - Modular architecture for easy extension |
| 70 | |
| 71 | **Proven Results** |
| 72 | - 8.97% improvement over few-shot baselines |
| 73 | - 15.75% improvement over literature-only approaches |
| 74 | - 80-84% hypothesis diversity (non-redundant insights) |
| 75 | - Human evaluators report significant decision-making improvements |
| 76 | |
| 77 | ## Core Capabilities |
| 78 | |
| 79 | ### 1. HypoGeniC: Data-Driven Hypothesis Generation |
| 80 | |
| 81 | Generate hypotheses solely from observational data through iterative refinement. |
| 82 | |
| 83 | **Process:** |
| 84 | 1. Initialize with a small data subset to generate candidate hypotheses |
| 85 | 2. Iteratively refine hypotheses based on performance |
| 86 | 3. Replace poorly-performing hypotheses with new ones from challenging examples |
| 87 | |
| 88 | **Best for:** Exploratory research without existing literature, pattern discovery in novel datasets |
| 89 | |
| 90 | ### 2. HypoRefine: Literature and Data Integration |
| 91 | |
| 92 | Synergistically combine existing literature with empirical data through an agentic framework. |
| 93 | |
| 94 | **Process:** |
| 95 | 1. Extract insights from relevant research papers (typically 10 papers) |
| 96 | 2. Generate theory-grounded hypotheses from literature |
| 97 | 3. Generate data-driven hypotheses from observational patterns |
| 98 | 4. Refine both hypothesis banks through iterative improvement |
| 99 | |
| 100 | **Best for:** Research with established theoretical foundations, validating or extending existing theories |
| 101 | |
| 102 | ### 3. Union Methods |
| 103 | |
| 104 | Mechanistically combine literature-only hypotheses with framework outputs. |
| 105 | |
| 106 | **Variants:** |
| 107 | - **Literature ∪ HypoGeniC**: Combines literature hypotheses with data-driven generation |
| 108 | - **Literature ∪ HypoRefine**: Combines literature hypotheses with integrated approach |
| 109 | |
| 110 | **Best for:** Comprehensive hypothesis coverage, eliminating redundancy while maintaining diverse perspectives |
| 111 | |
| 112 | ## Installation |
| 113 | |
| 114 | Install via pip: |
| 115 | ```bash |
| 116 | uv pip install hypogenic |
| 117 | ``` |
| 118 | |
| 119 | **Optional dependencies:** |
| 120 | - **Redis server** (port 6832): Enables caching of LLM responses to significantly reduce API costs during iterative hypothesis generation |
| 121 | - **s2orc-doc2json**: Required for processing literature PDFs in HypoRefine workflows |
| 122 | - **GROBID**: R |