promptpilot
installs/mo · pypistarsforks
How to install promptpilot?
doganarif/promptpilot
$git clone https://github.com/doganarif/promptpilotA fast, lightweight Python library and CLI tool for versioning, testing, and optimizing your AI prompts across multiple providers.
README.md
| 1 | # PromptPilot |
| 2 | |
| 3 | A fast, lightweight Python library and CLI tool for versioning, testing, and optimizing your AI prompts across multiple providers. |
| 4 | |
| 5 | [](https://pypi.org/project/promptpilot-cli/) |
| 6 | [](https://pypi.org/project/promptpilot-cli/) |
| 7 | [](https://github.com/doganarif/promptpilot/blob/main/LICENSE) |
| 8 | |
| 9 | ## 🚀 Quick Start |
| 10 | |
| 11 | ```bash |
| 12 | # Install from PyPI |
| 13 | pip install promptpilot-cli |
| 14 | |
| 15 | # Initialize a new prompt |
| 16 | promptpilot init my-summary --description "Summarize text in 3 paragraphs" |
| 17 | |
| 18 | # Run an A/B test |
| 19 | promptpilot abtest my-summary --input sample.txt --provider openai |
| 20 | ``` |
| 21 | |
| 22 | ## ✨ Features |
| 23 | |
| 24 | - **Version control** for your AI prompts, with or without Git |
| 25 | - **A/B testing** to compare prompts based on token usage and response quality |
| 26 | - **Multi-provider support** for OpenAI, Claude, Llama, and HuggingFace |
| 27 | - **Fast startup time** with lazy loading of dependencies |
| 28 | - **Live progress feedback** during operations |
| 29 | - **Extensible architecture** for custom providers and formatters |
| 30 | - **Python API** for integration into your own workflows |
| 31 | |
| 32 | ## 📋 Requirements |
| 33 | |
| 34 | - Python 3.11+ |
| 35 | - API keys for the services you want to use |
| 36 | |
| 37 | ## 🔧 Installation |
| 38 | |
| 39 | ```bash |
| 40 | pip install promptpilot-cli |
| 41 | ``` |
| 42 | |
| 43 | For development: |
| 44 | |
| 45 | ```bash |
| 46 | git clone https://github.com/doganarif/promptpilot.git |
| 47 | cd promptpilot |
| 48 | pip install -e . |
| 49 | ``` |
| 50 | |
| 51 | ## ⚙️ Configuration |
| 52 | |
| 53 | Create a `.env` file in your project root with your API keys: |
| 54 | |
| 55 | ```ini |
| 56 | # Required API keys (for the providers you use) |
| 57 | OPENAI_API_KEY=sk-... |
| 58 | ANTHROPIC_API_KEY=sk-ant-... |
| 59 | LLAMA_API_KEY=llm-... |
| 60 | |
| 61 | # Optional default models |
| 62 | PROMPTCTL_DEFAULT_MODEL=gpt-4o |
| 63 | PROMPTCTL_CLAUDE_MODEL=claude-3-opus-20240229 |
| 64 | PROMPTCTL_LLAMA_MODEL=llama-3-70b-instruct |
| 65 | |
| 66 | # Optional API URL for Llama |
| 67 | LLAMA_API_URL=https://api.llama-api.com |
| 68 | ``` |
| 69 | |
| 70 | PromptPilot will automatically detect and use this file. |
| 71 | |
| 72 | ## 📖 CLI Usage Examples |
| 73 | |
| 74 | ### Initialize a new prompt |
| 75 | |
| 76 | ```bash |
| 77 | promptpilot init text-extractor \ |
| 78 | --description "Extract key information from documents" \ |
| 79 | --author "Jane Smith" |
| 80 | ``` |
| 81 | |
| 82 | This creates `prompts/text-extractor.yml` with a template and metadata. |
| 83 | |
| 84 | ### Version Management |
| 85 | |
| 86 | ```bash |
| 87 | # Save current prompt state as a new version |
| 88 | promptpilot save summary -m "Added more detailed instructions" |
| 89 | |
| 90 | # Show difference between versions |
| 91 | promptpilot diff summary |
| 92 | |
| 93 | # List all versions of a prompt |
| 94 | promptpilot list summary |
| 95 | |
| 96 | # List all available prompts |
| 97 | promptpilot list |
| 98 | ``` |
| 99 | |
| 100 | ### Run an A/B test |
| 101 | |
| 102 | ```bash |
| 103 | # Basic A/B test with OpenAI |
| 104 | promptpilot abtest summary --input article.txt --provider openai |
| 105 | |
| 106 | # Testing with Claude |
| 107 | promptpilot abtest summary --input article.txt --provider claude --model claude-3-sonnet |
| 108 | |
| 109 | # Include full responses in the output |
| 110 | promptpilot abtest summary --input article.txt --include-responses |
| 111 | |
| 112 | # Output results in JSON format |
| 113 | promptpilot abtest summary --input article.txt --format json > results.json |
| 114 | ``` |
| 115 | |
| 116 | ## 💻 Python API Examples |
| 117 | |
| 118 | ### Basic A/B Testing |
| 119 | |
| 120 | ```python |
| 121 | from promptpilot.utils import load_prompt_versions |
| 122 | from promptpilot.models import Prompt |
| 123 | from promptpilot.providers import get_provider |
| 124 | from promptpilot.runner import ABTestRunner |
| 125 | |
| 126 | # Load two prompt versions |
| 127 | prev, curr = load_prompt_versions("summary") |
| 128 | |
| 129 | # Create prompt objects |
| 130 | prompt_a = Prompt(name="summary_v1", template=prev, version=1) |
| 131 | prompt_b = Prompt(name="summary_v2", template=curr, version=2) |
| 132 | |
| 133 | # Initialize provider (OpenAI by default) |
| 134 | provider = get_provider("openai", model="gpt-4o") |
| 135 | |
| 136 | # Run the A/B test |
| 137 | runner = ABTestRunner(provider) |
| 138 | input_text = open("article.txt").read() |
| 139 | |
| 140 | result = runner.run_test( |
| 141 | prompt_a=prompt_a, |
| 142 | prompt_b=prompt_b, |
| 143 | variables={"text": input_text}, |
| 144 | include_responses=True |
| 145 | ) |
| 146 | |
| 147 | # Display results |
| 148 | runner.display_results(result) |
| 149 | |
| 150 | # Get the winner |
| 151 | winner_name, token_count = runner.get_winner(result) |
| 152 | print(f"Winner: {winner_name} with {token_count} tokens") |
| 153 | ``` |
| 154 | |
| 155 | ### T |
People who liked this also viewed
6 picksType
- components-helperBased on the docs to provide code prompt files for vue component libraryPromptsNov 2023
- VibeCode-Protocol-SuiteA comprehensive suite of protocols, meta-prompts, and orchestration tools designed to streamline software development workflows, project management, and team collaboration. Includes the VibeCode Protocol Suite with web interface, user manuals, and reusable templates for efficient development processes.PromptsJul 2026
- Promptable-web-sdkWeb SDK for Promptable Website.PromptsDec 2022
- LLMFlexA python package for developing AI applications with local LLMs.PromptsJan 2025
- cli-repo-to-promptexport any codebase to a single LLM-ready markdown promptPromptsJul 2026
- Files2PromptVS Code extension to copy files for an LLM promptPromptsApr 2025
