$git clone https://github.com/colindoyle0000/llms-as-method-actorsOfficial implementation for paper LLMs as Method Actors: A Model for Prompt Engineering and Architecture with code, prompts, and sample outputs.
| 1 | # Official Repo of LLMs as Method Actors |
| 2 | |
| 3 | Official implementation for paper [LLMs as Method Actors: A Model for Prompt Engineering and Architecture](https://arxiv.org/abs/2411.05778) with code, prompts, and sample outputs. |
| 4 | |
| 5 | ## Setup |
| 6 | |
| 7 | 1. Setup your API key |
| 8 | |
| 9 | Store your OpenAI API key in environment variable ``OPENAI_API_KEY``. (To learn more about setting up an OpenAI API key, follow [this link](https://help.openai.com/en/articles/5112595-best-practices-for-api-key-safety)). If you would like to use Anthropic or Gemini models, store those keys in environmental variables ``ANTHROPIC_API_KEY`` and ``GEMINI_API_KEY``. |
| 10 | |
| 11 | 2. Install the necessary packages |
| 12 | |
| 13 | ```bash |
| 14 | pip install -r requirements.txt |
| 15 | ``` |
| 16 | |
| 17 | This will install all dependencies listed in `requirements.txt`. |
| 18 | |
| 19 | ## Tutorials |
| 20 | |
| 21 | The Jupyter notebook tutorial.ipynb can walk you through how to run each of the approaches from the paper on any Connections puzzle. |
| 22 | |
| 23 | ## Sample Outputs |
| 24 | |
| 25 | Sample outputs for each of the approaches are available in the “sample_outputs” folder as markdown files. For the more complicated approaches, markdown files are included that track the intermediary progress for each guess that was considered. Markdown files named “summary” and “summary_detailed” summarize the overall outcome and the results of the intermediary steps. |
| 26 | |
| 27 | The sample outputs were created by running the python scripts on the Connections puzzle from the day that the paper was announced on arXiv.org, November 11, 2024. |
| 28 | |
| 29 | The results were as follows: |
| 30 | |
| 31 | | Approach | Success? | # Correct Guesses | # Incorrect Guesses | |
| 32 | | --------------------------- | -------- | ----------------- | ------------------- | |
| 33 | | Vanilla | No | 0 | 4 | |
| 34 | | Chain-of-Thought | No | 1 | 4 | |
| 35 | | Chain-of-Thought (Scripted) | Yes | 4 | 3 | |
| 36 | | Actor | Yes | 4 | 0 | |
| 37 | | Actor-2 | Yes | 4 | 1 | |
| 38 | | Oneshot-o1 | No | 2 | 2 | |
| 39 | | Vanilla-o1 | Yes | 4 | 0 | |
| 40 | | Actor-o1 | Yes | 4 | 0 | |
| 41 | |
| 42 | ## Citations |
| 43 | |
| 44 | Please cite the paper and star this repo if you find it interesting or useful. Feel free to contact colin.doyle@lls.edu if you have any questions. |
| 45 | |
| 46 | ```bibtex |
| 47 | @misc{doyle2024method, |
| 48 | title={LLMs as Method Actors: A Model for Prompt Engineering and Architecture}, |
| 49 | author={Colin Doyle}, |
| 50 | year={2024}, |
| 51 | eprint={2411.05778}, |
| 52 | archivePrefix={arXiv}, |
| 53 | primaryClass={cs.CL} |
| 54 | } |
| 55 | ``` |