byfacebookresearch· 1 MCP server
Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios.
$git clone https://github.com/facebookresearch/meta-agents-research-environmentsInstalls into the current project.
Install meta-agents-research-environments by running `git clone https://github.com/facebookresearch/meta-agents-research-environments`, then use it for the current task and follow its documentation at https://github.com/facebookresearch/meta-agents-research-environments.
| 1 | # Meta Agents Research Environments (ARE) |
| 2 | |
| 3 | [](https://badge.fury.io/py/meta-agents-research-environments) |
| 4 | [](https://www.python.org/downloads/) |
| 5 | [](LICENSE) |
| 6 | |
| 7 | > A research environment for simulating complex, real-life tasks that require multi-step reasoning and dynamic adaptation. |
| 8 | |
| 9 | Meta Agents Research Environments (ARE) is a platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this research platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges. In particular, ARE runs the [Gaia2](#example-gaia2-benchmark) benchmark, a follow-up to [Gaia](https://arxiv.org/abs/2311.12983), evaluating a broader range of agent capabilities. |
| 10 | |
| 11 |  |
| 12 | |
| 13 | Looking for the container-based Gaia2-CLI benchmark stack? Start with |
| 14 | [gaia2-cli](gaia2-cli). |
| 15 | |
| 16 | ## Table of Contents |
| 17 | |
| 18 | - [Background](#background) |
| 19 | - [Install](#install) |
| 20 | - [Usage](#usage) |
| 21 | - [API](#api) |
| 22 | - [Contributing](#contributing) |
| 23 | - [License](#license) |
| 24 | |
| 25 | ## Background |
| 26 | |
| 27 | ARE addresses critical gaps in AI agent evaluation by providing: |
| 28 | |
| 29 | - **Dynamic Environments**: Scenarios that evolve over time with new information and changing conditions |
| 30 | - **Multi-Step Reasoning**: Complex tasks requiring 10+ steps and several minutes to complete |
| 31 | - **Real-World Focus**: Grounded situations that mirror actual real-world challenges |
| 32 | - **Comprehensive Evaluation**: The Gaia2 benchmark with 800 scenarios across multiple domains |
| 33 | |
| 34 | |
| 35 | ### Getting Started |
| 36 | |
| 37 | | | | |
| 38 | |---|---| |
| 39 | | **[Quick Start](https://facebookresearch.github.io/meta-agents-research-environments/quickstart.html)** | Get up and running with your first scenario in just a few minutes with step-by-step instructions. | |
| 40 | | **[Gaia2 Evaluation](https://facebookresearch.github.io/meta-agents-research-environments/user_guide/gaia2_evaluation.html)** | Build and evaluate your agents on the Gaia2 benchmark, a comprehensive suite of 800 dynamic scenarios across 10 universes. | |
| 41 | | **[Gaia2 Blog Post](https://huggingface.co/blog/gaia2)** | Learn more about Gaia2 on the Hugging Face blog. | |
| 42 | | **[Paper](https://ai.meta.com/research/publications/are-scaling-up-agent-environments-and-evaluations/)** | Read the research paper detailing the Gaia2 benchmark and evaluation methodology. | |
| 43 | | **[Demo](https://huggingface.co/spaces/meta-agents-research-environments/demo)** | [Try the ARE Demo on Hugging Face](https://huggingface.co/spaces/meta-agents-research-environments/demo) — Play around with the agent platform directly in your browser, no installation required! | |
| 44 | | **[Gaia2 Leaderboard](https://huggingface.co/spaces/meta-agents-research-environments/leaderboard)** | Check the self-published results from Gaia2 Benchmark runs. | |
| 45 | | **[Learn More](https://facebookresearch.github.io/meta-agents-research-environments/foundations/index.html)** | Dive deeper into the core concepts of agents, environments, apps, events, and scenarios. | |
| 46 | |
| 47 | |
| 48 | ## Install |
| 49 | |
| 50 | For complete installation instructions and setup options, see the [Installation Guide](https://facebookresearch.github.io/meta-agents-research-environments/user_guide/installation.html). |
| 51 | |
| 52 | ### Prerequisites |
| 53 | |
| 54 | First, install [uv](https://docs.astral.sh/uv/), a fast Python package installer and resolver. |
| 55 | |
| 56 | ### Quick Start with uvx |
| 57 | |
| 58 | The fastest way to get started is using `uvx` to run commands directly: |
| 59 | |
| 60 | ```bash |
| 61 | # Run Gaia2 benchmark scenarios |
| 62 | uvx --from meta-agents-research-environments are-benchmark gaia2-run --hf meta-agents-research-environments/gaia2 --hf_split validation -l 1 |
| 63 | |
| 64 | # Run custom scenarios |
| 65 | uvx --from meta-agents-research-environments are-run -s scenario_tutorial -a default |
| 66 | ``` |
| 67 | |
| 68 | All the commands in this README and the documentation are availab |