agenta

Documentation | Website | Slack

The Open source LLMOps Platform

Prompt playground, prompt management, evaluation, and observability

---

Documentation • Changelog • Website • Agenta Cloud

---

What is Agenta?

Agenta is a platform for building production-grade LLM applications. It helps engineering and product teams create reliable LLM apps faster.

Agenta provides end-to-end tools for the entire LLMOps workflow: building (LLM playground, evaluation), deploying (prompt and configuration management), and monitoring (LLM observability and tracing).

Features

• Prompt Playground: Experiment, iterate on prompts, and compare outputs from over 50 LLM models side by side (docs)
• Custom Workflows: Build a playground for any custom LLM workflow, such as RAG or agents. Enable all the team to easily iterate on its parameters and evaluate it from the web UI.
• LLM evaluation: Run evaluation suite from the webUI using predefined evaluators like LLM-as-a-judge, RAG evaluators, or custom code evaluators. (docs)
• Human evaluation: Collaborate with subject matter experts for human annotation evaluation, including A/B testing and annotating golden test sets.
• Prompt Management: Version your prompts and manage them across different environments (docs, quick start)
• LLM Tracing: Observe and debug your apps with integrations to most providers and frameworks (docs, quick start)
• LLM Monitoring: Track cost and latency and compare different deployments.

Getting Started

Agenta Cloud:

The easiest way to get started is through Agenta Cloud. It is free to signup, and comes with a generous free-tier.

Self-host:

mkdir agenta && cd agenta
curl -L https://raw.githubusercontent.com/agenta-ai/agenta/main/docker-compose.gh.yml -o docker-compose.gh.yml
docker compose -f docker-compose.gh.yml up -d

• Agenta uses port 80, to use a different port set the env var AGENTA_PORT. For instance export AGENTA_PORT=90
• Add --pull always to the last command to upgrade your version of Agenta.

Disabling Anonymized Tracking

By default, Agenta automatically reports anonymized basic usage statistics. This helps us understand how Agenta is used and track its overall usage and growth. This data does not include any sensitive information. To disable anonymized telemetry, follow these steps:

• For web: Set TELEMETRY_TRACKING_ENABLED to false in your agenta-web/.env file.
• For CLI: Set telemetry_tracking_enabled to false in your ~/.agenta/config.toml file.

Contributing

We warmly welcome contributions to Agenta. Feel free to submit issues, fork the repository, and send pull requests.

We are usually hanging in our Slack. Feel free to join our Slack and ask us anything

Check out our Contributing Guide for more information.

Contributors

Thanks goes to these wonderful people (emoji key):

Sameh Methnani	Suad Suljovic ? ?‍? ?	burtenshaw	Abram	Israel Abebe ? ?	Master X	corinthian ?
Pavle Janjusevic ?	Kaosi Ezealigo ?	Alberto Nunes ?	Maaz Bin Khawar ? ?‍?	Nehemiah Onyekachukwu Emmanuel	Philip Okiokio	Abhinav Pandey
Ramchandra Warang ?	Biswarghya Biswas	Uddeepta Raaj Kashyap	Nayeem Abdullah	Kang Suhyun	Yoon	Kirthi Bagrecha Jain
Navdeep	Rhythm Sharma	Osinachi Chukwujama	莫尔索	Agunbiade Adedeji	Emmanuel Oloyede	Dhaneshwarguiyan
Priyanshu Prajapati	Raviteja	Arijit	Yachika9925	Aldrin ️	seungduk.kim.2304	Andrei Dragomir
diego	brockWith	Dennis Zelada	Romain Brucker	Heon Heo	Drew Reisner	Ikko Eltociear Ashimine
Vishal Vanpariya	Youcef Boumar	LucasTrg ?	Ashraf Chowdury ?	jp-agenta ?	Mr Unhappy ? ?	Moreno Bonaventura ?

This project follows the all-contributors specification. Contributions of any kind are welcome!