Evaluate & Optimize LLM Pipelines

You can use Ellf with any AI provider, LLM API or local models. If you can do it in Python, you’ll be able to use it with Ellf! The built-in workflows include integrations with popular API providers like OpenAI, Google Gemini and TypeSafe, as well as local models via Hugging Face and spaCy.

You can access models via API requests with built-in secrets support for authentication, or run them directly from your data-private cluster. You and your coding assistant can also easily add custom workflows to connect to any internal or external service, and load, run or export custom models using any framework.

Ellf lets you create LLM-powered agents and assign them to annotation tasks, just like you would with human annotators. Agents support a range of different annotation types, like span annotation, text classification or binary feedback, and will automatically create annotated data, which is added to the dataset and available for review in the UI. You can also customize agents and add your own for different data types and using custom prompts and models.

Ellf lets you plug in your own data processing cluster hosted locally or on your cloud infrastructure and under your control. The cluster provides all data, models and custom code you use and create, and serves the annotation interface and performs actions like model training or data analysis.

This means that no sensitive data ever leaves your servers or has to pass through our servers in the process. Whenever data from the cluster is used and shown, like in the web app or CLI, Ellf makes two separate requests: one to our server and one to your cluster.

The cluster can be set up in your cloud account (e.g. AWS, Google Cloud or Azure) or on a local workstation in a few simple steps, without requiring specific DevOps or infrastructure expertise. If you’ve connected Ellf to your coding assistant like Claude Code, it can also set up and launch your cluster for you.