Evaluation

Open academic benchmark suite for stress-testing LLMs on contamination-resistant, domain-specific tasks.

Open-source eval and red-teaming framework for LLM apps, prompts, and RAG pipelines.

Autoresearch engine that iteratively rewrites code to optimize against a numeric evaluation metric.

Automatic LLM evaluator and leaderboard that benchmarks instruction-following with length-controlled win rates.

Open-source toolkit for testing, tracing, and monitoring production AI agents.

Enterprise AI observability and guardrails platform for monitoring agents, LLMs, and ML models in production.

End-to-end evaluation, simulation, and observability platform for shipping production-grade AI agents.

Prompt management and side-by-side LLM evaluation for OpenAI and Anthropic models.

LLM engineering platform combining a multi-model gateway with tracing, evals, and prompt management.

Private, expert-graded leaderboards from Scale AI that rank frontier LLMs on domains contaminated public benchmarks can no longer measure.

Open benchmark for evaluating multimodal web agents on realistic visual browsing tasks.

No-signup LLM playground for testing, comparing, and versioning prompts against your own API keys.

Terminal tool that scores hundreds of open LLMs against your actual CPU, RAM, and GPU and tells you which ones will run well.

Olympiad-level multi-discipline benchmark for stress-testing reasoning in LLMs and multimodal models.

Open-source LLM and agent observability platform with tracing, evals, and experimentation built on OpenTelemetry.

Agentic management platform for runtime guardrails, policy enforcement, and observability across LLM agents.

Open-source LLMOps platform for prompt engineering, evaluation, and observability in one workspace.

Public leaderboard from the OpenCompass project ranking open and closed LLMs across 100+ benchmarks.

Stack Overflow-derived benchmark for evaluating code LLMs on real-world programming questions.

Dynamic LLM benchmark that mixes web queries with existing datasets to mirror Chatbot Arena rankings at a fraction of the cost.

Head-to-head LLM battle arena with a public leaderboard for ranking AI models.

Independent benchmarking platform comparing AI models and inference providers across intelligence, speed, and cost.

Trustworthiness scoring layer that flags LLM hallucinations in real time.

LLM evaluation, observability, and prompt management platform for teams shipping production AI apps.