Reinforcement Learning Environments & Agent Evaluation for LLM Coding Agents
Bespoke Labs
Bespoke Labs builds reinforcement learning environments that frontier AI labs use to train and grade LLM coding agents. I led a team of 15–20 engineers building two of them, including one that drops an agent into a simulated machine learning platform with realistic production failures and checks whether it repairs them.
- Python
- Docker
- FastAPI
- PostgreSQL
- XGBoost
- pytest
- Claude Code
- Codex
Outcomes
- Led machine learning engineering for an agent evaluation platform that runs sandboxed, multi-turn agent sessions across several LLM providers and grades them automatically
- Built a pass-rate model that rejects 10%+ of weak tasks before human review, so reviewers spend their time on tasks worth keeping
- Main contributor to a cheat-resistant benchmark: hidden tests, CI promotion gates and locked-down containers, so an agent can only earn the reward by doing the work
- Built an environment linter as Claude Code and Codex skills that audits every task for ways an agent could game it



