Introduction

Training a strong coding agent is not a single training job. A realistic SWE pipeline has to mine useful GitHub changes, turn them into verifiable tasks, roll agents out in isolated environments, filter and convert trajectories, train models, and measure them with an evaluator. Each stage tends to grow its own scripts, storage layout, runtime assumptions, GPU requirements, and failure modes. The hard part is not only running the stages; it is keeping their handoffs reproducible as the data, repos, models, and infrastructure keep changing.

SWE-Lego-Live turns that loop into one agentic workflow. Instead of treating data construction, rollout, training, and evaluator measurement as disconnected jobs, it keeps them in a live pipeline where each stage can be checked, rerun, inspected, and extended by agents.

Highlights

• Live, growing dataset: download the SWE-Lego-Live dataset from Hugging Face, with bi-weekly updates that keep tasks and trajectories aligned with real repository activity.
• Fully agentic data pipeline: curate high-quality SWE and coding tasks across 8+ programming languages and 20+ task tags, then roll out trajectories with 4+ coding agents, including Claude Code, OpenCode, OpenHands, and Terminus.
• Unified, block-wise design: use agent plugin skills to interact with, customize, and extend each stage through a consistent block interface.

Project Design

The project is organized as a tree of blocks. A block is a contract for agentic workflows and one runnable unit in the pipeline. Each block owns its config, scripts, plugin skills, artifacts, dashboard, and pinned repository dependencies. Users can operate the workflow through the same plugin-skill lifecycle across blocks: :setup, :check, and :run.

The root block orchestrates four subblocks: curator builds verified SWE tasks, tracer rolls out agent trajectories, trainer turns trajectories into model checkpoints, and evaluator measures those checkpoints with shared rubrics.

• curator: curates high-quality SWE and coding tasks from GitHub PRs, issues, and online forums such as Stack Overflow. Carefully designed filtering requirements preserve high-quality instances across 8+ programming languages and 20+ task tags.
• tracer: collects high-quality trajectories with verified rewards, supporting rollouts across multiple coding scaffolds such as Claude Code, OpenCode, OpenHands, and Terminus.
• trainer: converts rollout traces into training-ready formats and launches end-to-end training workflows.
• evaluator: measures checkpoints on coding benchmarks and supports detailed analysis with carefully designed rubrics and tags.
• live dashboard: displays live statistics across tasks, trajectories, training status, and evaluator results. It also supports experiment comparison and deeper analysis over curated rubrics and tags.

Why Blocks?

Blocks make the pipeline easier to use and easier to change. Because every stage follows the same contract, users can run the whole system through plugin skills, restart a failed stage from its archived state, swap one block without breaking the others, and inspect artifacts through a consistent dashboard interface. See Block Design for the exact block anatomy and how the diagram maps onto the repository structure.

Where to go next

• Getting Started — the three skills (:setup, :check, :run) used to drive every block
• Block Design — the abstraction the whole pipeline is built on
• Block docs: curator · tracer · trainer · evaluator
• Reference — config.yaml schema and the archive format