DSPy and GEPA Trial

Overview

DSPy treats LLM pipelines as optimizable programs with signatures, modules, and teleprompters instead of hand-tuned prompt strings. GEPA and related optimizers search over prompts and demonstrations using task metrics (DSPy). GEPA (Genetic-Pareto) reflects on execution traces in natural language, proposes new instructions, and keeps a Pareto frontier of candidates; it is available as dspy.GEPA and as the standalone gepa package (GEPA paper).

Trial when prompt iteration is a bottleneck and you have labeled eval sets. Avoid black-box optimization without human review of final prompts and failure cases.

Adoption Signals

• The GEPA paper was accepted as an oral at ICLR 2026 and reports that GEPA outperforms GRPO by 6% on average and up to 20% with up to 35x fewer rollouts, and beats MIPROv2 by more than 10% (GEPA paper).
• Databricks reports that GEPA-optimized gpt-oss-120b exceeded baseline Claude Opus 4.1 and Claude Sonnet 4 on its IE Bench information-extraction suite while being 90x and 22x cheaper to serve (Databricks blog).
• GEPA is built into other platforms: MLflow's mlflow.genai.optimize_prompts() offers it as one of two optimizers from MLflow 3.5 (MLflow prompt optimization), and Google's adk optimize command applies GEPA to refine agent instructions (ADK agent optimization).
• Nubank's paper on customer-support agents at 100M-user scale uses GEPA optimization to make its LLM-as-judge evaluation more consistent (Nubank paper).
• DSPy has about 38,000 GitHub stars and an MIT licence (DSPy on GitHub) and about 5.2 million PyPI downloads in the 30 days to 30 September 2026 (PyPI Stats); the standalone GEPA repository has about 6,800 stars (GEPA on GitHub).

Risks

• Metric overfitting. GEPA optimizes whatever the metric rewards; without a separate validation set it reuses the training set, and DSPy logs a warning that this makes GEPA overfit prompts to the training set (dspy.GEPA source), and a weak metric produces prompts that game it.
• Optimization cost. Each run needs a budget (auto, max_full_evals or max_metric_calls) and a strong reflection model (dspy.GEPA docs); Databricks notes that its reported savings cover serving only and that GEPA runs took 2 to 3 hours (Databricks blog).
• Prompts tied to one model. An optimized prompt is tuned to the task model it was optimized against; a model upgrade or provider switch means re-running optimization and re-validating.
• Framework churn. DSPy 3.4 calls itself the LM transition release, with 3.5 as the migration deadline for legacy custom-LM integrations (DSPy 3.4.0 release notes).
• Opaque outputs for reviewers. Evolved instructions can be long and hard to read, which makes compliance review and change approval harder than for hand-written prompts.

Recommendation

Trial DSPy with GEPA where a task already has a measurable definition of good output, such as extraction, classification, routing or LLM-as-judge prompts, and at least a few hundred labeled examples split into training and validation sets. Treat the optimizer as a build step: version the eval set, the metric and the resulting prompt together, cap the metric-call budget, and have a person review the final prompt and its failure cases before it ships.

It is not worth introducing for open-ended generation without a reliable metric, or for teams that cannot yet run evals in CI. Teams already on MLflow or Google ADK can try GEPA through those integrations before taking on DSPy's programming model. Pin the DSPy version, plan for the 3.5 LM interface migration, and re-run optimization whenever the underlying model changes.

Sources

• DSPy docs
• DSPy GitHub
• DSPy API
• dspy.GEPA docs
• dspy.GEPA source code
• DSPy 3.4.0 release notes
• GEPA on GitHub
• GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning (arXiv)
• Databricks: Building state-of-the-art enterprise agents 90x cheaper with automated prompt optimization
• MLflow prompt optimization
• Google ADK agent optimization
• Nubank: Building Customer Support AI Agents at 100M-User Scale (arXiv)
• PyPI Stats: dspy