Model Development Lifecycle
Working definition of the MDLC across classical ML, generative AI and agentic systems.
Research question
What are the stages of the model development lifecycle, and how do they differ between classical ML, LLM applications and agentic systems?
Scope
- Stage definitions and their artifacts
- Differences in the unit of development (trained model vs prompt/system vs agent)
- Where humans make decisions in each variant
Working stage model
This repository uses seven stages. They are a working vocabulary, not a claim about industry consensus.
| Stage | Classical ML | LLM application | Agentic system |
|---|---|---|---|
| 01 Require | Prediction target, label definition | Task definition, quality bar | Goal, scope, permitted actions |
| 02 Plan | Feature & model approach | Model choice, retrieval design | Tool set, skills, autonomy level |
| 03 Build | Training pipeline, trained weights | Prompts, retrieval index, glue code | Agent config, skills, tool bindings |
| 04 Validate | Holdout metrics, fairness tests | Eval suites, red-teaming | Task-success evals, trajectory review |
| 05 Approve | Model risk sign-off | Launch review | Permission & autonomy approval |
| 06 Deploy | Batch / online scoring | API / app release | Runtime with sandbox & tool access |
| 07 Monitor | Drift, performance | Quality, cost, safety signals | Trajectory traces, action audit |
Sculley et al. argue that in real-world ML systems only a small fraction of the code is the model itself; the surrounding infrastructure (configuration, data collection, feature extraction, serving, monitoring) dominates.[1]
If that holds for agentic systems too, most MDLC investment will go to the surrounding system — context, tools, policy, evaluation and audit — rather than to the model.
Key findings
SR 26-2, the newest US bank model-risk guidance, describes a lifecycle that maps onto the working stages: statement of purpose (Require), data selection and testing (Build), validation before first use (Validate, Approve) and monitoring for changed data relevance and use beyond purpose (Deploy, Monitor). It covers non-generative models and excludes generative and agentic AI.[2]
MLflow 3 is positioned as one platform to instrument, register and monitor both training pipelines and multi-agent RAG systems (a single vendor's claim).[3]
An agent has no release event once it runs continuously: SR 26-2 asks for ongoing monitoring (the EU AI Act does too, for high-risk systems; see the continuous-evidence note), but no source found specifies a minimum for a low-risk agent or says who accepts it.[2]
Vendor lifecycles for agents are written in the same shape. Microsoft Foundry lists nine steps from choosing an agent type through versioning, tracing, evaluation, publishing and monitoring;[4] Google's agent evaluation runs from eval cases through inference, scoring and optimisation.[5] In the pages read, neither has a requirements stage; Require and Plan are the least covered stages in the vendor matrix.
The seven-stage vocabulary holds up against the documents read so far, but Require and Plan are thinly supported by tools, so the working model may overstate how much of those two stages is software today. Evidence is mostly vendor documentation.
Related notes
- Will the ML and LLM lifecycles converge? — control plane converges, data plane diverges (draft)
- Framing the MDLC research program
- Continuous evidence for always-on agents (draft)
- Unifying the control layer across data, feature and model platforms (draft)
Comparisons & patterns
- Vendor coverage matrix — stage coverage across six vendor profiles
- Agentic MDLC workflow — the seven stages with humans, agents, artifacts and control gates
- Benchmarking framework — one-variable experiments for testing workflow claims
References
- D. Sculley et al., “Hidden Technical Debt in Machine Learning Systems,” NeurIPS 2015. papers.nips.ccpaper · S-0001
- Federal Reserve, OCC, FDIC, SR 26-2 attachment, 2026-04-17; PDF read in full. linksupervisory guidance · S-0041
- MLflow, “MLflow 3” launch post, 2025-06. linkvendor blog · S-0010
- Microsoft Learn, “Agent development lifecycle in Microsoft Foundry.” linkvendor documentation · S-0079
- Google Cloud, “Agent evaluation,” Gemini Enterprise Agent Platform. linkvendor documentation · S-0091