Will the ML and LLM lifecycles converge? ML 与 LLM 生命周期会趋同吗？
Position: tooling converges in the control plane; build, evaluation and — for now — regulation diverge.
立场：工具在控制平面趋同；构建、评估以及（目前的）监管则在分化。
- Date · 日期
- 2026-09-30
- Status · 状态
- Draft — first sources attached; H5 revised after SR 26-2草稿 — 已附首批来源；因 SR 26-2 修订 H5
- Author · 作者
- 🌱
- Tags · 标签
- lifecycle, convergence, ml, llm, agents, governance
- Question · 研究问题
- Will the lifecycles for classical ML and for LLM / agentic systems converge into one, and if only partly, which parts?传统机器学习与 LLM / 智能体系统的生命周期会合而为一吗？如果只是部分趋同，是哪些部分？
- Why it matters · 意义
- It decides whether organizations buy one platform or several, who owns LLM systems, and whether existing model-risk processes can be extended rather than rebuilt.它决定了企业该买一个平台还是多个、LLM 系统归谁负责，以及现有模型风险流程能否扩展沿用。
- Addresses · 对应
- Q1 · theme T4 on the核心主题 T4，见 homepage / 首页
Executive summary 摘要
Probably not fully. Tooling vendors are explicitly unifying the control plane — one registry, one observability layer for classical ML and GenAI[4]. But the way systems are built has shifted from training to composition[5][6], evaluation and failure modes differ in kind[8][9], and agents need controls classical models never did[10]. The most important new evidence is regulatory: US banking regulators' revised model-risk guidance, issued 17 April 2026, explicitly places generative and agentic AI outside its scope[1]. My working position is now: convergence in tooling and internal governance; divergence in build, evaluation and, for now, formal regulation.
大概率不会完全趋同。工具厂商正在明确地统一控制平面——传统 ML 与生成式 AI 共用一个注册表、一套可观测性[4]。但系统的构建方式已从“训练”转向“组合”[5][6]，评估方法和失败形态有本质差异[8][9]，而智能体还需要传统模型从未需要过的管控[10]。最重要的新证据来自监管：美国银行监管机构于 2026 年 4 月 17 日发布的修订版模型风险指引，明确将生成式 AI 和智能体 AI 排除在适用范围之外[1]。我目前的立场因此调整为：工具与内部治理趋同；构建、评估以及（目前的）正式监管分化。
Key findings 主要发现
| # | Finding · 发现 | Points to · 指向 | Source · 来源 |
|---|---|---|---|
| F1 | SR 26-2 (17 Apr 2026) supersedes SR 11-7 and states that generative and agentic AI models are not within its scope; its principles apply to traditional and non-generative, non-agentic AI models.SR 26-2（2026-04-17）取代 SR 11-7，并声明生成式与智能体 AI 模型不在其适用范围内；其原则仅适用于传统模型及非生成式、非智能体的 AI 模型。 | diverge · 分化 | [1] |
| F2 | SR 26-2 also says a bank's risk management and governance practices should still guide controls for tools not covered by the guidance.SR 26-2 同时指出，对于指引未覆盖的工具，银行仍应依据自身的风险管理与治理实践来确定管控。 | converge · 趋同 | [1] |
| F3 | NIST published a Generative AI Profile (AI 600-1, July 2024) as a companion to the existing AI RMF 1.0, rather than a separate framework.NIST 发布的生成式 AI 配置文件（AI 600-1，2024 年 7 月）是现有 AI RMF 1.0 的配套文件，而非独立框架。 | converge · 趋同 | [2] |
| F4 | The EU AI Act sets separate obligations for providers of general-purpose AI models (Chapter V), applicable from 2 August 2025.欧盟《人工智能法案》第五章为通用 AI 模型提供者单独设定义务，自 2025 年 8 月 2 日起适用。 | diverge · 分化 | [3] |
| F5 | MLflow 3 (June 2025) is positioned as one platform to instrument, register and monitor both training pipelines and multi-agent RAG systems.MLflow 3（2025 年 6 月）定位为同一平台，可对训练流水线和多智能体 RAG 系统进行埋点、注册和监控。 | converge · 趋同 | [4] |
| F6 | In a survey of 600 US enterprise IT decision-makers, 51% of GenAI deployments used RAG and 9% of production models were fine-tuned.一项针对 600 名美国企业 IT 决策者的调查显示：51% 的生成式 AI 部署使用 RAG，仅 9% 的生产模型经过微调。 | diverge · 分化 | [6] |
Diagram 图示
- Top: the shared control plane, owned by model risk and the platform team.
- Middle: one lane per system type. Columns line up (input → build → artifact → evaluate → monitor), but the content of each cell differs.
- Between them: policy and gates flow down to every lane; evidence and approval requests flow up.
- Dashed link: hybrid systems. Diamond G: autonomy approval, a control only agents need[10]. Bottom: the foundation model vendor changes lanes 2 and 3 from outside the team[7].
- 上层：统一控制平面，由模型风险部门和平台团队负责。
- 中层：每类系统一条车道。各列对齐（输入 → 构建 → 产物 → 评估 → 监控），但每格的内容不同。
- 两层之间：策略与关卡向下作用于所有车道；证据与审批申请向上提交。
- 虚线：混合系统。菱形 G：自主程度审批，只有智能体需要的管控[10]。底部：基础模型供应商从团队外部改变车道 2 和 3[7]。
Arguments for convergence 支持趋同的论点
Tooling vendors are unifying the control plane. MLflow 3's launch states that a transformer training pipeline and a multi-agent RAG system can be instrumented with the same tools, deployed through the same registry and monitored with unified observability[4].
工具厂商正在统一控制平面。MLflow 3 的发布说明称，transformer 训练流水线与多智能体 RAG 系统可以用同一套工具埋点、通过同一个注册表部署、以统一的可观测性监控[4]。
Standards bodies extend rather than replace. NIST's Generative AI Profile is a companion to AI RMF 1.0[2], and SR 26-2 tells banks to apply their own governance practices to AI tools outside its scope[1].
标准机构选择扩展而非替换。NIST 的生成式 AI 配置文件是 AI RMF 1.0 的配套文件[2]；SR 26-2 也要求银行对其范围之外的 AI 工具沿用自身的治理实践[1]。
Real systems are already hybrid. LLM applications call classical classifiers for routing or risk scoring; classical pipelines consume embeddings. Hybrid systems blur lifecycle boundaries. (No source yet.)
真实系统本来就是混合的。LLM 应用调用传统分类器做路由或风险评分；传统流水线也在使用 embedding。混合系统模糊了生命周期的边界。（尚无来源。）
Arguments against full convergence 反对完全趋同的论点
Formal regulation is splitting, at least in US banking. SR 26-2 excludes generative and agentic AI models because they are “novel and rapidly evolving”[1]. The EU AI Act likewise creates a separate obligation set for general-purpose AI model providers[3].
正式监管正在分开，至少在美国银行业如此。SR 26-2 以生成式与智能体 AI 模型“新颖且快速演变”为由将其排除在外[1]。欧盟《人工智能法案》同样为通用 AI 模型提供者设立了单独的义务[3]。
Teams compose rather than train. Researchers argue state-of-the-art results increasingly come from compound systems of multiple components rather than single models[5]; in enterprise surveys, RAG (51%) far outpaces fine-tuning (9%)[6].
团队在组合而非训练。研究者认为，最先进的成果越来越多地来自由多个组件构成的复合系统，而非单一模型[5]；企业调查中 RAG（51%）远多于微调（9%）[6]。
| Dimension · 维度 | Classical ML · 传统 ML | LLM / agentic · LLM / 智能体 | Source · 来源 |
|---|---|---|---|
| Unit of development开发单元 | Trained weights训练出的权重 | Composition of prompts, retrieval, tools, agent configurationprompt、检索、工具、智能体配置的组合 | [5][6] |
| Evaluation评估 | Ground truth; stable metrics (AUC, RMSE)有标准答案；指标稳定（AUC、RMSE） | Open-ended outputs; LLM judges reach over 80% agreement with human preferences开放式输出；LLM 评审与人类偏好的一致率超过 80% | [8] |
| Source of change变更来源 | Mostly own retraining主要是自己重新训练 | Vendor model updates: GPT-4's accuracy on one task fell from 84% to 51% between March and June 2023供应商更新模型：GPT-4 在某项任务上的准确率在 2023 年 3 月到 6 月间从 84% 降至 51% | [7] |
| Failure modes失败形态 | Drift, performance decay漂移、性能下降 | Indirect prompt injection via retrieved data, enabling data theft and other attacks通过检索数据进行间接 prompt 注入，可导致数据窃取等攻击 | [9] |
| Agent-specific controls智能体特有管控 | —— | Constraining the action space and requiring approval; legibility; automatic monitoring; attributability; interruptibility约束行动空间并要求审批；可读性；自动监控；可归因性；可中断性 | [10] |
The classical-ML column is general practice, not a cited claim.
“传统 ML”一列为一般实践描述，未单独引用。
What converges, what diverges 趋同与分化
| Converges · 趋同 | Diverges · 分化 |
|---|---|
| Registry, tracing and observability tooling注册表、追踪与可观测性工具 [4] | How systems are built系统构建方式 [5][6] |
| Risk frameworks (extended, not replaced)风险框架（扩展而非替换） [2] | Evaluation methods评估方法 [8] |
| Internal governance practices内部治理实践 [1] | Formal supervisory guidance (for now)正式监管指引（目前） [1][3] |
| Approval gates and records审批关卡与记录 | Failure modes and controls失败形态与管控 [9][10] |
My interpretation 我的解读
“Converge or not” is the wrong framing; the question is which layer. The evidence so far splits cleanly: what vendors sell (tooling) is converging, while what regulators formally supervise is diverging. That gap is where internal governance has to do the work — SR 26-2 effectively hands GenAI and agent governance back to each bank's own control plane.
“趋同与否”这个问法不准确，关键是哪一层。目前的证据分得很清楚：厂商卖的东西（工具）在趋同，而监管机构正式监督的范围在分化。两者之间的空白要由企业内部治理来填补——SR 26-2 实际上把生成式 AI 和智能体的治理交还给了每家银行自己的控制平面。
Hypotheses 假设
H4. MDLC tooling will converge in the control plane (governance, approval, evidence, registry) while build and evaluation tooling stays specialized by system type. Supported by F5; not yet tested beyond one vendor.
H4.MDLC 工具会在控制平面（治理、审批、证据、注册表）趋同，而构建与评估工具会按系统类型保持专门化。F5 支持；尚未在单一厂商之外验证。
H5 revised. Original: model-risk functions will extend existing validation frameworks to LLM and agentic systems. F1 contradicts this for formal US banking guidance. Revised H5: formal supervisory guidance will treat generative and agentic AI separately in the near term, while organizations extend their internal governance control plane to cover them (F2).
修订 H5。原假设：模型风险部门会把现有验证框架扩展到 LLM 和智能体系统。F1 与此相悖（就美国银行业正式指引而言）。修订后的 H5：短期内，正式监管指引会把生成式与智能体 AI 分开处理，而企业会把内部治理的控制平面扩展覆盖它们（F2）。
Open questions 未解问题
Will US regulators issue separate guidance for generative and agentic AI, and will it reuse SR 26-2's structure?
美国监管机构会否为生成式与智能体 AI 单独发布指引？它会沿用 SR 26-2 的结构吗？
Who owns LLM applications inside enterprises: the ML platform team or application engineering?
企业里 LLM 应用归谁负责：ML 平台团队还是应用工程团队？
Is MLflow's unification typical? Do other platforms ship one registry for both, or separate products?
MLflow 的统一做法是否普遍？其他平台是共用一个注册表，还是拆成不同产品？
What would change my view? 什么会改变我的看法？
- Toward full convergence: enterprises broadly fine-tune or train their own models (the 9% figure[6] rising sharply); or regulators fold GenAI/agentic AI back into general model-risk guidance.
- Toward full divergence: major platforms split ML and LLM/agent products and registries; or dedicated GenAI/agent guidance appears with its own lifecycle requirements. Partly observed: F1, F4.
- 更偏向完全趋同：企业普遍开始微调或训练自己的模型（9% 这一比例[6]大幅上升）；或监管把生成式 / 智能体 AI 重新纳入通用模型风险指引。
- 更偏向彻底分化：主流平台把 ML 与 LLM / 智能体产品及注册表拆开；或出现带有独立生命周期要求的生成式 / 智能体专门指引。已部分出现：F1、F4。
References 参考文献
- Board of Governors of the Federal Reserve System, OCC, FDIC, SR 26-2: Revised Guidance on Model Risk Management, 17 April 2026 (supersedes SR 11-7). letter · attachment (scope: footnote 3)regulatory guidance · S-0006
- NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, 26 July 2024. nist.govstandard · S-0007
- Regulation (EU) 2024/1689 (Artificial Intelligence Act), Chapter V (general-purpose AI models) and Art. 113. eur-lex.europa.euregulation · S-0003
- MLflow, “MLflow 3” launch post, 9 June 2025. mlflow.orgvendor blog · S-0010
- M. Zaharia et al., “The Shift from Models to Compound AI Systems,” BAIR Blog, 18 February 2024. bair.berkeley.eduresearch blog · S-0008
- Menlo Ventures, “2024: The State of Generative AI in the Enterprise,” 20 November 2024 (survey of 600 US IT decision-makers). menlovc.comindustry survey · S-0009
- L. Chen, M. Zaharia, J. Zou, “How is ChatGPT's behavior changing over time?,” arXiv:2307.09009, 2023. arxiv.orgpaper · S-0011
- L. Zheng et al., “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” NeurIPS 2023 Datasets and Benchmarks. arxiv.orgpaper · S-0012
- K. Greshake et al., “Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection,” arXiv:2302.12173, 2023. arxiv.orgpaper · S-0013
- Y. Shavit et al. (OpenAI), “Practices for Governing Agentic AI Systems,” 2023, §4. openai.comwhite paper · S-0014
Last updated · 最后更新 2026-09-30. Revision: added sources; revised H5 after SR 26-2.修订：补充来源；因 SR 26-2 修订 H5。