MLflow 3
The open source MLflow community has reached a major milestone. Today, we're releasing MLflow 3, which brings production-ready generative AI capabilities to the platform that millions of developers trust for ML operations.
This isn't just another feature update. MLflow 3 fundamentally expands what's possible with open source ML tooling, addressing the observability and quality challenges that have made GenAI deployment feel like a leap of faith.
Major Updates
🎯 MLflow LoggedModel
MLflow 3 introduces a refined architecture with the new LoggedModel entity as a first-class citizen, moving beyond the traditional run-centric approach. This enables better organization and comparison of GenAI agents, deep learning checkpoints, and model variants across experiments.
Learn more about MLflow LoggedModel in the documentation.
🔗 Strong Lineage Support
Enhanced model tracking provides comprehensive lineage between models, runs, traces, prompts, and evaluation metrics. The new model-centric design allows you to group traces and metrics from interactive queries and automated evaluation jobs, enabling rich comparisons across model versions.
New GenAI Evaluation Suite
MLflow's evaluation and monitoring capabilities help you systematically measure, improve, and maintain the quality of your GenAI applications throughout their lifecycle. From development through production, use the same quality scorers to ensure your applications deliver accurate, reliable responses while managing cost and latency.
Learn more about the new GenAI evaluation suite in the documentation.
The new evaluation suite is available only in Managed MLflow on Databricks, with open source support coming soon. Interested in trying it out? Start a free Databricks trial to explore these features today.
⚡ Prompt Optimization
The MLflow Prompt Registry now includes prompt optimization capabilities, allowing you to automatically improve prompts using evaluation feedback and labeled datasets. This includes versioning, tracking, and systematic prompt engineering workflows.
Learn more about prompt optimization in the documentation.
📚 Revamped Documentation
The MLflow documentation has been fully redesigned to support two main user journeys: GenAI development and classic machine learning workflows. The new structure offers dedicated sections for GenAI features (including LLMs, prompt engineering, and tracing), and traditional ML capabilities such as experiment tracking, model registry, deployment, and evaluation.
Visit the new MLflow 3 documentation to explore the updated content and guides.
Other Features
Streaming API for ResponsesAgent: New streaming response capabilities through the ResponsesAgent class with predict_stream method, enabling real-time streaming for GenAI applications (docs)
Auto-tracing support for PydanticAI and smolagents: Enhanced auto-tracing integrations for emerging GenAI frameworks, providing seamless observability out of the box (PydanticAI docs and smolagentsdocs).
Add search_prompts API for prompt registry: New API functionality for searching and discovering prompts in the registry, making prompt management more efficient (API docs: search_prompts).
Support token tracking for OpenAI/LangChain auto-tracing: Enhanced tracing now captures detailed token usage and cost information for better observability and cost management.
Record environment metadata in tracing: MLflow automatically captures standard environment metadata like source name, Git commit hash, and execution type as tags on traces (docs).
UI support for video artifacts: The MLflow UI now supports viewing video files directly in the artifact viewer, expanding beyond traditional ML artifacts.
and many more: Numerous other enhancements across tracking, model registry, and UI components to improve usability, performance, and developer experience.
Breaking Changes
MLflow 3 includes several breaking changes as part of improving framework consistency and performance. Key changes include removal of MLflow Recipes, fastai and mleap flavors, and various deprecated API parameters.
For the complete list of breaking changes, visit the MLflow 3 Breaking Changes documentation.
Upgrade Recommendation
We recommend testing MLflow 3 in a separate environment before upgrading your production workflows to ensure compatibility with your existing setup.
Getting Started
pip install 'mlflow>=3.1'
Explore the new MLflow 3 documentation and try out the enhanced GenAI capabilities with our updated quickstart guides. The model-centric architecture and improved tracing make it easier than ever to build, evaluate, and deploy production-ready AI applications. Explore the MLflow 3 documentation to learn more about the new features and how to get started.
Full Changelog
For the complete list of all changes, bug fixes, and improvements in MLflow 3, visit the full changelog on GitHub.