MLflow 2.8.0

MLflow 2.8.0 includes several notable new features and improvements

• The MLflow Evaluate API has had extensive feature development in this release to support LLM workflows and multiple new evaluation modalities. See the new documentation, guides, and tutorials for MLflow LLM Evaluate to learn more.
• The MLflow Docs modernization effort has started. You will see a very different look and feel to the docs when visiting them, along with a batch of new tutorials and guides. More changes will be coming soon to the docs!
• 4 new LLM providers have been added! Google PaLM 2, AWS Bedrock, AI21 Labs, and HuggingFace TGI can now be configured and used within the AI Gateway. Learn more in the new AI Gateway docs!

Features:

• [Gateway] Add support for AWS Bedrock as a provider in the AI Gateway (#9598, @andrew-christianson)
• [Gateway] Add support for Huggingface Text Generation Inference as a provider in the AI Gateway (#10072, @SDonkelaarGDD)
• [Gateway] Add support for Google PaLM 2 as a provider in the AI Gateway (#9797, @arpitjasa-db)
• [Gateway] Add support for AI21labs as a provider in the AI Gateway (#9828, #10168, @zhe-db)
• [Gateway] Introduce a simplified method for setting the configuration file location for the AI Gateway via environment variable (#9822, @danilopeixoto)
• [Evaluate] Introduce default provided LLM evaluation metrics for MLflow evaluate (#9913, @prithvikannan)
• [Evaluate] Add support for evaluating inference datasets in MLflow evaluate (#9830, @liangz1)
• [Evaluate] Add support for evaluating single argument functions in MLflow evaluate (#9718, @liangz1)
• [Evaluate] Add support for Retriever LLM model type evaluation within MLflow evaluate (#10079, @liangz1)
• [Models] Add configurable parameter for external model saving in the ONNX flavor to address a regression (#10152, @daniellok-db)
• [Models] Add support for saving inference parameters in a logged model's input example (#9655, @serena-ruan)
• [Models] Add support for completions in the OpenAI flavor (#9838, @santiagxf)
• [Models] Add support for inference parameters for the OpenAI flavor (#9909, @santiagxf)
• [Models] Introduce support for configuration arguments to be specified when loading a model (#9251, @santiagxf)
• [Models] Add support for integrated Azure AD authentication for the OpenAI flavor (#9704, @santiagxf)
• [Models / Scoring] Introduce support for model training lineage in model serving (#9402, @M4nouel)
• [Model Registry] Introduce the copy_model_version client API for copying model versions across registered models (#9946, #10078, #10140, @jerrylian-db)
• [Tracking] Expand the limits of parameter value length from 500 to 6000 (#9709, @serena-ruan)
• [Tracking] Introduce support for Spark 3.5's SparkConnect mode within MLflow to allow logging models created using this operation mode of Spark (#9534, @WeichenXu123)
• [Tracking] Add support for logging system metrics to the MLflow fluent API (#9557, #9712, #9714, @chenmoneygithub)
• [Tracking] Add callbacks within MLflow for Keras and Tensorflow (#9454, #9637, #9579, @chenmoneygithub)
• [Tracking] Introduce a fluent login API for Databricks within Mlflow (#9665, #10180, @chenmoneygithub)
• [Tracking] Add support for customizing auth for http requests from the MLflow client via a plugin extension (#10049, @lu-ohai)
• [Tracking] Introduce experimental asynchronous logging support for metrics, params, and tags (#9705, @sagarsumant)
• [Auth] Modify the behavior of user creation in MLflow Authentication so that only admins can create new users (#9700, @gabrielfu)
• [Artifacts] Add support for using xethub as an artifact store via a plugin extension (#9957, @Kelton8Z)

Bug fixes:

• [Evaluate] Fix a bug with Azure OpenAI configuration usage within MLflow evaluate (#9982, @sunishsheth2009)
• [Models] Fix a data consistency issue when saving models that have been loaded in heterogeneous memory configuration within the transformers flavor (#10087, @BenWilson2)
• [Models] Fix an issue in the transformers flavor for complex input types by adding dynamic dataframe typing (#9044, @wamartin-aml)
• [Models] Fix an issue in the langchain flavor to provide support for chains with multiple outputs (#9497, @bbqiu)
• [Docker] Fix an issue with Docker image generation by changing the default env-manager to virtualenv (#9938, @Beramos)
• [Auth] Fix an issue with complex passwords in MLflow Auth to support a richer character set range (#9760, @dotdothu)
• [R] Fix a bug with configuration access when running MLflow R in Databricks (#10117, @zacdav-db)

Documentation updates:

• [Docs] Introduce the first phase of a larger documentation overhaul (#10197, @BenWilson2)
• [Docs] Add guide for LLM eval (#10058, #10199, @chenmoneygithub)
• [Docs] Add instructions on how to force single file serialization within the onnx flavor's save and log functions (#10178, @BenWilson2)
• [Docs] Add documentation for the relevance metric for MLflow evaluate (#10170, @sunishsheth2009)
• [Docs] Add a style guide for the contributing guide for how to structure pydoc strings (#9907, @mberk06)
• [Docs] Fix issues with the pytorch lightning autolog code example (#9964, @chenmoneygithub)
• [Docs] Update the example for mlflow.data.from_numpy() (#9885, @chenmoneygithub)
• [Docs] Add clear instructions for installing MLflow within R (#9835, @darshan8850)
• [Docs] Update model registry documentation to add content regarding support for model aliases (#9721, @jerrylian-db)

For a comprehensive list of changes, see the release change log, and check out the latest documentation on mlflow.org.