Databricks New Mosaic AI Capabilities to Help Build Production-Quality AI Systems and Apps

Organizations are struggling to transition Generative AI projects from pilot to full-scale production due to privacy, quality, and cost concerns. While foundation models have all significantly improved, they still struggle to produce high-quality results. The highest-performing models may still give responses that are inaccurate, unsafe, or expose confidential data. To address these challenges, organizations are going beyond deploying one extremely large model to deploying compound AI systems. This approach uses multiple components, including various models, retrievers, vector databases, and tools for evaluation, monitoring, security, and governance. As a result, compound AI systems offer much higher production quality, allowing organizations to deliver more accurate, safe, and governed AI applications efficiently.

“We believe that compound AI systems will be the best way to maximize the quality, reliability, and measurement of AI applications going forward, and may be one of the most important trends in AI in 2024,” says Matei Zaharia, Co-founder and CTO at Databricks. “Databricks is uniquely positioned to capitalize on these trends with the investments we’re making to improve quality, augmenting the model with real-time data and agents and tools to give it new capabilities it has little knowledge of.”

To help customers build production-quality Generative AI applications, Databricks is launching Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Tools Catalog, Mosaic AI Model Training, and Mosaic AI Gateway.

Databricks is introducing several new capabilities to help customers deploy enterprise-ready compound AI systems. RAG is a type of compound AI system, because it uses multiple components like a vector database, and tools for monitoring, evaluation, security, and governance to improve the accuracy of the LLM. Last month, Databricks announced the general availability of Mosaic AI Vector Search as a serverless vector database seamlessly integrated in the Data Intelligence Platform. Today, Databricks announces Mosaic AI Agent Framework, which makes it easy for developers to quickly and safely build high-quality RAG applications, using foundation models and their enterprise data. They can evaluate the quality of their RAG application with Mosaic AI Agent Evaluation, iterate quickly, and redeploy their application easily. Mosaic AI Agent Evaluation is an AI-assisted evaluation tool that automatically determines if outputs are high-quality and provides an intuitive UI to get feedback from human stakeholders. Collectively, these capabilities help organizations to deploy production-quality Generative AI solutions.

Compound AI systems often take advantage of tools as functions that equip these systems with new capabilities to interact with the world, such as intelligently generating and executing code, searching the web, calling APIs, and more. Mosaic AI Tools Catalog lets organizations govern, share, and register tools using Databricks Unity Catalog. This ensures models that are tool-enabled can use these in a secure and governed manner, as well as making these tools discoverable across the organization.

Mosaic AI Model Training enables fine-tuning for foundation models, increasing model quality and decreasing cost

Mosaic AI Model Training fine-tunes open source foundation models with an organization’s private data, giving it new knowledge that is specific to its domain or task. These fine-tuned models are fully owned and controlled by the customer and produce higher-quality results for specific use cases because they have been trained on the organization’s private data for specialized tasks. In addition to being more accurate for specific domains, smaller models fine-tuned by Model Training are also faster and less expensive to serve than larger proprietary models because they have fewer parameters and require less computing power.

Mosaic AI Gateway provides a unified interface to query, manage, and deploy any open source or proprietary model, enabling customers to easily switch the large language models (LLMs) that power their applications without needing to make complicated changes to the application code. It supports usage tracking and guardrails, letting organizations track who is calling the model, set up rate limits to control spending from their enterprise users, and filter for safety and personally identifiable information (PII) regardless of which model is being used. Finally, it offers built-in governance and monitoring to continuously help ensure quality.

“Corning is a materials science company — our glass and ceramics technologies are used in many industrial and scientific applications, so understanding and acting on our data is essential. We built an AI research assistant using Databricks Mosaic AI Agent Framework to index hundreds of thousands of documents including US patent office data,” said Denis Kamotsky, Principal Software Engineer at Corning. “Having our LLM-powered assistant respond to questions with high accuracy was extremely important to us — that way, our researchers could find and further the tasks they were working on. To implement this, we used Databricks Mosaic AI Agent Framework to build a Generative AI solution augmented with the US patent office data. By leveraging the Databricks Data Intelligence Platform, we significantly improved retrieval speed, response quality, and accuracy.”

“FordDirect is on the leading edge of the digital transformation of the automotive industry. We are the data hub for Ford and Lincoln dealerships, and we needed to create a unified chatbot to help our dealers assess their performance, inventory, trends, and customer engagement metrics. Databricks Mosaic AI Agent Framework allowed us to integrate our proprietary data and documentation into our Generative AI solution that uses RAG,” said Tom Thomas, VP of Analytics at FordDirect. “The integration of Mosaic AI with Databricks Delta Tables and Unity Catalog made it seamless to our vector indexes real-time as our source data is updated, without needing to touch our deployed model.”

"As a leading global manufacturer, Lippert leverages data and AI to build highly-engineered products, customized solutions and the best possible experiences," said Kenan Colson, VP Data & AI at Lippert. "Mosaic AI Agent Framework has been a game-changer for us, because it allowed us to evaluate the results of our GenAI applications and demonstrate the accuracy of our outputs while maintaining complete control over our data sources. Thanks to the Databricks Data Intelligence Platform, I'm confident in deploying to production.”

Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Model Training, and Mosaic AI Gateway are now in public preview. Mosaic AI Tools Catalog is in private preview.