MLOps Trenches

The Long Tail of ML Deployment

Tuhin Srivastava, Co-Founder and CEO of Baseten, talked about empowering software engineers with machine learning and AI.

Data scientists needing to learn engineering is quite a popular song in the ML space, but the flip side of this coin is that engineers also need to know about machine learning.

Why? From a future perspective, data scientists are a relic of the past. Back then, there wasn't a clear understanding or structure to get value from machine learning. It was more like a research function for the "data people" to figure out. This thinking had the so-called data science / ML tasks assigned to traditional analysts. Unfortunately, analysis and ML are very different things.

There is so much more leverage when engineers use machine learning because they can build their products without getting other people to build them. In the future, data people who became machine learning people are going to shrink back to being data people. Machine learning will become a massive part of every engineer's toolkit. It is now more important for engineers to learn how to adapt these things without understanding the in-depth working of a model, but they can choose the suitable and appropriate model instead of starting from scratch.

The barrier to entry for ML has been absolutely disintegrated, and the recent trend of anyone being able to use large language models is clear evidence.

Video || Spotify || Apple

End-to-End MLOps Pipeline

Aurimas Griciūnas, Senior Solutions Architect at Neptune.ai, talked about MLOps pipeline processes and tools.

MLOps is a function of different pipelines that make up the ML lifecycle. At its core, it defines the development velocity for machine learning products. Its development velocity is focused horizontally rather than vertically. Maturity, processes, tools, and organizational structure are some of the most important things to consider when actually implementing MLOps in an organization. The training and inference pipelines are the main building blocks of the MLOps pipeline.

ML project lifecycle comprises four concurrent lanes. The feedback loop, stage, artifact handover, and individual (technical professionals) involved.

The stage is broken down into a series of sub-stages throughout the cycle. It starts with the ideation stage, where technical professionals brainstorm whether an idea needs ML and what path to take to provide a solution. Then the experimentation stage, where data scientists carry out experiments to obtain a model that solves the challenges surrounding the idea using available data assets. Next, in the deployment stage, the machine learning engineer deploys the proof of concept (POC) model, which is in the form of a model binary. Then the monitoring stage involves packaging the entire machine learning system with application logging, monitoring, and feedback loop implementation.

Video

Scaling Experimentation for ML at Coinbase

A key tool for assessing the performance of machine learning (ML) models and enhancing product development is A/B testing. Traditional AB testing methods have presented many difficulties for Coinbase, such as restricted testing capacity and time-consuming experimentation procedures. To get around these obstacles, Coinbase created AB universes, a structure for dividing the user space to allow the execution of numerous concurrently conflicting tests.

This piece explores the concept of AB universes, their benefits, and how Coinbase integrated them into its ML systems to ultimately accelerate the adoption and growth of ML-driven services at Coinbase.

A/B Implementation at Coinbase
At Coinbase, they successfully integrated AB universes into two significant ML systems. The ML Feed Engine, which uses universes to power all recommendation systems at Coinbase, including the home feed, was first constructed. The ML Notification Engine, which employs ML to optimize notification delivery, was also switched to use universes.

Read Here

ML is on the Edge

This blog is featured by Wallaroo and written by Martin Bald, Sr Community Manager Wallaroo.AI.

According to Gartner, by 2025, more than 50% of enterprise-critical data will be created and processed outside the data center or cloud. As we start thinking about all the possibilities for AI and Machine Learning projects this will generate, we must remember that only 50% of AI initiatives reach an operational state (Production) within the enterprise. Not only that. Out of that 50%, only 10% produce meaningful ROI.

This is a crucial time to establish best practices and put in place processes and solutions to help make the ML models work more effectively by getting the models into production. This is especially important right now as the capabilities and potential of data are accelerating innovation through LLMs, Computer Vision, and AI at the Edge. Specifically, Edge devices will be generating the data created outside of the cloud at the beginning of this post. This blog examines some challenges Edge AI deployments face and best practices to overcome current assumptions and perceptions.

Read Here

Live LLM App Building Tutorial

Get hands-on building Large Language Model (LLM) apps in this live session on June 15. Yujian Tang, ML Ops Community Volunteer and Developer Advocate at Zilliz, will provide a guided tutorial and pro tips to help you create scale LLM apps.

This live session will explore Large Language Models (LLMs) 's primary challenges in production: high cost and a lack of domain knowledge. To tackle these hurdles head-on, the tutorial introduces vector databases as a solution to effectively address these problems by enabling efficient data injection and caching through vector embeddings.

Through hands-on exercises using LlamaIndex and Milvus, the open-source vector database from Zilliz, participants will develop an LLM application. You’ll walk away with the ability to optimize LLM workflows, improve efficiency, and harness the benefits of vector databases in real-world scenarios.

Key topics covered include the fundamentals of vector databases, LLMs' challenges with data, and effective strategies for resolving these issues. By the end of the tutorial, attendees will be equipped with the knowledge and skills to seamlessly integrate vector databases into LLM production pipelines, empowering them to maximize the performance and practicality of these language models.

Save your Spot