Engineering Labs recap

This talk will provide some background on the March 2022 MLOps EngineeringLab in collaboration with the team behind Flyte, then we will go into lightning talks given by the three winning teams.

Flyte is the backbone for large-scale Machine Learning and Data Processing (ETL) pipelines at Lyft, Spotify, BlackShark, and many others. It is used across business-critical applications ranging from ETA, Pricing, Mapping, Autonomous Vehicles, and many more. At its core is a Kubernetes native workflow engine that executes 10M+ containers per month as part of thousands of workflows.

The talk will focus on:
- What does a Simple ML User Journey look like?
- The sacrifices ML Practitioners have to do to achieve Robustness?
- How does Flyte help bridge the gap?

If you’re an organization deploying ML models (you must be if you’re reading this newsletter!), then responsible and fair AI should be one of your top priorities. We had a great time discussing the state of responsible AI with Krishnaram, who is literally one of the godfathers on the topic. He gave us incredibly useful frameworks and criteria for assessing how fair a model is and how you can deploy a minimum explainable product.

Additionally, if you want to help, you can fill this survey for the technical paper the Fiddler team is working on and the results will be shared with the community! Win Win!

As inference models become a core pillar of cloud native applications, organizations are looking for ways to operationalize these workloads in the cloud, edge and on prem. The real-time nature of many inference applications creates resource management challenges that we don’t see in training workloads. For example, Uber Eats users don’t want to wait to learn how long it will take to receive their order until after the food arrives. Unfortunately, most of us can't throw unlimited compute resources at the problem to guarantee low latency and fast response times.

So, how can MLOps Engineers maximize reliability and performance within the constraints of their available resources? How can they manage the lifecycle of re-training, updating and deploying models as the size of the total deployment grows?

Whether you’re already using tools like Seldon and Triton or you’re getting ready to deploy your first application, it pays to learn about the challenges you’ll encounter in the next stage of growth. Check out this free guide to ensure you're prepared to meet SLAs and stay in control of your growing inference workloads.

Last week we sat down with Maarten and Jovan, the creators and maintainers of Vaex, a powerful new data processing tool.

Vaex empowers data scientists to process more data than can fit in RAM, opening the door to analyzing terabytes of data on a laptop, without a cluster. Maarten walked us through the motivation and history, and Jovan stepped us through a coding example, from based arithmetic operations, to custom numpy functions, even training a scikit learn model, all on over 120 million rows of data (>100GB), on a laptop, in seconds.

If you’re looking to level up your data science skills and add some magic to your toolbox, you should check out their full talk.