MLOps Jungle

Round 2 with Katen Umare, CEO and Founder at Union.ai. His forte is in the dark arts of pipelines. 🪄

Pipelines are often thought about only in the Data Engineering sense. Still, they actually exist everywhere from the software to the business. As humans, we can easily think this way because they are great organizational tools for handling problems. They enable a visual breakdown of the entire process involved in an approach. Each piece gets classified into a step. They also promote the scalability and independence of the components that exist within them.

SWE vs ML Pipelines
The static/stateless nature of traditional software vs dynamics of machine learning software. ML has a more dynamic kick because they are driven by rapid changes to provide more optimal solutions to a problem.

Ketan argues their diametrically opposite traits make the existing set of tooling that we have for handling traditional software processes unsuited to capture machine learning software processes. Current orchestration tooling doesn't take into account the fundamental assumption of the constant flux of things. That tweak changes the entire concept of the way to think about pipelines.

Machine learning software-based pipelines push the power of modality down to the user to increase that flexibility. This is core because of the high number of iterations that occur in the build process.

Airflow vs Flyte
There are still debates around how well airflow fits the profile as a machine learning orchestrator. Especially now that we have other emerging orchestrators like flyte.

Airflow has a data engineering spin to it. This means adaptability to Machine Learning software, but the fundamental core of its design (i.e data engineering) doesn't cover the whole story.

Flyte tries to capture the intrinsic assumptions that exist in ML. This is essential to addressing the complexities that are involved.

At the recent Bristrol in-person meetup. We had Luke Marsdon an MLOps Consultant and one of the pioneers of the MLOps community.

The MLOps tooling landscape is mental. Keeping track of all the tools out there is nearly impossible. On the other side of the coin lies attempting to integrate each and every one of these tools together.

Approaching tooling Integration in MLOps
An alternative to going Tom Cruise on the integration of every MLOps tool that exists would be to look at patterns that already exist for integration when they are plugged into each other.

Don't reinvent the wheel.

Borrow patterns from DevOps, Security, and Observability. This perspective points out the dependencies of MLOps on infrastructure, which can now be used to drive the integration path for MLops.

All that sounds a bit familiar? I agree.

Last Thursday, there was an online ask-me-anything session with two senior ML infrastructure engineers at Netflix, Kedar Sadekar (low-level ML infrastructure, algorithms) and Romain Cledat (high-level ML infrastructure, tooling).

Spoiler Alert. They did not tell us when stranger things season 5 comes out.

Within one hour, the guests answered 44 questions ( =1.3 answers per minute!)

Questions ranged from ML infrastructure and operations at Netflix, what the team has gained from open-sourcing their internal ML framework, how they manage model training pipelines at Netflix, how they deploy and monitor models, and many many more!

Not too late to see all the insights. Join #ask-me-anything channel to read the session, and stay tuned for the next session with Union.ai team (maintainers of Flyte) on August 18th!