MLflow VS Kubeflow

In the latest coffee session, we came back to our best-performing podcast ever with Byron Allen. He is a senior consultant at Servian.

Yea, I Know. Kubeflow and MLflow again. 2022 edition. Conversations around these two tools never end.

What makes MLFlow and KubeFlow? These two MLOps Lifecycle popstars have been topping the trend charts for a couple of years now. Pointing out their differences is key to understanding what made them popular.

At its core, it goes something like this. MLflow is a straightforward package that enables users to handle the experiment phase in the MLOps Lifecycle. KubeFlow is more of an end-to-end platform for MLOps. It's core feature is centered around the orchestration of ML pipelines. Kubeflow pipelines. An abstraction on top of Argo workflows, which we discussed at length a week ago. However, KF is not a one trick pony. It supports much more than pipelines.

Nuances: Kubeflow has a tightly coupled structure that is dependent on Kubernetes and DevOps practices. This makes setup and integration with other tools a bit more rigid. It is ideal when working in a cross-functional team because in practice it's meant to be used by a data scientist/ML engineer, but they might not necessarily be experienced in Dev/Ops.

MLFlow has more of a decoupled structure that enables a very high level of flexibility. It makes it easy to swap out and integrate functionalities with other tools. This gives more liberty to the way it is used.

We break down all the differences and how the tools have evolved in the latest coffee session.

Blood sweat and tears were put into the first annual AI Infrastructure Ecosystem report.

With hundreds of ML infrastructure tools on the market, how do you make sense of them all? Welp, this report aims to provide a comprehensive and clear overview of the AI/ML infrastructure landscape.

Random Tangent - Intelligence has been a tremendous competitive advantage throughout history.

It’s not just about having it, it's about wielding it effectively.

Over the last decade, intelligence has undergone a profound change. It’s no longer just confined to our heads.

Machine learning makes it possible to productionize intelligence for previously unsolvable challenges.

Take a moment. Step back. Remember. we are writing the history books as we go forth.

/end tangent

P.S. the team that created this report is looking to hear your feedback. What are your thoughts? Hit reply and give me your unfiltered hot take.

Rasgo recently launched a free SQL generator.

For those unfamiliar with what a SQL Generator is. Think template for SQL queries, which lets you customize column names and table structure, choose the operation that you want to do, and then, it constructs the syntax for you in different "flavors" of SQL.

Never stress about the subtle differences between DATEDIFF() vs DATE_DIFF() again!

The Story? People were searching on Google and Stack Overflow for required SQL syntax and wasting precious time that could be used for data analysis. The Rasgo team wanted to help - no strings, paywalls, or email collections attached.

Select the transform you’re running and the column names of your table. The SQL generator will automatically give you the SQL you need in copy/paste form.

Pretty Kewl.

Last week, Tecton announced its $100 million Series C funding, proving that the MLOps market is still hot 🔥🔥🔥

This round was led by new investor Kleiner Perkins with participation from strategic investors Databricks and Snowflake Ventures, previous investors Andreessen Horowitz and Sequoia Capital, and new investors Bain Capital Ventures and Tiger Global.

Tecton plans to use this investment to further deliver on customer value, scale both engineering and go-to-market teams, and continue helping enterprises experience firsthand the significant uplift of leveraging real-time or streaming data in predictive data products.

We're having an outstanding AMA session with Romain Cledat and Kedar Sadekar, senior engineers building ML infrastructure at Netflix! The event is held in our Slack channel #ask-me-anything. Join in and ask any question, or schedule one if you can't attend this Thursday at 7pm CET/10am PST.

Romain Cledat, is a Ph.D. senior software engineer in the Machine Learning Infrastructure group at Netflix that is responsible for Metaflow among other things. 3+ years at Netflix, ex-Facebook, ex-Intel.

Romain Cledat: Every ML Model Deserves To Be A Full Micro-service | PyData New York 2019 (the infrastructure used at Netflix enables data-scientists to quickly deploy their model as a microservice to perform inference or integrate with other systems)
ML with Metaflow and Kubernetes: Prototype to production on Amazon EKS - AWS re:Invent 2021
Ahmad Houri: How Netflix Metaflow Helped Us Build Real-World Machine Learning Services 2021 (example use-case with Metaflow).

Kedar Sadekar, a senior software engineer in the Personalization Infrastructure team at Netflix that builds scalable, distributed computing systems for the algorithmic engineers that help improve member personalization. 11+ years at Netflix. Key talks:

Scalable Logging and Tracking 2012
Spark and Spark Streaming at Netflix 2015
Heterogeneous Workflows With Spark At Netflix 2016 (on a Netflix framework that enables Data Scientists to design and run machine learning pipelines that seamlessly exchange data and artifacts between Spark jobs and other components)
Fact Store at Scale for Netflix Recommendations 2018 (how Netflix uses historical facts data to generate features required by the machine learning model)