Automate Everything in MLOps

Letters. Words. Sentences. Paragraphs.

For context: Last week the reading club met for the 3rd time and talked about Software Engineering for Machine Learning: A Case Study.

As always, great turnout with mentally stimulating conversations. I default to Daniel for some in-game commentary, however this week he only caught the 2nd half. He was still able to formulate some key takeaways.

"The one thing that kinda stood out to me, as an unanswered question, is how to find the right level of abstraction for monitoring/understanding models across various stakeholders.

That was my big takeaway, thinking about how to design stuff that enables not so technical stakeholders but also lets the power users dig into the details"

Korri Jones had this to say:

"Team Makeup - Data Scientists & Engineers should be on the same team, under the same leadership in order to accelerate MLOps efforts. And that this will eliminate "Hand-offs" and change them to "Handshakes" between the teams."

A question unanswered thus far in the slack channel is around automation.

"The importance of having automated end-to-end pipelines was brought up in this paper as well as in Google's Rules of ML.

For those who have mature tooling around this, was it built or bought? What lessons have you learned? For those who don't, what are the gaps that current tool options still have?"

Maybe you can help answer that question in slack?

Deploy Models at Scale in the Cloud

Vishnu Prathish came to talk to us about MLOps and time series forecasting models. For those of you wondering, Vishnu's company Innovyze is tackling challenges around water infrastructure.

Some of the physical pipes he is building ML models for are wooden. Data is being gathered manually once a day, then emailed to his team. I do not envy them. If you ever were to complain about your data, just remember Vishnu making it happen against all odds!

What to expect from the convo?

Why he prefers cloud-native tooling
What does his almost 100% automated retraining pipeline look like
Why some problems cant be solved with ML
How his team creates digital twins to gain insights

Some of my favorite quotes:

"A pump is a pump is a pump." Referring to how you don't get much diversity introduced into water pumps. Crack the code once and you are smooth sailing. Unless you haven't got the data. Coincidentally, that also happens.

Data science is a messy process. One word: reproducibility. Reproducibility is a must for Innovyze. the whole business is centered around ML.

The Incredible convo with Vishnu can be found here on video and podcast here.

Remember to never complain about your data unless someone is emailing you 3 days after the fact with the data points.

War Stories Galore

This week, Demetrios and I had the pleasure of talking to Nick Masca, the head of Data Science at Marks and Spencer Group, the iconic retailer. A conversation with MLOps war stories. Better said, a war story conversation. The kind that informs modern MLOps best practices.

Nick shared how to make MLOps organizational changes at large companies. I loved one tidbit he mentioned--"it's an evolution, not a revolution". That's a frank observation about the speed of practical change. As we all know it doesn't happen overnight.

Another great learning Nick shared focused on the value of delivering incremental results regularly. Oftentimes, ML projects suffer because of a focus on delivering too much too soon. This can then lead to a trough of disappointment with the way things actually pan out. Nick shared his experience on how to avoid such pitfalls with us so you dont have to learn the hard way. I highly recommend listening to his tactical advice on YouTube or on Anchor!

Till next time,
Vishnu

NO MEETUP THIS WEEK

We will be starting the amazing apply() conference at the normal meetup time on Wednesday. It will last until about 3pm PST. I know for the Euro folks like myself that's late. But I'm staying up. It's gonna be worth it.

Some of the talks I am most looking forward to:

Wes McKinney - Tying the Room Together: Apache Arrow and the Next Generation of Data Analytics Systems
Chip Huyen - Machine Learning is going real time
Orr Shilon - A Point in Time: Mutable Data in Online Inference
Stefan Krawczyk - Hamilton: a Micro Framework for Creating Dataframes

The talk that has got me the most intrigued, I also have the highest expectations for: "The Only Truly Hard Problem in MLOps" by Todd Underwood and Niall Murphy. You may remember Todd from this coffee session. And Niall? Oh yeah, he wrote the book on being an SRE. Don't let me down fellas!

Check out the full agenda here. See you there!

Recurring Themes

In this newsletter alone we have talked about automating retraining pipelines twice. You can imagine how much I hear this topic come up on a weekly basis.

Two coffee sessions stand out to me for having valuable insights around when and why ML pipelines should be automated. I grabbed a few quotes from those sessions and threw them on a medium post.

The nice part about being able to stand of the shoulders of giants, I don't have to come up with anything original. I get to quote the ones who are actually putting in the work!

Top Quotes

‘You don’t want to jump directly to automated processes at the start. With anything in life, I think you want to do a thing yourself first, so you discover any issues or pitfalls. Do that, and it’s easier to feel confident enough to automate.' - Luigi Patruno

'In some cases, you can use automation from the outset by focusing on the boring bits.' - Neal Lathia