LLMs - One-Sized to Verticalized

LLMs in Focus: From One-Size Fits All to Verticalized Solutions // Venky Ganti and Laurel Orr // MLOps Podcast #196

Looking at the Home Alone franchise, it's easy to see that sequels are hard.

Thankfully, we're talking about text to SQL in this episode. So, although it comes with its challenges, at least you know no matter how much it goes wrong, you won't end up with Home Alone 3.

Here to help you avoid the disaster that is Home Alone: The Holiday Heist are Laurel and Venky from Numbers Station. We went into what the different challenges are, but also where you can get unequivocal gains and what you can do to make sure that you set yourself up for success.

We also talked all about open-source AI and what some of the downsides are of one-size-fits-all LLMs, why they're not ideal, and how to make this trade-off between size and performance.

So, enjoy a nice quiet night in watching the episode. Just don't create a series of unhinged, life-threatening booby traps around your house before you do.

Video || Spotify || Apple

Darek Kłeczek from Weights and Biases // Michelle Marie Conway from Lloyds Banking Group // Oliver Chipperfield from M-KOPA // Mark Huang from Gradient

In an episode sure to interest Warren G and Nate Dogg, we dig in to regulated environments, and how to manage your models in them.

We all have to deal with regulations to some extent, so listening to experts share how they do it gives some great insights.

They swap tips on making sure data science projects play nice with legal and risk management rules, stress the need for clear explanations, transparency, and keeping the conversation flowing with business folks. Plus, they dig into how tools can help manage models, crank up productivity, and amp up teamwork.

So, don't hit the Eastside, hit the link below instead.

You can watch it here.

LLMOps: The Good, The Bad, The Ugly // Stefan Sabev // IRL Meetup #58 Edinburgh

Although the title of this is a classic Western film, it actually brings to mind a more seasonal offering; A Christmas Carol.

Stefan begins with the Ghost of LLMs Past, talking about the release of the "Attention Is All You Need" paper in June 2017.

For the Ghost of LLMs Present he discusses the the impact of LLMs on various industries such as finance, HR, and education.

There’s a healthy dose of Scrooge in there too, as he looks at cost and optimization, outlining different approaches, including self-hosting models and fine-tuning for specific use cases.

Then for the Ghost of LLMs Yet To Come he concludes by encouraging the audience to explore and build creative applications using LLM technology.

And if ghost stories are too scary for you, he does address concerns around privacy, safety, and "hallucinations” and how they can be managed.

As Tiny Tim might say, “AI bless us, every one!”

Check it out here!

💡Job of the week

Staff/Principal Backend Engineer // Linea (San Francisco/hybrid, US)

Our groundbreaking product enables data engineers to generate pipelines directly from data science notebooks, track data flows and lineage, and swiftly identify and resolve issues.

Responsibilities:

Lead key product feature design and development for Linea Platform to meet a diverse set of data engineering requirements.
Define product requirements with PMs and lead the implementation of a high-performing product.
Lead the design and implementation of a scalable system to support existing and upcoming features.

Requirements:

Hands-on experience in designing large scale systems (e.g. microservices, SQS, Kafka, etc.) and APIs
Familiarity with data pipeline orchestration frameworks such as Airflow, Prefect, Oozie, etc.
Experience with at least one of the cloud infrastructure (AWS, Azure, GCP, Kubernetes)

LLMOps: Why Does It Matter?

At this time of year, there’s a lot of talk about Noël. This blog goes the other way and it’s all about the L - LLMOps to be precise.

It outlines what LLMOps is, and some key ways it differs from MLOps, including prompt versioning, finetuning, and computational resources.

And, because of these differences, it helpfully outlines what could be considered components of LLMOps, like model monitoring, and the benefits of LLMOps such as scalability.

I'm sure if you read it, you'll have an 'L'-evated understanding of LLMOps!

Huge thanks to Samhita Alla from Union.ai for the contribution.