|
|
|
|
Great to see you've opened this, but you should be reading this later!
Right now you should be joining in Day One of our AI in Production conference!
If you don't manage to join in live, you'll be able to catch up and join our 20,000 subscribers on our YouTube
channel!
I know, 20,000! That's amazing - huge thank you to all of you! 💖Really makes up for the lack of Valentine's cards I got.
|
|
|
|
|
|
|
|
|
LLM
Evaluation // Aparna Dhinakaran // MLOps Podcast #210
Golden Retrievers.
Not just super furry animals, but also a product name I need to trademark after being inspired during this podcast.
It was great a chat about that classic headache: evaluation.
We get into custom evaluations, particularly concerning application needs, the Phoenix evaluations library, LLMs as judges, score evals vs classification evals, and more.
Aparna even shares her ‘hot takes’ about how finetuning an open-source model versus a private model ends up slowing down product development.
And while we’re on evaluation, we’re conducting a new evaluation survey. We’d love it if you could take five minutes and fill it out.
There’s also a blog below to learn all about the survey we did in September 2023, so feel free to evaluate that!
|
|
|
Enterprise Model Management Course from Weights &
Biases
|
|
|
|
|
|
|
In the rapidly evolving field of machine learning, the difference between leading and lagging behind often hinges on the efficiency and sophistication of your model management practices. If you're seeking to transform your approach to managing, versioning, and deploying machine learning models, the Weights & Biases AI Academy's Enterprise Model Management new free course is a great place to start.
This course offers a deep dive into storing, versioning, and evaluating your models with the strategies and tools used by top enterprise companies today. Learn from industry experts Hamel Husain and Noa Schwartz, who bring real-world insights and case studies, ensuring you gain practical, applicable knowledge.
With a curriculum that covers everything from leveraging the Weights & Biases Model Registry to mastering automation with
webhooks, this course is tailored to empower you with the skills to streamline your ML workflows, enhance collaboration across teams, and ensure your models are deployed with confidence and precision.
Whether you're looking to improve model governance, automate your ML pipelines, or simply stay ahead in the competitive field of machine learning, this course will equip you with the knowledge and tools to achieve just that. Join us to unlock the full potential of your models and workflows, setting a new standard for excellence in your projects.
Enroll here for free
|
|
|
|
|
|
|
Ads Ranking Evolution at Pinterest // Aayush Mudgal // MLOps Podcast #211 From evaluation in the last podcast to evolution in this one.
It’s no easy task either, and Aayush was really honest about how they’ve been doing it at Pinterest from 2018 to today.
He talks about the transition from
cascading and scalding to Spark jobs due to scalability and flexibility needs, switching from Tensorflow one to Pytorch, and the transition from XGBoost-based GBTDs to deep learning models in recommendation systems. Plus the shift in their optimization strategy and the process of identifying users in a buying state and figuring out which ads to present to them.
Through each of these transitions they’re always thinking about the ROI every time they evolve, or as my dad used to say, “Is the juice worth the squeeze?”
Orange-juice glad you listened to the podcast now?
|
|
|
💡Job of the weekSenior Data Scientist: Consultant and Training // AlignAI (US, remote)AlignAI offers innovative solutions to enhance data-driven decision-making. They are seeking a Senior Data Scientist for a dual role in consulting and leading training programs, focusing on developing and deploying machine learning models and educational materials. Responsibilities: - Conduct exploratory data analysis and model development.
- Create and deliver training materials on machine learning concepts.
- Guide data science projects utilizing Large Language Models.
Requirements: - 4+ years of experience in data science and machine learning.
- Proven track record of deploying ML models into production.
- Strong Python coding skills and experience in customer project scoping.
|
|
|
MLOps Community IRL Meetup
|
|
|
|
|
Individual Projects on LLMs // IRL Meetup #64 Lagos
Some quick-fire projects, including:
- automating data analysis using an LLM agent to extract, transform, and load data from Cargo
- an AI finance solution for small and medium enterprises
- a pyramid text summarization model utilizing a transformer model to summarize dialogues, detailing the workflow from data injection to model training and UI integration
- a semantic search model for finding books based on user prompts
Plenty there to inspire you!
|
|
|
|
|
|
|
🖊️ Brilliant Bloggers: Connect and Contribute!What makes this community unique is the amazing members and their continual involvement. And we'd love you to add your spark.
Whether it's sharing your writing flair or lending a hand with proofreading and editing, every bit counts.
Find out more in our guides and join us in the writing community on Slack.
|
|
|
How to Adapt Your LLM for Question Answering with Prompt-TuningFine-tuning is fine, but this blog may prompt you to look closer at prompt-tuning. It contrasts prompt-tuning with traditional fine-tuning,
emphasizing its efficiency and adaptability. It includes detailed explanations of prompt tuning and p-tuning mechanisms, dataset preprocessing, and formatting for model training. It also covers practical steps for implementing these methods using the SQuAD dataset. Oh, and be sure to follow the prompts below some of the pictures to get the full experience! With thanks to Anish Shah from Weights and Biases for their contribution.
|
|
|
Evaluation Survey InsightsWay back in the mists of time (September 2023) we conducted a survey to gather insights on evaluating LLMs. This blog is a summary of some key insights, including:- Budget Allocation for LLMs: 81% of surveyed professionals had dedicated budgets for LLMs. There's money there!
- Preference in Model Selection: Smaller, open-source models for ease of deployment or OpenAI for superior performance?
- Challenges in LLM Evaluation: Measuring output quality, addressing data scarcity, and managing hallucinations.
Plus we’re conducting a new evaluation survey. It'd be great if you could take a few minutes to fill it out.
|
|
|
|
|
Add your profile to our jobs board here
|
|
|
|
|
|
|
|
|
|
|
