Product Dimension

On this podcast, we had Ethan Rosenthal, engineering manager at Square, talk about ML solutions from the product perspective.

Product Managers X ML Products
Developing a product is already hard enough with regular "software engineers," and it becomes even more complex when you top that with data/ML people.

A product manager might not get the technical bits and pieces of what/how the engineers might want to approach the solution.

When building ML products, product managers need a certain level of data science intuition and engineering understanding. This doesn't necessarily imply being technical.

At the core of product management, being very smart and able to make good decisions makes a good product manager.

Large Language Models
Lately, many NLP applications might have different bespoke solutions for various tasks like questions and answering, text classification can be solved using one large language model.

There are trade-off that exists between using large language models to handle many tasks together, over using single language models for single tasks. It's easy to get wacky outputs from these large language models.

From a product manager's perspective, the big question is, "With the errors that these models make, what use cases are they applicable to at their current stage of technology?"

Figuring out cases where the results of these models are less costly is crucial.

How much you rely on the human in the loop is a way to navigate how conservative you have to be with the model.

On this meetup, we had Senior Consultant Marouen Hizaoui and Senior Consultant/MLOps Lead Mo Basirati from Reply. They shared common MLOps challenges from their experience.

Reply MLOps
Reply is a company that offers consulting, technology, and agency services across industries to clients all over the world.

To Reply, MLOps is a set of concepts and best practices that aims to build a reliable and efficient machine learning system.

MLOps principles are broken down into six different categories, which include: versioning, testing, monitoring, automation, deployment, and reproducibility.

The target market for Reply cuts across a wide range of industry sectors, from big tech companies to non-tech companies.

MLOps Problems
In MLOps, people tend to think more about the latest tools and technology, but in practice, the principles are more important.

The team structure is one problem that might appear trivial and can easily get overlooked, but it is very important in implementing MLOps.

Most companies' typical team structure setup is to create two different sub-teams for data and operations. This isolation causes their operations, workflows, and processes to be siloed into two worlds. They have different managers, budgets, toolings, e.t.c.

This brings about communication problems, the development environment problems, e.t.c.

Pair programming, weekly meetings, setting conventions and standards, and increasing awareness about the situation are some of the solutions to help address team structure challenges.

At the last Lisbon meet-up, we had Catarina Silva, Staff Engineer at Unbabel, Luís Silva, Team Lead AI Platform at Outsystems, Jorge Pessoa, Software and Data Engineer at Ntropy, and Pedro Coelho, EMEA ML Engineering Leader at Zendesk. They discussed the gains and benefits of machine learning platforms thinking in developing ML solutions.

Different Organizational Archetypes
Development performance is affected by the organizational archetype that is implemented. This could either be functional-oriented or market-oriented teams.

Functional-oriented teams optimize for expertise, where a specific team has a specific function. There is divisional labor since they are separated into different groups.

Market-oriented teams optimize for responding quickly to customer needs, having functional customer roles, and independently developing testing and deployment to production.

Ups and Downs
In a nutshell, they serve different purposes at different times. The market-oriented approach is extremely fast at providing value and scales well. There is fluid collaboration because all the context and knowledge regarding the AI service exists within the product team. At the end of the journey, the platform team will help manage the end-to-end lifecycle.

This helps the product team focus on iterating a better version of the product and abstract the complexities for the marketing teams.

Time to market is super fast.

The functional approach comes in handy when there is a need to make do with the available skillset and manage time.

And based on the specific scenario, you don't have to build more teams to handle more features. In summary, many teams aren't needed in the functional approach, but if the context is switched, many things might not work as intended.