Dr. Waleed Kadous, Head of Engineering at Anyscale, shares from his long years of experience, the reoccurring circle of events in the development and application of ML.

The revolution of ML has been great over the years, especially the constant solutions for reoccurring issues like scalability. Recently ML has rapidly gone mainstream, with the event of LLMs and huge models. Unfortunately, this journey is taking us down memory lane of familiar scalability problems.

On one hand, the reoccurring loop of getting more computational power and doing crazier stuff which requires more computational power is a large part of the problem. Complex or huge amount of data and the requirements for real-time application is the other part of the story.

MLOps Level of Maturity
Making infrastructural and architectural choices as a new ML company is crucial. There is the option of using commercial tools to build in-house or exploring open-source products for building.

It depends a little bit on the focus of the company. The other piece is the ability and competence of the team to build quality ML systems.

Building from scratch isn't worth it unless your core business is ML like Snorkel or OpenAI. Given that flexibility and scalability are the most important things for a new company, going for off-the-shelf infrastructural and architectural options is better. However, more often than not, off-the-shelf solutions provide only either flexibility or scalability which makes selecting hard.

This blog was written by Sangwoo Shim, Co-founder, and CTO MakinaRocks.

We are living in the age of artificial intelligence (AI), a technology that has made its way into every industry and is advancing at an unprecedented pace.

Epitomizing the innovations in AI is the hyper-scale AI model. The number of parameters, which serves as an indicator of the scale of machine learning (ML) models, doubled in just a year from the 50 million of Google’s Transformer in 2017 to the 100 million of OpenAI’s GPT in 2018. The number of parameters of AI models, which used to multiply tenfold each year, reached 170 billion with GPT-3 in 2020 and 1.6 trillion with Google’s Switch Transformer in 2021, heralding the dawn of hyper-scale AI computing.

This blog talks about hyper-scale AI models and how they can be used for multiple applications.

Mind Versus Data
Progress in the last decade shows that the success of an ML system depends largely on the data it was trained on. Instead of focusing on improving ML algorithms, most companies focus on managing and improving their data. Despite the success of models using massive amounts of data, many are skeptical of the emphasis on data as the way forward.

Mind might be disguised as inductive biases or intelligent architectural designs. Data might be grouped together with computation since more data tends to require more computation. In theory, you can both pursue architectural designs and leverage large data and computation, but spending time on one often takes time away from another.

Jason Dunn, a Senior Product Manager at Tecton. He works us through how Tecton is enabling data scientists, ML engineers, and data engineers to improve their feature engineering workflow through Tecton's feature platform.

Ultimately building real-time ML systems is hard and maintaining them is even harder. It's easy to wind up with very siloed, air-prone, and brittle systems that are difficult to scale. Tecton thinks of how feature engineering can better improve machine learning models to drive better predictions and outcomes in real-time for the end user.

Tecton's Feature Platform is primarily motivated towards improving future engineering workflows. By default, it supports real-time, streaming, or batch data sources. At its core, it comprises a feature repository, feature store, and feature engine.

The feature repo end goal is to ensure that you don't go outside the lines of how you think about your current Git Ops workflow like
some CI/CD aspects etc., while the feature engine is used for transformation. Then the feature store is where all the features live, and can be directly accessed in production.

Tecton 0.6 is focused on making the feature repo, feature store, and feature engine flexible in terms of your workflows to be able to integrate into notebook-driven development. This speeds up time to product launches, using a seamless iteration loop.

It's designed to enable engineers to test features directly within notebooks from the beginning while generating training data, productionizing those features through the standard GitOps workflow, and sharing and discovering these features. These features can then be accessed online directly without additional iteration loops.