GPT, A Gateway Drug

LLMs at Scale

On this podcast, we had Nils Reimers, Director of Machine Learning at Cohere.

LLMs Security
The guild of LLMs is to make humans more productive by automating uninteresting tasks. Unfortunately, bad actors can augment these LLM's capabilities. This brings up some security concerns that are quite challenging to manage like data poisoning, PII data manipulation, personalized spam, phishing emails, hate speech, etc.

However, with security principles, there is a trade-off between the model provider and the user. Either model providers respect the privacy of the user by not having access to their data due to the inability to properly protect user data or they have access to user data but are able to effectively shield user data from the bad actors.

One major drawback of using LLMs in production is the latency issue. Usually, there is the non-subjective time it takes to get an answer back from the model and there is the perceived time. Model providers apply all sorts of tricks to address the latency gaps and improve the user experience when using these models.

They could either make the output wait time fun by engaging users with some quizzes, fact tells etc. Or they can make the waiting time seem shorter by streaming the results to the users as the model generates an output, rather than a typical one-batch output.

This idea is to keep the users occupied while the model generates an output.

Video || Spotify || Apple

Agentic Relationship Management

Ashe Magalhaes, founder of Hearth AI, talked about technical approaches to agentic product relationship management.

Hearth AI is a stealth start-up that aims to build an agent that augments the human experience. Agents are Al systems that leverage large language models (their simulated world model) and human feedback to determine which actions to take and in which order.

Agentic products describe a class of products that execute a series of observations, "thoughts, and actions to meet an objective.

Despite being glued to our screens, the ability to efficiently manage personal and professional networks still presents some unprecedented levels of complexity:

• Dropping relevant communication threads
• Wasting time on manual data entry tasks
• Losing key network details
• Failing to match people with the right opportunities at night times
• Letting important relationships grow cold

However, agentic relationship management products that leverage Al agents can be used to learn, understand, and act on behalf of the user. Hearth Al will synthesize and manage your network complexity to ensure that people focus on their connections.

YouTube

Data Privacy and Security

Diego Oppenheimer, Partner and CEO at Factory, moderated a panel session with Vin Vashishta, C-level Technical Strategy Advisor and Founder at V Squared, Saahil Jain, Engineering Manager at You.com, Gevorg Karapetyan, Co-Founder and CTO at ZERO Systems, and Shreya Rajpal Founding Engineer at Predibase.

When thinking about the implications of data privacy / data security with respect to LLMs, it's no longer about the data itself, it is the patterns within the data that can be uncovered. But it also creates vulnerabilities for anyone with a significant amount of data out in public. The ways these foundational language model APIs are used need to be framed to address the different concerns that exist today with data privacy.

A lot of these models are still susceptible to hallucinations and technical work still needs to be done to reduce hallucinations. Hallucinations don't nullify the usefulness of these models. The goal is to ensure that users are presented with the right expectations when using language products or tools. Also, we need to ensure that the language products or tools don't mislead users and that users know how to responsibly use it.

Video || Spotify || Apple

Fine-Tuning LLMs

We will have Mark Kim-Huang, Co-Founder and Head of AI at Preemo inc., on this podcast to talk about LLMs best practices.

With the open source community releasing foundational models at a blistering pace, there has never been a better time to develop an AI-powered product. In this talk, we walk you through the challenges and state-of-the-art techniques that can help you fine-tune your own LLMs. Additionally, we will provide guidance on how to determine when a small model would be more appropriate for your use case.