Healthy Data Schizophrenia

David Bayliss the Data Alchemist at LexisNexis, took us on a joy ride in TARDIS (the police phone booth that can travel space and time from Dr. Who). We explored the past, present, and future of LexisNexis Risk Analytics

Feeling like a cool kid cuz LexisNexis rhymes? Yea me too.

LexisNexis is a risk analytics company that plays an essential role in the world of security, authentication, and privacy by using Data science/ML.

They happened to be lurking everywhere - financial markets, credit requests, insurance markets. Hiding in plain sight as they say.

LexisNexis Use Cases and Diversity
The devil is in the data. At scale, a lot of variables come into play. Variables that could threaten the quality and competence of working with big data and big data engines.

It turns out that time travelers at LexisNexis have invested significant time and effort in addressing these issues with the use of Metalanguage, low-level algorithms, and DSL.

This introduces an abstraction layer that can manage and process whatever is thrown at the system, both internally and externally.

The Tardis Iron Box
In simple words, it is a universal data science virtual machine that allows the use of any data science tool and style within it. Although the output from this magic box only comes in one form, "Metadata".

The coolest part is that it gives the power to carry out huge data work on a "common" computer before performing the computational workloads/processes in the cloud.

It enables traceability, security, and repeatability.

But this newfound tech is basically "old tech". What do I mean by that?

Technically, the Tardis Iron Box is built on low-level code and the idea is to separate the handling of each process.

Perks of time travel, I guess.

We had a cool hang out with Kemal, a senior machine learning Engineer at Beat.

No more Kubeflow! We want Argo!

Argo Workflows is a language agnostic tool that enables the running of workflows on Kubernetes. It also supports the use of containers natively.

It extends its pipelining capabilities to make ML workflows easier, with batch computation, artifacts/metrics logging, and solution adaptability.

The matrimony of Argo and Kubeflow
By the power vested in me, I pronounce it to you that Argo isn't dependent on Kubeflow, though they have established Kubernetes consent.

There is no need to be a Kubernetes pro before being able to use Argo..... just consider it as one of those abstraction layer thingy that Kubernetes has.

An important takeaway. Keep each piece simple when trying to solve a problem.

The rise of deep learning has extended its reach into the category of search. Deep learning models are fueling a fundamental shift in how data can be represented, and this shift is from text-based representations to vector-based representations, called vector embeddings

Vector embeddings are numerical representations of unstructured data that require a different way of storing, indexing, and retrieving information.

This application of AI is used to transform unstructured data (images, text, video) into structured data, and very useful applications can be built once these vector embeddings are in place, including recommendation systems, semantic search applications, interactive Q&A experiences, and object recognition capabilities.

The industry is expecting that over 80% of enterprise data will be unstructured by 2025, and that’s a lot of data that can be converted to vectors for modern vector similarity search applications.

Redis has native capabilities for AI developers to generate, store, index, and query vector embeddings.

Rediscover Redis for Vector Similarity Search.