Recommender Systems Using Hugging Face & NVIDIA
Learn to implement a recommender system with Hugging Face and NVIDIA in this short Transformers4Rec tutorial. Includes an easy-to-follow Colab.
Created on November 15|Last edited on January 10
Comment
There's no question that recommender systems have changed the way we consume content, purchase consumer goods, and create or maintain relationships online. With a few clicks, and sometimes even without explicitly teaching a model our preferences, we can have a never-ending stream of personalized content delivered straight to our screens. But how exactly do these systems work, and what implications do they have for the future of content personalization, how we shop, and who we connect with online?
Here's What We'll Be Covering
Here's What We'll Be CoveringWhat Is A Recommender System?Hugging Face and Recommender SystemsNLP, Hugging Face's Transformers, and RecSysA Working Example of a Recommendation System in ColabRecommended Reading
To understand recommender systems, it's first important to understand the problem they're trying to solve. In a world of infinite choices, we're constantly faced with the challenge of turning this multitude of 'choice' into something that just 'fits': finding the content that's most relevant to us, buying products that not only fits our needs but matches our wants or preferences, etc. That's where recommender systems come in.
So let's start by answering a critical question:
What Is A Recommender System?
Recommender systems are algorithms that use data from our past behavior and use it to predict what we might want in the future. Based on our past interactions, they recommend new content that we're likely to be interested in. The two most common types of recommender systems are content-based and collaborative filtering.
- Content-based recommender systems recommend items based on their similarity to other items that we've interacted with in the past. So, if we've watched a lot of romantic comedies, a content-based recommender system might recommend a new romantic comedy that's similar to the ones we've already seen.
- Collaborative filtering recommender systems, on the other hand, recommend items based on the similarity of our past behavior to other people's past behavior. So, if we've watched a lot of romantic comedies and a lot of other people who've watched romantic comedies have also watched a particular drama, a collaborative filtering recommender system might recommend that drama to us.
There are a few different ways to implement a recommender system, but at its core, it's all about using data to find patterns and make predictions.
The implications of recommender systems are far-reaching in everything from the socks you buy, to the jobs you're shown on LinkedIn, to the news stories you read. They have the potential to completely change the way we discover new content and consume media. For better or for worse, recommender systems are here to stay. And as they continue to evolve, they will only become more and more sophisticated. So, it's important to understand how they work and what implications they have for the future of content personalization.
Hugging Face and Recommender Systems
A library that you can use in conjunction with the Hugging Face ecosystem is NVIDIA's Transformers4Rec tool. The library is designed to help developers build efficient and accurate recommender systems for sequential and session-based recommendation tasks.
The library provides a set of tools that allow developers to easily train and deploy recommender models. The library is also designed to work with a variety of data sources, including relational databases, NoSQL databases, and text data sources.
The Transformers4Rec library is based on the Transformer architecture, which is a deep learning technique that is used to learn representations of data in context. The transformer architecture is a general-purpose architecture that can be used for a variety of tasks, including natural language processing, image recognition, and recommender systems. The transformer architecture is well-suited for recommender systems because it can learn the relationships between items in a dataset.
Recall that the Transformer model is based on the idea of self-attention, which is the ability to attend to multiple parts of a data set simultaneously. This allows the model to learn from data more effectively than other models, such as recurrent neural networks (RNN) or long short-term memory (LSTM) models.
The Transformer model has been shown to outperform dozens of earlier model classes on a wide range of tasks, including machine translation and language modeling. In particular, Transformers out-perform kNN-based and HMM-based recsys models as well as numerous other classes of older recommendation methods.
Of interest to those of us who work with interconnected data is that Transformer models also handle long-term dependencies better than non-Attention-based models. This is due to the fact that the Transformer model uses a global attention mechanism, which allows it to consider the entire data set when making predictions.
The Transformers4Rec library includes a number of features that make it a powerful tool for building recommender systems. The library includes a set of pre-trained models that can be used to quickly train new recommender models. The library also includes a set of tools that allow developers to easily deploy recommender models. The library is designed to work with a variety of data sources, including the option to accept as input or generate as output parquet files and generate outputs via the protobuf text format.
NLP, Hugging Face's Transformers, and RecSys
Transformers4Rec relies on the Hugging Face (HF) Transformers library to easily experiment with the latest and state-of-the-art NLP Transformer architectures for sequential and session-based recommendation tasks and deploy those models into production. HF Transformers was established with the goal of providing standardized implementations of Transformer architectures that are produced by the research community, often within days or weeks of their publication.
As Transformers become more widely used as universal computation engines, expect to see additional domains where Transformers excel beyond recommendation systems and the areas shown in the diagram below.
The library is designed for both research and production and is composed of three building blocks:
- A tokenizer, which converts raw text to sparse index encodings
- A transformer architecture, and
- A head for NLP tasks, like Text Classification, Generation, Sentiment Analysis, Translation, Summarization.
Transformers4Rec only uses the transformer architectures building block (b) and their configuration classes from HF Transformers. Additionally, Transformers4Rec provides additional blocks necessary for recommendation, e.g., input features normalization and aggregation, and heads for recommendation and sequence classification/prediction.
The Trainer class was also extended to allow for the evaluation with common recsys metrics, e.g., mean reciprocal rank, normalized discounted cumulative gain, and more.
A Working Example of a Recommendation System in Colab
We've taken the original four datasets – two e-commerce and two news datasets – that were used to create the Transformers4Rec model that were not directly downloadable from the NVIDIA researcher's work – the original URLs had reached download limits – and made that data accessible to the community.
In addition, we've constructed a Notebook to show you how to use wandb to log the training tasks the researchers ran when building and training the Transformers4Rec model. You can see performance curves below for a wide range of metrics important in the recsys and information retrieval domains.
Below, using the Weights & Biases dynamic charting you can see a training run that we undertook while training the Transformers4Rec model. You too can train the model from scratch using the Colab Notebook provided below. Or if you'd rather utilize the Transformer4Rec model from NVIDIA - without doing the training below - you can check out various recsys Notebooks / tasks from NVIDIA in their repository on Github.
Run set
7
And speaking of recommendations. Here are some other articles you may be interested in.
Recommended Reading
A Step-by-Step Guide to Tracking HuggingFace Model Performance
This article provides a quick tutorial for training Natural Language Processing (NLP) models with HuggingFace and visualizing their performance with W&B.
A Gentle Introduction to Reinforcement Learning With An Example
This article provides a primer on reinforcement learning with an autonomous driving example with OpenAI Gym and Stable Baselines3 to tie it all together.
Hyperparameter Search for HuggingFace Transformer Models
In this article, we will explore how to perform hyperparameter search for pre-trained HuggingFace transformer models, making use of Weights & Biases Sweeps.
Classification Loss Functions: Comparing SoftMax, Cross Entropy, and More
Sometimes, when training a classifier, we can get confused about the last layer to put on our neural networks. This article helps you understand how to do it right.
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.