At Criteo we a run a large-scale recommender system, capable of generating billions of relevant recommendations daily. Deep Learning provides powerful models that shine when applied to data-rich problems, so we evidently study this class of algorithms in our recommendation system.
This year Criteo AI Lab is presenting its latest advancements on applying deep learning to recommender systems at the 2017 RecSys Deep Learning workshop:
- Specializing Joint Representations for the task of Product Recommendation from Thomas Nedelec, Elena Smirnova, Flavian Vasile (Criteo AI Lab)
- Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks from Elena Smirnova, Flavian Vasile (Criteo AI Lab)
Content2vec: A unified product representation for recommender systems
We have been experimenting with the use of product embeddings in order to continually improve our recommendation engine. We started by running Prod2vec in production and then worked on enhancing the product representations by using product categories as side information. However, one has access to far more information than product collaborative filtering (CF) and product categories data. In the same spirit of previous content-based approaches that were developed for recommender systems, we were looking for an simple way to integrate this information (images, product title, product detailed description…) into the CF-based product vector. Incorporating content data is important since it can help in cases where the original CF data is too sparse (e.g. new products, cold start issues…). This is why we designed Content2vec, a multi-modular architecture that can integrate and use all the available product signals. The goal of the framework is to reach a unified product representation that gathers all information available on the products to enable us to do better recommendations.
Practically, each module is fed with one of the types of signals available on the product (such as its image) and builds a vector representation specialized to predict the probability of two products being connected by a user event (e.g. co-visited, co-bought). For most of the modules, the architecture is a multi-layer Convolutional Neural Network.
We decided to go for a multi-modular architecture and not build a fully end-to-end network because the modular design allows us to deploy it in a large scale production environment, that requires us to be able to have stable performance under model retraining, be able to easily add new signal, to easily upgrade the processing of an existing signal and to easily debug performance issues by looking at each components independently.
We also introduce a new type of learning unit that we call the Cross-Interaction Unit (CIU): this representation helps us to model all interactions that exist between the modality-specific embeddings with respect to the initial optimization objective and to approximate interaction terms between the products that cannot be explained by a linear combination of the modality-specific similarities.
We report the performance of our architecture on two datasets that we extracted from the Amazon dataset. It is a first step in order to be able to build a unified representation of the products. In the future, we are looking forward seeing new ways to build good product representation and understanding whether content approaches can work for cross-category recommendations.
Contextual Recurrent Neural Networks for Recommendation
At Criteo, we compute recommendations personalized according to each user behaviour. To recommend the most relevant products, we strive at understanding the user and his shopping needs.
Recently, Recurrent Neural Networks (RNNs) have shown that Deep Learning models can provide useful user representations for recommendation. The main idea behind RNNs is to keep track of internal state that summarizes the sequence of items that user has interacted with so far. In this approach, state acts as a internal memory that automatically extracts useful information from the sequence.
Besides the actual historical items, user sequences are often characterized by various contextual information such as the type of user-item interactions, the time gaps between events and the time of day for each interaction. This contextual information has the potential to greatly influence the recommendation. For example, in the context of a sale, items similar to a sold item are complementary items.
To account for contextual information in RNN, we propose to modify the behaviour of the RNN at two levels. First, input and output layers are conditioned on context by combining the context embedding with the item embedding. Second, model dynamics are parametrized with the hidden unit transitions as a function of context information. We compare variants of proposed architecture with sequential and non-sequential baselines and show that our method significantly outperforms all of the other methods on two e-commerce datasets, the YooChoose open dataset and an internal dataset. We observe especially large improvements on hard cases, such as sale events and non-historical items.
For further details, please see our papers, and, if you are attending RecSys 2017, make sure to stop by live presentations on August 27.
We are delighted to sponsor RecSys 2017 as advancing state-of-the-art in recommender systems is one of our key focus areas. If you want to learn about research at Criteo, please come to talk to us at our booth.
We look forward to seeing you in Como.