After working on theoretical questions in high-dimensional statistics as an academic at UC, Berkeley, I felt I wanted to see what statistics and machine learning were at industrial scale. This is why I joined Criteo in September 2017 to contribute to large-scale machine learning in industry.
Friends in France told me that Criteo had positions for visiting professors, offering a research-oriented environment within a 3,000-employee company. Following a few interviews with people in Paris and Palo Alto, as research is spread between the US and France, I was excited to have the opportunity to join. Initially, I thought this would only be for one year.
I thought that my general area of expertise would be useful and helpful but my academic work would not necessarily be immediately applicable to problems at Criteo, though some seemed to have potential. I did not know exactly what I would be working on and I expected and was looking forward to having to learn quite a few new things. I also expected that I would have an adjustment period and indeed had one that lasted a few months. I felt I was in a supportive environment, which helped, but it had been a while since I was in this situation (back to PhD/post-doc days pretty much…) so it was a bit unsettling while it lasted.
I found the research environment diverse and exciting. Members of the research team have expertise in e.g. various areas of machine learning and statistics, are deeply knowledgeable about the inner workings of Criteo and its machine learning engine. As we were sitting together, it was easy to pick someone else’s brain to get perspective, or to suggest ideas to others. The diversity of viewpoints was very enriching, especially in drawing connections I would not necessarily have thought of on my own. I learned a lot about various topics connected (sometime only loosely!) to my main initial research interests by just being able to listen to and participate in conversations. Having generally shared purposes in research made for a very nice collaborative environment.
How to combine long-term research and business goals?
I also enjoyed the fact that I was given time to do long term research. I became interested and involved in auction theory during my stay at Criteo. That has little to do with what I was doing before, but it is very interesting, challenging intellectually as well as practically important. This is how I got hooked. I was able to attend as many seminars as I wanted while in Paris on a variety of different topics. My involvement in this new topic went like this. A series of lectures on auction theory were organized at Ecole Polytechnique (in part by Vianney Perchet, who works part time at Criteo). Tim Roughgarden gave those lectures and we were encouraged to attend them if interested. Several of us did.
We then decided to organize a reading group on the topic at Criteo. We went thoroughly through one of the main PhD-level textbooks. We had bi-weekly meetings with people from research and a few people on the more engineering/implementation side. This allowed us to then really start reading the current literature and it took us around 3–4 months to start doing research-level work. It turned out to be one of my activities at Criteo. We have contributed to academic research, since we are encouraged to submit papers to conferences and journals. We have also brought our expertise to bear internally on ongoing engineering and business improvements within the company. Our work has covered the whole arc of industrial research from new, rich, academic ideas to practical industrial solutions, exchanging with engineers, academic colleagues, etc… I found this deeply satisfying.
Thanks to discussions with colleagues at Criteo, I also heard about and started contributing (with them!) to new areas of research, like fairness in machine learning.
Impact on the company’s products.
I think the current size of Criteo is an asset in this regard: it is still at a size where one can make significant impact in a limited amount of time, but big enough that one can think at a fairly high level about various challenges.
There are a lot of research problems to be tackled and a great variety of people to talk to. I chose to be involved in multiple problems at the same time, at different levels. For instance, I was impressed to attend a meeting with 50-or-so people from engineering, business and research about problems that had surfaced from the business side with the bootstrap. Resampling is one of my areas of research so I got involved. I worked closely with Thomas Nedelec, who is finishing his CIFRE PhD, and Feng Li who came from his team to the research team for a 3-month “research bootcamp”. Together we were able to design a good solution that fixed the problem at no new engineering cost.
This feeling of making my first “real contribution” to Criteo was very gratifying. Since then, I have of course tried to lend a hand when I could, and on a few occasions have acted as an internal consultant as many of us do in the research team. I also appreciated that Criteo encourages publication and communication of research results, so I felt that during all my time at Criteo, I kept contributing to the broader research community.
It has also been informative and sobering to see what it implies to do statistics and machine learning at the scale of data Criteo is working with. Knowing the possibilities and limitations of industrial-scale data science changed the way I think about some of those problems.
Key takeaways
As a result, my perspective on what are important future problems in machine learning and statistics has changed quite a bit since I arrived at Criteo.
It’s also been really enriching for me to see how management works in a tech company. The whole process of setting up goals, achieving them, tracking progress is very different from academia. It has also been instructive to see how hiring and career management/promotions are handled.
Seeing how technical communication is done inside the company has been interesting as well. This turned out to be quite different from academia, with much more emphasis on important details and less stepping back/big picture idea talks. It makes sense in an engineering-heavy environment. It may also make it a bit more difficult for newcomers to get to the gist of a problem, at least those with my background and experience. Maybe years of teaching help in that direction…
Space design, mainly organized around open spaces, was also something I was curious about. It took me several weeks to get used to the open space. It is obviously great for collaboration and exchanging ideas, less so for working on things that require intense sustained focus, such as doing a proof. I am now able to do both in that environment, and there are also quiet rooms that address this potential challenge.
I have enjoyed working in a positive environment and having many smart people to talk to, both on the engineering and research sides; everything is very fluid at Criteo, which is nice. I haven’t found the culture inside the company, or at least what I saw, to be very different from similar environments in academia. I also liked the fact that established opinions or solutions are regularly challenged, usually in a bottom-up process, with questions and solutions backed up by careful thinking and data analysis.
My experience after a year was very positive and I extended my leave from UC, Berkeley for another year. I am spending this second year in Criteo’s offices in Palo Alto and have the same general impression here as I had in Paris. Of course, details are a bit different in Paris and in the US as the office is much bigger in Paris, but Criteo’s enjoyable culture is very present here too.
Be part of the experience!
In the end, I’ve had a lot of fun, felt that I contributed to the company and research team, opened new avenues for research for myself, made new friends and collaborators and came out impressed with the company and everyone I met. I also have a new perspective on what industrial data science is likely to be, which I think is invaluable for academics working in this area.
Author: Noureddine El Karoui