Five Criteos attended on March 27th & 28th the conference ScaledML 2019 at the Computer History Museum in Mountain View. It was the 4th edition of this conference focusing on Scaling ML models, data, algorithms, & infrastructure.
The lineup of speakers was impressive: Two Turing Award Winners, the creators of TensorFlow, PyTorch, Spark, Caffe, TensorRT, OpenAI, etc.
The level of the conference was pretty high and we can’t address each talk here… so, here is a summary of the highlights of the conference according to the Criteo delegation.
- Domain-Specific Architectures for Deep Neural Networks (David Patterson, Google/UC Berkeley): very instructive talk about the genesis and evolution of TPU. It demonstrates how doing one thing (matrix multiplication) but doing it extremely well (e.g. by using systolic arrays, an algorithm dating from 1979) helps going beyond the end of Moore’s law.
- AI and Security: Lessons and Future Direction (Dawn Song, UC Berkeley): with this thorough presentation of current threats on ML systems, Dawn Song reminded us that all the ML systems are there to serve a purpose and that adversaries can today easily divert them from it if no counter-measures are put in place. Eye-opening talk.
- Responsible Data Science (Lise Getoor, UCSC): the 2nd part of Lise Getoor talk highlighted the challenges of ethics in ML. As for any engineering practice, Machine Learning can raise some important ethical questions as demonstrated by recent history (COMPAS, Google image classification issues, HUD complaint on Facebook, etc.). Being aware of those challenges is the first step needed to take proper actions to avoid similar problems in the future. Must watch talk for any ML practitioner.
- MLFlow: An Introduction (Matei Zaharia, Databricks/Stanford): The presenter highlighted major pain points that many ML development teams experience. MLFlow seeks to reduce these pain points and create a more uniform development process among collaborators. MLFlow can work with all ML libraries and languages, allowing maximum flexibility.
- Machine Learning Trends at Quora (Adam D’Angelo, Quora): A very insightful talk about lessons learned while building an ML platform and organization around it. The key findings were on how to optimize for efficiency while operating as a relatively small company in such a labour intensive field as AI.
- GPT-2 (Ilya Sutskever, OpenAI): Ilya Sutskever explained the genesis of GPT-2, the language model that OpenAI trained and that produces impressive pieces of text. The model made the news recently for its achieved results but also for the decision of OpenAI to do only a partial release of the model by fear of the usage that could be done of it.
Some of the videos can be found here.
A big thank you to the organizing team and to the different speakers for this very interesting event.
See you next year!
Authors: Renaud Bauvin, Michael Nation, Oleksandr Oleinikov
If this kind of topics interests you, Criteo is hiring!