As technological innovation drives the development of AI systems, their capacity to produce outputs, influence, or make decisions in an increasing number of contexts may raise key questions about trustworthiness in their design, deployment, or usage. While the concept of trustworthiness in AI can have different meanings depending on the angle taken, recent advancements have propelled regulatory frameworks, such as the EU’s AI Act, into the spotlight. In addition to shaping industry practices, these legislative developments also stimulate and necessitate new avenues for scientific research.

The goal of this symposium is to exchange among key stakeholders of the scientific and policy communities—on what trustworthy AI means in theory and in practice, in particular in the context of the open Internet. Through this dialogue, we aim to address privacy, fairness, accountability, transparency, and explainability as the foundation of trustworthy AI.

This symposium held in anticipation and under the label of the Paris AI Action Summit, will showcase key initiatives from Criteo and other leading organizations. Through two general sessions and an accessible scientific session, we will explore how research initiatives enable us to anticipate and address regulatory needs, ultimately contributing to a more responsible, innovative, and, in the end, trustworthy ecosystem.

9:30 – 9:40 | Opening by Diarmuid Gill – Chief Technology Officer at Criteo

9:40 – 12:30 | 1st session – The various faces of Trustworthy AI

• 9:40 – 10:00 | Bertrand Braunschweig – Scientific coordinator of the Confiance.ai community: The five walls of AI, five years later

• 10:00 – 11:00 | Panel with Q&A: What is the definition of trustworthy AI?

  • Bertrand Braunschweig – Scientific coordinator of the Confiance.ai community

  • Marie-Aude Aufaure – President of Datarvest

  • Armand Lacombe – AI policy officer (chargé de mission IA) at Service of numerical economy (Service de l’économie numérique), DGE

  • Panel moderated by Nathalie Laneret – VP Government Affairs & Public Policy at Criteo

• 11:00 – 11:20 | Coffee

• 11:20 – 12:30 | Invited talks

  • Gaël Varoquaux – Researcher (directeur de recherche) at Inria: AI, from tech to public policy

  • Hugo Richard – Senior researcher at Criteo AI Lab: A story about local differential privacy

  • Eloi Zablocki – Researcher at Valeo.ai: Where? Why? What? Explaining vision-based driving models

12:30 – 13:30 | Lunch

13:30 – 15:30: 2nd session – Trustworthy AI on the open Internet

• 13:30 – 14:00 | Lyline Lim – Head of Impact & Sustainability at PhotoRoom: Understand AI to better trust AI – How Photoroom trained a foundation image model and deployed it to millions of users

• 14:00 – 15:00 | Practical use cases of trustworthy AI in the advertising industry

  • Maxime Vono – Staff researcher at Criteo AI Lab: privacy on the open internet

  • Mariia Vladimirova – Senior researcher at Criteo AI Lab: fairness in advertising

• 15:00 – 15:30 | Oana Goga – Researcher (directrice de recherche) at Inria: Measuring and mitigating risks with AI-driven information targeting

15:30 – 16:00 | Coffee

16:00 – 17:30 | 3rd session – Scientific session: Academic partners for trustworthy AI

• 16:00 – 16:30 | Matthieu Cord – Professor at Sorbonne Université & scientific director of Valeo.ai: Explainability framework for large multimodal models

• 16:30 – 17:00 | Vianney Perchet – Professor at ENSAE & principal researcher at Criteo AI Lab: Trading-off price for data quality to achieve fair online allocation

• 17:00 – 17:30 | Aurélien Bellet – Researcher (directeur de recherche) at Inria: Auditing privacy in machine learning

17:30 – 17:50 | Closing

• Mathieu Morgensztern – Founder of AdaptivAI: From theory to practice, the keys to a responsible approach to AI in the business world

• Romain Lerallut – VP of Engineering at Criteo and head of the Criteo AI Lab

17:50 – 21:00 | Cocktails & scientific posters

Several scientific posters from Criteo and partners will be featured during the cocktail, with paper authors to explain and popularize their work.

• Song Duong, faithfulness in NLP (SCOPE: A Self-supervised framework for Improving Faithfulness in Conditional Text Generation)

• Maxime Vono, dataset valuation (DU-Shapley: A Shapley Value Proxy for Efficient Dataset Valuation)

• Mariia Vladimirova, fairness in advertising (FairJob: A Real-World Dataset for Fairness in Online Systems)

• Ilana Sebag, privacy in gen AI (Differentially Private Gradient Flow based on the Sliced Wasserstein Distance)

• Alain Rakotomamonjy, federated learning (Personalised Federated Learning On Heterogeneous Feature Spaces)

• Corentin Pla and Hugo Richard, user-level privacy (Distribution-Aware Mean Estimation under User-level Local Differential Privacy)

• Simon Mauras, fairness in job allocation (The Price of Fairness in Bipartite Matching)

• Matilde Tullii, privacy in advertising (Open Research Challenges for Private Advertising Systems Under Local Differential Privacy)

The event will be moderated by Romain Lerallut – VP of Engineering at Criteo and head of the Criteo AI Lab.

All sessions will be held in English.

• Bertrand Braunschweig – Scientific coordinator of the Confiance.ai community.

  • Speaker (1st session) and panelist

  • Talk title: The five walls of AI, five years later

  • Abstract: Back in 2020, I started talking and writing about what I called « the five walls of AI », walls that AI is likely to smash into if we don’t take precautions. Any one of these five walls is capable of putting an end to its progress, which is why it is essential to know what they are and to seek answers in order to avoid the famous third winter of AI, a winter that would follow on from the first two in 197x and 199x, during which AI research and development came to a virtual standstill for lack of budget and community interest. These walls are trustworthiness, energy, cybersecurity, human-computer interaction, and inhumanity. Five years later, I will take a look back and see if these walls are still in front of us, taking into account the progress made since.

  • Bio: After a career as a researcher and project manager in simulation and AI in the field of energy, Bertrand Braunschweig headed the ICT department of ANR, two Inria research centers (Rennes then Saclay), produced two editions of the Inria white paper on AI and coordinated the research component of the national artificial intelligence strategy. He is now an independent consultant and provides scientific support to various organizations including as scientific coordinator of the Confiance.ai community and programs. He is also the coeditor of the forthcoming European standard on AI risk management, in support of the European AI Act.

• Armand Lacombe – AI policy officer (chargé de mission IA) at Service of numerical economy (Service de l’économie numérique), DGE

  • Panelist

  • Bio: Armand Lacombe is a policy officer at the Directorate-General for Enterprise, where he works in a team dedicated to the building of public policies for AI. He focuses on issues such as setting up of the AI regulation in line with the AI Act, ensuring safety and promoting standardization. He holds an engineering degree from Ecole Polytechnique and a PhD in computer science.

• Marie-Aude Aufaure – President of Datarvest

  • Panelist

  • Bio: Marie-Aude Aufaure is in charge of the big data professional training at CentraleSupelec and scientific advisor for innovative companies involved in digital technologies. She also acts as an independent expert for the European Commission. Her areas of expertise cover Digital Technologies: Big Data, Business Intelligence, Semantic Technologies, Data Mining and Machine Learning. Marie-Aude Aufaure, who obtained a PhD in computer science from the University of Paris 6, joined CentraleSupelec in 2003. She was head of the academic chair in Business Intelligence funded by SAP from 2008 to 2014, and member of the laboratory of Mathematics Applied to Systems. Before joining CentraleSupelec, she worked in industry and in academia, in particular at INRIA where she was associated partner from 2001 to 2014. She was involved in many European and National collaborative projects as scientific manager, and has supervised fifteen PhD students.

•  Gaël Varoquaux – Researcher (directeur de recherche) at Inria

  • Speaker (1st session)

  • Talk title: AI: from tech to public policy

  • Abstract: Historically an actor of tech, a machine-learning coder, I have been in policy-advisory missions over the last two years, such as the French National AI commission or the writing of the International Scientific Report on the Safety of Advanced AI. Talking with many experts and stakeholders brings to light a broader picture of the impact of AI in our world, and how it is perceived. I will summarize what I learned.

  • Bio: Gaël Varoquaux is a research director working on data science at Inria (French computer science national research) where he leads the Soda team. He is also co-founder and scientific advisor of Probabl. Varoquaux’s research covers fundamentals of artificial intelligence, statistical learning, natural language processing, causal inference, as well as applications to health, with a current focus on public health and epidemiology. He also creates technology: he co-founded scikit-learn, one of the reference machine-learning toolboxes, and helped build various central tools for data analysis in Python. Varoquaux has worked at UC Berkeley, McGill, and university of Florence. He did a PhD in quantum physics supervised by Alain Aspect and is a graduate from Ecole Normale Superieure, Paris.

• Hugo Richard – Senior researcher at Criteo AI Lab

  • Speaker (1st session)

  • Talk title: A story about local differential privacy

  • Abstract: This talk is an introduction to local differential privacy. The introduction is made through the lens of a study of abortion rates in North Carolina in 1970 which is the first real-life experimental study using randomised response, a locally differentially private introduced in 1965 by Warner. The definition of local differential privacy is presented as well as the necessary trade-offs between privacy and accuracy. In a second-part, the research questions associated with multiple interactions are briefly discussed.

  • Bio: Hugo Richard did his PhD under the supervision of Bertrand Thirion (INRIA, Mind) on the unsupervised analysis of neuroimaging data. He then did a post-doc under the supervision of Vianney Perchet (Inria Fairplay, ENSAE/Crest, Criteo) on bandits and competitive analysis. He is now a Senior Research Scientist at Criteo AI Lab. His current research interests touch several areas of machine learning: bandits, competitive analysis and differential privacy.
    

• Eloi Zablocki – Researcher at Valeo.ai

  • Speaker (1st session)

  • Talk title: Where? Why? What? Explaining vision-based driving models

  • Abstract: This talk explores various explainability methods for vision-based driving models. It begins with input-attribution methods (“where”), progresses to textual explanations that clarify reasoning (“why”), and concludes with counterfactual explanations highlighting decision-driving elements through minimal input changes (“what”). The role of generative models in enhancing explainability is a central focus. Additionally, the talk addresses challenges in evaluating explanations, introducing evaluation approaches based on human user studies and automated pipelines leveraging generative models.

  • Bio: Eloi Zablocki is a research scientist at Valeo.ai, where he has been working since 2020. Before that, he earned his Ph.D. from Sorbonne University, on multi-modal machine learning with language and vision. Prior to this, he studied at École Polytechnique and ENS Paris-Saclay. His research interests include vision and language, multi-sensor scene understanding and forecasting, and model explainability.

• Lyline Lim – Head of Impact & Sustainability at PhotoRoom

  • Speaker (2nd session)

  • Talk title: Understand AI to better trust AI – How Photoroom trained a foundation image model and deployed it to millions of users
    

  • Abstract: This talk will unveil the behind the scenes of image foundation models. It will tackle why Photoroom trained their own foundational model instead of using an open source model, how Photoroom trained such a model from scratch, and the struggles along the way: data, compute, scaling, gathering feedback, etc.
    

  • Bio: Lyline Lim is currently the Head of Impact and Sustainability at PhotoRoom, where she focuses on responsible AI topics, including public affairs, social impact, sustainability and diversity. She joined PhotoRoom in May 2022 and previously held positions as Chief of Staff where she launched the B2B business unit. Her technical background includes more than 10 years of experience as an engineer in image processing and working in various tech companies like Algolia and GoPro.
    

• Mariia Vladimirova – Senior researcher at Criteo AI Lab

  • Speaker (2nd session)

  • Talk title: Fairness in advertising

  • Abstract: Ensuring fairness in AI-driven advertising is a critical challenge in the ad-tech industry, where vast amounts of data are utilized to predict user behavior. Biases present in historical data can inadvertently perpetuate social inequalities, leading to ethical, legal, and business risks. Addressing these challenges, Criteo has taken a pioneering step by releasing the FairJob Dataset, the first large-scale fairness-aware dataset specifically designed for digital advertising. This dataset aims to foster research and innovation in mitigating bias within real-world advertising scenarios. Through our efforts, we seek to raise awareness, set new benchmarks for ethical AI practices, and drive meaningful progress in creating more equitable and responsible AI systems.

  • Bio: Mariia Vladimirova is a senior researcher at Criteo working towards trustworthiness in artificial intelligence (AI). She leads a research initiative focused on algorithmic fairness, actively contributes to academic advancements in AI, and provides technical and scientific inputs to policy makers in the context of the AI regulations. Beyond her research, she is a passionate advocate for gender diversity and inclusivity, serving as an ambassador for both the Women in Tech community at Criteo and the Women in Machine Learning and Data Science society.

• Maxime Vono – Staff researcher at Criteo AI Lab

  • Speaker (2nd session)

  • Talk title: Privacy on the open internet

  • Abstract: TBD

  • Bio: Maxime Vono is a Staff Researcher Lead within Criteo AI Lab, the artificial intelligence laboratory of Criteo. He manages an hybrid team of machine learning (ML) engineers and researchers, working on building next-gen bidding engines robust to data fragmentation and the loss of user signal. Research areas of interest include privacy-preserving ML, cohort-based and multi-task learning. Before joining Criteo, he was a Research Scientist within the Lagrange Research Center of Huawei, working with Eric Moulines (French Academy of Sciences) on federated learning and differential privacy. He obtained his Ph.D. in statistics in 2020 and was a Visiting Researcher within the Statistics Department of Oxford University in 2019 working with Arnaud Doucet (Senior Staff Researcher at Google DeepMind and Full Professor at Oxford).

• Oana Goga – Researcher (directrice de recherche) at Inria

  • Speaker (2nd session)

  • Talk title: Measuring and mitigating risks with AI-driven information targeting

  • Abstract: In this talk, I will discuss risks associated with online advertising and micro-targeting and present methodological approaches for measuring and mitigating these risks. I will provide insights from some of our latest results on political ad micro-targeting, marketing to children, and tracking. I will discuss how our measurement studies informed European lawmaking, how citizens can help, and how researchers interested in assessing risks with online platforms can take advantage of the latest EU laws.

  • Bio: Oana Goga is a Research Director at Inria (French Institute for Research in Computer Science and Automation) and a member of the Inria CEDAR team and the Laboratoire d’Informatique d’Ecole Polytechnique (LIX). She investigates risks for humans and society brought by online platforms and their deployments of AI, such as advertising technologies. She looks at risks ranging from privacy to disinformation and manipulation to child protection. Her research is interdisciplinary, and she works with economists, social scientists, and legal scholars. Her work has influenced European law, and she has served as an external expert for the European Commission on problems related to data access in the Digital Services Act (DSA). She received the CNRS Bronze Medal in 2024, the Lovelace-Babbage Award from the French Science Academy and the French Computer Society in 2023, and she received an ERC Starting Grant in 2022 that aims to measure and mitigate the impact of AI-driven information targeting. Her recent research received several awards, including the Andreas Pfitzmann Best Student Paper Award at PETS 2024, the Honorable Mention Award at The Web Conference in 2020, and the CNIL-Inria Award for Privacy Protection 2020.

• Matthieu Cord – Professor at Sorbonne Université & scientific director of Valeo.ai

  • Speaker (3rd session)

  • Talk title: Explainability framework for large multimodal models

  • Abstract: Multimodal large language models (LLMs) have achieved remarkable proficiency in understanding multimodal inputs, fueling extensive research aimed at developing increasingly powerful models. Despite recent advancements towards the interpretability of these models, understanding internal representations of LMMs remains largely a mystery. We propose a framework for monitoring LLMs based on a dictionary learning approach, applied to latent token representations.  The elements of the learned dictionary correspond to our proposed concepts. We demonstrate that these concepts are semantically grounded in both vision and text and are useful for interpreting representations of test samples. Furthermore, we analyze the evolution of these representations during fine-tuning, enabling us to trace changes in encoded concepts across modalities. We also demonstrate the use of shift vectors to capture these concept changes, highlighting the practical implications of our findings for model steering.

  • Bio: Matthieu Cord is a professor at Sorbonne University and scientific director of Valeo.ai. His academic research is carried out at the Institute of Intelligent Systems and Robotics (ISIR), where he heads the Machine Learning team (MLIA). He currently holds a chair in the national AI program at Sorbonne’s SCAI center.  He is an honorary member of the Institut Universitaire de France and served for three years as an AI expert at the CNRS and ANR. His research focuses on computer vision, machine learning and artificial intelligence.

• Vianney Perchet – Professor at ENSAE & principal researcher at Criteo AI Lab

  • Speaker (3rd session)

  • Talk title: Trading-off price for data quality to achieve fair online allocation

  • Abstract: We consider the problem of online allocation subject to a long-term fairness penalty, but under additional privacy issue. Indeed, we do not assume that the decision-maker observes the protected attributes — which is often unrealistic in practice. Instead, we shall assume that they can purchase data that help estimate them from sources of different quality (maybe of different levels of privacy); and hence reduce the fairness penalty at some cost. After introducing, formalizing and analyzing the static problem (where all parameters are known), we investigate the dynamical version that we formalized as multi-armed bandit problem. We will provide high level insight of the solution and the kind of performance guarantees that can be achieved.

  • Bio: Vianney Perchet is a professor at the Centre de recherche en économie et statistique (CREST) at the ENSAE since october 2019. Mainly focusing on the interplay between machine learning and game theory, his themes of research are at the junction of mathematics, computer science and economics. The spectrum of his interest ranges from pure theory (say, optimal rates of convergence of algorithms) to pure applications (modeling user behavior, optimisation of recommender systems, etc.) He is also part-time principal researcher in the Criteo AI Lab, in Paris, working on efficient exploration in recommender systems.

• Aurélien Bellet – Researcher (directeur de recherche) at Inria

  • Speaker (3rd session)

  • Talk title: Auditing privacy in machine learning

  • Abstract: Machine learning models trained on personal data may unintentionally reveal sensitive information. This talk presents two approach to “audit” the privacy of machine learning. First, I will argue that Membership Inference Attacks (MIA) provide a versatile tool for assessing privacy risks of releasing machine learning models, and to provide formal privacy lower bounds. Second, I will a present
    cryptography-based approach allowing a third-party to securely verify that a potentially malicious model trainer did train with the claimed privacy guarantees, without access to the model or the data.

  • Bio: Aurélien Bellet is a senior researcher at Inria, France. His current research focuses on the design of privacy-preserving machine learning algorithms in centralized and decentralized settings. Aurélien has served as area chair for top conferences such as ICML, NeurIPS and AISTATS. He co-organized several international workshops on machine learning and privacy as well as the 10th edition of the French pluridisciplinary conference on privacy protection (APVP). He also co-organizes FLOW, an online seminar on federated learning with 1000+ registered attendees.

• Mathieu Morgensztern – Founder of AdaptivAI

  • Closing

  • Talk title: From theory to practice, the keys to a responsible approach to AI in the business world

  • Abstract: This talk will conclude the symposium with a practical study of the challenges of implementing AI within companies and organizations in a responsible way. Mathieu Morgensztern, founder of AdaptivAI, will share his feedback to raise the right questions and formalize guidelines on mastering tools and risks, securing models before and after deployment, and getting employees on board. 

Event organized by Criteo, with (in alphabetical order):

• Marieke Buis, Senior PR & Communication Manager France;

• Joaquín Engelmo Moriche, Staff Developper Relations Engineer;

• Jean-Yves Franceschi, Senior Researcher;

• Nathalie Laneret, VP Government Affairs & Public Policy;

• Dainora Magnier, R&D Support Coordinator;

• Mariia Vladimirova, Senior Researcher.