PhD Position F/ M Workflow Provenance and Its Application to Explainable and Transparent Artificial Intelligence

Contexte et atouts du poste

Supervisory Team

Location and Mobility

The thesis will be hosted by the KerData team at the Inria research center of Rennes. Rennes is the capital city of Britanny, in the western part of France. It is easy to reach thanks to the high-speed train line to Paris. Rennes is a dynamic, lively city and a major center for higher education and research: 25% of its population are students.

This thesis will include collaborations with international partners from the USA, thus research visits to and from the collaborator's teams are expected.

The KerData team in a nutshell for candidates

Principales activités

Envisioned Approach

To explore what AI workflow provenance metadata can tell us about AI transparency and explainability,
we will build upon previous work and active research of the members in the supervisory team in
the USA and France.

For Aim A, the research methodology centers on analysing the AI model life-cycle, enabling
technologies and infrastructures, using our previous work on taxonomies for neural network metadata
[RCLJT22] as a starting point. We described the structure of a neural network with an architectural
taxonomy capturing the number, type (e.g., convolution or pooling), shape, and order of layers;
and the hyperparameters associated with each layer (e.g., kernel, stride, and padding for convolutional
or pooling layers). The architectural taxonomy is independent of the data, thus allowing for comparison
across datasets. We also defined a behavioral taxonomy throughout training covering the training
parameters (e.g., learning rate and batch size); the criterion used for gain or loss; the method used for
training and the measurement used for fitness; and the type of learning curve including, for example,
designations of late-learners and never-learners. Following a similar approach, we aim to formalize
similar taxonomies for other machine learning methods. This work will be complementary to our ongoing
collaboration with the Workflows and Ecosystems Group from Oak Ridge National Laboratory
(ORNL), in which we are exploring extensions to foundational work on provenance taxonomies with
a focus on system telemetry metadata. A combination of both approaches is necessary to deliver a
comprehensive formal framework suitable for developers and practitioners.

Aim B will build upon E2CLab [RCAV21], our solution for reproducible workflow execution with
support for capturing provenance and monitoring metadata. Currently, E2Clab includes a provenance
service that delegates system monitoring to third-party libraries in a non-intrusive way. However,
E2CLab will have to be extended to provide fine-grained access to cross-layer metadata via multiple
dedicated services. We hypothesize that such design will enable a more efficient deployment since multiple
services can have separation of concerns in the scope of the metadata they capture (i.e., hardware,
system, model and application). In addition, we expect to leverage previous work [KRCLJT22] to develop
a new mechanism to capture model-specific metadata as part of this service suite. Significant
efforts will be necessary to design a high-performance metadata storage middleware suitable to connect
E2Clab with Flowcept, a data integration system that captures and queries workflow provenance
developed by our ORNL collaborators. Finally, our preliminary results suggest algorithmic improvements
will be necessary to optimize Flowcept and ensure it is not introducing overheads in the overall
workflow execution.

For Aim C, we will start by analysing our collection of neural network record trails under the light
of the new taxonomy from Aim A. In these previous work we amassed and annotated the life-cycle
of 6,000 randomly-generated NNs across their generation, training, and validation stages [RCLJT22].
The resultant record trails, comprising both structural and learning curve data, were systematically
organized in tabular text files. These record trails constitute a valuable curated collection of provenance
information encompassing architecture, metadata, and performance metrics. Our collaborators from
the Massachusetts Institute of Technology (MIT) are currently carrying out a large survey on opensource
models that includes metadata, size and other existing metrics. In a similar approach to our
previous work, we plan to generate record trails from the models provided by our collaborators, and
we will enrich them with comprehensive metadata captured using the proof-of-concept from Aim B.
We will apply causal inference techniques on the taxonomy-structured metadata to understand the
feature strength on these data and the causal relationships between the architectural features (e.g.,
hyperparameters, number of layers, type of layers), behavioral features (e.g., final accuracy, accuracy
curve) and other elements mapped to the taxonomies from Aim A. This approach will enable us to
draw significant insights into the determinants of model interpretability, and what this can inform
about transparency and explainability. We will systematically document the analysis procedure into
a methodology for (i) the categorization of metadata into the aforementioned taxonomies, (ii) the
extraction of key model features, and (iii) the analysis of causal relationships.

References

[BBFM23] Elisa Bertino, Suparna Bhattacharya, Elena Ferrari, and Dejan Milojicic. Trustworthy
ai and data lineage. IEEE Internet Computing, 27:5–6, 2023.
[JRO] Fariha Tasmin Jaigirdar, Carsten Rudolph, Gillian Oliver, David Watts, and Chris
Bain. What information is required for explainable ai? : A provenance-based research
agenda and future challenges. In 2020 IEEE 6th International Conference on
Collaboration and Internet Computing (CIC), pages 177–183, 2020.
[KNHJ] Amruta Kale, Tin Nguyen, Frederick C Harris Jr, Chenhao Li, Jiyin Zhang, and
Xiaogang Ma. Provenance documentation to enable explainable and trustworthy ai:
A literature review. Data Intelligence, 5:139–162, 2023.
[KRCLJT22] Ariel Keller Rorabaugh, Silvina Ca´ıno-Lores, Travis Johnston, and Michela Taufer.
Building high-throughput neural architecture search workflows via a decoupled fitness
prediction engine. IEEE Transactions on Parallel and Distributed Systems,
33:2913–2926, 2022.
[MCSAGBS21] Mar¸cal Mora-Cantallops, Salvador S´anchez-Alonso, Elena Garc´ıa-Barriocanal, and
Miguel-Angel Sicilia. Traceability for trustworthy ai: A review of models and tools.
Big Data and Cognitive Computing, 5, 2021.
[RCAV21] Daniel Rosendo, Alexandru Costan, Gabriel Antoniu, and Patrick Valduriez. E2clab:
Reproducible analysis of complex workflows on the edge-to-cloud continuum. In IPDPS
2021-35th IEEE International Parallel and Distributed Processing Symposium, 2021.
[RCLJT22] Ariel Keller Rorabaugh, Silvina Ca´ıno-Lores, Travis Johnston, and Michela Taufer.
High frequency accuracy and loss data of random neural networks trained on image
datasets. Data in Brief, 40:107780, 2022.
[SAL] Renan Souza, Leonardo G Azevedo, V´ıtor Louren¸co, Elton Soares, Raphael Thiago,
Rafael Brandao, Daniel Civitarese, Emilio Vital Brazil, Marcio Moreno, Patrick Valduriez,
et al. Workflow provenance in the lifecycle of scientific machine learning, 2022.
[SS23] Marius Schlegel and Kai-Uwe Sattler. Mlflow2prov: extracting provenance from machine
learning experiments. In Proceedings of the Seventh Workshop on Data Management
for End-to-End Machine Learning, pages 1–4, 2023.
[WDA] Mark D Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton,
Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino
da Silva Santos, Philip E Bourne, et al. The fair guiding principles for scientific
data management and stewardship. Scientific data, 3:1–9, 2016.

Compétences

Required:

Appreciated:

Avantages

Rémunération

Monthly gross salary amounting to 2100 euros for the first and second years and 2190 euros for the third year