Tiramisu enables the manipulation of knowledge coded within deep learning models. Using a pre-trained deep network model, Tiramisu can be used to extract the internal representations generated by that model for any data set (e.g., a set of images as percieved by a CNN). Once those initial representations are built, Tiramisu offers a variety of transforming operations, allowing the user to generate its own knowledge embedding space. Beyond transformations, Tiramisu also offers the possibility of applying data mining algorithms on any stage of the process, making the knowledge coded within deep networks exploitable by these techniques.
Tiramisu, implemented entirely in Python, is designed to perform efficiently both on a simple desktop computer and in high-performance clusters. On the former, the limited RAM memory resources are taken in consideraton, and complemented, if needed, with a persistence policy transparent to the user. On the latter, parallelism is provided through PyCOMPSs, allowing Tiramisu to scale for an arbitrary amount of distributed computational resources. Currently Tiramisu supports the Caffe deep learning toolkit as a source of data embeddings. Caffe is therefore a requirement for most of Tiramisu use cases. Persistency is provided through HDF5.
Currently, Tiramisu can build two types of embeddings: vectors and graphs. Vector embeddings originate from CNN activations, and can be transformed through various operations (reduce, collapse, binarize, normalize, etc. see API for details). Graph embeddings originate from vector embeddings, and can be exploited using a graph processing library (e.g., NetworkX).
Related Publications
2016
- Garcia-Gasulla, D; Béjar, J; Cortés, U; Ayguadé, E; Labarta, J; "A Visual Embedding for the Unsupervised Extraction of Abstract Semantics", accepted for publication at Cognitive Systems Research, 2016. (a step-by-step description of a typical Tiramisu workflow, with the purpose of image clustering using a convolutional neural network)
- Garcia-Gasulla, Dario; Moreno, Jonatan; Ramos-Pollán, Raúl; Casadiegos Barrios, Romel; Béjar, Javier; Cortés, Ulises; Ayguadé, Eduard; Labarta, Jesús; Suzumura, Toyotaro; "On the Representativeness of Convolutional Neural Networks Layers", Artificial Intelligence Research and Development, vol. 288, pp 29 - 38, 2016, IOS Press. (a study of the behavior of vector embeddings for various layers within a CNN)
- D. Garcia-Gasulla, J. Moreno-Vázquez, J.A. Espinosa-Oviedo, J. Conejero, G. Vargas-Solar, R.M. Badia, U. Cortés, T. Suzumura; "Efficient and versatile data analytics for deep networks", 7th International Supercomputing Conference in Mexico (ISUM'16), 2016. (brief overview of the Tiramisu architecture and components)
Acknowledgements
This project is partially supported by the Joint Study Agreement no. W156463 under the IBM/BSC Deep Learning Center agreement, by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316-P project and by the Generalitat de Catalunya (contracts 2014-SGR-1051), and by the Core Research for Evolutional Science and Technology (CREST) program of Japan Science and Technology Agency (JST).