Autonomic Systems and e-Business Platforms

Overview: 

The team performs high-level research in today’s eBusiness applications that have to deal with critical IT challenges in areas such as Big Data, Cloud Computing, Business Analytics, High Performance Computing (Supercomputing), Sustainable Computing and Cognitive Computing.

Objectives: 

The goal of this team is to conduct research on autonomic and intelligent resource management policies based on Self-Management strategies as the way to improve the adaptability, efficiency and productivity of current middleware layers. 
Our aim is to extend the possibilities of hing performance computing beyond their usual so far scopes.
In these research fields the team produces top research publications as well as software components and resource management policies that can be applied at middleware level. The group targets execution platforms composed of high-productivity hybrid multi-core systems and advanced storage architectures deployed in large-scale distributed environments. The group is also working in the design of novel big data algorithms to enable massive scale data and video analytics in large-scale clusters running several middleware in order to make context computable. 

GROUP MANAGER: 

     TORRES, JORDI

Research Lines: 

In order to manage the overwhelming scientific agenda of the group we have organized our activity in 6 areas:

 

Data-centric Computing Area:

The focus of this area is to accelerate the processing of data-driven workloads, including large analytics as well as stream processing, in heterogeneous execution frameworks. The work will focus on the following research topics:

  • Exploring systems and software strategies for leveraging high-performance in-memory key/value databases to accelerate data intensive tasks. The work will adopt the Scalabe Key Value Store (SKV) projects as the key/value store.
  • Building clouds which via their programmability at multiple layers and the embracing of hardware heterogeneity can host a variety workloads and can optimize resource configuration for these workloads. The core platform used for prototyping is OpenStack.
  • Develop mechanisms for an automated characterization of cost-effectiveness of Big Data deployments, such as Hadoop, to explore how runtime performance, and therefore its price, are critically affected by relatively simple software and hardware configuration choices. The group architected and maintains the Aloja portal.
  • Explore novel architectures of the emerging IoT stream processing platforms, that provide the capabilities of data stream composition, transformation and filtering in real time. The group architected and maintains the servIoTicy platform.
  • Building hardware prototypes (Minerva) as a group platform for running BigData workloads, exploring how to accelerate computation while keeping the cost of the prototype low, leveraging commodity hardware on the back end and high-end components in the front end.

For more information you can contact the area Lead Scientist:  CARRERA PEREZ, DAVID

Involved Team:

  • AMARAL, MARCELO - RESIDENT STUDENT
  • BERRAL, JOSEP LLUIS – POSTDOC RESEARCHER
  • CALL, AARON - SUPPORT ENGINEER
  • CUGAT, JOSEP -UPC MASTER STUDENT
  • KHOUZAMI, NESRINE - RESIDENT STUDENT
  • PEREZ, JUAN LUIS - JUNIOR RESEARCHER
  • POGGI, NICOLAS - POSTDOC RESEARCHER
  • POLO, JORDA - POSTDOC RESEARCHER
  • VILLALBA NAVARRO, ALVARO - JUNIOR RESEARCHER

 

Energy-aware Computing Area: 

The goal of this area is to develop management algorithms for virtualised Data Centres in a large-scale distributed ecosystem running heterogeneous workloads that optimize their operation with respect to energy and ecological efficiency.  The work in this area is grouped in the following main lines:

  • Models for the assessment and forecasting of energy and ecological efficiency in a virtualised Data Centre at different levels
  • Policies for the optimization of the scheduling and placement of Virtual Machines (VMs) in physical nodes considering the energy and ecological efficiency factors
  • Policies for the selection of Data Centre for remote placement of Virtual Machines (VMs) in a Data Centre ecosystem considering the energy and ecological efficiency factors
  • Integration of the cooling and power supply subsystems in the energy management strategy of Data Centres
  • Integration of renewable energy sources in the energy management strategy of Data Centres       

For more information you can contact the area Lead Scientist: GUITART FERNANDEZ, JORDI 

Involved Team:

  • CANUTO, MAURO - RESEARCH SUPPORT ENGINEER
  • MACIAS LLORET, MARIO - ASSOCIATE RESEARCHER
  • ORTIZ, DAVID - RESEARCH SUPPORT ENGINEER
  • SUBIRATS, JOSEP - RESEARCH SUPPORT ENGINEER

 

Data-driven Scientific Computing Area: 

The goal of this area is to design resource management strategies for Big Data applications, defining policies that enable distributed data stores to meet high-level performance goals. We focus on scientific applications, like those from life science domain, which data generation and accesses bound both precision and performance. During next years the main work of this research activity will be developed as part of these four main threads:

  • Propose novelty resource management strategies as query-driven data model, which focus on adapting the data model to the particular type of accesses implemented by the applications. We also aim to consider the intrinsic of continuous data streaming with real time requirements. This kind of environment also raises the challenge of defining an execution framework that is able to digest this kind of input data streams.
  • Work on the definition of programmer interfaces that hide the intrinsic of data storage and management.
  • Collaborate with other research areas in BSC to design and develop a suite of tools capable to monitor the execution of Big Data applications, which will guide system administrators and programmers to detect bottlenecks and misconfigurations.
  • Start to consider network usage of Big Data applications analyzing it and detecting how optical networks can benefit the performance of this application. The output of this analysis will allow us to propose policies to exploit the presence of optical networks.
  • Create a set of plugin modules based on our research results in order to be added to state-of-the-art open source NoSQL platforms. After this integration the comprehensive software package will be integrated in the BSC Big Data tools that the BSC department Computer Science will develop with the conjoint work of our research group and the groups of Storage and Grid. 
  • Hecuba: A project that aims to design and develop strategies to facilitate programmers the efficient usage of data stores for big data applications. For example, we will provide programmers with a software layer that will decouple data models from data layouts.

For more information you can contact the area Lead Scientist: BECERRA, YOLANDA 

Involved Team:

  • HERNANDEZ, ROGER - JUNIOR ENGINEER 
  • CUGNASCO, CESARE - RESEARCH SUPPORT ENGINEER
  • ALOMAR, GUILLEM- JUNIOR DEVELOPER

 

Data Analytics Computing Area: 

The goal of this area is to develop new algorithms for big data analytics in a large-scale clusters running several middlewares, such as spark, hadoop, storm or theano.  The work in this area is grouped in 3 main research lines:

  • Citizen as a sensor: Enhance new algorithms for social event detection in a city using citizen mobile data (e.g. Social Networks data, cell phone data, etc.) and open data retrieved from the city services (e.g. Public transport system, traffic, etc.). This research line comprises research from human mobility patterns to city services optimisation. 
  • Deep Learning: Enhance new deep neural network algorithms to solve a large variety of problems related with traffic network optimisation and text processing.
  • Recommendation systems: Enhance new recommendation systems using big data architectures.      

For more information you can contact the area Lead Scientist: NIN GUERRERO, JORDI

Involved Team:

  • ARANDA, JORDI  - RESEARCH SUPPORT ENGINEER
  • CEA, DANIEL - RESEARCH SUPPORT ENGINEER
  • CORDERO, JOSE ALEJANDRO - PHD.ASSOCIATE STUDENT
  • CAPDEVILA, JOAN - PHD. ASSOCIATE STUDENT 

 

Multimedia Big Data Computing Area:

The goal of this area is to design novel big data distributed computing systems to enable massive scale image and video analytics. The work in this area is grouped in 3 main lines:

  • Large scale visual concept detection and annotation
  • High-performance and scalable indexing of massive scale image and video
  • Multimedia big data analytics (recommendation, trend detection and latent user attribute inference)

For more information you can contact the area Lead Scientist: TOUS, RUBEN

Team:

  • MORA, DANI – UPC MASTER STUDENT
  • SULCA, OMAR – UPC MASTER STUDENT
  • QUIMI, JORGE – UPC MASTER STUDENT
  • CONESE, ALESSIO – UPC MASTER STUDENT

 

Barcelona Spark Lab:

Although our research group is working in different big data platforms, recently the Apache Spark open-source in-memory computing framework is the focus of a number of initiatives in our research agenda. Under the Barcelona Spark Lab we want to collecting and organizing all research and dissemination initiatives around Spark carried out in our group.

Also BSC is working with its dissemination through different initiatives such as leading the Barcelona Spark Meetup with more than 300 people at this moment.

 

Projects/Areas: 

Current involved projects:

  • COMPOSE (Collaborative Open Market to Place Objects at your Service) (2012-2015). COMPOSE is a FP7-ICT-2011.1.2 (ref.  317862) EU Funded Project, coordinated by IBM Haifa (IL) with the following partners: CREATE-NET (IT), Fraunhofer Institute FOKUS (DE), The Open University (UK), Barcelona Supercomputing Center (ES), INNOVA S.p.A (IT), University of Passau (DE), U-Hopper (IT), GEIE ERCIM (W3C) (FR), Fundació Privada Barcelona Digital Centre Tecnològic (Bdigital) (ES), Abertis Telecom (ES), and EVRYTHNG (CH). COMPOSE  aims at enabling new services that can seamlessly integrate real and virtual worlds through the convergence of the Internet of Services (IoS) with the Internet of Things (IoT). COMPOSE will achieve this through the provisioning of an open and scalable marketplace infrastructure, in which smart objects are associated to services that can be combined, managed, and integrated in a standardised way to easily and quickly build innovative applications.

 

  • BSC - IBM BGAS SoW (2013-2016) is a joint research project between researchers at Barcelona Supercomputing Center (BSC) and the  "Scalable Data Centric Computing" group at IBM Research - Watson Lab. This project aims at exploring systems and software strategies for leveraging in-memory key/value databases to accelerate data intensive tasks, with particular attention to the IBM BlueGene Active Storage (BGAS) architecture and the Scalable Key/value Store (SKV) as the key/value store.

 

  • BSC - IBM Heterogeneous Clouds SoW (2013-2016) is a joint research project between researchers at Barcelona Supercomputing Center (BSC) and the  "Middleware and Virtualization Management" group at IBM Research - Watson Lab. This is a project focused on building clouds which via their programmability at multiple layers and the embracing of hardware heterogeneity can host a variety workloads and can optimize resource configuration for these workloads. The project will explore the applicability of the so-called "Software Defined Environments (SDE)" to HPC workloads as it has been previously done with transactional and data analytics workloads.

 

  • Aloja (2014-2016) ALOJA is project funded by Microsoft Research through the BSC- Microsoft Research Center (bscmsrc.eu) that aims to provide automated optimization to Hadoop's performance under different hardware deployments options and software parameters.  We are also exploring new hardware architectures, both on-premise or in the cloud (either IaaS or PaaS) and what is the best configuration option for a given Hadoop job (or job type).  Part of the project includes a public (vendor neutral) Web platform with a repository of Hadoop benchmarks and data analysis tool.  We currently have over 4500 Hadoop benchmark executions both from our local clusters as well as in Azure (IaaS), on which we base our research.

 

  • EuroServer (FP7-ICT-2013-10 European Project, Grant Agreement no: 610456): Green Computing Node for European Micro-servers. Goal: Design and build a drastically improved energy- and cost-efficient solution suitable across both cloud data-centres and embedded application workloads. Our contribution: Optimise the local placement of Virtual Machines (VMs) within a physical node in a single Data Centre aiming for energy efficiency, by exploiting ARM low-power architectures

 

  • RenewIT (FP7-SMARTCITIES-2013 European Project, Grant Agreement no: 608679): Advanced concepts and tools for renewable energy supply of IT Data Centres. Goal: Develop a simulation tool to evaluate energy performance of different technical solutions that integrate renewable energy supply in IT Data Centres. Our contribution: Optimise both the local placement of Virtual Machines (VMs) in physical nodes and the selection of Data Centre for remote placement of VMs aiming for ecological efficiency, by exploiting the usage of green energy and the interaction with energy supply and cooling systems

 

  • ASCETiC  (FP7-ICT-2013-10 European Project, Grant Agreement no: 610874): Adapting Service lifeCycle towards EfficienT Clouds. Goal: Definition and integration of explicit measures of energy and ecological requirements into the design and development process for software. Our contribution: Optimise both the local placement of Virtual Machines (VMs) in physical nodes and the selection of Data Centre for remote placement of VMs aiming for energy efficiency, by focusing on the interaction and information exchange among Cloud layers during the whole service lifecycle for better optimization

 

  • Severo Ochoa Distinction (January 2012- January 2016): The Barcelona-Supercomputing Center-Centro Nacional de Supercomputación (BSC-CNS) has been accredited as Severo Ochoa Centre of Excellence, the award with which the Spanish Ministry recognizes leading research centres in Spain and international reference organisations in their respective areas. The award  will enable the execution of an ambitious research project which involves designing the hardware, software and applications to provide future solutions to the social challenges arising in health and climate change. Our contribution: provide the applications with resource management strategies according to their data management requirements and contribute to the design and development of an integrated stack of software to support the execution of the applications

 

  • Lightness: (FP7- Future Networks – 2012) Low latency and hIGH Throughput dynamic NEtwork infraStructures for high performance datacentre interconnects. Goal: design, implementation and experimental evaluation of a high-performance network infrastructure for data centres, where innovative photonic switching and transmission solutions are deployed. Our contribution: of the network usage of Big Data applications and detecting how optical networks can benefit the performance of this kind of applications

 

  • BSC-CA: The main goal of this project is to provide methods, a decision support system, an open source IDE and run-time environment for the high-level design, early prototyping, semi-automatic code generation, and automatic deployment of applications on multi-Clouds with guaranteed QoS. Our contribution: Create an automatic text analysing tool able to extract QoS information about different cloud providers using public information obtained from different websites, such as stack overflow

 

Previous involved projects:

  • IBM SOW-Active Storage Fabrics (ASF) is a collection of components that surround a parallel in-memory database (PIMD). PIMD is a parallel client, parallel server, key/value object store. This research is part of the MareIncognito research framework between IBM and BSC.
  • OPTIMIS aims at optimizing cloud services using techniques that take advantage of an architectural framework and a development toolkit that take trust, risk, eco-efficiency, cost and legal issues into account. Our group contributes in the self-management of Cloud infrastructures using business information.
  • Barrelfish project, which is a new research operating system being built from scratch to explore how to structure an OS for future multi- and many-core systems. The design principles of Barrelfish are motivated by two closely related trends in hardware design: first, the rapidly growing number of cores, which leads to a scalability challenge, and second, the increasing diversity in computer hardware, requiring the OS to manage and exploit heterogeneous hardware resources.
  • VENUS-C is focused on developing and deploying a Cloud Computing service for research and industry communities in Europe by offering an industrial-quality service-oriented platform based on virtualization technologies. Our group contributes with tools that allow user scenarios to exploit the facilities of Cloud infrastructures.
  • NUBA project (Normalized Usage of Business-oriented Architectures) (2009-2012). NUBA is a strategic research  program (MITyC TSI-020301-2009-30) funded by the Avanza2  R&D Plan of the Spanish Ministry of Industry, Tourism and Trade and coordinated by  Telefonica I+D with 8 partners. The aim of NUBA is to advance the state-of-the-art in business models and technology for the real-time deployment  of federated Cloud platforms, integrating infrastructure from different providers, to execute elastic  business services with  the required QoS and minimizing the energy consumption. http://nuba.morfeo-project.org.

PEOPLE

PUBLICATIONS AND COMMUNICATIONS

2010

Macías M, Guitart J. Using Resource-level Information into Nonadditive Negotiation Models for Cloud Market Environments. 12th IEEE/IFIP Network Operations and Management Symposium (NOMS'10). 2010 :325-332.
Goiri Í, Julià F, Guitart J, Torres J. Checkpoint-based Fault-tolerant Infrastructure for Virtualized Service Providers. 12th IEEE/IFIP Network Operations and Management Symposium (NOMS'10). 2010 :455-462.
Torres J, Ayguadé E, Carrera D, Guitart J, Beltran V, Becerra Y, Badia RM, Labarta J, Valero M. BSC contributions in Energy-aware Resource Management for Large Scale Distributed Systems. 1st Year Workshop of the COST Action IC0804 on Energy Efficiency in Large Scale Distributed Systems. 2010 :76-79.
Berral JL, Goiri Í, Nou R, Julià F, Guitart J, Gavaldà R, Torres J. Towards Energy-aware Scheduling in Data Centers using Machine Learning. 1st International Conference on Energy-Efficient Computing and Networking (e-Energy'10). 2010 :215-224.

Pages