The big data platform developed in the project, which combines Earth observation data with information from unmanned aerial vehicles, has applications in various sectors such as agriculture, water management, journalism and border security
BSC optimizations allows for greater efficiency and cost savings in the evolving landscape of Earth Observation (EO) and data-driven decision-making
Barcelona Supercomputing Center - Centro Nacional de Supercomputación (BSC-CNS) researchers have taken a significant step towards enhancing access to Earth Observation (EO) data and related services as part of the EU-funded CALLISTO project, which concluded on 30 December 2023.
This three-year collaboration saw the BSC´s Data-Centric Computing team optimize frameworks used in the CALLISTO big data platform. These optimizations were tested in agriculture and land control use cases and yielded an 80x speed-up in information retrieval and cost savings through the more efficient use of HPC resources that take advantage of artificial intelligence (AI) technology.
The CALLISTO project, a collaborative effort involving 17 partners across 7 EU countries and South Korea, sought to bridge the gap between Copernicus Data and Information Access Services providers and application end users. It developed an interoperable Big Data platform combining EO data with crowdsourced and geo-referenced information from Unmanned Aerial Vehicles. The platform was tested in real-world cases, demonstrating its applicability for a range of sectors, including agriculture, water management, journalism, and border security.
CALLISTO leveraged cutting-edge technologies, including, Artificial Intelligence (AI), Distributed Computing, Unmanned Aerial Vehicles, Data Fusion, Semantic Image Segmentation, and Ontologies. The project introduced 10 tools evaluated in four pilot use cases: Common Agricultural Policy (CAP) Monitoring, Water Quality Assessment, Satellite Journalism, and Land Border Change Detection.
BSC optimizations
BSC ensured that the large amounts of data used in the CALLISTO project could be processed as quickly and efficiently as possible while retaining readability and reuse in code. By focusing on the efficient use of hardware and the distributed computing aspect of the project, BSC researchers optimized software, pursued parallelization methods and efficiency. This acceleration helps platform users make well-informed and strategic decisions in cases like Common Agricultural Policy (CAP) Monitoring and Land Border Change Detection.
Figure 1: Graphical representation of speed up and scalability of CALLISTO’s Unmanned Autonomous Vehicle path planner with BSC optimizations
These accelerations (Figure 1) demonstrate the scalability possibilities when High-Performance Computing (HPC) resources are efficiently harnessed. In addition to the 80x speed-up, the results can be applied to any machine or deployment. Code optimizations and successfully streamlined software and hardware promise an additional potential 22x acceleration across multiple machines with further optimizations. The BSC´s contributions extend beyond these immediate achievements and include a methodological guide applicable to a myriad of applications that promises up to a 4x faster AI model training when coupled with HPC infrastructure.
BSC Project Investigator, Alberto Gutierrez Torre, explains, “By eliminating code bottlenecks and making a smart use of hardware, the BSC was able to improve the performance of CALLISTO Big Data platform components and reduce computational cost.“
CALLISTO has established a legacy that remains valuable for all stakeholders seeking to explore the emerging landscape of EO, geospatial applications and data-driven decision-making.
About CALLISTO
CALLISTO (Copernicus Artificial Intelligence Services and data fusion with other distributed data sources and processing at the edge to support DIAS and HPC infrastructures, GA 101004152) is an EU-funded project that sought to bridge the gap between Copernicus Data and Information Access Services (DIAS) providers and application end users through Artificial Intelligence (AI) solutions. It counted with the support of 17 partners from 7 EU countries and South Korea, coordinated by Serco Italia S.p.A. (SERCO), Italy, (Ethniko Kentro Erevnas kai Technologikis Anaptyxis (CERTH), Greece, Fraunhofer Gesellschaft zur Förderung der angewandten Forschung e.V. (Fraunhofer), Germany, CS GROUP (CS), France, Barcelona Supercomputing Center - Centro Nacional de Supercomputacion (BSC), Spain, Institut für Angewandte Informatik (InfAI) e.V., Germany, ACCELIGENCE LTD (ACCELI), Cyprus, Ethniko Asteroskopeio Athinon (NOA), Greece, Deutsche Welle (DW), Germany, De Watergroep, Belgium, Institut royal des Sciences naturelles de Belgique (RBINS), Belgium, Società Metropolitana Acque Torino S.p.A (SMAT S.p.A), Italy, Nurogames GmbH (NURO), Germany, European Union Satellite Centre (SATCHEN), Spain, DRAXIS Environmental S.A. (DRAXIS), Greece, DreVen SRL, Belgium, Korea University (KU), South Korea, from various industries, SMEs and academia. It received a EU grant of €3,999,954. CALLISTO has made significant progress towards enhancing access to Earth Observation (EO) data and developing innovative geolocation-based services.
Photo: Testing Unmanned Aerial Vehicles in the CAP use case in Cyprus (George Choumos, National Observatory of Athens).