European supercomputer MareNostrum 5 starts up at BSC

21 December 2023

MareNostrum 5 represents the largest investment ever made by Europe in a scientific infrastructure in Spain

The new European supercomputer MareNostrum 5, one of the most complete and versatile machines in the world at the service of the scientific community and the only one with two systems on the list of the 20 most powerful supercomputers on the planet, was inaugurated today at the Barcelona Supercomputing Center - Centro Nacional de Supercomputación (BSC-CNS).

Thanks to its unique computational architecture, MareNostrum 5 will advance science in all areas, from the development of digital twins of the planet Earth and the human body, to the search for new treatments for diseases such as cancer, the design of healthier and more sustainable cities, or the search for new energy sources and new materials.

MareNostrum 5 represents the largest investment ever made by Europe in a scientific infrastructure in Spain with a total cost of 202 million euros, of which 151.4 million euros correspond to the acquisition of the machine, financed jointly by the European High Performance Computing Joint Undertaking (EuroHPC JU), through the EU's Connecting Europe Facility and the Horizon 2020 research and innovation programme, as well as by the participating states: Spain -through the Ministry of Science, Innovation and Universities and the Generalitat de Catalunya-, Turkey and Portugal.

"We share the aim of making Barcelona a European scientific reference, for the benefit of Barcelona, Catalonia and the whole of Spain; because what is good for Barcelona and Catalonia is good for Spain and for Europe. The BSC is a centre of excellence in the new knowledge economy that will generate hundreds of jobs directly and thousands indirectly, and will train the professionals of today and tomorrow", stressed the President of the Government, Pedro Sánchez.

"Catalonia today takes an unprecedented step forward with the MareNostrum 5, one of the major objectives of European strategic autonomy and of Catalonia's economic and social future. Today we are making available to Catalonia's knowledge ecosystem a supercomputer that includes 4,480 state-of-the-art accelerated chips, each with more power than the entire first supercomputer that was launched not yet 20 years ago. A potential that will be at the service of science, but also has a clear link with the citizens and with the needs of the present and the future", said the President of the Generalitat, Pere Aragonès.

Anders Dam Jensen, Executive Director of the EuroHPC JU, said: "I am delighted to see MareNostrum 5 inaugurated and ready to serve European research in the new year. This day marks another major achievement for European supercomputing, and for us at the EuroHPC JU, represents the end of our first chapter of procuring European supercomputers. All eight machines in the first generation of EuroHPC systems are now out in the world, and ready for access by European researchers to push the boundaries of scientific and technological innovation.”

"Supercomputers are instruments at the service of science and engineering, accelerators of theory that allow the development of digital twins in very diverse and essential fields for society, such as climate change or precision medicine. We are very proud of the arrival of the new MareNostrum 5, one of the most important machines in Europe and the world to advance in the great challenges of science, such as the digital twin of the Earth, which we seek to develop as part of the Destination Earth project, one of Europe's great bets to combat climate change; or human digital twins, which will serve to create more efficient drugs to treat diseases that are difficult to treat today, among others. The presentation of MareNostrum 5 paves the way for the next MareNostrum 6, which we hope will incorporate European technology, a milestone that could be a reality in 5- or 6-years’ time," said BSC director Mateo Valero.

The launch of the new MareNostrum 5 consolidates the BSC as one of the world's leading supercomputing centres, with more than 900 employees, most of whom are researchers in four scientific departments: Computer Science, Life Sciences, Earth Sciences and Computational Applications for Science and Engineering.

Accelerating science to 314,000 billion calculations per second

MareNostrum 5 is owned by the EuroHPC JU. With a maximum total performance of 314 petaflops, equivalent to the capacity to perform up to 314,000 (European) billion calculations per second, MareNostrum5 joins two other EuroHPC systems: Lumi (Finland) and Leonardo (Italy) as the only three pre-exascale supercomputers in Europe. The capabilities and versatility of this new supercomputer will be instrumental in providing Europe with state-of-the-art supercomputing technology and accelerating the capacity for artificial intelligence research, enabling new scientific breakthroughs that will help solve global challenges.

The technology company Eviden has been the supplier selected after a tender to provide the new supercomputer, which also incorporates technology from other companies such as Lenovo, IBM, Intel and Nvidia, and in whose installation the German consultancy firm Partec has also participated.

A heterogeneous machine with state-of-the-art supercomputing technology

MareNostrum 5's unique architecture is designed to provide researchers with the best technology available to answer the big questions in science. It is a heterogeneous machine that combines two distinct systems: a general-purpose partition, dedicated to classical computing, and an accelerated partition, designed to push the frontiers of knowledge in artificial intelligence. Both systems, separately, rank among the top 20 supercomputers worldwide, 19th and 8th respectively, making the BSC the only supercomputing centre in Europe with two entries in the top 20 of the LINPACK ranking of the 500 most powerful supercomputers on the planet.

The general purpose (CPU) partition is the world's largest based on Intel's iconic x86 computing architecture, with a peak performance of 45.4 petaflops (45,400 European billion calculations per second) thanks to the new Sapphire Rapids processor. This part of the supercomputer, manufactured by Lenovo and ranked 19th in the LINPACK benchmark, is specially designed to solve complex scientific problems because of its ability to divide the machine's resources to run multiple tasks or programmes simultaneously, rather than being dedicated exclusively to one task. This allows greater flexibility and improves efficiency, as different users or projects can use the supercomputer at the same time depending on their needs.

The accelerated partition (GPU), manufactured by Eviden, is the third most powerful in Europe and eighth in the world according to the LINPACK benchmark, with a peak performance of 260 petaflops (260,000 European billion computations per second). This part of the supercomputer is powered by 4,480 next-generation NVIDIA Hopper processors, hardware accelerators designed to perform specific tasks more efficiently than general-purpose processors, enabling the acceleration of computationally intensive computations in critical research areas such as artificial intelligence and numerical simulation. These latest-generation chips, each about 8 cm2 in size, have more power than the entire MareNostrum 1 installed in 2004, which occupied the entire chapel of Torre Girona, about 180 m2, and was the fourth fastest in the world.

The new MareNostrum 5 also stands out for its large storage capacity, increasing from the 15 Pbytes available in MareNostrum 4 to 650 Pbytes. The system offers a net capacity of 248 Pbytes on disk which is complemented by a tape-based long-term storage solution with an additional 402 Pbytes of capacity, thanks to IBM's Spectrum Scale file system.

In addition, MareNostrum 5 has an interconnection network based on InfiniBand NDR200, which allows all the nodes of the system, more than 8,000 in total, to cooperate by exchanging information to solve the most complex problems, notably improving the speed of operation, in time and consumption, with respect to MareNostrum 4.

Use and applications to serve all areas of science

The main use of MareNostrum 5 is for research by Spanish and European scientists, although it will also be available for corporate research under special conditions. Access is through competitive, peer-reviewed calls for proposals, with projects prioritised according to their importance. The distribution is based on the investment made by the EuroHPC Joint Undertakingand the participating states. Thus, EuroHPC JU will carry out calls at European level to access 50% of the available time, while the rest will be distributed among Spain, Turkey and Portugal. In the case of Spain, access will be through the calls of the Spanish Supercomputing Network (RES). The number of projects that will be able to access the system increases with respect to MareNostrum 4, but there will still be excellent projects competing for resources that will not be able to access them.

By increasing the computing power, system memory and number of cores, MareNostrum 5 will help solve more and more complex problems. For example, climate change simulations will be able to have higher resolution, moving from representing phenomena on spatial scales of hundreds of kilometres to include processes occurring on scales of a few kilometres, making predictions much more accurate and reliable. In this sense, thanks to MareNostrum 5, Spain will play a key role in the European Union's Destination Earth project, whose objective is to develop a complete virtual replica of planet Earth that will make it possible to predict the effects of climate change, and to create and test scenarios for more sustainable development.

It will also be possible to tackle much more complex problems of artificial intelligence and big data analysis. For example, it will be possible to generate massive language models by training much larger neural networks with hundreds of billions of parameters, using infinitely larger datasets than today. This represents a disruptive change in the field and opens up hitherto unthinkable possibilities.

In addition to climate predictions and large language models, MareNostrum 5 is specifically designed to support European medical research in the design of new drugs, vaccine development and simulations of virus propagation. It will also be a crucial tool for materials science and engineering, which can benefit from the potential of the new supercomputer in areas such as simulation-based aircraft design and optimisation and data management for safer, cleaner and more efficient aviation. Similarly, the new European supercomputer will be used to advance the simulation of processes for generating new forms of energy such as nuclear fusion.

Integration with two of southern Europe's first quantum computers

MareNostrum 5 is also expected to integrate two quantum computers in the coming months: the first quantum computer of the Spanish Supercomputing Network (RES), which is part of the Quantum Spain initiative promoted by the Ministry of Digital Transformation through the Secretary of State for Digitalisation and Artificial Intelligence (SEDIA); and one of the first European quantum computers, after the EuroHPC JU selected the BSC as one of the six centres that will host the first European quantum computing network. Both quantum computers will be among the first to operate in southern Europe.

Prior to its launch, MareNostrum 5 has been recognised as the Top Supercomputing Achievement of 2023 by HPCWire, the most prestigious international supercomputing magazine, for the way it "combines traditional supercomputing with artificial intelligence to drive discoveries and expand new frontiers of science with the development of digital twins to study the climate and progress towards more personalised medicine".