# Computer Applications in Science & Engineering Department

### From AnnualReport

**The aim of the Computer Applications in Science & Engineering (CASE) Department is to identify, engage and support user communities in science and engineering that are potential users of High Performance Computing, boosted by its own research lines in High Performance Computational Mechanics.**

## Overview

The applications developed by the CASE department are truly multidisciplinary, requiring depth of expertise in many fields. In order to successfully develop these applications, the skills of the CASE team in numerical methods and parallel programming must be complemented by experts in appropriate areas. The Department therefore develops collaborations with other scientific groups, especially those with expertise in areas which the BSC-CNS Groups do not have extensive experience. Examples of institutions with strong research links with CASE include CIEMAT, CSIC, IAC, ICFO, IMDEA and different universities.

The research field of CASE is High Performance Computational Mechanics, which requires a deep background in Computer Science, Physics and Numerical Methods. Major research areas are Computational Fluid Dynamics and Solid Mechanics, Ab-initio DFT and TD-DFT molecular dynamics, Seismic Imaging and Parallel Programming. Major application areas are Aerospace, High Energy Physics (plasma core and edge transport, plasma wall interaction), Biomechanics (Cardiovascular and Respiratory systems), Geophysics and Atmospheric flows.

## Organisational Structure

The CASE Department is led by José María Cela, and although there is high interactivity amongst all the scientists in the Department, the research lines fall naturally in two main Groups; Physical & Numerical Modelling (PNM) and High Performance Computational Mechanics (HPCM).

Each Group consists of 8-10 people at any given time, comprising several senior scientists, post and pre-doctoral
students and visiting scientists. PNM research lines are horizontal and HPCM lines are vertical, in the sense that the
PNM Group is in charge of developing the core components which are then assembled and modified as required by
the HPCM Group into applications tailor-made to meet specific project needs.

## Key Projects

In 2009, the CASE Department carried out work under the scope of the following projects:

**EU funded projects:**

- EUFORIA: Generating a software infrastructure for support of the ITER design community.
- ETSF: Generating a software infrastructure for support of the spectroscopy community; in particular, work was undertaken on the scalability of OCTOPUS.
- W2PLASTIC: Magnetic Sorting and Ultrasound Sensor Technologies for Production of High Purity Secondary Polyolefins from Waste. The physical problem to be simulated was modeled and validated.
- DEISA: Continued collaboration, leading work packages 5 and 9, on "Enabling of Applications".
- PRACE: Several codes (ALYA, BSIT, CPMD, EUTERPE) were analysed and tuned for PRACE prototypes.

**Enterprise funded projects:**

- Kaleidoscope (REPSOL): With the goal of developing the most powerful seismic imaging tools, an RTM application 10 times faster than any other implementation was developed, specifically developed for running in both IBM's Cell/B.E. processor and GPUs.
- MareIncognito (IBM): Leading work package 1 for applications porting, some codes (such as SIESTA) were scaled for the MareIncognito architecture.

**Nationally funded projects:**

- Supercomputación y e-Ciencia (CONSOLIDER): Coordination of the project, whose aim is to develop a set of scientific Grand Challenges for Petaflop supercomputers and design the architecture of those machines. Some of the applications were also developed with the collaboration of CASE researchers.
- ATMOST (Plan Nacional): This project started in 2009 and aims to model ashes and contaminant dispersion in the atmosphere.

**The CASE department also develops two international collaboration projects in the area of biomechanics:**

- Airflow in the Human Respiratory System: In collaboration with both the Aeronautics and Bioengineering Departments at Imperial College London. A simulation of the complete human respiratory system, including the air surrounding the face, has been carried out. This project is being partially supported by HPC-Europa European project.
- Cerebral Hemodynamics Model: In collaboration with the CFDLab George Mason University, USA, the Krasnow Institute for Advanced Studies, George Mason University, USA, the Inova Fairfax Hospital, Virginia, USA, the National Center for Computational Biology, UCLA, USA and the Brain Research Institute, Melbourne, Australia. A model of the arterial system of the brain is being developed. A deflated preconditioner with conjugate gradient solver was used to accelerate the pressure solver in this simulation. A speed-up of 10 with respect to the original solver was obtained.

## Scientific Output

For additional information, please see the **Detailed Report of Research Activities 2009 for the CASE Department**.

Although a large part of the Department´s work is private and in-confidence and therefore cannot be published, some important
research results of the Department have been presented in congresses
and conference lectures as well as a number of scientific publications,
including: FEF09 and ParCFD-09, SEG and EAGE conferences in Geophysics, International
Conference on Mathematics and Continuum mechanics, as well as
the 1st International Conference on Computational & Mathematical Biomedical Engineering.

## Physical and Numerical Modelling (PNM)

The PNM Group researches basic themes, such as numerical modelling of physical phenomena, stabilisation techniques, algorithms and solution strategies, parallelisation strategies, coupled problems with domain decomposition methods, optimisation algorithms and error estimation techniques. In addition, PNM researchers investigate pre-process, post-process, data management and visualisation topics.

The research lines within PNM cover the full range of techniques required to simulate a physical problem, usually governed by partial or ordinary differential equations. The main areas of investigation are:

- Mathematical modelling of a given physical process.
- Numerical modelling of the mathematical equations - space and time discretisation: high order time integration schemes; variational multi-scale; finite element; domain decomposition (Chimera, non-overlapping meshes); turbulence models; PIC methods; Spectral methods.
- Numerical algorithms to solve the discrete equations efficiently, or to couple a set of algorithms to solve complex physical problems: explicit and implicit schemes, monolithic and fractional algorithms, preconditioners and multigrid.
- Efficient implementation in a computational mechanics code: distributed/shared memory parallelisation with MPI/OpenMP, code optimisation; architecture dependent implementation (VMX, Cell).
- Code performance analysis and optimisation.

The PNM has recently started a new research line focused on the design of applications specially designed for their use in Social Sciences and policy analysis areas. The group is developing a new simulator capable of executing Agent-Based Models of human societies in a HPC environment, in order to explore:

- Emergence of behavioral patterns in human societies, understood as complex systems.
- Interaction between societies and their relationship with environment and landscape.
- Impact of change in human groups and population dynamics (both ancient and present).
- Design of artificial societies as models to understand human behavior.
- Methodological and theoretical foundations of social simulation.

These topics are analyzed from a multidisciplinary approach, as CASE joins efforts with research groups belonging to different disciplines, with diverse perspectives of social interaction (i.e. Archaeology, Demography, Economy, Heritage, History and Sociology).

**Within these areas, in 2009 the PNM Group mainly followed these lines:**

- Efficient solving strategies: preconditioners, parallel strategies, fractional schemes.
- Octree-like structured mesh generator, now undergoing testing.
- Stabilisation algorithms for compressible flows for a wide range of Mach numbers.
- A domain decomposition Chimera scheme for different fluid problems
- A numerical implementation of Electrophysiology problems in unstructured meshes.
- A module of Alya for solving large-strain solid mechanics in the total Lagrangian formulation.
- A free surface module using level sets and a bi-fluid formulation.

## High Performance Computational Mechanics (HPCM)

The HPCM Group conducts application research and development in different science and technology domains where simulations are needed: aerospace, bio-mechanics, solid state physics, high energy physics, geophysics, environment, meteorology, etc.

The activities of the HPCM Group are driven by direct interaction with users and industry. Usually the core problem requires modelling of physical processes which then must be solved by intensive numerical calculation. The principal application fields that have been developed to date are:

- Bio-mechanics: Hemodynamics, respiratory system air low, cardiac simulation.
- Geophysics: seismic imaging and oil reservoir simulations.
- Plasma Physics.
- Atmospheric flows: mesoscale and urban environments.
- Energetically Efficient Building Design.
- Ab-inito DFT and TDDFT molecular dynamic simulations.
- CFD: subsonic and supersonic flows, free surface problems, coupled problems.

**Within these fields, in 2009, the HPCM Group developed:**

- A large-strain solid mechanics model for anisotropic cardiac tissue. This model will be coupled with the electrophysiology model to creat a cardiac simulator.
- Hemodynamics simulations of the arterial brain system.
- Airflow simulations of the whole respiratory system during normal breathing cycles.
- An RTM seismic imaging facility on GPUs.
- A dynamic atmospheric mesoscale parallel code.
- A parallel version of SIESTA code with better load balancing and sparse iterative eigensolvers.
- Free surface parallel solver for sailing boats.