BSC offers to the HPC community a set of tools that helps developers to program and execute their applications efficiently on distributed computational infrastructures.
This release includes support for Python 3 and for Python virtual environments, a persistent worker with memory-to-memory transfers for the C binding, and a default implementation of the persistent storage API based on Redis DB.
Attend a live PyCOMPSs demo at the BSC booth # A1412
The Workflows and Distributed Computing team at the Barcelona Supercomputing Center is proud to announce a new release, version 2.3 (codename Daisy), of the programming environment COMPSs.
This version of COMPSs, available from today, updates the result of the team’s work in the last years on the provision of a set of tools that helps developers to program and execute their applications efficiently on distributed computational infrastructures such as clusters, clouds and container managed platforms. COMPSs is a task based programming model known for notably improving the performance of large scale applications by automatically parallelizing their execution.
COMPSs has been available for the last years to the MareNostrum supercomputer users and to the Spanish Supercomputing Network and has been adopted in several research projects such as OPTIMIS, VENUS-C, EUBrazil OpenBio, EUBrazil CloudConnect, EUBra-BIGSEA, transplant, EGI and ASCETIC. In these projects COMPSs has been applied to implement use cases provided by different communities across diverse disciplines as biomedicine, engineering, biodiversity, chemistry, astrophysics and earth sciences. Currently it is also under extension and adoption in applications in the projects NEXTGenIO, MUG, mf2C, TANGO, CLASS, ExaQUte, LANDSUPPORT the BioExcel CoE, and the EXPERTISE ETN, as well as in a research contract with FUJITSU.
The new release comes with support for Python 3, while still giving backward compatibility to Python 2. Additionally, Python virtual environments are now supported by PyCOMPSs. This enables a user to work with several environments, each of them with different modules, packages, versions, or dependencies, installed. These environments can be activated or deactivated by the user according to their actual requirements. Another new feature for PyCOMPSs is that now can be run as a Python module from the Python interpreter.
Also, the worker side of the runtime has been extended to implement a memory-to-memory transfer mechanism for C applications that reduces the overhead of the tasks’ execution. With this memory-to-memory transfers, objects in memory do not need to be serialized and written to files when exchanging data between tasks.
The existing Storage API that enables the storage of memory objects in persistent memory in a transparent way for COMPSs applications has been now extended with a default implementation based on Redis which is distributed with the PyCOMPSs/COMPSs release 2.3.
COMPSs 2.3 comes with other minor new features, extensions and bug fixes.
At ISC-HPC 2018, PyCOMPSs/COMPSs will have a presence at the BSC booth, with presentations and demos. A live demo on the use of PyCOMPSs with the Jupyter notebook will be delivered at the BSC booth no. A1412, on Tuesday 26th at 11.15.
COMPSs has had around 1000 downloads last year and is used by around 20 groups in real applications. COMPSs has recently attracted interest from areas such as image recognition, genomics and biodiversity, where specific courses and dissemination actions have been performed.
The packages and the complete list of features are available in the Downloads page. A virtual appliance is also available to test the functionalities of COMPSs through a step-by-step tutorial that guides the user to develop and execute a set of example applications.
Additionally, a user guide and papers published in relevant conferences and journals are available.
For more information on COMPSs please visit our webpage: http://www.bsc.es/compss