COMP Superscalar

Big Data Distributed Computing Programming Models

COMP Superscalar (COMPSs) is a framework which aims to ease the development and execution of parallel applications for distributed infrastructures, such as Clusters, Clouds and containerized platforms.

Software Author: 

Workflows and Distributed Computing Group

Contact:

Jorge Ejarque (jorge [dot] ejarque [at] bsc [dot] es)

Rosa M. Badia (rosa [dot] m [dot] badia [at] bsc [dot] es)

Support mailing list (support-compss [at] bsc [dot] es)

Software Cost: 

COMP Superscalar is distributed under Apache License version 2

Primary tabs

COMPSs Download Form
Please, fill the following form in order to access this download:
2 + 3 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
COMPSs VM Download Form
Please, fill the following form in order to access this download:
17 + 0 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.

2.7 (Latest Version)

COMP Superscalar version 2.7 (Hyacinth) Release date: June-2020

Release Notes

New features

  • Runtime:
    • New trace events and cfg to see task constraints.
    • New task constraint option  for storage bandwidth.
    • Directories as task dependency type.
    • New IO tasks which can be overlapped with computational tasks.
    • New "weight" parameter property to prioritise data in data location schedulers.
    • New "keep rename" parameter property to enable/disable conversion to original names at worker.
    • New API call to check if a file exists or it is going to be generated by a task.
    • Support for MPI+OpenMP hybrid tasks.
  • Python:
    • Support for Python type hinting.
    • Support for executing Jupyter notebooks in myBinder.
    • PyCOMPSs player - a container based environment to easily install and use PyCOMPSs/COMPSs.
  • DDS-2:
    • Combination and execution of multiple operations within a single task.
    • New methods and optimizations in DDS class.

Improvements:

  • Change in MPI tasks syntax. ComputingNodes property has been changed to processes.
  • New flags in enqueue_compss to provide a port range to deploy workers (--worker_port_range), generate a core dump (--gen_coredump) and allow JMX connections for JVM profiling (--jmx_port).
  • Support for collections with direction OUT.
  • Support for short labels in traces (Requires latest Paraver version for a correct visualization).
  • Avoid failure in compss_wait_on_file if the file has not been previously used by a task.
  • Support for task with "isReplicated" properties in shared file systems.
  • Optimizations in data synchronizations in shared file systems.
  • Support for TCS as job scheduler in clusters.
  • Configuration files for Irene, Salomon and Starlife clusters.
  • Several bug fixes.

Known Limitations

  • Tasks that invoke Numpy and MKL may experience issues if a different MKL threads count is used in different tasks. This is due to the fact that MKL reuses  threads in the different calls and it does not change the number of threads from one call to another. This can be also happen with other libraries implemented with OpenMP.
  • C++ Objects declared as arguments in a coarse-grain tasks must be passed in the task methods as object pointers in order to have a proper dependency management.
  • Master as worker is not working for executions with persistent worker in C++.
  • Coherence and concurrent writing in parameters annotated with the "Concurrent" direction must be managed by the underlaying distributed storage system.
  • Delete file calls for files used as input can produce a significant synchronization of the main code.
  • Defining a parameter as OUT is only allowed for files and collection files.
  • There is an issue with hwloc and Docker which could affect to python mpi workers. Fixing it require to upgrade the hwloc version used by the MPI runtime.

For further information please refer to COMPSs Documentation

Check Installation manual for details about how to install from the repository

Read this document before downloading the VM image: COMPSs VM Instructions

Docker Image pull command:

docker pull compss/compss:2.7

Old Versions

2.6

2.5

2.4

2.3

2.2

2.1

2.0