GRID superscalar Run-time

Run-time features


  • Data dependence analysis
The core functionality of the GRID superscalar run-time is the Execute function. A data dependence analysis is performed inside this function to build the tasks DAG. The data dependence analysis is performed for the parameters that are input/output files. However, we are working to extending the data dependence analysis to all function parameters.

Data dependencies are classified as Read after Write (RaW), Write after Read (WaR) and Write after Write (WaW). Applying renaming to the filenames when a WaR or WaW dependence exists eliminates the dependence and increases the DAG concurrency.

  • File renaming
GRID superscalar run-time handles the renaming, maintaining at each moment for each renamed file the original filename and for each original filename which is the last renamed filename, and taking this renaming into account in the data dependence analysis. The run-time also keeps track of the server where each file is located. Files are transferred only on demand and if required. Together with a locality-aware scheduling policy, the number of file transfers is largely reduced.

  • Disk sharing
Another feature included in the run-time is the disk sharing awareness. By disk sharing we mean, for example, multinode servers with disks mounted through systems such as NFS. If two (or more) servers are mounted through NFS or any other type of system, the file transfers between them can be reduced to a local copy. Furthermore, if the same working directory is used, the run-time detects that the copies are not necessary and file transfers are not required.

  • Task submission
The task submission is performed inside the Execute function. If the task that is currently being called does not have any unresolved data dependence, and a hardware resource is available (provided by a broker), then the task is submitted for execution. If any of the previous conditions are not accomplished, then the task will be inserted in one of the internal queues. Once the current task has been executed, the output parameter will be sent from the server to the client host.

  • Checkpointing
Another feature included in the run-time library is a inter-task level checkpointing. This checkpointing mechanism is transparent to the application developer. The run-time tracks the tasks that have finished correctly, and in case any failures occurs, the application can be restarted from the failure point. Additionally to the task identifiers the run-time keeps copies of the required files to ensure a safely recovery.

Originally, GRID superscalar used as a underlying middleware Globus Toolkit 2.x. Recently, GRID superscalar has been extended to allow Globus Toolkit 4, Ninf-G and ssh/scp. You can find specific information about these extensions in the following links: