Team created to publish applications during COMPSs Tutorials, and share them among participants.

This team is to publish workflows executed from WPs that are not the three main pillars of the project.

Distributed computing aims to offer tools and mechanisms that enable the sharing, selection, and aggregation of a wide variety of geographically distributed computational resources in a transparent way. The research done in this team is based on the past expertise of the group, and on extending it towards the aspects of distributed computing that can benefit from this expertise. The team at BSC has a strong focus on programming models and resource management and scheduling in distributed computing ...

It explores the modelling of natural catastrophes – in particular, earthquakes and their associated tsunamis shortly after such an event is recorded.

It develops innovative adaptive workflows for climate and for the study of Tropical Cyclones (TC) in the context of the CMIP6 experiment, including in-situ analytics.

It focuses on the construction of DigitalTwins for the prototyping of complex manufactured objects integrating state-of-the-art adaptive solvers with machine learning and data-mining, contributing to the Industry 4.0 vision.

Project that aims to create the NeuroPlat portal for neurodrug design

Provenance registration is becoming more and more important, as we increase the size and number of experiments performed using computers. In particular, when provenance is recorded in HPC environments, it must be efficient and scalable. In this paper, we propose a provenance registration method for scientific workflows, efficient enough to run in supercomputers (thus, it could run in other ...

Session during the Innovative HPC workflows for industry (https://eflows4hpc.eu/event/innovative-hpc-workflows-for-industry/) that describes how Workflow Provenance is recorded with COMPSs: the background on the tools used, how the recording has been designed, and how to use it and inspect metadata.

Name: Incrementation and Fibonacci Access Level: public License Agreement: Apache2 Platform: COMPSs

Description

Brief Overview: Demonstrates COMPSs task parallelism with increment and Fibonacci computations. Helps to understand COMPSs.

Detailed Description:

1. Performs multiple increments of input values in parallel using COMPSs.
2. Concurrently calculates Fibonacci numbers using recursive COMPSs tasks.
3. Demonstrates task synchronization via compss_wait_on.

Execution

...

Calculates the Fibonacci series up to a specified length.

Name: Matmul GPU Case 1 Cache-ON Contact Person: cristian.tatu@bsc.es Access Level: public License Agreement: Apache2 Platform: COMPSs Machine: Minotauro-MN4

Matmul running on the GPU leveraging COMPSs GPU Cache for deserialization speedup. Launched using 32 GPUs (16 nodes). Performs C = A @ B Where A: shape (320, 56_900_000) block_size (10, 11_380_000)             B: shape (56_900_000, 10)   block_size (11_380_000, 10)             C: shape (320, 10)                block_size ...

Name: Matmul GPU Case 1 Cache-OFF Contact Person: cristian.tatu@bsc.es Access Level: public License Agreement: Apache2 Platform: COMPSs 3.3 Machine: Minotauro-MN4

Matmul running on the GPU without Cache. Launched using 32 GPUs (16 nodes). Performs C = A @ B Where A: shape (320, 56_900_000) block_size (10, 11_380_000)             B: shape (56_900_000, 10)   block_size (11_380_000, 10)             C: shape (320, 10)                block_size (10, 10) Total dataset size 291 ...

Name: K-Means GPU Cache OFF Contact Person: cristian.tatu@bsc.es Access Level: public License Agreement: Apache2 Platform: COMPSs Machine: Minotauro-MN4

K-Means running on GPUs. Launched using 32 GPUs (16 nodes). Parameters used: K=40 and 32 blocks of size (1_000_000, 1200). It creates a block for each GPU. Total dataset shape is (32_000_000, 1200). Version dislib-0.9

Average task execution time: 194 seconds

Name: K-Means GPU Cache ON Contact Person: cristian.tatu@bsc.es Access Level: public License Agreement: Apache2 Platform: COMPSs Machine: Minotauro-MN4

K-Means running on the GPU leveraging COMPSs GPU Cache for deserialization speedup. Launched using 32 GPUs (16 nodes). Parameters used: K=40 and 32 blocks of size (1_000_000, 1200). It creates a block for each GPU. Total dataset shape is (32_000_000, 1200). Version dislib-0.9

Average task execution time: 16 seconds

Name: Dislib Distributed Training - Cache ON Contact Person: cristian.tatu@bsc.es Access Level: public License Agreement: Apache2 Platform: COMPSs Machine: Minotauro-MN4

PyTorch distributed training of CNN on GPU and leveraging COMPSs GPU Cache for deserialization speedup. Launched using 32 GPUs (16 nodes). Dataset: Imagenet Version dislib-0.9 Version PyTorch 1.7.1+cu101

Average task execution time: 36 seconds

Name: Dislib Distributed Training - Cache OFF Contact Person: cristian.tatu@bsc.es Access Level: public License Agreement: Apache2 Platform: COMPSs Machine: Minotauro-MN4

PyTorch distributed training of CNN on GPU. Launched using 32 GPUs (16 nodes). Dataset: Imagenet Version dislib-0.9 Version PyTorch 1.7.1+cu101

Average task execution time: 84 seconds

Lysozyme in water full COMPSs application run at MareNostrum IV, using full dataset with two workers

PyCOMPSs implementation of Probabilistic Tsunami Forecast (PTF). PTF explicitly treats data- and forecast-uncertainties, enabling alert level definitions according to any predefined level of conservatism, which is connected to the average balance of missed-vs-false-alarms. Run of the Kos-Bodrum 2017 event test-case with 1000 scenarios, 8h tsunami simulation for each and forecast calculations for partial and full ensembles with focal mechanism and tsunami data updates.

PyCOMPSs implementation of Probabilistic Tsunami Forecast (PTF). PTF explicitly treats data- and forecast-uncertainties, enabling alert level definitions according to any predefined level of conservatism, which is connected to the average balance of missed-vs-false-alarms. Run of the Boumerdes-2003 event test-case with 1000 scenarios, 8h tsunami simulation for each and forecast calculations for partial and full ensembles with focal mechanism and tsunami data updates.

Name: Random Forest Contact Person: support-compss@bsc.es Access Level: public License Agreement: Apache2 Platform: COMPSs Machine: MareNostrum4 This is an example of Random Forest algorithm from dislib. To show the usage, the code generates a synthetical input matrix. The results are printed by screen. This application used dislib-0.9.0

Lysozyme in water sample COMPSs application

Cluster Comparison COMPSs application

Cholesky factorisation COMPSs application

K-means COMPSs application

Wordcount reduce version COMPSs application

Wordcount merge version COMPSs application

Lysozyme in water full COMPSs application, using dataset_small

Lysozyme in water full COMPSs application, using dataset_small