JUWELS Booster is the Fastest AI Supercomputer in Europe

A research team of AI consultants from JSC and the Karlsruhe Institute of Technology (KIT) succeeded in measuring the speed of AI applications running on the JUWELS Booster as well as on the HoreKa (Hochleistungsrechner Karlsruhe) supercomputer. They were able to perform the fastest ever and most computationally intensive AI calculations in Europe. As a submission entry in the MLPerf Training HPC competition, they trained two machine learning (ML) models, utilizing the high performance of the NVIDIA A100 GPUs and the Infiniband network connection installed in both machines. On JUWELS, they used up to 3072 GPUs and achieved an average computational performance of approximately 100 PFlop/s with automatic mixed precision.

The benchmark competitions are compute-intensive deep learning models. The Cosmoflow model predicts cosmological parameters from the outcome of cosmological simulations. DeepCam identifies regions of tropical storms in meteorological data. Training both models is quite data-intensive. The datasets (amounting to 8 terabytes and 5 terabytes) were stored in JSC’s file system in two large HDF5 files and the training algorithm read both files sequentially. The large main memory of the JUWELS Booster ensures that it only needs to be read once. The training was performed using the ML framework PyTorch, which makes use of automatic mixed precision.

The submission entry is a result of a collaboration between researchers from the Steinbuch Centre for Computing at KIT within Helmholtz AI, the cooperation unit of Helmholtz centres across Germany. Their AI consulting support will help to find new AI applications in science or to improve existing ones.

Contact: Dr. Stefan Kesselheim, s.kesselheim@fz-juelich.de