Merlin6 is the newest generation of the central PSI HPC clusters. In addition to Merlin6, surviving parts of the legacy Merlin5 system are available and maintained on a best effort basis.
The initial installment of Merlin6 was deployed in 2019. The cluster was designed to be extensible regarding the addition of compute nodes and storage. In addition to the main cluster's CPU based resources the system also contains a smaller partition of GPU resources for biology research (Cryo-EM analysis) and machine learning applications.
| Service | Hardware | |||
|---|---|---|---|---|
| Solution | Blade | Description | ||
|
Computing nodes (Merlin6) |
4 x HPE Apollo k6000 Chassis |
(24 blades per chassis) |
72 x Two Intel® Xeon® Gold 6152 Scalable Processor @ 2.10GHz (2 x 22 cores per node, HT-enabled, 384GB RAM, NVMe /scratch) 24 x Two Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz (2 x 24 cores per node, 18 x 768GB + 6 x 384GB RAM, NVMe / scratch) HPC Network based on Dual Port Infiniband ConnectX-5 EDR (1 x 100Gbps). Standard network 1 x 10Gbps. |
|
|
Computing nodes (Merlin5) |
Chassis |
(16 blades per chassis) |
Two Intel® Xeon® Processor E5-2670 @ 2.60GHz (2 x 8 cores, no-HT, 64GB RAM, SAS /scratch) HPC Network based on Single Port Infiniband ConnectX-3 QDR (1 x 40Gbps). Standard network 1Gbps. |
|
|
Login nodes |
Single Blade
|
Two Intel® Xeon® Gold 6152 Scalable Processor @ 2.10GHz (2 x 22 cores, HT-enabled, 384GB RAM, NVMe /scratch) HPC Network based on Dual Port Infiniband ConnectX-5 EDR (2 x 100Gbps). Standard network 2 x 10Gbps.
|
||
| 1 x HPE Proliant DL380 Gen9 |
Two Intel(R) Xeon(R) CPU E5-2697A v4@ 2.60GHz (2 x 16 cores, HT-enabled, 512GB RAM, SAS /scratch) HPC Network based on Dual Port Infiniband ConnectIB FDR (1 x 56Gbps). Standard network 1 x 1Gbps. |
|||
| Storage nodes |
Lenovo Distributed Storage Solution for IBM Spectrum Scale 1 x Lenovo DSS G240 Building block |
1 x ThinkSystem SR630 (Mgmt node) 2 x ThinkSystem SR650 (IO nodes) |
ThinkSystem SR630: Two Intel(R) Xeon(R) Gold 5118 Scalable Processor @ 2.30Ghz (2 x 12 cores, HT-enabled, 96GB RAM
Building block 1:
|
|
Other details
HPC Network
Merlin5 Infiniband is based on QDR (40Gbps) and FDR (56Gbps) technologies for MPI communication as well as for storage access. Merlin5 is connected to Merlin6 through FDR (56 Gbps) (MPI)
Merlin6 Infiniband is based on EDR (100Gbps) technology for MPI communication as well as for storage access. Infiniband bandwidth between chassis provide up to 1200Gbps
Network devices
AFS
AFS is available at PSI under the 'psi.ch' domain. This is mounted in the Merlin6 cluster thanks to the Auristor client. AFS contains personal user information as well as the software stack used in the Merlin clusters. AFS is mounted through the standard Ethernet network.
Home directories
Home directories are mounted under the PSI Central NFS service, providing up to 10GB capacity to each each user, with daily snapshots for one week. This is mounted through the standard Ethernet network.
HPC storage
The main storage is based on the IBM's General Parallel FileSystem Spectrum Scale, suitable for HPC environments. This is mounted through the Infiniband network for high performance and low latency.
O.S.
All nodes and servers are running RedHat Enterprise Linux.
Middleware
Remote Desktop
For remote desktop access, the newest login nodes are running NoMachine Terminal Server.
Batch system
The workload manager is based on the Slurm Workload Manager. The Merlin6 Slurm configuration allows running from single core based jobs up to MPI based jobs, allowing to scale up when running over multiple nodes.