Skip to main content

Nord3v2

What has changed?​

If you are coming from the old Nord III system, there are some relevant changes that you should take into account:

  • Upgraded O.S. (Red Hat Enterprise Linux 8.4)
  • Transition from LSF to SLURM! (SLURM 21.08.2)

These changes, however, will have some implications for all users. The main implications are:

  • Old jobscripts will have to be adapted to SLURM.
  • Nord3's software won't be available. You will need to request what you need.

System Overview​

Nord III version 2 is a supercomputer based on Intel SandyBridge processors, iDataPlex Compute Racks, a Linux Operating System and an Infiniband interconnection.

The current Peak Performance is 251,6 Teraflops. The total number of processors is 12,096 Intel SandyBridge-EP E5-2670 cores at 2.6 GHz (756 compute nodes) with at least 24.2 TB of main memory. See below a summary of the system:

  • 9 iDataPlex compute racks. Each one composed of:
    • 84 IBM dx360 M4 compute nodes
    • 4 Mellanox 36-port Managed FDR10 IB Switches
    • 2 BNT RackSwitch G8052F (Management Network)
    • 2 BNT RackSwitch G8052F (GPFS Network)
    • 4 Power Distribution Units
  • All IBM dx360 M4 node contain:
    • 2x E5-2670 SandyBridge-EP 2.6GHz cache 20MB 8-core
    • 500GB 7200 rpm SATA II local HDD
    • One of the following RAM configurations:
      • Default nodes: 32 GB/node
      • Medium memory nodes: 64 GB/node
      • High memory nodes: 128 GB/node
  • 15 PB of GPFS disk storage
  • Interconnection Networks
    • Infiniband Mellanox FDR10: High bandwidth network used by parallel applications communications (MPI)
    • Gigabit Ethernet: 10GbitEthernet network used by the GPFS Filesystem.

The operating system is Red Hat Enterprise Linux 8.4.

As we said before, there are 3 different types of nodes, each one with a different amount of memory available.

However, right now there is only one type of node available. All three types of nodes will be added in the future.

Connecting to the new Nord III​

The first thing you should know is your username and password. Once you have a login and its associated password you can get into the cluster through one of the following login nodes:

  • nord1.bsc.es
  • nord2.bsc.es
  • nord3.bsc.es
  • nord4.bsc.es

You must use Secure Shell (ssh) tools to login into or transfer files into the cluster. We do not accept incoming connections from protocols like telnet, ftp, rlogin, rcp, or rsh commands. Once you have logged into the cluster you cannot make outgoing connections for security reasons.

To get more information about the supported secure shell version and how to get ssh for your system (including windows systems) see the Appendices. Once connected to the machine, you will be presented with a UNIX shell prompt and you will normally be in your home ($HOME) directory. If you are new to UNIX, you will need to learn the basics before doing anything useful.