Storage

Now that master server is installed, it is time to configure the basic io servers, using NFS.
To do this, you will need to install the io server with Centos 7.2 also, and configure repository. You can install this server manually (USB stick, DVD, etc), or start like a compute node: install it using PXE.
Have a look at managing cluster chapter on how to install a server using PXE.

First thing to do on the nfs node is to configure network on it. We will use a static ip for io nodes.

Go to /etc/sysconfig/network-script, and edit ifcfg-enp0s3 to the following:

DEVICE="enp0s3"
NAME="enp0s3"
TYPE="Ethernet"
NM_CONTROLLED=no
ONBOOT="yes"
BOOTPROTO="static"
IPADDR="10.0.1.1"
NETMASK=255.255.0.0

Now restart network to activate the interface:

systemctl restart network

Lets disable NetworkManager, and Firewall:

systemctl stop NetworkManager
systemctl stop firewalld
systemctl disable NetworkManager
systemctl disable firewalld

We need to setup repository client, to be able to install packages.

From master (batman), just upload repository file to the nfs node:

scp /etc/yum.repos.d/os_base.local.repo root@10.0.1.1:/etc/yum.repos.d/os_base.local.repo

For record, file was:

[os_base.local]
name=os_base.local.repo
baseurl=http://10.0.0.1/repo/os_base.local.repo
gpgcheck=0
enabled=1

Login again on nfs node,

Also remove CentOs original repository files, but for safety, backup them, and update yum :

mkdir /etc/yum.repos.d.old/
mv /etc/yum.repos.d/CentOS-* /etc/yum.repos.d.old
yum clean all
yum update

This server will host the /home and /hpc-software of the whole cluster. /home is the main users directory, where they will be able to upload and compile their code, and also set their configuration. It will be accessible on each login and compute nodes. /hpc-software contains all shared tools and libraries to be available on each compute and login nodes.

Install needed packages:

yum install nfs-utils

/home should already be there, make /hpc-software dir:

mkdir /hpc-softwares

Then, edit nano /etc/exports to tell nfs server which directories to export:

/hpc-softwares  10.0.0.0/16(rw,no_root_squash,sync)
/home 10.0.0.0/16(rw,no_root_squash,sync)

And start nfs server:

systemctl enable rpcbind
systemctl enable nfs-server
systemctl start rpcbind
systemctl start nfs-server

That's all for basic NFS server. You can tune it if needed, or proceed to compute nodes installation.

Edit /etc/default/nfs-kernel-server and add more workers (to handle more connections) :

# Number of servers to start up
RPCNFSDCOUNT=8

Then restart nfs.

Increase buffer sizes on client side, especially for large files manipulations networks. In fstab (on compute/login nodes), use rsize=524288,wsize=524288 for mount options.

Use nfsV3 instead of v4. In fstab, add vers=3 for mount options.

Increase MTU (see network). This parameter is very important as it has a major impact on performances (9000 is a good start for Ethernet, and 65536 for IPoverIB Infiniband).