QuantaStor Ceph NFS: kernel vs Ganesha

QuantaStor Ceph requires Ganesha NFS server if NFS is used. Since we ship one ISO with the ability to run ZFS or Cehp or both, we ship QuantaStor with the ability to run kernel NFS or Ganesha NFS or both.

NFS servers typically listen on TCP Port 2049.  Since they cannot both use the same port number, by default kernel NFS uses 2049 and Ganesha NFS uses 2249.

Unfortunately some NFS clients cannot be configured to use a non-standard port number like 2249.

If you have deployed Ceph, need NFS NAS, and have clients that must use port 2049, then the following steps can be used to disable kernel NFS and reconfigure Ganesha NFS to use port 2049.

These steps must be done on all Ceph nodes in the QuantaStor cluster.

Disable Kernel NFS

The following procedure will need to be performed on all Ceph nodes in the QuantaStor cluster

• Blacklist the kernel module by creating /etc/modprobe.d/blacklist-nfsd.conf file containing the following lines:
  
  • blacklist nfsd
    blacklist nfs_acl
    blacklist lockd
    blacklist grace
• Disable kernel NFS in systemctl via the following commands:
  
  • systemctl stop nfs-server.service
  • systemctl stop nfs-kernel-server.service
  • systemctl disable nfs-server.service
  • systemctl disable nfs-kernel-server.service

Reconfigure Ganesha NFS to use Port 2049

• Inside /etc/ganesha/ganesha.conf
  • Change line:     nfs_port = 2249;
  • To:            nfs_port = 2049;

Reboot the System

• All nodes in the QuantaStor Ceph cluster will need to be rebooted

Check Kernel NFS Services are Disabled and Ganesha is on Port 2049

• On all nodes in the cluster
• lsmod | grep nfsd
  • This should return nothing
• netstat -anp | grep ganesha
  • The following line should show be returned:
  • tcp6       0      0 :::2049                 :::*                    LISTEN      27484/ganesha.nfsd 

Script to facilitate this process

• disable-kernel-nfs-config-ganesha-port2049.sh
• #create blacklist-nfsd.conf
  echo "blacklist nfsd
  blacklist nfs_acl
  blacklist lockd
  blacklist grace" > /etc/modprobe.d/blacklist-nfsd.conf
  
  #disable kernel nfs services is systemctl
  systemctl stop nfs-server.service
  systemctl stop nfs-kernel-server.service
  systemctl disable nfs-server.service
  systemctl disable nfs-kernel-server.service
  
  #change ganesha.conf port from 2249 to 2049
  sed -i "s/nfs_port = 2249/nfs_port = 2049/g" /etc/ganesha/ganesha.conf
  
  #request reboot
  echo "Please reboot this node to enable Ganesha NFS on port 2049"

After these steps are completed, you can mount clients using syntax like "mount -t nfs <QuantaStor site VIF IP address>:<NFS share name> <local client mount point directory>