Subject:
It's possible that if an NFS share get's unmounted then data can be written to the unmounted mount point directory on the local hard disk.
This can cause two issues:
1. If it is not corrected the local filesystem can fill up.
2. Once the NFS share is remounted the data that was written will not be seen but will still use space on the disk.
Detail:
Environment:
Upstream clients use FTP to continually send files to a FTP Server which has a NFS mount to a QuantaStor share.
Problem:
Sometimes the FTP Server's local disk fills up.
Possible cause:
The FTP server NFS mount gets disconnected from the QuantaStor share and files are written locally to the FTP server.
This could happen if the QuantaStor NFS share is disable, the FTP server gets rebooted, and the upstream clients continue to send files. If the FTP server process on the FTP server is still running, it will write the files to the local disk instead of QuantaStor.
Fix to prevent this from occurring:
Changing the local mount point on the FTP server to be immutable will prevent writes to that directory unless that directory is successfully mounted to QuantaStor. In this state, if not mounted, the FTP server process will throw errors when trying to write files locally. A monitoring solution can catch these and a remount of the QuantaStor will fix the problem.
- ( replace all items below "<....>" with the proper value
- Login to the FTP server
- Become root via
- sudo su -
- Find the mount point via
- mount | grep nfs
- The first item is the NFS server IP and share, the second is the mount point.
- To set the mount point immutable, it must be unmounted. Remember this will prevent the FTP server software from receiving files from the FTP clients and must be done during a scheduled outage.
- umount <mount point>
- Set mount point immutable
- chattr +i <mount point>
- Test this by
- cd <mount point>
- touch hello
- This should return an error
- Remount the mount point via
- mount -a (if mount point is in /fstab)
- or manually
- Test again
- cd <mount point>
- touch hello
- This should work
At this point, the FTP server has been set to not allow file writes to the mount point.
Comments