Troubleshooting NFS Mount Issues in Linux

Network File System (NFS) is a protocol which allows a system to share directories and files with others over a network. By using NFS, users and programs can access files on remote systems almost as if they were local files.

This post refers how to mount the network share in our local system and what are all the common issues and how to generally troubleshoot connectivity and config issues.

NFS Client Configuration

1. Install the required nfs packages if not already installed on the server Read more

# rpm -qa | grep nfs-utils


# yum install nfs-util

2. Use the mount command to mount exported file systems. Syntax for the command:

# mount -t nfs -o options host:/remote/export /local/directory 

——————- advertisements ——————-  

———————————————————

Example :

# mount -t nfs -o ro,nosuid remote_host:/home /remote_home

This example does the following:
– It mounts /home from remote host (remote_host) on local mount point /remote_home.
– File system is mounted read-only and users are prevented from running a setuid program (-o ro,nosuid options).

3. Update /etc/fstab to mount NFS shares at boot time.

# vi /etc/fstab


remote_host:/home      /remote_home nfs        ro,nosuid           0            0

Troubleshooting NFS connectivity issues

Depending on the client and the issue, wide range of error messages can appear while trying to mount an NFS share, it might also take forever to mount, or even mount normally but the mount points will be empty.Below are the common errors we face in the client side while mounting the NFS/NAS shares.

——————- advertisements ——————-  

———————————————————

Error 1: 

mount: mount to NFS server 'NFS-Server' failed: System Error: No route to host.

This can be caused by the RPC messages being filtered by either the host firewall, the client firewall, or a network switch. Verify if a firewall is active and if NFS traffic is allowed. Normally nfs is using port 2049.

  1. Check the show mount output of the server to verify the filesystem has exported for the client ip.
# showmount –e <NFS server IP > | grep –I  <clientIP>

Check the port Connectivity of the NFS server using telnet

# telnet <NFS server IP> 2049

 

Error 2:

mount_nfs: can't mount / from 1.2.3.4 onto /mnt: RPC prog. not avail

Error: “mount clntudp_create: RPC: Port mapper failure – RPC: Unable to receive

The Linux NFS implementation requires that both the NFS service and the portmapper (RPC) service be running on both the client and the server. Check it like this:

——————- advertisements ——————-  

———————————————————

            # rpcinfo -p
     program vers proto   port  service

    100000    4   tcp    111  portmapper

    100000    3   tcp    111  portmapper

    100000    2   tcp    111  portmapper

    100000    4   udp    111  portmapper

    100000    3   udp    111  portmapper

    100000    2   udp    111  portmapper...

          # ]# systemctl status rpcbind
  •  rpcbind.service - RPC bind service
             Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; indirect; vendor preset: enabled)

            Active: active (running) since Fri 2018-05-18 12:39:15 IST; 2s ago

            Process: 15222 ExecStart=/sbin/rpcbind -w $RPCBIND_ARGS (code=exited,      status=0/SUCCESS)

 Main PID: 15223 (rpcbind)

            CGroup: /system.slice/rpcbind.service

           └─15223 /sbin/rpcbind -w

 

May 18 12:39:15 nfsserver systemd[1]: Starting RPC bind service...

May 18 12:39:15 nfsserver systemd[1]: Started RPC bind service.

If not, start it with the commands give below.

# systemctl start rpcbind

——————- advertisements ——————-  

———————————————————

Error 3: 

Error: “NFS Stale File Handle”

Unlike traditional Linux file systems that allow an application to access an open file even if the file has been deleted using unlink or rm, NFS does not support this feature. An NFS file is deleted immediately. Any program which attempts to do further I/O on the deleted file will receive the “NFS Stale File Handle” error. For example, if your current working directory is an NFS directory and is deleted, you will see this error at the next shell prompt.

To refresh the client’s state with that of the server you may do a lazy unmount the mount point and remount it

# umount -l /mnt/mount_point

or kill the process, which references the mounted file system:

# fuser -k [mounted-filesystem].

——————- advertisements ——————-  

———————————————————

Error 4:

Error: “Access Denied” or “Permission Denied

Check the export permissions for the NFS file system. You can do this from the client:

# showmount -e server_name

 

Error 5:

Error: “rpc mount export: RPC: Timed out

Unable to access file system at [NFS SERVER]: rpc mount export: RPC: Timed out This is caused by DNS name resolution issue. NFS(RPC) needs reverse name resolution. If NFS server or client cannot resolve their name, this error occurs. In case gets the error message, check DNS configuration and /etc/hosts configuration.

 

Hope we have covered almost all the regular errors and steps for solving those. Please share your thoughts in the comments section. If you want us to add any additional issues-resolution, kindly let us know.

Thanks for reading..!

LINUX- Active Directory Integration

Most of the organisation uses Active directory domain services for user administration and management.Like windows machines, Linux servers also can authenticate and managed via active directory. In this tutorial, we are describing how to join a Linux server in to an active directory domain.

 Environment Prerequisites

Read more

  • Microsoft Windows Active Directory.
  • Linux host – RHEL
  • Below Packages needed to be installed on Linux host
  • Samba (version 3):
    • samba3x
    • samba3x-client
    • samba3x-winbind
    • samba3x-common
    • And  packages that might be needed to meet dependencies
  • Kerberos:
    • krb5-workstation
    • krb5-libs
    • And packages that might be needed to meet dependencies
  • PAM:
    • pam_krb5
  • NTP:

——————- advertisements ——————-  

———————————————————

   Configuration

This section describes the technical configuration of how to add Linux host as member of a Microsoft Windows Active Directory domain.Technical steps are below.

1. Update  the FQDN in /etc/hosts

It’s highly recommended to update  /etc/hosts with Acive directory FQDN. If something happens to DNS ,system can still resolve out to it.

2. Update the Host name – /etc/sysconfig/network

where “master” is the RHEL host name and “ADserver “is the ADDS (Active directory domain service) Server name.

3. Update the DNS – /etc/resolve.conf

Set the system’s search domain and point to the AD DNS server in /etc/resolv.conf

4. Synchronise the Time – /etc/ntp.conf

Its mandatory to have time synchronization between the domain server and its client. To achieve this, edit the ntp server details in the ntp.conf.

——————- advertisements ——————-  

———————————————————-

5. Update the Samba and krb configuration using authconfig-tui

Check if necessary packages are installed and backup the below configuration file 

/etc/krb5.conf

/etc/samba/smb.conf

Execute the command authconfig-tui. You will get the below text user interface. Fill in the field as below

Once You checked the necessary fields mentioned above, click on Next

——————- advertisements ——————-  

———————————————————-

Update the Kerberos setting as per your environment and click next.

Modify the Samba settings and click Ok.

Verify the configuration

Validate and update the additional information on the Kerberos and samba configuration files

  1. Verify /etc/krb5.conf

2.Update /etc/samba/smb.conf for ID management

Update idmap config range as below as well as backend connection as rid. This is to keep      same UID for the users across the domain. Please insert if these lines are not present

——————- advertisements ——————-  

———————————————————-

3. verify /etc/nsswitch.conf

In order to tell the system to use winbind for authentication, add winbind to passwd and group in /etc/nsswitch.conf as below if it is not already get updated

Join the server to the domain

To join the server in domain, under the specific OU , use the below command

#net ads join createcomputer=Datacenter-FI/Linux_Servers -U <admin id>

Replace the OU names accroding to your environment (Datacenter-Fi/Linux_servers is based on my test environment).You should be having an admin ID created in the AD already to join the computer.

Restart the service

Once joined to the domain , restart the  winbind service

#systemctl restart winbind

——————- advertisements ——————-  

———————————————————-

Restrict Access only to a specific AD group

To restrict access to the server for a specific AD group is possible via editing the file /etc/security/pam_winbind.conf .

 

Edit the line require_membership_of  and add the SIDs of the group which needs access to this server by comma separated.

Enable the Home directory on first login

Enable oddjobd to create home directory automatically in the initial login with default permissions of 700

# authconfig –enablemkhomedir –update

Verify Your Access

We have completed the AD integration in the server. now test your access with your AD id and password.

eg: login [email protected] and password – AD password.

Hope this helps you. Please have your queries and suggestions in the comments section below.

Linux Swap Space Creation and Monitoring

Overview

This Post is intended to understand the swap creation, monitoring and extending in Redhat Linux.

Swap space is a restricted amount of physical memory that is allocated for use by the operating system when the amount of physical memory (RAM) is full. If the system needs more memory resources and the RAM is full, inactive pages in memory are moved to the swap space. While swap space can help machines with a small amount of RAM, it should not be considered a replacement for more RAM. Swap space is located on hard drives, which have a slower access time than physical memory. Read more

Recommended System Swap Space
In years past, the recommended amount of swap space increased linearly with the amount of RAM in the system. But because the amount of memory in modern systems has increased into the hundreds of gigabytes, it is now recognized that the amount of swap space that a system needs is a function of the memory workload running on that system. However, given that swap space is usually designated at install time, and that it can be difficult to determine beforehand the memory workload of a system, Redhat recommend determining system swap using the following table.

Amount of RAM in the System Recommended Amount of Swap Space
4GB of RAM or less a minimum of 2GB of swap space
4GB to 16GB of RAM a minimum of 4GB of swap space
16GB to 64GB of RAM a minimum of 8GB of swap space
64GB to 256GB of RAM a minimum of 16GB of swap space
256GB to 512GB of RAM a minimum of 32GB of swap space

Note : On most distributions of Linux, it is recommended that you set swap space while installing the operating system

 

How to Monitor Swap Space

We shall look at different commands and tools that can help you to monitor your swap space usage in your Linux systems as follows

Using the swapon Command

To view all devices marked as swap in the /etc/fstab file you can use the –all option. Though devices that are already working as swap space are skipped

If you want to view a summary of swap space usage by device, use the – summary (swapon –s) option.

[[email protected] ~]# swapon –summary
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       2097148 0       -1
[[email protected] ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       2097148 0       -1
Note :- Use –help option to view more options and information.
Using /proc/swaps

The /proc filesystem is a process information pseudo-file system. It actually does not contain ‘real’ files but runtime system information, for example system memory, devices mounted, hardware configuration and many more.

[[email protected] ~]# cat /proc/swaps

Filename                                Type            Size    Used    Priority

/dev/dm-1                               partition       2097148 0       -1

[[email protected] ~]#

Using ‘free’ Command
The free command is used to display the amount of free and used system memory. Using the free command with -h option, which displays output in a human readable format.
[[email protected] ~]# free -h
              total        used        free      shared  buff/cache   available
Mem:           7.6G        674M        6.5G        9.8M        507M        6.7G
Swap:          2.0G          0B        2.0G
 Using top Command
To check swap space usage with the help of ‘top’ command
Using the vmstat Command
This command is used to display information about virtual memory statistics
[[email protected] ~]# vmstat
procs ———–memory———- —swap– —–io—- -system– ——cpu—–
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 6791708   2784 516484    0    0     7     0   24   23  0  0 100  0  0
ADDING SWAP SPACE
Sometimes it is necessary to add more swap space after installation
You have three options: create a new swap partition, create a new swap file, or extend swap on an existing LVM2 logical volume. It is recommended that you extend an existing logical volume
Extending Swap on an LVM2 Logical Volume
To extend an LVM2 swap logical volume(suppose /dev/mapper/centos-swap is our swap volume)
1. Disable swapping for the associated logical volume:
[[email protected] ~]# swapoff -v /dev/mapper/centos-swap
swapoff /dev/mapper/centos-swap
[[email protected] ~]# swapon -s
2. Resize the LVM2 logical volume by 256 MB
 [[email protected] ~]# lvresize /dev/mapper/centos-swap -L +256M
  Size of logical volume centos/swap changed from 2.00 GiB (512 extents) to 2.25 GiB (576 extents).
  Logical volume centos/swap successfully resized.
3. Format the new swap space
[[email protected] ~]# mkswap /dev/centos/swap
mkswap: /dev/centos/swap: warning: wiping old swap signature.
Setting up swapspace version 1, size = 2359292 KiB
no label, UUID=5e487401-9ae0-4e1d-adff-2346edfc6244
4. Enable the extended logical volume
[[email protected] ~]# swapon -va
swapon /dev/mapper/centos-swap
swapon: /dev/mapper/centos-swap: found swap signature: version 1, page-size 4, same byte order
swapon: /dev/mapper/centos-swap: pagesize=4096, swapsize=2415919104, devsize=2415919104
5. Test that the logical volume has been extended properly
[[email protected] ~]# free -h
              total        used        free      shared  buff/cache   available
Mem:           7.6G        677M        6.5G        9.8M        507M        6.7G
Swap:          2.2G          0B        2.2G
[[email protected] ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       2359292 0       -1
Creating an LVM2 Logical Volume for Swap
To add a swap volume group (suppose /dev/centos/swap2 is the new volume)
1. Create the LVM2 logical volume of size 256 MB
[[email protected] ~]# lvcreate centos -n swap2 -L 256M
  Logical volume “swap2” created.
2. Format the new swap space
[[email protected] ~]# mkswap /dev/centos/swap2
Setting up swapspace version 1, size = 262140 KiB
no label, UUID=6ea40455-47a0-46bf-844e-ec0ebd4a4e6a
3. Add the following entry to the /etc/fstab file
/dev/mapper/centos-swap2 swap                    swap    defaults        0 0
4. Enable the extended logical volume
[[email protected] ~]# swapon –va
swapon /dev/mapper/centos-swap2
swapon: /dev/mapper/centos-swap2: found swap signature: version 1, page-size 4, same byte order
swapon: /dev/mapper/centos-swap2: pagesize=4096, swapsize=268435456, devsize=268435456
5. Verify the swap space
[[email protected] ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       2097148 0       -1
/dev/dm-3                               partition       262140  0       -2
Creating a Swap File
To Add a swap file
1. Determine the size of the new swap file in megabytes and multiply by 1024 to determine the number of blocks. For example, the block size of a 64 MB swap file is 65536.
2. At a shell prompt as root, type the following command with count being equal to the desired block size:
[[email protected] ~]# dd if=/dev/zero of=/swapfile bs=1024 count=65536
65536+0 records in
65536+0 records out
67108864 bytes (67 MB) copied, 0.0893063 s, 751 MB/s
[[email protected] ~]# ls -ld /swapfile
-rw-r–r–. 1 root root 67108864 May 17 16:38 /swapfile
[[email protected] ~]# du -sh /swapfile
64M     /swapfile
3. Change the permissions of the newly created file
[[email protected] ~]# chmod 0600 /swapfile
4. Setup the swap file with the command
[[email protected] ~]# mkswap /swapfile
Setting up swapspace version 1, size = 65532 KiB
no label, UUID=8a404550-e8a3-4f2b-9daf-137fc34f7b6d
5. Edit /etc/fstab and enable the newly added swap space
/swapfile          swap            swap    defaults        0 0
[[email protected] ~]# swapon -va
swapon /swapfile
swapon: /swapfile: found swap signature: version 1, page-size 4, same byte order
swapon: /swapfile: pagesize=4096, swapsize=67108864, devsize=67108864
6. Verify the swap space created.
[[email protected] ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       2097148 0       -1
/dev/dm-3                               partition       262140  0       -2
/swapfile                               file    65532   0       -3
Hope this has helped you ..
Thanks!!!!