How to debug NFS issues

  • 7011571
  • 03-Jan-2013
  • 03-Jan-2013

Environment


SUSE Linux Enterprise Server 11
SUSE Linux Enterprise Server 10
Novell Open Enterprise Server 11 (OES 11) Linux
Novell Open Enterprise Server 2 (OES 2) Linux
SUSE Linux Enterprise Desktop 10
SUSE Linux Enterprise Desktop 11

Situation

A NFS server or client on SUSE Linux Enterprise of Novell OES doesn't work as expected. The logs don't provide any details about what goes wrong.

Resolution

It depends on the type of issue, but if it seems to be a sole NFS issue it makes sense to debug the issue in the following order:

  1. Set up the environment so that the problematic behavior can be reproduced immediately. It only makes sense to enable additional debugging when the issue is directly reproduced and no other distracting logs have to be read and analysed. So the basic steps should always be:
    1. Enable debugging
    2. Promptly reproduce the issue
    3. Disable debugging
  2. rpcdebug is very helpful in analysing the NFS or RPC communication. If you want to get additional logging information from the NFS Server aka nfsd use this command:

    rpcdebug -m nfsd all:

    for NFS client analysis use:

    rpcdebug -m nfs all

    Please note that the previous command uses "nfs" instead of "nfsd" as a parameter to option -m.
    In rare cases the RPC protocol needs to be debugged and that can be achieved by using:

    rpcdebug -m rpc all

    The additional logging information will appear in /var/log/messages or can be viewed with the command "dmesg".

    Disable the respective debugging options again by using the option -c in front of the trailing all:

    rpcdebug -m nfsd -c all
    rpcdebug -m nfs -c all
    rpcdebug -m rpc -c all

    Please note that it can be helpful when analysing a NFS server issue to enable the debugging on the client at the same time to see how the communication looks on both sides.

  3. If all of the above rpc commands didn't provide enough or detailed information, advanced logging on kernel level with sysctl should be considered. It is not recommended to mix rpcdebug and the following sysctl commands since this would fill up the logs rapidly and would make them hardly readable. As a start only one of them should be used for debugging purposes.

    To enable additional logging for nfsd use:

    sysctl -w sunrpc.nfsd_debug=1023

    for an NFS client issue:

    sysctl -w sunrpc.nfs_debug=1023

    and finally for RPC:

    sysctl -w sunrpc.rpc_debug=1023

    These commands also lead to additional messages in /var/log/messages that can be viewed via "dmesg" as well.

    The respective debugging can be disabled by using the same commands like above, but change "1023" to "0":

    sysctl -w sunrpc.nfsd_debug=0

    or

    sysctl -w sunrpc.nfs_debug=0

    or

    sysctl -w sunrpc.rpc_debug=0